Open surgical instrument detecting and positioning method and device

Abstract

The invention discloses a method for detecting and positioning an open surgical instrument. The method comprises the following steps: acquiring a surgical image data set; the surgical image data set comprises a plurality of surgical image data; the surgical image data comprises a surgical instrument image in an operation stage and corresponding label information; the label information comprises a surgical instrument category and a minimum outer surrounding frame position coordinate; preprocessing the operation image data set to obtain a preprocessed data set; performing training processing on a preset surgical instrument detection model by utilizing the pre-processing data set to obtain a trained surgical instrument detection model; and processing the acquired surgical image information by using the trained surgical instrument detection model to obtain surgical instrument category information and surgical instrument position information.

Description

Technical Field

[0001] The present invention relates to the fields of computer image processing and artificial intelligence, and in particular to a method and device for detecting and positioning open surgical instruments. Background Art

[0002] For doctors who use surgical instruments to perform open surgery, accurate and reasonable evaluation of the doctor's surgical operation is conducive to improving the doctor's surgical skills and improving shortcomings and deficiencies. At present, computer vision technology has developed rapidly and matured, and it can realize the image detection of surgical instruments, providing a feasible technical approach for doctors to evaluate the quality of surgery.

[0003] Current research on surgical instrument detection based on computer vision mainly focuses on minimally invasive surgery, such as robot-assisted surgery and laparoscopic surgery. There is less research on surgical instrument detection for open surgery. Minimally invasive surgery does not require additional acquisition equipment, and the view under the endoscope is the complete doctor's operating perspective. Compared with minimally invasive surgery, open surgery acquisition equipment requires additional installation and is collected from a third-party perspective. Due to ethical and privacy reasons, it is difficult to obtain open surgery data sets.

[0004] The difficulties in the field of open surgical instrument detection are mainly concentrated on the occlusion and angle change problems caused by the two-dimensional visual image acquisition perspective. The acquisition equipment of most surgical images or videos is fixed, and occlusion between surgical instruments and organs and tissues is prone to occur in the acquisition field of view. In open surgery, there is also occlusion and truncation of surgical instruments by doctors' hands. Frequent occlusion will lead to incomplete shapes of surgical instruments. Fixed acquisition perspective causes surgical instruments to frequently change angles as doctors operate. The same type of surgical instruments in different frames have the characteristics of variable shapes. In addition, the surgical consumables in open surgery are small in size and flexible, resulting in flexible volume proportions of surgical consumables. The above characteristics bring difficulties to the detection of open surgical instruments. The current surgical instrument detection algorithm has low detection accuracy for surgical instruments and surgical consumables with variable shapes and flexible volumes. Therefore, it is necessary to develop high-precision open surgical instrument detection and positioning methods to address the above detection difficulties. Summary of the invention

[0005] The current surgical instrument detection algorithm has low detection accuracy for surgical instruments and surgical consumables with variable shapes and flexible volumes in open surgery. In order to solve the above technical difficulties, the present invention provides an open surgical instrument detection and positioning method and device.

[0006] In a first aspect of the embodiment of the present application, a method for detecting and positioning an open surgical instrument is disclosed, comprising:

[0007] S1, obtaining a surgical image data set; the surgical image data set includes a plurality of surgical image data; the surgical image data includes images of surgical instruments in an operation stage and corresponding label information; the label information includes surgical instrument categories and minimum outer bounding box position coordinates;

[0008] S2, preprocessing the surgical image dataset to obtain a preprocessed dataset;

[0009] S3, using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model;

[0010] S4, using the trained surgical instrument detection model to process the collected surgical image information to obtain surgical instrument category information and surgical instrument location information.

[0011] The preprocessing of the surgical image data set to obtain a preprocessed data set includes:

[0012] S21, performing data cleaning processing on the surgical image data set to obtain a cleaned data set;

[0013] S22, normalizing the cleaned data set to obtain a normalized data set;

[0014] S23, performing uniform distribution processing on the normalized data set to obtain a preprocessed data set.

[0015] The uniform distribution processing is performed on the normalized data set to obtain a preprocessed data set, including:

[0016] S231, performing statistics on the value distribution of the normalized data set to obtain value distribution information; the value distribution information includes a number of value intervals and a data distribution probability of each value interval;

[0017] S232, obtaining the median value of the normalized data set;

[0018] S233, taking the median value as the center of the preset value interval and taking the total probability of the interval being greater than a set probability threshold as the goal, determining N value intervals on the left and N value intervals on the right of the median value;

[0019] S234, constructing a preprocessed data set using all data in the normalized data set whose values ​​are located in the N value intervals to the left and the N value intervals to the right of the median value.

[0020] The surgical instrument detection model comprises: a feature extraction unit, a feature analysis unit, a feature fusion unit and a feature detection unit;

[0021] The feature extraction unit includes a first convolution module, a second convolution module, a first feature transformation module, a third convolution module, a second feature transformation module, a fourth convolution module, a third feature transformation module, a fifth convolution module, a fourth feature transformation module, a pyramid pooling module, and a channel attention module;

[0022] The feature analysis unit includes a first upsampling layer module, a first splicing layer module, a first feature conversion module, a second upsampling layer module, a second splicing layer module, and a second feature conversion module;

[0023] The feature fusion unit includes a sixth convolution module, a third splicing layer module, a third feature conversion module, a seventh convolution module, a fourth splicing layer module, and a fourth feature conversion module;

[0024] The feature detection unit includes a first detection head module, a second detection head module, a third detection head module and a fusion detection module.

[0025] The output end of the first convolution module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the first feature transformation module, the output end of the first feature transformation module is connected to the input end of the third convolution module, and the output end of the third convolution module is connected to the input end of the second feature transformation module; the output end of the second feature transformation module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the third feature transformation module; the output end of the third feature transformation module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the fourth feature transformation module; the output end of the fourth feature transformation module is connected to the input end of the pyramid pooling module; the output end of the pyramid pooling module is connected to the input end of the channel attention module;

[0026] The output end of the first upsampling layer module is connected to the input end of the first splicing layer module; the output end of the first splicing layer module is connected to the input end of the first feature conversion module; the output end of the first feature conversion module is connected to the input end of the second upsampling layer module; the output end of the second upsampling layer module is connected to the input end of the second splicing layer module; the output end of the second splicing layer module is connected to the input end of the second feature conversion module;

[0027] The output end of the sixth convolution module is connected to the input end of the third splicing layer module; the output end of the third splicing layer module is connected to the input end of the third feature conversion module; the output end of the third feature conversion module is connected to the input end of the seventh convolution module; the output end of the seventh convolution module is connected to the input end of the fourth splicing layer module; the output end of the fourth splicing layer module is connected to the input end of the fourth feature conversion module;

[0028] The output end of the first detection head module, the output end of the second detection head module and the output end of the third detection head module are respectively connected to the input end of the fusion detection module;

[0029] The output end of the second feature transformation module is connected to the input end of the second splicing layer module;

[0030] The output end of the third feature transformation module is connected to the input end of the first splicing layer module;

[0031] The output end of the first feature conversion module is connected to the input end of the third splicing layer module;

[0032] The output end of the channel attention module is connected to the input end of the first upsampling layer module and the input end of the third concatenation layer module respectively;

[0033] The output end of the second feature conversion module is connected to the input end of the sixth convolution module and the input end of the first detection head module respectively;

[0034] The output end of the third feature conversion module is connected to the input end of the second detection head module;

[0035] The output end of the fourth feature conversion module is connected to the input end of the third detection head module;

[0036] The input end of the first convolution module is used to receive surgical image information;

[0037] The first detection head module is used to output category information and position information of a surgical instrument of a first scale;

[0038] The second detection head module is used to output category information and position information of the surgical instrument of a second scale;

[0039] The third detection head module is used to output category information and position information of surgical instruments of a third scale;

[0040] The fusion detection module is used to perform fusion processing on the output of the first detection head module, the output of the second detection head module and the output of the third detection head module to obtain a surgical instrument category prediction value and a surgical instrument position prediction value.

[0041] The feature transformation module includes a first convolution submodule, a second convolution submodule, a segmentation layer submodule, a bottleneck layer submodule combination, a splicing layer submodule, and a third convolution submodule; the bottleneck layer submodule combination includes n serially connected bottleneck layer submodules;

[0042] In the feature transformation module, the output end of the first convolution sub-module is respectively connected to the input end of the bottleneck layer sub-module combination, the input end of the segmentation layer sub-module and the input end of the splicing layer sub-module; the output end of the segmentation layer sub-module is connected to the input end of the bottleneck layer sub-module combination; the output end of the bottleneck layer sub-module combination is connected to the input end of the splicing layer sub-module; the output end of the second convolution sub-module is connected to the input end of the splicing layer sub-module; the output end of the splicing layer sub-module is connected to the input end of the third convolution sub-module.

[0043] The method of using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model includes:

[0044] Initialize the number of training iterations; use the preprocessed data set as the training data set; use the surgical image data in the preprocessed data set as training data of the training data set;

[0045] Inputting the training data in the training data set as input data into a surgical instrument detection model;

[0046] The input data is processed using the surgical instrument detection model to obtain a prediction value; the prediction value includes a surgical instrument category prediction value and a surgical instrument position prediction value; the surgical instrument position prediction value is a prediction value of the position coordinates of the minimum outer bounding box of the surgical instrument;

[0047] Performing difference calculation processing on the obtained predicted value and the label information corresponding to the input data to obtain a difference value;

[0048] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0049] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0050] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0051] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0052] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0053] When the model training state does not meet the training termination condition, the parameters of the surgical instrument detection model are updated using the parameter update model, the training iteration number value is increased by 1, and the execution is triggered to input the training data in the training data set as input data into the surgical instrument detection model;

[0054] When the model training state satisfies the training termination condition, the training process of the surgical instrument detection model is completed to obtain a trained surgical instrument detection model.

[0055] In a second aspect of the embodiment of the present application, an open surgical instrument detection and positioning device is disclosed, the device comprising:

[0056] A memory storing executable program code;

[0057] a processor coupled to the memory;

[0058] The processor calls the executable program code stored in the memory to execute the open surgical instrument detection and positioning method.

[0059] In a third aspect of the embodiments of the present application, a computer storable medium is disclosed, wherein the computer storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the open surgical instrument detection and positioning method.

[0060] In a fourth aspect of the embodiments of the present application, an information data processing terminal is disclosed, and the information data processing terminal is used to implement the open surgical instrument detection and positioning method.

[0061] The beneficial effects of the present invention are:

[0062] The present invention mainly focuses on the problem of surgical instrument detection in open surgery. By constructing a surgical instrument detection model suitable for surgical instrument targets, the problem of difficulty in detecting surgical instruments and surgical consumables with variable shapes and flexible volumes is solved, thereby improving the accuracy and robustness of surgical instrument detection.

[0063] The present invention designs a surgical instrument detection model, and specifically designs a feature extraction unit, a feature analysis unit, a feature fusion unit and a feature detection unit; through the feature extraction unit, feature extraction is performed on the image target, and the feature extraction ability of the model for the input image is enhanced by adjusting the input of the residual module and adding convolution modules of different channels; the feature analysis unit is used to assign different weights to the features extracted by the backbone network, thereby improving the utilization efficiency of the feature map; in the feature fusion unit, a feature conversion module is used to improve the efficiency of feature fusion and make the model lighter; the proposed two-stage bounding box regression loss function assigns different attentions at different stages of training, performs indiscriminate learning on all samples in the early stage of training, and in the later stage of training, the training resources are inclined to difficult-to-detect samples and low-quality samples, thereby effectively improving the training efficiency.

[0064] The present invention performs surgical instrument detection based on computer vision technology, automatically identifies and locates the categories of surgical instruments appearing in images or videos based only on surgical images or videos, and can effectively learn detailed tiny features. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0066] Figure 2 It is a block diagram of the components of the surgical instrument detection model of the present invention;

[0067] Figure 3 A block diagram of the composition of the feature transformation module of the surgical instrument detection model of the present invention;

[0068] Figure 4 It is a block diagram of the composition of the pyramid pooling module of the surgical instrument detection model of the present invention;

[0069] Figure 5 This is a block diagram of the composition of the convolution module of the surgical instrument detection model of the present invention. DETAILED DESCRIPTION

[0070] In order to better understand the content of the present invention, an embodiment is given here.

[0071] Figure 1 1 is a flow chart for implementing the method of the present invention; Figure 2 It is a block diagram of the components of the surgical instrument detection model of the present invention; Figure 3 A block diagram of the composition of the feature transformation module of the surgical instrument detection model of the present invention; Figure 4 It is a block diagram of the composition of the pyramid pooling module of the surgical instrument detection model of the present invention; Figure 5 This is a block diagram of the composition of the convolution module of the surgical instrument detection model of the present invention.

[0072] In view of the problem that the current surgical instrument detection algorithm has low detection accuracy for surgical instruments and surgical consumables with variable shapes and flexible volumes in open surgery, the present invention discloses a method and device for detecting and positioning open surgical instruments.

[0073] In a first aspect of the embodiment of the present application, a method for detecting and positioning an open surgical instrument is disclosed, comprising:

[0074] S1, obtaining a surgical image data set; the surgical image data set includes a plurality of surgical image data; the surgical image data includes images of surgical instruments in an operation stage and corresponding label information; the label information includes surgical instrument categories and minimum outer bounding box position coordinates;

[0075] S2, preprocessing the surgical image dataset to obtain a preprocessed dataset;

[0076] S3, using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model;

[0077] S4, using the trained surgical instrument detection model to process the collected surgical image information to obtain surgical instrument category information and surgical instrument location information;

[0078] The preprocessing of the surgical image data set to obtain a preprocessed data set includes:

[0079] S21, performing data cleaning processing on the surgical image data set to obtain a cleaned data set;

[0080] S22, normalizing the cleaned data set to obtain a normalized data set;

[0081] S23, performing uniform distribution processing on the normalized data set to obtain a preprocessed data set;

[0082] The uniform distribution processing is performed on the normalized data set to obtain a preprocessed data set, including:

[0083] S231, performing statistics on the value distribution of the normalized data set to obtain value distribution information; the value distribution information includes a number of value intervals and a data distribution probability of each value interval;

[0084] S232, obtaining the median value of the normalized data set;

[0085] S233, taking the median value as the center of the preset value interval and taking the total probability of the interval being greater than a set probability threshold as the goal, determining N value intervals on the left and N value intervals on the right of the median value;

[0086] S234, constructing a preprocessed data set using all data in the normalized data set whose values ​​are located in the N value intervals to the left and the N value intervals to the right of the median value;

[0087] The surgical instrument detection model comprises: a feature extraction unit, a feature analysis unit, a feature fusion unit and a feature detection unit;

[0088] The feature extraction unit includes a first convolution module, a second convolution module, a first feature transformation module, a third convolution module, a second feature transformation module, a fourth convolution module, a third feature transformation module, a fifth convolution module, a fourth feature transformation module, a pyramid pooling module, and a channel attention module;

[0089] The feature analysis unit includes a first upsampling layer module, a first splicing layer module, a first feature conversion module, a second upsampling layer module, a second splicing layer module, and a second feature conversion module;

[0090] The feature fusion unit includes a sixth convolution module, a third splicing layer module, a third feature conversion module, a seventh convolution module, a fourth splicing layer module, and a fourth feature conversion module;

[0091] The feature detection unit comprises a first detection head module, a second detection head module, a third detection head module and a fusion detection module;

[0092] The output end of the first convolution module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the first feature transformation module, the output end of the first feature transformation module is connected to the input end of the third convolution module, and the output end of the third convolution module is connected to the input end of the second feature transformation module; the output end of the second feature transformation module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the third feature transformation module; the output end of the third feature transformation module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the fourth feature transformation module; the output end of the fourth feature transformation module is connected to the input end of the pyramid pooling module; the output end of the pyramid pooling module is connected to the input end of the channel attention module;

[0093] The output end of the first upsampling layer module is connected to the input end of the first splicing layer module; the output end of the first splicing layer module is connected to the input end of the first feature conversion module; the output end of the first feature conversion module is connected to the input end of the second upsampling layer module; the output end of the second upsampling layer module is connected to the input end of the second splicing layer module; the output end of the second splicing layer module is connected to the input end of the second feature conversion module;

[0094] The output end of the sixth convolution module is connected to the input end of the third splicing layer module; the output end of the third splicing layer module is connected to the input end of the third feature conversion module; the output end of the third feature conversion module is connected to the input end of the seventh convolution module; the output end of the seventh convolution module is connected to the input end of the fourth splicing layer module; the output end of the fourth splicing layer module is connected to the input end of the fourth feature conversion module;

[0095] The output end of the first detection head module, the output end of the second detection head module and the output end of the third detection head module are respectively connected to the input end of the fusion detection module;

[0096] The output end of the second feature transformation module is connected to the input end of the second splicing layer module;

[0097] The output end of the third feature transformation module is connected to the input end of the first splicing layer module;

[0098] The output end of the first feature conversion module is connected to the input end of the third splicing layer module;

[0099] The output end of the channel attention module is connected to the input end of the first upsampling layer module and the input end of the third concatenation layer module respectively;

[0100] The output end of the second feature conversion module is connected to the input end of the sixth convolution module and the input end of the first detection head module respectively;

[0101] The output end of the third feature conversion module is connected to the input end of the second detection head module;

[0102] The output end of the fourth feature conversion module is connected to the input end of the third detection head module;

[0103] The input end of the first convolution module is used to receive surgical image information;

[0104] The first detection head module is used to output category information and position information of a surgical instrument of a first scale;

[0105] The second detection head module is used to output category information and position information of the surgical instrument of a second scale;

[0106] The third detection head module is used to output category information and position information of surgical instruments of a third scale;

[0107] The fusion detection module is used to perform fusion processing on the output of the first detection head module, the output of the second detection head module and the output of the third detection head module to obtain a surgical instrument category prediction value and a surgical instrument position prediction value;

[0108] The feature transformation module includes a first convolution unit, a second convolution unit, a segmentation layer unit, a bottleneck layer unit combination, a splicing layer unit, and a third convolution unit; the bottleneck layer unit combination includes n serially connected bottleneck layer units;

[0109] In the feature transformation module, the output end of the first convolution unit is connected to the input end of the bottleneck layer unit combination, the input end of the segmentation layer unit and the input end of the splicing layer unit respectively; the output end of the segmentation layer unit is connected to the input end of the bottleneck layer unit combination; the output end of the bottleneck layer unit combination is connected to the input end of the splicing layer unit; the output end of the second convolution unit is connected to the input end of the splicing layer unit; the output end of the splicing layer unit is connected to the input end of the third convolution unit;

[0110] In the present invention, the composition of each convolution unit, convolution module and convolution layer is as follows: Figure 5 shown.

[0111] The method of using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model includes:

[0112] Initialize the number of training iterations; use the preprocessed data set as the training data set;

[0113] Inputting the training data in the training data set as input data into a surgical instrument detection model;

[0114] The input data is processed using the surgical instrument detection model to obtain a prediction value; the prediction value includes a surgical instrument category prediction value and a surgical instrument position prediction value; the surgical instrument position prediction value is a prediction value of the position coordinates of the minimum outer bounding box of the surgical instrument;

[0115] Performing difference calculation processing on the obtained predicted value and the label information corresponding to the input data to obtain a difference value;

[0116] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0117] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0118] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0119] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0120] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0121] When the model training state does not meet the training termination condition, the parameters of the surgical instrument detection model are updated using the parameter update model, the training iteration number value is increased by 1, and the execution is triggered to input the training data in the training data set as input data into the surgical instrument detection model;

[0122] When the model training state satisfies the training termination condition, the training process of the surgical instrument detection model is completed to obtain a trained surgical instrument detection model.

[0123] The parameter updating model is:

[0124]

[0125] θ←θ+v;

[0126] In the formula, x (i) is the i-th training data in the training data set, y (i) is the label information of the i-th training data in the training data set, is the loss function, v is the parameter update value, θ is the parameter of the surgical instrument detection model, η is the initial parameter learning rate, α is the momentum angle parameter, 0≤α≤1, It means to find the partial derivative of variable θ, f(x (i) ; θ) represents the predicted value obtained by the surgical instrument detection model for the i-th training data in the training data set, f(·) is the calculation function corresponding to the surgical instrument detection model; exp represents the power operation of the constant e; η and α are preset values.

[0127] The first scale is smaller than the second scale, the second scale is smaller than the third scale, the first scale is 80×80×256, the second scale is 40×40×512, and the third scale is 20×20×512;

[0128] The difference value satisfies the convergence condition, which means that the difference value is less than a preset convergence threshold; the difference value does not satisfy the convergence condition, which means that the difference value is not less than the preset convergence threshold.

[0129] The difference calculation process may be implemented using a loss function.

[0130] The loss function may adopt a cross entropy loss function.

[0131] The bottleneck layer unit includes two 2D convolution modules connected in series;

[0132] The goal of determining the N value intervals on the left and the N value intervals on the right of the median value is to ensure that the sum of the data distribution probabilities of the N value intervals on the left and the N value intervals on the right of the median value is greater than the set probability threshold;

[0133] The set probability threshold may be 0.9;

[0134] The preprocessed data set is obtained by uniformly distributing the normalized data set, so as to ensure that the training set has the same data distribution as the entire database, thereby eliminating the impact of unbalanced data characteristics.

[0135] The fusion detection module is used to perform fusion processing on the output of the first detection head module, the output of the second detection head module, and the output of the third detection head module to obtain a surgical instrument category prediction value and a surgical instrument position prediction value, including:

[0136] The fusion detection module performs splicing processing on the output of the first detection head module, the output of the second detection head module and the output of the third detection head module to obtain a splicing vector, performs high-confidence detection processing on the splicing vector to obtain a surgical instrument category prediction value and a surgical instrument position prediction value;

[0137] The high confidence detection process may adopt a non-maximum suppression method or a confidence threshold filtering method, etc.

[0138] In the feature transformation module of the present invention, its input is calculated through three channels. The first channel is calculated by the first convolution block and the residual module, and the number of output channels is equal to the number of input channels; the second channel is calculated by the second convolution block, and the output channel becomes twice the input channel; the third channel is calculated by the third convolution block, and the input and output channels remain unchanged. The outputs of the three channels are spliced ​​through the splicing block, and the output is integrated into an output with the same channel dimension as the input through the fourth convolution block output; a network composed of convolution blocks and feature transformation modules is used to perform multiple stages of feature extraction; a channel attention module is used to process the features extracted by the aforementioned network, and the channel attention module is used to optimize the features. The attention is learned by a 1-dimensional convolution with a kernel of k and a sigmoid function to obtain the weights of each channel. The interaction between channels takes into account each channel and the k nearest neighbor channels of each channel, and the calculation is shown in (1).

[0139] ω=σ(C1D k (y)) (1)

[0140] Where C1D represents 1D convolution and σ is the sigmoid function. It is worth noting that the kernel size of the 1D convolution is selected by the adaptive kernel size selection network, and the channel attention module constructs a nonlinear mapping function φ(k)=γ*kb between the number of channels C and the convolution kernel size k. Therefore, the equation relationship between the kernel size k and the channel dimension C is shown in formula (2).

[0141]

[0142] Among them, |T| odd represents the nearest neighbor odd number of t. The channel attention module can select different kernel sizes through nonlinear mapping according to the size of the channel dimension C, which determines the coverage of the inter-channel dependency and makes the channel attention operation more efficient and lightweight.

[0143] The input of the feature conversion module is processed by two paths. The first path passes the input through the first convolution block to obtain the output; the second path passes the input through the second convolution block and the residual network to obtain the output. The outputs of the two paths are spliced ​​by the splicing block and then processed by the third convolution block to obtain the final output.

[0144] During the training process of the surgical instrument detection model, a two-stage loss function is used for training, as shown in (3) and (4):

[0145]

[0146] in, is the CIoU loss, B represents the predicted box, B gt For the real frame. is the IoU loss, i.e., the intersection over union loss between the predicted box and the true box, and is calculated as shown in formula (5). is the distance loss, is the aspect ratio loss. Formulas (6) and (7) are the calculations of distance loss and aspect ratio loss of CIoU respectively. is the WIoU loss, and r is the non-monotonic focusing coefficient, which is used to balance the gradient gain. is the distance attention, calculated as shown in (8). is the IoU loss, calculated as shown in formula (5):

[0147]

[0148] Among them, |B∩B gt | is the intersection area between the predicted box and the true box, |B∪B gt | is the union area of ​​the predicted box and the true box.

[0149]

[0150] Among them, ρ 2 (p,p gt ) is the Euclidean distance between the center point of the predicted box and the true box, c 2 It is the diagonal distance between the predicted box and the real box.

[0151]

[0152] Among them, w gt ,h gt , are the width and height of the real box respectively. w, h are the width and height of the predicted box respectively. α is the similarity weight used to measure the aspect ratio.

[0153]

[0154] Among them, x and y are the horizontal and vertical coordinates of the center point of the prediction box, respectively. gt ,y gt W are the horizontal and vertical coordinates of the center point of the real frame. g and H g Respectively represent the width and height of the minimum bounding box of the predicted box and the real box. g and H g Removed from the computation graph to avoid blocking of convergence.

[0155] The data cleaning process includes filling missing values, smoothing noise data, and smoothing or deleting outliers; the smoothed noise data is obtained by first determining the noise data, and then smoothing the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the observed data, or greater than the measurement upper limit of the sensor of the observed data. The outlier point can be determined by the Kalman filter method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0156] The data normalization process is to map data of different value ranges into a specified value range, and its mathematical expression is:

[0157]

[0158] Among them, t is a data value in the data set, t max is the maximum value of all data in the data set, t min is the minimum value of all data in the data set, t 归一化 is the value of data t after normalization;

[0159] In a second aspect of the embodiment of the present application, an open surgical instrument detection and positioning device is disclosed, the device comprising:

[0160] A memory storing executable program code;

[0161] a processor coupled to the memory;

[0162] The processor calls the executable program code stored in the memory to execute the open surgical instrument detection and positioning method.

[0163] In a third aspect of the embodiments of the present application, a computer storable medium is disclosed, wherein the computer storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the open surgical instrument detection and positioning method.

[0164] In a fourth aspect of the embodiments of the present application, an information data processing terminal is disclosed, and the information data processing terminal is used to implement the open surgical instrument detection and positioning method.

[0165] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting and positioning an open surgical instrument, characterized in that: include: S1, obtaining a surgical image data set; the surgical image data set includes a plurality of surgical image data; the surgical image data includes images of surgical instruments in an operation stage and corresponding label information; the label information includes surgical instrument categories and minimum outer bounding box position coordinates; S2, preprocessing the surgical image dataset to obtain a preprocessed dataset; S3, using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model; S4, using the trained surgical instrument detection model to process the collected surgical image information to obtain surgical instrument category information and surgical instrument location information.

2. The open surgical instrument detection and positioning method according to claim 1, characterized in that: The preprocessing of the surgical image data set to obtain a preprocessed data set includes: S21, performing data cleaning processing on the surgical image data set to obtain a cleaned data set; S22, normalizing the cleaned data set to obtain a normalized data set; S23, performing uniform distribution processing on the normalized data set to obtain a preprocessed data set.

3. The open surgical instrument detection and positioning method according to claim 2, characterized in that: The uniform distribution processing is performed on the normalized data set to obtain a preprocessed data set, including: S231, performing statistics on the value distribution of the normalized data set to obtain value distribution information; the value distribution information includes a number of value intervals and a data distribution probability of each value interval; S232, obtaining the median value of the normalized data set; S233, taking the median value as the center of the preset value interval and taking the total probability of the interval being greater than a set probability threshold as the goal, determining N value intervals on the left and N value intervals on the right of the median value; S234, constructing a preprocessed data set using all data in the normalized data set whose values ​​are located in the N value intervals to the left and the N value intervals to the right of the median value.

4. The open surgical instrument detection and positioning method according to claim 2, characterized in that: The surgical instrument detection model comprises: a feature extraction unit, a feature analysis unit, a feature fusion unit and a feature detection unit; The feature extraction unit includes a first convolution module, a second convolution module, a first feature transformation module, a third convolution module, a second feature transformation module, a fourth convolution module, a third feature transformation module, a fifth convolution module, a fourth feature transformation module, a pyramid pooling module, and a channel attention module; The feature analysis unit includes a first upsampling layer module, a first splicing layer module, a first feature conversion module, a second upsampling layer module, a second splicing layer module, and a second feature conversion module; The feature fusion unit includes a sixth convolution module, a third splicing layer module, a third feature conversion module, a seventh convolution module, a fourth splicing layer module, and a fourth feature conversion module; The feature detection unit includes a first detection head module, a second detection head module, a third detection head module and a fusion detection module.

5. The open surgical instrument detection and positioning method according to claim 4, characterized in that: The output end of the first convolution module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the first feature transformation module, the output end of the first feature transformation module is connected to the input end of the third convolution module, and the output end of the third convolution module is connected to the input end of the second feature transformation module; the output end of the second feature transformation module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the third feature transformation module; the output end of the third feature transformation module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the fourth feature transformation module; the output end of the fourth feature transformation module is connected to the input end of the pyramid pooling module; the output end of the pyramid pooling module is connected to the input end of the channel attention module; The output end of the first upsampling layer module is connected to the input end of the first splicing layer module; the output end of the first splicing layer module is connected to the input end of the first feature conversion module; the output end of the first feature conversion module is connected to the input end of the second upsampling layer module; the output end of the second upsampling layer module is connected to the input end of the second splicing layer module; the output end of the second splicing layer module is connected to the input end of the second feature conversion module; The output end of the sixth convolution module is connected to the input end of the third splicing layer module; the output end of the third splicing layer module is connected to the input end of the third feature conversion module; the output end of the third feature conversion module is connected to the input end of the seventh convolution module; the output end of the seventh convolution module is connected to the input end of the fourth splicing layer module; the output end of the fourth splicing layer module is connected to the input end of the fourth feature conversion module; The output end of the first detection head module, the output end of the second detection head module and the output end of the third detection head module are respectively connected to the input end of the fusion detection module; The output end of the second feature transformation module is connected to the input end of the second splicing layer module; The output end of the third feature transformation module is connected to the input end of the first splicing layer module; The output end of the first feature conversion module is connected to the input end of the third splicing layer module; The output end of the channel attention module is connected to the input end of the first upsampling layer module and the input end of the third concatenation layer module respectively; The output end of the second feature conversion module is connected to the input end of the sixth convolution module and the input end of the first detection head module respectively; The output end of the third feature conversion module is connected to the input end of the second detection head module; The output end of the fourth feature conversion module is connected to the input end of the third detection head module; The input end of the first convolution module is used to receive surgical image information; The first detection head module is used to output category information and position information of a surgical instrument of a first scale; The second detection head module is used to output category information and position information of the surgical instrument of a second scale; The third detection head module is used to output category information and position information of surgical instruments of a third scale; The fusion detection module is used to perform fusion processing on the output of the first detection head module, the output of the second detection head module and the output of the third detection head module to obtain a surgical instrument category prediction value and a surgical instrument position prediction value.

6. The open surgical instrument detection and positioning method according to claim 5, characterized in that: The feature transformation module includes a first convolution submodule, a second convolution submodule, a segmentation layer submodule, a bottleneck layer submodule combination, a splicing layer submodule, and a third convolution submodule; the bottleneck layer submodule combination includes n serially connected bottleneck layer submodules; In the feature transformation module, the output end of the first convolution sub-module is respectively connected to the input end of the bottleneck layer sub-module combination, the input end of the segmentation layer sub-module and the input end of the splicing layer sub-module; the output end of the segmentation layer sub-module is connected to the input end of the bottleneck layer sub-module combination; the output end of the bottleneck layer sub-module combination is connected to the input end of the splicing layer sub-module; the output end of the second convolution sub-module is connected to the input end of the splicing layer sub-module; the output end of the splicing layer sub-module is connected to the input end of the third convolution sub-module.

7. The open surgical instrument detection and positioning method according to claim 1, characterized in that: The method of using the preprocessed data set to train a preset surgical instrument detection model to obtain a trained surgical instrument detection model includes: Initialize the number of training iterations; use the preprocessed data set as the training data set; use the surgical image data in the preprocessed data set as training data of the training data set; Inputting the training data in the training data set as input data into a surgical instrument detection model; The input data is processed using the surgical instrument detection model to obtain a prediction value; the prediction value includes a surgical instrument category prediction value and a surgical instrument position prediction value; the surgical instrument position prediction value is a prediction value of the position coordinates of the minimum outer bounding box of the surgical instrument; Performing difference calculation processing on the obtained predicted value and the label information corresponding to the input data to obtain a difference value; Determine whether the difference value satisfies a convergence condition, and obtain a first determination result; When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result; When the second judgment result is no, determining that the model training state does not meet the training termination condition; When the second judgment result is yes, determining that the model training state satisfies the training termination condition; When the first judgment result is yes, determining that the model training state satisfies a training termination condition; When the model training state does not meet the training termination condition, the parameters of the surgical instrument detection model are updated using the parameter update model, the training iteration number value is increased by 1, and the execution is triggered to input the training data in the training data set as input data into the surgical instrument detection model; When the model training state satisfies the training termination condition, the training process of the surgical instrument detection model is completed to obtain a trained surgical instrument detection model.

8. An open surgical instrument detection and positioning device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the open surgical instrument detection and positioning method according to any one of claims 1 to 7.

9. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the open surgical instrument detection and positioning method according to any one of claims 1 to 7.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the open surgical instrument detection and positioning method as described in any one of claims 1 to 7.

Images