A computer vision-based surgical consumable identification method

Abstract

The present application relates to the field of image recognition, and specifically to a surgical consumable recognition method based on computer vision. The present application adopts an adaptive receptive field convolution strategy and a frequency domain branch complement, realizes double-dimensional complementary representation of spatial structure and material attribute. The present application constructs two kinds of graph structures of association graph, utilizes a graph attention network to carry out modal information propagation, applies an optimal transmission mechanism to cross-graph alignment of spatial-frequency domain features, and overcomes the defect that a traditional point-by-point matching method is easily disturbed by local noise; the present application explicitly extracts gradient information through the difference between shallow and deep features, and adds it back to the original feature flow in the form of residual, effectively making up for the problem of edge detail loss in the deep network. The present application adopts a bidirectional reconstruction measurement strategy for final classification, establishes strict symmetry constraints, and realizes high-precision recognition under the condition of only a small amount of labeled samples.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image recognition, and particularly relates to a surgical consumable identification method based on computer vision. BACKGROUND

[0002] Intelligent management of surgical consumables is an important part of the construction of a smart medical system. With the rapid development of surgical techniques, the types and specifications of various surgical consumables (including sutures, hemostatic materials, implants, catheters, electrotomes, surgical blades, etc.) are increasingly diverse. The appearance of the surgical consumables is highly similar, the size difference is great, the material properties are diverse, and the surgical consumables often face complex imaging conditions such as reflection, obstruction, and shadow in the operating room environment. At present, the operating room mainly relies on manual visual counting, barcode / QR code scanning, or radio frequency identification tags to manage consumables. The above methods have defects such as low efficiency and easy errors, tedious operation and the need to scan one by one, label pasting changing the physical form of the consumables, and high cost. Some hospitals have begun to try to introduce computer vision technology to achieve automatic identification of consumables. However, most existing visual recognition methods directly apply general image classification models, lack specialized modeling of fine-grained features of surgical consumables (such as needle tip curvature, surface texture, material spectral characteristics, boundary morphology, etc.), and thus the recognition accuracy and robustness are difficult to meet the clinical application requirements when facing the real surgical consumable recognition task with extremely subtle differences between categories and limited labeled data. SUMMARY

[0003] In view of the above, in order to overcome the defects of the prior art, the application provides a surgical consumable identification method based on computer vision, which breaks through the limitations of the traditional single-path visual identification framework and realizes systematic improvement from three dimensions of "multi-view feature extraction", "global optimal alignment" and "edge sensitive enhancement". At the feature extraction level, the method discards the fixed receptive field convolution operation and instead adopts an adaptive receptive field convolution strategy, which enables the network to dynamically adjust the size and shape of the convolution kernel according to the actual size and morphology of the consumable, and simultaneously introduces a frequency domain branch to automatically filter the discriminative frequency components related to the material through a learnable spectral mask, realizing double-dimensional complementary representation of spatial structure and material attribute. At the feature fusion level, the method constructs two kinds of graph structures, namely a spatial adjacency graph and a frequency correlation graph, uses a graph attention network for intra-modal information propagation, and for the first time applies an optimal transport mechanism to the cross-graph alignment of spatial-frequency features - this method can establish an optimal correspondence between the two sets of features at the global level, fundamentally overcoming the defect that the traditional point-by-point matching method is easily disturbed by local noise. At the edge information processing level, the method designs an edge enhancement module based on difference operation, which explicitly extracts gradient information through the difference between shallow and deep features, and adds it back to the original feature stream in the form of residual, effectively making up for the loss of edge details in the deep network. In addition, the method also uses a bidirectional reconstruction measurement strategy for final classification - requiring the query sample and the class support set to be able to reconstruct each other, establishing a more stringent symmetry constraint, and realizing high-precision identification under the condition of only a small amount of labeled samples.

[0004] The application provides a surgical consumable identification method based on computer vision, which specifically comprises the following steps:

[0005] Obtaining a surgical consumable image to be identified and a surgical consumable image of a known consumable category;

[0006] Inputting the surgical consumable image into a spatial domain feature extraction branch and a frequency domain feature extraction branch respectively, performing adaptive receptive field convolution processing on the surgical consumable image by the spatial domain feature extraction branch to extract a spatial feature map, and performing frequency domain transformation on the surgical consumable image by the frequency domain feature extraction branch and extracting a frequency domain feature map by using a learnable spectral mask;

[0007] The spatial domain feature extraction branch performs adaptive receptive field convolution processing on the surgical consumable image to extract a spatial feature map, comprising:

[0008] Extracting an input feature map of the surgical consumable image by a convolutional neural network, performing global average pooling on the input feature map along the vertical direction and the horizontal direction respectively to obtain a vertical direction global description vector and a horizontal direction global description vector;

[0009] predict a vertical direction kernel scale according to the vertical direction global description vector, and predict a horizontal direction kernel scale according to the horizontal direction global description vector;

[0010] construct a sampling grid according to the vertical direction kernel scale and the horizontal direction kernel scale, and resample the input feature map based on the sampling grid;

[0011] perform convolution operation on the resampled feature map and an adaptive convolution kernel to obtain an adaptive receptive field convolution output;

[0012] perform modulation and bias processing on the adaptive receptive field convolution output, and perform channel enhancement through a channel attention mechanism to obtain the spatial feature map.

[0013] The frequency domain feature extraction branch performs frequency domain transformation on the surgical consumable image and extracts a frequency domain feature map using a learnable frequency spectrum mask, including:

[0014] perform two-dimensional discrete cosine transformation on each input feature map channel of the surgical consumable image to obtain an initial frequency spectrum representation;

[0015] upsample the learnable frequency spectrum mask to the same resolution as the initial frequency spectrum representation, and perform Sigmoid activation and completion processing on the upsampled learnable frequency spectrum mask to obtain a frequency spectrum weighting mask;

[0016] multiply the initial frequency spectrum representation and the frequency spectrum weighting mask element by element to obtain discriminative frequency spectrum features;

[0017] perform two-dimensional inverse discrete cosine transformation on the discriminative frequency spectrum features to obtain the frequency domain feature map.

[0018] perform adaptive fusion on the spatial feature map and the frequency domain feature map to obtain a fusion feature map; including:

[0019] concatenate the spatial feature map and the frequency domain feature map along the channel dimension to obtain a concatenated feature map;

[0020] process the concatenated feature map through a convolution subnetwork to generate two-channel attention score maps, the two channels corresponding to the spatial feature map and the frequency domain feature map, respectively;

[0021] perform Softmax normalization on the two-channel attention score maps position by position to obtain spatial branch fusion weights and frequency domain branch fusion weights;

[0022] perform position-by-position weighted combination on the spatial feature map and the frequency domain feature map according to the spatial branch fusion weights and the frequency domain branch fusion weights to obtain the fusion feature map.

[0023] constructing a two-path graph structure according to the fused feature map, the two-path graph structure comprising a spatial adjacency graph and a frequency domain association graph;

[0024] constructing a two-path graph structure according to the fused feature map, comprising:

[0025] taking each spatial position in the fused feature map as a node to obtain a node sequence;

[0026] calculating a spatial distance according to spatial coordinates of each node in the fused feature map, constructing an edge weight of the spatial adjacency graph based on the spatial distance and a Gaussian kernel function, and connecting each node in the spatial adjacency graph to only nodes with a spatial distance less than a preset neighborhood radius;

[0027] calculating a feature distance according to similarity between feature vectors of each node, constructing an edge weight of the frequency domain association graph based on the feature distance and a Gaussian kernel function, and connecting each node in the frequency domain association graph to only nodes with a feature distance less than a preset threshold.

[0028] encoding the spatial adjacency graph and the frequency domain association graph by using a graph attention network to obtain a spatial graph node representation and a frequency domain graph node representation;

[0029] aligning and fusing the spatial graph node representation and the frequency domain graph node representation by an optimal transport mechanism to obtain an aligned and fused feature, comprising:

[0030] mapping the spatial graph node representation and the frequency domain graph node representation to a spatial graph probability distribution and a frequency domain graph probability distribution by using a learnable projection matrix respectively;

[0031] calculating a cost matrix between the spatial graph node representation and the frequency domain graph node representation, each element in the cost matrix being a cosine distance between corresponding nodes;

[0032] solving an optimal transport matrix by using a Sinkhorn algorithm based on the spatial graph probability distribution and the frequency domain graph probability distribution as edge distribution constraints, with a transmission cost on the cost matrix being minimized as an objective;

[0033] re-weighting the spatial graph node representation according to the optimal transport matrix, supplementally enhancing the frequency domain graph node representation, and summing the two to obtain the aligned and fused feature.

[0034] inputting the aligned and fused feature into an edge enhancement module to enhance edge information, to obtain an edge enhanced feature, comprising:

[0035] The aligned fusion features are input into multiple deep separable convolution branches with different receptive fields, the outputs of each branch are spliced along the channel dimension, and then compressed through point-wise convolution to obtain multi-scale intermediate features;

[0036] The multi-scale intermediate features are sequentially subjected to first convolution processing to obtain shallow layer features, and the shallow layer features are subjected to average pooling and deep convolution processing to obtain deep layer features;

[0037] The deep layer features are subtracted from the shallow layer features to obtain shallow layer differential enhancement features, and the deep layer features are subtracted from the deep layer features after average pooling and deep convolution to obtain deep layer differential enhancement features;

[0038] The shallow layer differential enhancement features and the deep layer differential enhancement features are spliced and subjected to convolution processing, and then added to the aligned fusion features in residual to obtain the edge enhancement features.

[0039] Based on the edge enhancement features, the distance between the query sample and each category support set is calculated through a bidirectional feature reconstruction metric strategy, and the category of the surgical consumable image is determined according to the distance;

[0040] The distance between the query sample and each category support set is calculated through a bidirectional feature reconstruction metric strategy, comprising:

[0041] The feature of the query sample is represented as a query feature, and the features of all samples in each category support set are spliced along the spatial dimension to obtain support features of each category;

[0042] The query feature and the support feature are projected into query key-value triplets and support key-value triplets through a learnable projection matrix, respectively;

[0043] The attention weight is calculated according to the query key-value triplets and the support key-value triplets, the support key-value triplets are weighted using the attention weight, the query reconstruction feature is reconstructed, and the first reconstruction error between the query key-value triplets and the query reconstruction feature is calculated;

[0044] The reverse attention weight is calculated according to the support key-value triplets and the query key-value triplets, the query key-value triplets are weighted using the reverse attention weight, the support reconstruction feature is reconstructed, and the second reconstruction error between the support key-value triplets and the support reconstruction feature is calculated;

[0045] The first reconstruction error and the second reconstruction error are weighted and summed to obtain the distance between the query sample and each category support set.

[0046] The category of the surgical consumable image is determined according to the distance, comprising:

[0047] The distance between the query sample and each category support set is taken as a negative value, divided by a preset temperature coefficient, and the probability of the query sample belonging to each category is calculated by a Softmax function.

[0048] The category with the maximum probability is selected as the recognition result of the surgical consumable image.

[0049] The query sample refers to the edge enhancement feature corresponding to a single surgical consumable image whose category to be identified is unknown, and each category support set refers to the edge enhancement feature set of a small number of labeled samples contained in each known consumable category in the training data, which is used as a category reference prototype and compared with the query sample in a bidirectional reconstruction manner.

[0050] In the training stage, the network parameters are optimized end-to-end by using a classification loss, an alignment loss and an edge consistency loss to form a total loss function;

[0051] The classification loss is a cross-entropy loss, which is used to drive the network to correctly classify;

[0052] The alignment loss is the sum of the KL divergence between the spatial graph probability distribution and the frequency domain graph probability distribution, which is used to constrain the distribution consistency in the optimal transport process;

[0053] The edge consistency loss is the mean square error between the edge strength map output by the edge enhancement module and the edge ground truth map extracted from the original image, which is used to supervise the enhancement quality of the edge information.

[0054] The above-mentioned scheme has the following beneficial results:

[0055] (1) The application breaks through the limitation of the traditional single path visual recognition framework, realizes systematic improvement from the three dimensions of "multi-view feature extraction", "global optimal alignment" and "edge sensitive enhancement". In the feature extraction layer, the method discards the fixed receptive field convolution operation, and instead adopts an adaptive receptive field convolution strategy, so that the network can dynamically adjust the size and shape of the convolution kernel according to the actual size and shape of the consumables, and at the same time introduces a frequency domain branch to automatically filter the discriminative frequency components related to the material through a learnable spectral mask, realizing the complementary representation of spatial structure and material attribute in two dimensions. In the feature fusion layer, the method constructs two kinds of graph structures of spatial adjacency graph and frequency correlation graph, uses graph attention network for modal information propagation, and for the first time applies the optimal transport mechanism to the cross-graph alignment of spatial-frequency features - this method can establish the optimal correspondence between the two sets of features at the global level, fundamentally overcoming the defect that the traditional point-by-point matching method is easily disturbed by local noise. In the edge information processing layer, the method designs an edge enhancement module based on difference operation, which explicitly extracts gradient information through the difference between shallow and deep features, and adds it back to the original feature stream in the form of residual, effectively making up for the loss of edge details in the deep network. In addition, the method also uses a bidirectional reconstruction measurement strategy for final classification - requiring the query sample and the class support set to be able to reconstruct each other, establishing a more stringent symmetry constraint, and realizing high-precision recognition under the condition of only a small amount of labeled samples. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of a surgical consumable recognition method based on computer vision is provided.

[0057] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0059] Embodiment one, refer to Figure 1 A surgical consumable recognition method based on computer vision, specifically comprising the following steps:

[0060] Obtaining surgical consumable images to be recognized and surgical consumable images of known consumable categories;

[0061] The surgical consumables image is input into the spatial domain feature extraction branch and the frequency domain feature extraction branch respectively. The spatial domain feature extraction branch performs adaptive receptive field convolution processing on the surgical consumables image to extract spatial feature maps. The frequency domain feature extraction branch performs frequency domain transformation on the surgical consumables image and extracts frequency domain feature maps using a learnable spectral mask.

[0062] The spatial feature map and the frequency domain feature map are adaptively fused to obtain a fused feature map;

[0063] A dual-path graph structure is constructed based on the fused feature map, the dual-path graph structure including a spatial adjacency graph and a frequency domain correlation graph;

[0064] The spatial adjacency graph and the frequency domain association graph are encoded using a graph attention network to obtain spatial graph node representations and frequency domain graph node representations, respectively.

[0065] The spatial graph node representation and the frequency domain graph node representation are aligned and fused across graphs using the optimal transmission mechanism to obtain alignment and fusion features.

[0066] The alignment and fusion features are input into the edge enhancement module to enhance edge information, resulting in edge enhancement features;

[0067] Based on the edge enhancement features, the distance between the query sample and each category support set is calculated using a bidirectional feature reconstruction metric strategy, and the category of the surgical consumable image is determined based on the distance.

[0068] The query sample refers to the edge enhancement features corresponding to a single surgical consumable image whose category is unknown. Each category support set refers to the set of edge enhancement features of a small number of labeled samples contained in each known consumable category in the training data, which serves as a category reference prototype and is used for bidirectional reconstruction and comparison with the query sample.

[0069] Example 2: Based on the above examples, this example uses a spatial domain feature extraction branch to perform adaptive receptive field convolution processing on the surgical consumables image to extract a spatial feature map, including:

[0070] The input feature map of the surgical consumables image is extracted by a convolutional neural network. Global average pooling is then performed on the input feature map along the vertical and horizontal directions to obtain the vertical global description vector and the horizontal global description vector.

[0071] Predict the vertical kernel scale based on the vertical global description vector, and predict the horizontal kernel scale based on the horizontal global description vector;

[0072] A sampling grid is constructed based on the vertical kernel scale and the horizontal kernel scale, and the input feature map is resampled based on the sampling grid;

[0073] The resampled feature map is convolved with an adaptive convolution kernel to obtain the adaptive receptive field convolution output.

[0074] The adaptive receptive field convolution output is modulated and biased, and channel enhancement is performed through a channel attention mechanism to obtain the spatial feature map.

[0075] The frequency domain feature extraction branch performs frequency domain transformation on the surgical consumables image and extracts frequency domain feature maps using a learnable spectral mask, including:

[0076] A two-dimensional discrete cosine transform is performed on each channel of the input feature map of the surgical consumables image to obtain an initial spectral representation;

[0077] The learnable spectral mask is upsampled to the same resolution as the initial spectral representation, and the upsampled learnable spectral mask is then subjected to Sigmoid activation and completion processing to obtain a spectral weighted mask.

[0078] The initial spectral representation is multiplied element-wise by the spectral weighting mask to obtain the discriminative spectral features;

[0079] The discriminative spectral features are subjected to a two-dimensional inverse discrete cosine transform to obtain the frequency domain feature map.

[0080] Example 3: Based on the above examples, this example adaptively fuses the spatial feature map and the frequency domain feature map to obtain a fused feature map, including:

[0081] The spatial feature map and the frequency domain feature map are concatenated along the channel dimension to obtain a concatenated feature map;

[0082] The concatenated feature map is processed by a convolutional sub-network to generate a two-channel attention score map, the two channels corresponding to the spatial feature map and the frequency domain feature map, respectively;

[0083] The attention score maps of the two channels are normalized at each position using Softmax to obtain the spatial branch fusion weights and the frequency domain branch fusion weights.

[0084] Based on the spatial branch fusion weight and the frequency domain branch fusion weight, the spatial feature map and the frequency domain feature map are combined position-by-position with weights to obtain the fused feature map.

[0085] Example 4: This example is based on the above examples, and constructs a dual-path graph structure according to the fused feature map, including:

[0086] Each spatial location in the fused feature map is treated as a node, resulting in a node sequence;

[0087] The spatial distance is calculated based on the spatial coordinates of each node in the fused feature graph. The edge weights of the spatial adjacency graph are constructed based on the spatial distance and the Gaussian kernel function. Each node in the spatial adjacency graph is only connected to nodes whose spatial distance is less than a preset neighborhood radius.

[0088] The feature distance is calculated based on the similarity between the feature vectors of each node. The edge weights of the frequency domain association graph are constructed based on the feature distance and the Gaussian kernel function. Each node in the frequency domain association graph is only connected to nodes whose feature distance is less than a preset threshold.

[0089] Example 5: Based on the above examples, this example performs cross-graph alignment and fusion of the spatial graph node representation and the frequency domain graph node representation using an optimal transmission mechanism, including:

[0090] The spatial graph node representation and the frequency domain graph node representation are respectively mapped to spatial graph probability distribution and frequency domain graph probability distribution through learnable projection matrices;

[0091] Calculate the cost matrix between the spatial graph node representation and the frequency domain graph node representation, where each element in the cost matrix is ​​the cosine distance between the corresponding nodes;

[0092] Based on the spatial graph probability distribution and the frequency domain graph probability distribution as edge distribution constraints, and with the goal of minimizing the transmission cost on the cost matrix, the optimal transmission matrix is ​​solved by the Sinkhorn algorithm.

[0093] The spatial graph node representation is semantically reweighted based on the optimal transfer matrix, and the frequency domain graph node representation is supplemented and enhanced. The weighted sum of the two is then used to obtain the aligned fusion feature.

[0094] Example 6: Based on the above examples, this example inputs the alignment and fusion features into the edge enhancement module to enhance edge information, obtaining edge enhancement features, including:

[0095] The aligned and fused features are input into multiple depthwise separable convolutional branches with different receptive fields. The outputs of each branch are concatenated along the channel dimension and then the channels are compressed by pointwise convolution to obtain multi-scale intermediate features.

[0096] The multi-scale intermediate features are sequentially subjected to a first convolution process to obtain shallow features, and the shallow features are then subjected to average pooling and depth convolution processes to obtain deep features.

[0097] Subtracting the shallow features from the deep features yields shallow differential enhancement features; subtracting the deep features from the deep features after performing average pooling and depthwise convolution yields deep differential enhancement features.

[0098] The shallow differential enhancement feature and the deep differential enhancement feature are concatenated and then processed by convolution. The residuals of the concatenated feature and the alignment fusion feature are added to obtain the edge enhancement feature.

[0099] Example 7: Based on the above examples, this example calculates the distance between the query sample and the support sets of each category using a bidirectional feature reconstruction metric strategy, including:

[0100] The features of the query sample are represented as query features, and the features of all samples in each category support set are concatenated along the spatial dimension to form the support features of each category.

[0101] The query features and the support features are projected into query key-value triples and support key-value triples respectively using a learnable projection matrix;

[0102] Attention weights are calculated based on the query key-value triples and the support key-value triples. The support key-value triples are then weighted using the attention weights to reconstruct the query reconstruction features. The first reconstruction error between the query key-value triples and the query reconstruction features is then calculated.

[0103] Calculate the back attention weights based on the support key-value triples and the query key-value triples, use the back attention weights to weight the query key-value triples, reconstruct the support reconstruction features, and calculate the second reconstruction error between the support key-value triples and the support reconstruction features.

[0104] The first reconstruction error and the second reconstruction error are weighted and summed to obtain the distance between the query sample and the support sets of each category.

[0105] Determining the category of the surgical consumable image based on the distance includes:

[0106] The distance between the query sample and each category support set is negatively divided by a preset temperature coefficient, and the probability of the query sample belonging to each category is calculated using the Softmax function.

[0107] The category with the highest probability is selected as the recognition result of the surgical consumable image.

[0108] Example 8: Based on the above examples, this example uses a total loss function composed of classification loss, alignment loss, and edge consistency loss to perform end-to-end optimization of network parameters during the training phase.

[0109] The classification loss is the cross-entropy loss, which is used to drive the network to classify correctly;

[0110] The alignment loss is the sum of the KL divergences between the spatial graph probability distribution and the frequency domain graph probability distribution, used to constrain the distribution consistency in the optimal transmission process;

[0111] The edge consistency loss is the mean square error between the edge intensity map output by the edge enhancement module and the edge ground truth map extracted from the original image, and is used to supervise the enhancement quality of edge information.

[0112] Example 9: This example, based on the above examples, describes the complete workflow of the intelligent surgical consumables identification system.

[0113] I. System Deployment and Application Scenarios:

[0114] The intelligent surgical consumables identification system described in this embodiment is deployed in the consumables management workstation of the operating room of a tertiary hospital. This workstation is equipped with a high-resolution industrial camera, supplemental lighting equipment, and an industrial control computer with a GPU. Before the system was put into use, the hospital's operating room management personnel selected eight categories of common surgical consumables from the warehouse—including absorbable sutures, non-absorbable sutures, metal hemostatic clips, absorbable hemostatic clips, disposable electrosurgical scalpels, surgical blades, medical gauze pads, and medical catheter connectors—selecting five different specifications or models for each category. System operators used the industrial camera to capture multiple images of each sample from different angles and orientations. Each image was labeled by experienced operating room nurses, constructing a support set of five labeled samples for each consumable category, which was stored in the system database as a reference for identification.

[0115] After the support set was built, the system officially entered the daily use phase. One morning, an operating room nurse placed a batch of used surgical consumables awaiting inventory on a workbench in the designated recognition area. The industrial camera automatically triggered an image, capturing a color image containing various consumables. The system first preprocessed the image—scaling it to a uniform size, performing mean normalization to eliminate lighting differences, and converting the image from the RGB color space to separate it into red, green, and blue channels before sending it to the subsequent processing flow.

[0116] II. Processing procedure of the spatial domain feature extraction branch:

[0117] The preprocessed image is simultaneously fed into both the spatial domain feature extraction branch and the frequency domain feature extraction branch for parallel processing. In the spatial domain feature extraction branch, the system first performs global average pooling on the input feature map. Taking a real-world scenario as an example: the input feature map has 64 channels, a height of 80 pixels, and a width of 80 pixels. The system averages all 80 rows of pixels in each channel along the height direction to obtain a 64-dimensional vertical global description vector, which actually reflects the overall grayscale distribution characteristics of the consumable image in the vertical direction. Similarly, it averages along the width direction to obtain a 64-dimensional horizontal global description vector. For images of long and thin suture needles, the vertical and horizontal global description vectors exhibit significant anisotropy; while for gauze blocks with a near-square shape, the description vectors in the two directions are more balanced.

[0118] The system inputs the vertical global description vector into a prediction network consisting of convolutional layers, batch normalization layers, and LeakyReLU activation layers. This network outputs a scalar value between 0 and 1. Assuming the system's preset maximum kernel size is 9, and the prediction network outputs a value of 0.75, the system multiplies this value by 8 and adds 1, resulting in a vertical kernel scale of 7. Similarly, the horizontal prediction network might output 0.65, yielding a horizontal kernel scale of 6.2. The system divides these two kernel scale values ​​by the stride parameter, rounds them to an odd number, and obtains discrete convolutional kernel sizes—for example, 7 pixels vertically and 5 pixels horizontally. In other words, the system adaptively selects a 7×5 asymmetric convolutional kernel size for the current image.

[0119] Next, the system constructs a normalized sampling grid based on the predicted 7×5 convolution kernel size. This grid samples 7 points evenly in the vertical direction and 5 points evenly in the horizontal direction, with the coordinates of all points limited to between -1 and +1, forming a basic grid to guide the convolution sampling position. The system further calculates an offset field based on the deviation of the two kernel scale values ​​from the center value, allowing the sampling points to undergo a slight positional shift to accommodate the curvature or irregular shape of the consumable. For a curved suture needle, the offset field causes the sampling points to shift accordingly along the bending direction of the needle, enabling subsequent convolution operations to better conform to the actual contour of the needle.

[0120] The system resamples the input feature map using a pre-constructed sampling grid and offset field, remapping the features originally sampled on the regular grid to adaptively adjusted sampling positions, resulting in a resampled feature map. The system then constructs an adaptive convolutional kernel with dimensions equal to the number of input channels multiplied by 7 multiplied by 5 multiplied by the number of output channels. The parameters of this kernel were learned from a large number of images during system training. After convolving the resampled feature map with this adaptive convolutional kernel, the system further applies modulation enhancement to the output—first, a modulation map of the same size as the convolutional output is generated, where each value is constrained to between -1 and +1 using the Tanh function to enhance or suppress responses at specific spatial locations; simultaneously, a bias map is generated as a global offset. Both the modulation map and the bias map originate from further convolutional processing of the input feature map, allowing the modulation and bias process to be dynamically adjusted according to the input content. After modulation enhancement, the system weights each channel through a channel attention mechanism: global average pooling is performed on the feature map to obtain the global description value of each channel, and channel attention weights between 0 and 1 are generated through convolutional layers and the Sigmoid function. These weights are multiplied by all spatial locations of the corresponding channel so that the channel that contributes more to the recognition task receives a higher response.

[0121] After the above-mentioned adaptive receptive field convolution, modulation enhancement and channel attention triple processing, the spatial domain feature extraction branch outputs a spatial feature map, which fully preserves the shape contour, surface texture and spatial structure information of the consumable.

[0122] III. Processing procedure of the frequency domain feature extraction branch:

[0123] In parallel with the spatial domain branch, the frequency domain feature extraction branch processes the same input image. The system independently performs a two-dimensional discrete cosine transform on each channel of the input image. Taking the red channel as an example, the system multiplies each pixel value in the channel with the cosine basis function and accumulates them to obtain the coefficient values ​​at each frequency position in the spectral representation. The cosine basis function changes gently at low frequencies, corresponding to smooth regions and overall brightness distribution in the image; it changes drastically at high frequencies, corresponding to edges and detailed textures in the image. After the two-dimensional discrete cosine transform, the system obtains the initial spectral representations of the three channels. Each spectral representation has the same spatial resolution as the input image, but the value at each position does not represent pixel intensity, but rather the energy level of the corresponding frequency component.

[0124] The system loads a pre-learned spectral mask template with dimensions of 1 x 3 x 16 x 16 and very few parameters. The system upsamples this template using bilinear interpolation to the exact same resolution as the initial spectral representation—for example, if the initial spectral representation is 80 x 80, the mask template is upsampled to 3 x 80 x 80. The upsampled template is then constrained to between 0 and 1 using the sigmoid function, and the result is subtracted from a tensor of all ones to obtain the final spectral weighted mask. In this mask, values ​​close to 0 indicate that the frequency component should be preserved, and values ​​close to 1 indicate that the frequency component should be suppressed. This "suppression-based" mask design allows the network to retain all frequency components in the initial training phase and gradually learn to suppress irrelevant frequencies as training progresses.

[0125] Taking the distinction between metal and absorbable hemostatic clips as an example: metal hemostatic clips have regular transverse serrations on their surface, exhibiting stable energy peaks in the high-frequency region of the spectrum; absorbable hemostatic clips are made of polymer materials, with a smooth, serrated surface, and their spectral energy is mainly concentrated in the low- and mid-frequency regions. After extensive training, the system learns a spectral mask that takes low values ​​in specific high-frequency regions corresponding to metal consumables to preserve these discriminative frequencies, and takes high values ​​in frequency regions corresponding to background textures to suppress interference.

[0126] The system multiplies the initial spectral representation element-wise with a spectral weighting mask, preserving discriminative frequency components while suppressing irrelevant or interfering frequency components. Subsequently, the system performs a two-dimensional inverse discrete cosine transform on the weighted spectral representation, converting the filtered frequency information back to the spatial domain to obtain a frequency domain feature map. While this feature map resembles a regular image in form, it actually carries material attribute information filtered by the spectrum, offering a unique advantage in distinguishing different material consumables.

[0127] IV. Adaptive fusion of spatial and frequency domain features:

[0128] After the spatial domain and frequency domain branches output spatial and frequency domain feature maps respectively, the system enters the feature fusion stage. Assuming that the spatial size of both feature maps is 80 by 80 and the number of channels is 64, the system first concatenates the two along the channel dimension to obtain a concatenated feature map with 128 channels.

[0129] The system processes the stitched feature map using a lightweight convolutional sub-network consisting of three convolutional layers. The first convolutional layer compresses 128 channels to 64 channels, the second layer maintains 64 channels, and the third layer compresses the 64 channels to 2 channels. Each convolutional layer is followed by a LeakyReLU activation function. After processing by this sub-network, the system obtains a 2-channel, 80x80 attention score map. The two channels of this score map correspond to the initial scores of the spatial and frequency domain features, respectively.

[0130] The system independently performs Softmax normalization on each spatial location of the attention score map. Assuming that at a certain spatial location, the spatial feature score map value is 2.3 and the frequency domain feature score map value is 1.7, the system calculates 2.3 raised to the power of the natural constant to approximately 9.97, and 1.7 raised to the power of the natural constant to approximately 5.47. The sum of these two is 15.44. The fusion weight of the spatial branch at this location is approximately 0.65 (9.97 divided by 15.44), and the fusion weight of the frequency domain branch at this location is approximately 0.35 (5.47 divided by 15.44). The system repeats the above normalization calculation for all 6400 spatial locations, resulting in two sets of fusion weight maps of size 64 x 80 x 80—one corresponding to the spatial branch and the other to the frequency domain branch.

[0131] Finally, the system multiplies the spatial feature map by the spatial branch fusion weights and the frequency domain feature map by the frequency domain branch fusion weights, then adds them element-wise to obtain the fused feature map. Each spatial location in this fused feature map is a weighted combination of spatial and frequency domain features, with the weights automatically determined by the system based on the content of that location—the spatial branch has a higher weight at material edges and in areas with rich textures; the frequency domain branch has a higher weight in areas with obvious material features. This adaptive fusion strategy allows the system to dynamically select the optimal feature source.

[0132] V. Construction and coding of dual-path cross-modal graph structure:

[0133] The fused feature map is fed into the graph construction module. The system treats each spatial location in the fused feature map as a graph node. If the size of the fused feature map is 64 channels x 80 x 80, the system will expand it into 6400 nodes, each represented by a 64-dimensional feature vector.

[0134] The system first constructs a spatial adjacency graph. For any two nodes, the system obtains their spatial coordinates in the original feature map and calculates the Euclidean distance between them. The system presets the spatial neighborhood radius to 5 pixels and only establishes connections between node pairs whose distance is less than this radius. For node pairs that meet the distance condition, the system calculates the Gaussian kernel function value as the edge weight with spatial distance as the independent variable; the closer the nodes are, the greater the edge weight. After the above processing, each node in the spatial adjacency graph is only connected to nodes in its surrounding local region, reflecting the spatial local structure of the consumable image.

[0135] The system constructs a frequency domain association graph in parallel. For any two nodes, the system calculates the Euclidean distance between their feature vectors as a measure of feature similarity. The system pre-determines that each node only establishes connections with the eight other nodes whose feature distances are smallest. For node pairs satisfying this condition, the system calculates a Gaussian kernel value as the edge weight, using the feature distance as the independent variable; the more similar the features of the nodes, the larger the edge weight. The frequency domain association graph captures the semantic relationships between different spatial locations; even if two locations are far apart, if their features are similar, a connection will be established in the frequency domain association graph.

[0136] After the graph structure is constructed, the system uses a multi-head graph attention network to encode the two types of graphs separately. Each attention head of the graph attention network independently calculates the attention coefficients between nodes—for the target node and one of its neighboring nodes, the system performs linear transformations on their feature vectors, concatenates them, then performs a dot product with the attention parameter vector, and activates them with LeakyReLU to obtain an unnormalized attention score. The system performs Softmax normalization on the attention scores of all neighboring nodes of the target node to obtain normalized attention coefficients. These coefficients reflect the relative importance of neighboring nodes to the target node. Each attention head uses the normalized attention coefficients to perform a weighted sum of the transformed features of neighboring nodes, and the result is used as the output of that head after activation. The system concatenates the outputs of the four attention heads along the feature dimension to form the output of this layer of the graph attention network. The system stacks two layers of graph attention networks, with the output of the first layer serving as the input of the second layer, to obtain the final spatial graph node representation and frequency domain graph node representation.

[0137] VI. Optimal Transmission Cross-Graph Alignment and Fusion:

[0138] Spatial graph node representation and frequency domain graph node representation describe the fusion characteristics from the perspectives of spatial adjacency and frequency domain correlation, respectively. The system achieves fine alignment and fusion of the two through the optimal transmission mechanism.

[0139] The system first maps the spatial graph node representation and the frequency domain graph node representation to a low-dimensional semantic space using learnable projection matrices. The mapping results are then Softmax normalized along the node dimension, resulting in a probability distribution of nodes in each graph across semantic categories. Both the spatial graph probability distribution and the frequency domain graph probability distribution have dimensions equal to the number of nodes multiplied by the number of semantic categories. Each row represents the probability distribution of a node across all semantic categories, and the sum of all rows is 1.

[0140] The system then constructs a cost matrix. The rows of the cost matrix correspond to nodes in the spatial graph, and the columns correspond to nodes in the frequency domain graph. The element in the i-th row and j-th column of the matrix is ​​defined as the cosine distance between the i-th node in the spatial graph and the j-th node in the frequency domain graph—that is, 1 minus the normalized dot product of their eigenvectors. A cosine distance closer to 0 indicates greater similarity between the two nodes and a lower alignment cost; a cosine distance closer to 1 indicates greater difference between the two nodes and a higher alignment cost.

[0141] The goal of optimal transmission is to minimize transmission cost while satisfying edge distribution constraints. The system uses the spatial graph probability distribution as row edge constraints—requiring the sum of each row of the transmission matrix to equal the probability quality of the corresponding spatial graph node; and the frequency domain graph probability distribution as column edge constraints—requiring the sum of each column of the transmission matrix to equal the probability quality of the corresponding frequency domain graph node. The system solves this constrained optimal transmission problem using the Sinkhorn algorithm: first, the transmission matrix is ​​initialized as a matrix of all 1s divided by the number of nodes; then, row normalization (dividing each row by its sum to make the row sum equal to the corresponding row edge distribution) and column normalization (dividing each column by its sum to make the column sum equal to the corresponding column edge distribution) are performed alternately. Simultaneously, an entropy regularization term is introduced to make the transmission matrix smooth and differentiable. After dozens of iterations, the system converges to obtain the optimal transmission matrix. The element values ​​of this matrix reflect the optimal correspondence strength between spatial graph nodes and frequency domain graph nodes.

[0142] The system obtains spatial features after frequency-domain perspective reweighting by multiplying the optimal transfer matrix by the spatial graph nodes. It also obtains frequency-domain features after spatial perspective enhancement by multiplying the transpose of the optimal transfer matrix by the frequency graph nodes. Adding these two features yields aligned and fused features. In these features, the information from the spatial and frequency domain perspectives establishes a fine semantic correspondence through optimal transfer, achieving true cross-perspective fusion.

[0143] VII. Differential enhancement processing of the edge enhancement module:

[0144] The aligned and fused features are then fed into the edge enhancement module for edge information enhancement. The system first inputs the aligned and fused features into a 3x3 depthwise separable convolution for preliminary feature extraction. The depthwise separable convolution decomposes the standard convolution into channel-wise depthwise convolution and pointwise 1x1 convolution, which significantly reduces the computational load.

[0145] The initially extracted features are input in parallel into four depthwise separable convolutional branches with different receptive fields, and the kernel sizes are 5x5, 7x7, 9x9, and 11x11, respectively. For a feature of size 64 channels x 80x80, the four branches output 64-channel x 80x80 feature maps. The system concatenates the outputs of the four branches along the channel dimension to obtain a 256-channel feature map, and then compresses the number of channels back to 64 using a 1x1 depthwise separable convolution to obtain multi-scale intermediate features. This feature integrates semantic information from different receptive fields ranging from 5 pixels to 11 pixels, capturing both fine local edge details and perceiving a wider range of boundary context.

[0146] The system then performs differential edge enhancement on the multi-scale intermediate features. Specifically, the system first extracts shallow features through a concatenation of 3x3 and 1x1 depthwise separable convolutions; then, it performs average pooling on the shallow features (with a 3x3 kernel size and a stride of 1, and padding to maintain the spatial dimensions) and finally extracts deep features through 1x1 depthwise separable convolutions. Shallow features retain more spatial details, while deep features contain higher-level semantic information.

[0147] The core operation of differential enhancement is as follows: for any input feature, the system calculates the difference between that feature and the feature extracted from deep features. The system performs the above differential operation on shallow features and deep features respectively—subtracting the result of deep feature extraction from the shallow feature yields a shallow differential enhanced feature, which highlights gradient change information in the spatial domain; subtracting the result of deep feature extraction from the deep feature yields a deep differential enhanced feature, which strengthens the semantic-level boundary response.

[0148] The system concatenates shallow and deep differential enhancement features along the channel dimension, compresses the channels using a 1x1 depthwise separable convolution, and then adds them element-wise with the original aligned and fused features (i.e., residual connections) to obtain edge enhancement features. Residual connections ensure that the original feature information can be losslessly transmitted to the module output, while the differential enhancement part is superimposed as supplementary information, which strengthens the edge response without losing the original discriminative features.

[0149] VIII. Output of bidirectional reconstruction measurement classification and recognition results:

[0150] Edge enhancement features are fed into the bidirectional reconstruction metric module for classification decisions. The system loads pre-built support sets for eight categories of surgical consumables from the database, with each category containing features from five labeled samples.

[0151] Taking a specific query sample as an example—assuming the sample is actually an absorbable suture, but the system doesn't know its category beforehand—the system expands the edge enhancement features of the query sample into a feature matrix of 6400 spatial locations multiplied by 64 channels, obtaining the query feature matrix. For the absorbable suture category, the system concatenates the feature matrices of the five supporting samples of this category along the spatial dimension, obtaining a support feature matrix of 32000 spatial locations multiplied by 64 channels.

[0152] The system projects the query feature matrix and support feature matrix into query key, query value, and query tensor, and support key, support value, and support tensor, respectively, using a learnable projection matrix. The projected feature dimension is 64-dimensional. The system calculates the product of the query tensor and the support key matrix and scales it by dividing by the square root of the projection dimension. The result is then Softmax normalized along the spatial dimension of the support samples to obtain the spatial attention weight matrix of the query to the support set, with dimensions 6400 x 32000, representing the attention intensity of each position in the query to each position in the support set. This attention weight matrix is ​​multiplied by the support value matrix to obtain the query reconstructed feature matrix, which is the query feature reconstructed from the support set. The system calculates the squared Frobenius norm between the query value matrix and the query reconstructed feature matrix as the first reconstruction error; the smaller this error, the higher the matching degree between the query sample and the absorbable suture category.

[0153] The system symmetrically computes the inverse reconstruction: It calculates and scales the product of the support tensor and the query key matrix, performs Softmax normalization along the query space dimension to obtain the inverse attention weight matrix for the support tensor to the query. This matrix is ​​multiplied by the query value matrix to obtain the support reconstruction feature matrix. The squared Frobenius norm between the support value matrix and the support reconstruction feature matrix is ​​calculated as the second reconstruction error. The system then weights and sums the first and second reconstruction errors with learnable weights to obtain the comprehensive distance between the query sample and the absorbable stitching category.

[0154] The system calculates the aforementioned comprehensive distance for all eight categories, resulting in eight distance values. Assuming the distance for absorbable sutures is 0.82, for non-absorbable sutures 1.35, for metal hemostatic clips 2.67, for absorbable hemostatic clips 2.89, for electrosurgical scalpels 3.54, for scalpel blades 3.21, for medical gauze pads 4.12, and for medical catheter connectors 3.76, the system negatives these eight distance values ​​and divides them by a temperature coefficient (1.0), then converts them into eight probability values ​​using the Softmax function. The calculated probability is approximately 0.71 for absorbable sutures, approximately 0.21 for non-absorbable sutures, and all other categories have probabilities below 0.03. The system selects the category with the highest probability—absorbable sutures—as the identification result.

[0155] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0156] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0157] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for identifying surgical consumables based on computer vision, characterized in that, include: Acquire images of surgical consumables to be identified and images of surgical consumables of known categories; The surgical consumables image is input into the spatial domain feature extraction branch and the frequency domain feature extraction branch respectively. The spatial domain feature extraction branch performs adaptive receptive field convolution processing on the surgical consumables image to extract spatial feature maps. The frequency domain feature extraction branch performs frequency domain transformation on the surgical consumables image and extracts frequency domain feature maps using a learnable spectral mask. The spatial feature map and the frequency domain feature map are adaptively fused to obtain a fused feature map; A dual-path graph structure is constructed based on the fused feature map, the dual-path graph structure including a spatial adjacency graph and a frequency domain correlation graph; The spatial adjacency graph and the frequency domain association graph are encoded using a graph attention network to obtain spatial graph node representations and frequency domain graph node representations, respectively. The spatial graph node representation and the frequency domain graph node representation are aligned and fused across graphs using the optimal transmission mechanism to obtain alignment and fusion features. The alignment and fusion features are input into the edge enhancement module to enhance edge information, resulting in edge enhancement features; Based on the edge enhancement features, the distance between the query sample and each category support set is calculated using a bidirectional feature reconstruction metric strategy, and the category of the surgical consumable image is determined based on the distance. The query sample refers to the edge enhancement features corresponding to a single surgical consumable image whose category is unknown. Each category support set refers to the set of edge enhancement features of a small number of labeled samples contained in each known consumable category in the training data, which serves as a category reference prototype and is used for bidirectional reconstruction and comparison with the query sample.

2. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, The spatial domain feature extraction branch performs adaptive receptive field convolution processing on the surgical consumables image to extract spatial feature maps, including: The input feature map of the surgical consumables image is extracted by a convolutional neural network. Global average pooling is then performed on the input feature map along the vertical and horizontal directions to obtain the vertical global description vector and the horizontal global description vector. Predict the vertical kernel scale based on the vertical global description vector, and predict the horizontal kernel scale based on the horizontal global description vector; A sampling grid is constructed based on the vertical kernel scale and the horizontal kernel scale, and the input feature map is resampled based on the sampling grid; The resampled feature map is convolved with an adaptive convolution kernel to obtain the adaptive receptive field convolution output. The adaptive receptive field convolution output is modulated and biased, and channel enhancement is performed through a channel attention mechanism to obtain the spatial feature map.

3. The surgical consumables identification method based on computer vision according to claim 2, characterized in that, The frequency domain feature extraction branch performs frequency domain transformation on the surgical consumables image and extracts frequency domain feature maps using a learnable spectral mask, including: A two-dimensional discrete cosine transform is performed on each channel of the input feature map of the surgical consumables image to obtain an initial spectral representation; The learnable spectral mask is upsampled to the same resolution as the initial spectral representation, and the upsampled learnable spectral mask is then subjected to Sigmoid activation and completion processing to obtain a spectral weighted mask. The initial spectral representation is multiplied element-wise by the spectral weighted mask to obtain the discriminative spectral features; The discriminative spectral features are subjected to a two-dimensional inverse discrete cosine transform to obtain the frequency domain feature map.

4. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, Adaptive fusion of the spatial feature map and the frequency domain feature map yields a fused feature map, including: The spatial feature map and the frequency domain feature map are concatenated along the channel dimension to obtain a concatenated feature map; The concatenated feature map is processed by a convolutional sub-network to generate a two-channel attention score map, the two channels corresponding to the spatial feature map and the frequency domain feature map, respectively; The attention score maps of the two channels are normalized at each position using Softmax to obtain the spatial branch fusion weights and the frequency domain branch fusion weights. Based on the spatial branch fusion weight and the frequency domain branch fusion weight, the spatial feature map and the frequency domain feature map are combined position-by-position with weights to obtain the fused feature map.

5. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, Constructing a dual-path graph structure based on the fused feature map includes: Each spatial location in the fused feature map is treated as a node, resulting in a node sequence; The spatial distance is calculated based on the spatial coordinates of each node in the fused feature graph. The edge weights of the spatial adjacency graph are constructed based on the spatial distance and the Gaussian kernel function. Each node in the spatial adjacency graph is only connected to nodes whose spatial distance is less than a preset neighborhood radius. The feature distance is calculated based on the similarity between the feature vectors of each node. The edge weights of the frequency domain association graph are constructed based on the feature distance and the Gaussian kernel function. Each node in the frequency domain association graph is only connected to nodes whose feature distance is less than a preset threshold.

6. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, Cross-graph alignment and fusion of the spatial graph node representation and the frequency domain graph node representation are performed using an optimal transmission mechanism, including: The spatial graph node representation and the frequency domain graph node representation are respectively mapped to spatial graph probability distribution and frequency domain graph probability distribution through learnable projection matrices; Calculate the cost matrix between the spatial graph node representation and the frequency domain graph node representation, where each element in the cost matrix is ​​the cosine distance between the corresponding nodes; Based on the spatial graph probability distribution and the frequency domain graph probability distribution as edge distribution constraints, and with the goal of minimizing the transmission cost on the cost matrix, the optimal transmission matrix is ​​solved by the Sinkhorn algorithm. The spatial graph node representation is semantically reweighted based on the optimal transfer matrix, and the frequency domain graph node representation is supplemented and enhanced. The weighted sum of the two is then used to obtain the alignment fusion feature.

7. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, The alignment and fusion features are input into the edge enhancement module for edge information enhancement to obtain edge enhancement features, including: The aligned and fused features are input into multiple depthwise separable convolutional branches with different receptive fields. The outputs of each branch are concatenated along the channel dimension and then the channels are compressed by pointwise convolution to obtain multi-scale intermediate features. The multi-scale intermediate features are sequentially subjected to a first convolution process to obtain shallow features, and the shallow features are then subjected to average pooling and depth convolution processes to obtain deep features. Subtracting the shallow features from the deep features yields shallow differential enhancement features. Subtracting the deep features from the deep features after performing average pooling and depthwise convolution yields deep differential enhancement features. The shallow differential enhancement feature and the deep differential enhancement feature are concatenated and then processed by convolution. The residuals of the concatenated feature and the alignment fusion feature are added to obtain the edge enhancement feature.

8. The surgical consumables identification method based on computer vision according to claim 1, characterized in that, The distance between the query sample and the support sets of each category is calculated using a bidirectional feature reconstruction metric strategy, including: The features of the query sample are represented as query features, and the features of all samples in each category support set are concatenated along the spatial dimension to form the support features of each category. The query features and the support features are projected into query key-value triples and support key-value triples respectively using a learnable projection matrix; Calculate attention weights based on the query key-value triples and the support key-value triples, use the attention weights to weight the support key-value triples, reconstruct the query reconstruction features, and calculate the first reconstruction error between the query key-value triples and the query reconstruction features. Calculate the back attention weights based on the support key-value triples and the query key-value triples, use the back attention weights to weight the query key-value triples, reconstruct the support reconstruction features, and calculate the second reconstruction error between the support key-value triples and the support reconstruction features. The first reconstruction error and the second reconstruction error are weighted and summed to obtain the distance between the query sample and the support sets of each category.

9. The surgical consumables identification method based on computer vision according to claim 8, characterized in that, Determining the category of the surgical consumable image based on the distance includes: The distance between the query sample and each category support set is negatively divided by a preset temperature coefficient, and the probability of the query sample belonging to each category is calculated using the Softmax function. The category with the highest probability is selected as the recognition result of the surgical consumable image.

10. A method for identifying surgical consumables based on computer vision according to any one of claims 1-9, characterized in that, The method further includes: During the training phase, the network parameters are optimized end-to-end using an overall loss function composed of classification loss, alignment loss, and edge consistency loss. The classification loss is the cross-entropy loss, which is used to drive the network to classify correctly; The alignment loss is the sum of the KL divergences between the spatial graph probability distribution and the frequency domain graph probability distribution, used to constrain the distribution consistency in the optimal transmission process; The edge consistency loss is the mean square error between the edge intensity map output by the edge enhancement module and the edge ground truth map extracted from the original image, and is used to supervise the enhancement quality of edge information.

Images