Surgical instrument abnormality responsibility identification system based on artificial intelligence

Abstract

The present invention relates to the field of image data processing technology, and in particular to an artificial intelligence-based surgical instrument abnormality responsibility identification system, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the following steps: obtaining video frame images during surgery, obtaining feature key points and feature key point descriptors in each video frame image; obtaining a first attention degree of the feature key point based on the matching of the feature key point descriptors and the position distribution; obtaining a second attention degree of the feature key point based on the position information and similarity between each feature key point and other feature key points; obtaining an attention degree image corresponding to each video frame image based on the first attention degree and the second attention degree, and performing surgical instrument abnormality responsibility identification based on the attention degree image. The present invention enables subsequent doctors to have more accurate results in surgical instrument responsibility identification.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an artificial intelligence-based surgical instrument abnormality responsibility identification system. Background Art

[0002] During the operation, if there is an abnormality of the surgical instrument or the wrong use of the surgical instrument, it is necessary to determine the responsibility for the corresponding phenomenon, and the surgical instrument needs to be located and identified before responsibility identification. The CenterNet network is commonly used to locate and identify surgical instruments. However, since the positioning and identification results need to be processed for responsibility identification of abnormal situations of surgical instruments in the future, and the surgical instruments are operated by relevant doctors, the input image of the recognition neural network needs to ensure that a high level of attention is paid to the contact part between the doctor and the surgical instrument. The existing method often uses regional segmentation images to perform different annotations on different regions. The inaccurate regional edge division of this method may lead to the inability to guarantee the degree of attention to the contact part, making the subsequent doctor's identification of surgical instrument responsibility less accurate. Summary of the invention

[0003] In order to solve the technical problem that the existing image annotation method is relatively inaccurate, resulting in inaccurate results of subsequent doctors' identification of surgical instrument responsibilities, the purpose of the present invention is to provide a surgical instrument abnormality responsibility identification system based on artificial intelligence, and the technical solution adopted is as follows:

[0004] Obtain video frame images during the surgical process, and obtain edge points belonging to surgical instruments in each video frame image; perform key point detection on each video frame image, and obtain characteristic key points and descriptors of the characteristic key points in each video frame image according to the overlap between the key points and edge points detected in each video frame image and the key point detection results;

[0005] According to the matching status and position distribution of the descriptors of the feature key points in each video frame image and its adjacent video frame images, the first attention degree of each feature key point in each video frame image is obtained;

[0006] According to the position information and similarity between each feature key point in each video frame image and other feature key points, a second attention degree of each feature key point in each video frame image is obtained;

[0007] An attention image corresponding to each video frame image is obtained according to the first attention level and the second attention level, and responsibility for abnormality of surgical instruments is identified according to the attention image.

[0008] Preferably, obtaining the first attention degree of each feature key point in each video frame image according to the matching status and position distribution of the descriptors of the feature key points in each video frame image and its adjacent video frame images specifically includes:

[0009] Any feature key point in any video frame image is recorded as the target key point. The feature matching pair of the target key point is obtained according to the similarity of the descriptors between the feature key points in the adjacent video frame images of the video frame image where the target key point is located. The normalized value of the Euclidean distance between the two feature key points in the feature matching pair in the same video frame image is used as the first attention degree of the target key point.

[0010] Preferably, obtaining a feature matching pair of the target key point according to the similarity of descriptors between feature key points in adjacent video frame images of the video frame image where the target key point is located specifically includes:

[0011] The target key point is taken as the left node, each feature key point in the video frame image adjacent to the video frame image where the target key point is located is taken as the right node, the similarity between the descriptors corresponding to the left node and the right node is taken as the edge value between the nodes, and the KM algorithm is used to match the key points to obtain a feature matching pair of the target key point; the feature matching pair includes a left node and a right node.

[0012] Preferably, obtaining the second attention level of each feature key point in each video frame image according to the position information and similarity between each feature key point and other feature key points in each video frame image specifically includes:

[0013] For any video frame image, the embedding vector sequence of each feature key point is obtained according to the distance between each feature key point and other feature key points and the neighborhood information;

[0014] For any feature key point, the similarity sequence of the feature key point is obtained according to the similarity of the embedding vector sequence between the feature key point and other feature key points;

[0015] The principal component analysis algorithm is used to obtain the principal component direction of the similarity sequence of the feature key points, and the angle between the principal component direction and the horizontal right direction is negatively correlated and normalized to obtain the second attention degree of the feature key points.

[0016] Preferably, obtaining the embedding vector sequence of each feature key point according to the distance between each feature key point and other feature key points and the neighborhood information specifically includes:

[0017] For any video frame image, each feature key point is taken as a node, and the Euclidean distance between every two feature key points is taken as the edge weight to construct the initial graph structure; the initial graph structure is input into the Node2Vec algorithm to obtain the embedding vector of each node in the initial graph structure under different preset neighborhood range values, and all the embedding vectors of each node constitute an embedding vector sequence to obtain the embedding vector sequence of each feature key point.

[0018] Preferably, obtaining the similarity sequence of feature key points according to the similarity of the embedding vector sequences between the feature key points and other feature key points specifically includes:

[0019] Record any feature key point as a selected key point, and record any other feature key point in the video frame image except the selected key point as a reference key point;

[0020] Calculate the similarity between the embedding vectors of the selected key point and the reference key point under the same neighborhood range value, and obtain the initial similarity between the selected key point and the reference key point under the corresponding neighborhood range value; calculate the average of the initial pixel points of the selected key point and all reference key points under the same neighborhood range value, and obtain the comprehensive similarity of the selected key point under the neighborhood range value; construct the similarity sequence of the selected key point with the comprehensive similarities of the selected key point under all neighborhood range values.

[0021] Preferably, obtaining the attention image corresponding to each video frame image according to the first attention level and the second attention level specifically includes:

[0022] For any video frame image, the product of the first attention degree and the second attention degree of each feature key point is taken as the global attention degree of each feature key point. The global attention degree is used to perform bilinear interpolation processing on the pixel points belonging to the same type of area in the video frame image to obtain the attention degree image corresponding to the video frame image.

[0023] Preferably, the performing key point detection on each video frame image separately specifically includes: performing key point detection on each video frame image separately using a SIFT algorithm.

[0024] Preferably, obtaining the characteristic key points and the descriptors of the characteristic key points in each video frame image according to the overlap of the key points and the edge points detected in each video frame image and the key point detection result specifically includes:

[0025] The edge points in the key point detection results using the SIFT algorithm are used as feature key points, and the key point descriptions obtained by using the SIFT algorithm for key point detection are used as descriptors of the feature key points.

[0026] Preferably, the step of separately obtaining edge points belonging to the surgical instrument in each video frame image specifically includes: for any video frame image, obtaining key points in the video frame image using the Hessian matrix, and recording the key points belonging to the surgical instrument as edge points.

[0027] The embodiments of the present invention have at least the following beneficial effects:

[0028] The present invention first obtains the edge points belonging to the surgical instrument in the video frame image during the operation, and then determines the characteristic key points and descriptors in the video frame image through the coincidence of the edge points and the key points, so that the characteristic key points contain the characteristic information of the target part, that is, the characteristic information of the surgical instrument, and the key information in the image. The subsequent analysis results of the characteristic key points are more representative and reduce the amount of calculation to a certain extent. Then, the matching situation and position distribution of the descriptors of the characteristic key points in each video frame image and its adjacent video frame images are analyzed to obtain the first attention degree of the characteristic key points. The first attention degree reflects the attention degree of each characteristic key point from the aspect of the moving distance characteristics of the characteristic key points. Further, the position information and similarity between each characteristic key point and other characteristic key points in each video frame image are analyzed to obtain the second attention degree. The second attention degree reflects the degree of attention of the characteristic key point from the aspect of the similarity between the characteristic key points in the neighborhood of the characteristic key point. Finally, the attention degree image is obtained by combining the attention degree of the two aspects, which can pay more attention to the contact part between the doctor and the surgical instrument, help to obtain the partial information with the doctor's operation, facilitate the subsequent responsibility identification, and make the subsequent doctor's responsibility identification result of the surgical instrument more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a flowchart of the steps of a method performed by an artificial intelligence-based surgical instrument abnormality responsibility identification system provided by the present invention. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of a surgical instrument abnormality responsibility identification system based on artificial intelligence proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] The following is a detailed description of a specific solution of a surgical instrument abnormality responsibility identification system based on artificial intelligence provided by the present invention in conjunction with the accompanying drawings.

[0034] One embodiment of the present invention provides a surgical instrument abnormality responsibility identification system based on artificial intelligence, the system is used to achieve Figure 1 The specific steps are as follows:

[0035] Step 1: Obtain video frame images during the operation, and obtain the edge points belonging to the surgical instruments in each video frame image; perform key point detection on each video frame image, and obtain the characteristic key points and the descriptors of the characteristic key points in each video frame image based on the overlap of the key points and edge points detected in each video frame image and the key point detection results.

[0036] Existing target detection methods often use the CenterNet network for target detection, and the CenterNet network uses heat maps as labeled input images. The center point area corresponding to each target part in each heat map has a Gaussian kernel function, and the Gaussian kernel function is centrally symmetric. In order to pay more attention to the information of the contact part between the doctor and the surgical instrument during the positioning and identification process of the surgical instrument, it is necessary to pay more attention to the part of the surgical instrument close to the doctor's operating hand. In order to avoid too much calculation, the key points of the key parts are extracted in the embodiment of the present invention.

[0037] First, obtain the video frame images during the operation, and obtain the edge points belonging to the surgical instrument in each video frame image. In this embodiment, for any video frame image, the key points in the video frame image are obtained using the Hessian matrix, and the key points belonging to the surgical instrument are recorded as edge points. Since the subsequent task of the CenterNet network is to perform target detection, it is necessary to delete the key points of the non-target part of the video frame image, that is, to delete the key points of the non-surgical instrument part that are detected.

[0038] In other embodiments, each video frame image can be downsampled first, which can reduce the dimension of the features and retain effective information, thereby avoiding overfitting to a certain extent. The downsampling parameter settings specifically include: the downsampling process uses a 2*2 window for sliding window processing, and uses mean pooling with a step size of 1 to obtain the downsampling thickness value. Multiple downsampling processes can be used to obtain multiple downsampled images, and the sampling stops after 50 times. Then, the key point detection is performed on each downsampled image using the Hessian matrix, and then the key points that do not belong to the edge of the surgical instrument in each downsampled image are deleted to obtain the edge points of the surgical instrument in the video frame image.

[0039] It should be noted that the local extreme points in each video frame image can be obtained through the Hessian matrix, and then the points overlapping with the target can be screened out to obtain edge points. In order to more accurately obtain the attention of the surgical instrument part, it is necessary to further detect the key points of the video frame image, that is, use the SIFT algorithm to detect the key points of each video frame image separately. Among them, the key points found by the SIFT algorithm are some very prominent points that do not change due to factors such as lighting, affine transformation and noise, such as corner points, edge points, bright spots in dark areas or dark spots in bright areas.

[0040] Furthermore, the part of the key point detection result using the SIFT algorithm that belongs to the edge point is the part that may need higher attention. The edge point in the key point detection result using the SIFT algorithm is used as the feature key point. At the same time, the description of each key point can be obtained in the process of key point detection using the SIFT algorithm. The description of each feature key point obtained in the SIFT algorithm is directly used as the descriptor of the feature key point, which can be used for subsequent feature analysis of the feature key point. Among them, the feature key point represents the key points that belong to the target part and are of more concern in each video frame image.

[0041] Step 2: According to the matching conditions and position distribution of the descriptors of the feature key points in each video frame image and its adjacent video frame images, the first attention degree of each feature key point in each video frame image is obtained.

[0042] It should be noted that surgical instruments are in a state of constant movement during the operation. The greater the moving distance of the surgical instruments, the greater the probability of transfer operations between different doctors. Therefore, the identification and judgment of these surgical instruments becomes more important. Therefore, it is necessary to pay more attention to the key points with large moving distances in the video frame images of adjacent frames, which will help to accurately identify the doctor's information and facilitate subsequent responsibility identification.

[0043] In order to accurately obtain the moving distance of the feature key point in each video frame image in two adjacent video frame images, it is necessary to first match each feature key point in each video frame image, and the matching process needs to obtain the feature information corresponding to the feature key point. In this embodiment, the KM algorithm is used for key point matching, and at the same time, the descriptor of each feature key point is used as the feature information of each feature key point to participate in the matching.

[0044] Any feature key point in any video frame image is recorded as the target key point, and the feature matching pair of the target key point is obtained according to the similarity of the descriptors between the feature key points in the adjacent video frame images of the video frame image where the target key point is located. Specifically, the target key point is taken as the left node, each feature key point in the adjacent video frame image of the video frame image where the target key point is located is taken as the right node, and the similarity between the descriptors corresponding to the left node and the right node is taken as the edge value between the nodes. The KM algorithm is used to match the key points to obtain the feature matching pair of the target key point; the feature matching pair includes a left node and a right node.

[0045] Among them, in this embodiment, the video frame image where the target key point is located is matched with the feature key point in the next adjacent video frame image. In particular, the last video frame image collected does not have the next adjacent video frame image, and no feature analysis is performed on it in this embodiment. At the same time, there are various methods for calculating the similarity between feature information, and the implementer can choose according to the specific implementation scenario, for example, Pearson correlation coefficient, cosine similarity, etc., are all existing technologies and will not be introduced in detail here.

[0046] Furthermore, the normalized value of the Euclidean distance between the two feature key points in the feature matching pair in the same video frame image is used as the first attention degree of the target key point. For example, assuming that the video frame image where the target key point is located is the i-th video frame image, and the other feature key point in the feature matching pair of the target key point is recorded as the matching key point of the target key point, then the video frame image where the matching key point is located is the i+1-th video frame image, and the pixel point at the corresponding position of the matching key point in the i-th video frame image is obtained, and then the Euclidean distance between the target key point and the pixel point is calculated. In other embodiments, the pixel point at the corresponding position of the target key point in the i+1-th video frame image can also be obtained, and then the Euclidean distance between the pixel point and the matching key point is calculated.

[0047] In this embodiment, the maximum and minimum value method is used to normalize the calculated Euclidean distance. In other embodiments, the implementer can make a selection according to the specific implementation scenario, and can also use the ratio of the calculated Euclidean distance to the maximum value of all Euclidean distances as the normalized value.

[0048] According to the same method, the first attention degree of each feature key point in each video frame image can be obtained, and then the first attention degree reflects the degree of attention of each feature key point from the moving distance characteristics of the feature key point. The larger the value of the first attention degree, the greater the moving distance of the feature key point in the video frame image of the adjacent frame, and the greater the attention degree should be given to it. The smaller the value of the first attention degree, the smaller the moving distance of the feature key point in the video frame image of the adjacent frame, and the smaller the attention degree should be given to it.

[0049] Step three, according to the position information and similarity between each feature key point in each video frame image and other feature key points, obtain the second attention degree of each feature key point in each video frame image.

[0050] It should be noted that the part where the doctor's hand contacts the surgical instrument is often a relatively flat area of ​​the surgical instrument, and the key points of this part are often also relatively common key points, that is, the difference is smaller than all other key points. For example, when a scalpel is used, the part that the doctor contacts is the back of the knife or the side of the knife, not the tip of the knife. The purpose of this embodiment is to improve the accuracy of the procedure responsibility identification results while positioning and identifying the surgical instrument, so it is necessary to pay more attention to the part where the scalpel contacts the doctor.

[0051] It is understandable that the feature key points that are more common in the part of the doctor's body that is not in contact with the surgical instrument are similar to the feature information of most other feature key points. Therefore, after combining the neighborhood information, the similarity between the feature key points that are more common in the part of the doctor's body that is not in contact with the surgical instrument and other feature pixel points changes less. However, the feature key points that are more prominent in the surgical instrument, such as the tip of the scalpel, have a large difference in feature information with other feature key points. After combining more common key points in the neighborhood, the similarity between these more prominent feature key points and other feature key points changes more.

[0052] Based on this, by analyzing the position information and similarity between the feature key points, greater attention can be paid to the feature key points in the contact part between the scalpel and the doctor.

[0053] First, in this embodiment, a method for obtaining the second attention level of each feature key point is described by taking any video frame image as an example. That is, for any video frame image, an embedding vector sequence of each feature key point is obtained according to the distance between each feature key point and other feature key points and neighborhood information.

[0054] Specifically, each feature key point is taken as a node, and the Euclidean distance between every two feature key points is taken as the edge weight to construct an initial graph structure; the initial graph structure is input into the Node2Vec algorithm to obtain the embedding vector of each node in the initial graph structure under different preset neighborhood range values, and all the embedding vectors of each node constitute an embedding vector sequence to obtain the embedding vector sequence of each feature key point.

[0055] Among them, the Node2vec algorithm is an algorithm for learning node embedding representation. Node embedding representation is the process of representing each node as a vector, so that the distance between nodes in the vector space can reflect their relationship in the graph structure. Through different preset neighborhood range values, the network neighborhood of each node in different ranges is obtained. The network neighborhood is a fixed-size neighbor set used to consider each graph node in the Node2vec algorithm.

[0056] In this embodiment, the preset neighborhood range values ​​are 1, 2, 3, ..., 9 and 10 respectively, so that the embedding vector corresponding to each node in the initial graph structure corresponding to the video frame image under different neighborhood range values ​​can be obtained. That is, in this embodiment, each node corresponds to 10 embedding vectors, and constitutes an embedding vector sequence of the feature key points corresponding to the node, and in the embedding vector sequence, the embedding vectors are arranged in ascending order according to their corresponding neighborhood range values.

[0057] For example, when the neighborhood range value is 1, the embedding vector represents the node integrating the information of the nodes in the neighborhood range with the smallest edge weight with the node, and thus the embedding vector is obtained; when the neighborhood range value is 2, the embedding vector represents the node integrating the information of the nodes in the two neighborhood ranges with the smallest edge weight with the node. Based on this, the embedding vectors of the feature key points corresponding to each node under each neighborhood range value respectively represent the features of the feature key points combined with different neighborhood range information. Among them, the embedding vectors corresponding to the feature key points belonging to the contact part between the doctor and the surgical instrument are similar to the features between most of the embedding vectors, while the embedding vectors corresponding to the few feature key points that are more prominent in the surgical instrument part are somewhat different from the embedding vectors of other feature key points.

[0058] Based on this, the similarity between each feature key point and other feature key points in each video frame image is analyzed. That is, for any feature key point in any video frame image, the similarity sequence of the feature key point is obtained based on the similarity of the embedding vector sequence between the feature key point and other feature key points.

[0059] Specifically, any feature key point is recorded as a selected key point, and any other feature key point in the video frame image except the selected key point is recorded as a reference key point; the similarity between the embedded vectors of the selected key point and the reference key point under the same neighborhood range value is calculated to obtain the initial similarity between the selected key point and the reference key point under the corresponding neighborhood range value; the average of the initial pixel points of the selected key point and all reference key points under the same neighborhood range value is calculated to obtain the comprehensive similarity of the selected key point under the neighborhood range value; the comprehensive similarities of the selected key point under all neighborhood range values ​​constitute the similarity sequence of the selected key point.

[0060] In this embodiment, the cosine similarity of the embedding vectors of the selected key point and the reference key point under the same neighborhood range value is calculated as the corresponding initial similarity, that is, when the neighborhood range value is 1, the cosine similarity between the embedding vectors of the selected key point and the reference key point under the neighborhood range value of 1 is calculated. According to the same method, the selected key point and the reference key point have corresponding initial similarities under 10 different neighborhood range values, and then, the selected key point and each of the other reference key points have corresponding initial similarities under 10 different neighborhood range values. When the neighborhood range value is 1, the average value of the initial similarities corresponding to the selected key point and all the other reference key points is calculated to obtain the comprehensive similarity. According to the same method, the comprehensive similarities corresponding to the selected key point under 10 different neighborhood range values ​​can be obtained, forming a similarity sequence of the selected key point.

[0061] Since the embedding vectors corresponding to the characteristic key points of the contact part between the doctor and the surgical instrument in the video frame image are similar to the characteristics between most of the embedding vectors, and these key points account for a large proportion, the principal component analysis method can be used to obtain the characteristic information of these key points.

[0062] That is, the principal component analysis algorithm is used to obtain the principal component direction of the similarity sequence of the feature key points, and the angle between the principal component direction and the horizontal right direction is negatively correlated and normalized to obtain the second attention of the feature key points. Among them, the PCA algorithm is used to process the similarity sequence of the selected key points to obtain multiple eigenvalues ​​and eigenvectors corresponding to the eigenvalues. The method represented by the eigenvector corresponding to the maximum eigenvalue is the principal component direction.

[0063] In this embodiment, the t-th characteristic key point in the video frame image is taken as the selected key point, and the calculation formula of the second attention degree of the selected key point can be expressed as: G t =exp(-α t ), where G t represents the second attention of the t-th feature key point, α tIt represents the angle between the principal component direction of the similarity sequence of the t-th feature key point and the horizontal right direction, and exp() represents an exponential function with the natural constant e as the base.

[0064] α t The larger the value of , the greater the difference between the main direction of the similarity sequence of the t-th feature key point and the 0° direction, which further indicates that the t-th feature key point is less likely to belong to the contact part between the doctor and the surgical instrument, and the smaller the corresponding second attention value. t The smaller the value of , the smaller the difference between the main direction of the similarity sequence of the t-th feature key point and the 0° direction, which further indicates that the t-th feature key point is more likely to belong to the contact part between the doctor and the surgical instrument, and the larger the corresponding second attention value. The second attention reflects the degree to which the feature key point needs to be paid attention to from the similarity between the neighborhood feature key points of the feature key point.

[0065] Step 4: Obtain an attention image corresponding to each video frame image according to the first attention level and the second attention level, and identify the responsibility for abnormality of the surgical instrument according to the attention image.

[0066] Since the feature key points are only some edge points belonging to the surgical instrument, the attention degree of the feature key points belonging to the same surgical instrument can be used as the original value. The original value can be used to perform bilinear interpolation processing on other pixel points in the same surgical instrument to obtain the corresponding attention degree, and obtain a heat map as the input of the CenterNet network for target detection.

[0067] Based on this, an attention image corresponding to each video frame image is obtained according to the first attention and the second attention. Specifically, for any video frame image, the product of the first attention and the second attention of each feature key point is used as the global attention of each feature key point, and the pixel points belonging to the same type of area in the video frame image are bilinearly interpolated using the global attention to obtain the attention image corresponding to the video frame image. In this embodiment, the attention image is a heat map.

[0068] Furthermore, the attention image corresponding to each video frame image is input into the conventional CenterNet network to obtain the target box of each target in the image, that is, the annotated image containing the target box. The main part of the target box is the surgical instrument, but it also contains some information about the doctor's operating hand, so as to facilitate subsequent responsibility identification based on the target box.

[0069] The labeled image obtained by the network output is input into the recognition neural network to obtain the corresponding fault category and the person responsible for the fault. The training process of the recognition neural network includes: the input of the recognition neural network is the video frame image and the labeled image containing the target box, and the output is whether it is abnormal and the corresponding person responsible for the abnormality. The training adopts the cross entropy loss function. The training process is an existing method and will not be introduced in detail here.

[0070] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A surgical instrument abnormality responsibility identification system based on artificial intelligence, comprising a memory and a processor, It is characterized in that The processor executes the computer program stored in the memory to implement the following steps: Obtain video frame images during the surgical process, and obtain edge points belonging to surgical instruments in each video frame image; perform key point detection on each video frame image, and obtain characteristic key points and descriptors of the characteristic key points in each video frame image according to the overlap between the key points and edge points detected in each video frame image and the key point detection results; According to the matching status and position distribution of the descriptors of the feature key points in each video frame image and its adjacent video frame images, the first attention degree of each feature key point in each video frame image is obtained; According to the position information and similarity between each feature key point in each video frame image and other feature key points, a second attention degree of each feature key point in each video frame image is obtained; Obtaining an attention image corresponding to each video frame image according to the first attention level and the second attention level, and identifying the responsibility for abnormality of the surgical instrument according to the attention image; The obtaining, according to the matching conditions and position distribution of the descriptors of the feature key points in each video frame image and its adjacent video frame images, the first attention degree of each feature key point in each video frame image specifically includes: Any feature key point in any video frame image is recorded as a target key point, and a feature matching pair of the target key point is obtained according to the similarity of the descriptors between the feature key points in the adjacent video frame images of the video frame image where the target key point is located, and the normalized value of the Euclidean distance between the two feature key points in the feature matching pair in the same video frame image is used as the first attention degree of the target key point; The obtaining of the second attention level of each feature key point in each video frame image according to the position information and similarity between each feature key point and other feature key points in each video frame image specifically includes: For any video frame image, the embedding vector sequence of each feature key point is obtained according to the distance between each feature key point and other feature key points and the neighborhood information; For any feature key point, the similarity sequence of the feature key point is obtained according to the similarity of the embedding vector sequence between the feature key point and other feature key points; The principal component analysis algorithm is used to obtain the principal component direction of the similarity sequence of the feature key points, and the angle between the principal component direction and the horizontal right direction is negatively correlated and normalized to obtain the second attention degree of the feature key points; The detecting of key points of each video frame image respectively specifically includes: detecting key points of each video frame image respectively by using SIFT algorithm; The method of obtaining the characteristic key points and the descriptors of the characteristic key points in each video frame image according to the overlap of the key points and the edge points detected in each video frame image and the key point detection result specifically includes: The edge points in the key point detection results using the SIFT algorithm are used as feature key points, and the key point descriptions obtained by the key point detection using the SIFT algorithm are used as descriptors of the feature key points; The method of respectively obtaining edge points belonging to the surgical instrument in each video frame image specifically includes: for any video frame image, using the Hessian matrix to obtain key points in the video frame image, and recording the key points belonging to the surgical instrument as edge points.

2. According to claim 1, a surgical instrument abnormality responsibility identification system based on artificial intelligence, It is characterized in that The step of obtaining a feature matching pair of the target key point based on similarities between descriptors of feature key points in adjacent video frame images of the video frame image where the target key point is located specifically includes: The target key point is taken as the left node, each feature key point in the video frame image adjacent to the video frame image where the target key point is located is taken as the right node, the similarity between the descriptors corresponding to the left node and the right node is taken as the edge value between the nodes, and the KM algorithm is used to match the key points to obtain a feature matching pair of the target key point; the feature matching pair includes a left node and a right node.

3. According to claim 1, a surgical instrument abnormality responsibility identification system based on artificial intelligence, It is characterized in that The step of obtaining an embedding vector sequence of each feature key point according to the distance between each feature key point and other feature key points and neighborhood information specifically includes: For any video frame image, each feature key point is taken as a node, and the Euclidean distance between every two feature key points is taken as the edge weight to construct the initial graph structure; the initial graph structure is input into the Node2Vec algorithm to obtain the embedding vector of each node in the initial graph structure under different preset neighborhood range values, and all the embedding vectors of each node constitute an embedding vector sequence to obtain the embedding vector sequence of each feature key point.

4. According to claim 3, a surgical instrument abnormality responsibility identification system based on artificial intelligence, It is characterized in that The obtaining of the similarity sequence of the feature key points according to the similarity of the embedding vector sequences between the feature key points and other feature key points specifically includes: Record any feature key point as a selected key point, and record any other feature key point in the video frame image except the selected key point as a reference key point; Calculate the similarity between the embedding vectors of the selected key point and the reference key point under the same neighborhood range value, and obtain the initial similarity between the selected key point and the reference key point under the corresponding neighborhood range value; calculate the average of the initial pixel points of the selected key point and all reference key points under the same neighborhood range value, and obtain the comprehensive similarity of the selected key point under the neighborhood range value; construct the similarity sequence of the selected key point with the comprehensive similarities of the selected key point under all neighborhood range values.

5. According to claim 1, a surgical instrument abnormality responsibility identification system based on artificial intelligence, It is characterized in that The obtaining, according to the first attention degree and the second attention degree, an attention degree image corresponding to each video frame image specifically includes: For any video frame image, the product of the first attention degree and the second attention degree of each feature key point is taken as the global attention degree of each feature key point. The global attention degree is used to perform bilinear interpolation processing on the pixel points belonging to the same type of area in the video frame image to obtain the attention degree image corresponding to the video frame image.

Images