An operating room consumable usage analysis method and system

Abstract

The application discloses a kind of operating room consumable use analysis method and system, it is related to medical instrument management technical field, method includes: obtaining operation video and force data in surgical instrument operation process;According to the force data, generate operation reference;According to the force data and the operation reference, identify key mechanical event and target timestamp;According to the target timestamp, extract the video frame corresponding to the key mechanical event from the operation video;Determine the two-dimensional position of surgical instrument tip on the video frame;According to the two-dimensional position of surgical instrument tip, generate target anatomical position in three-dimensional medical image model;According to the target timestamp, the force data and the target anatomical position, generate event evidence chain.The application realizes consumable use analysis, and improves accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical device management technology, and in particular to a method and system for analyzing the use of consumables in the operating room. Background Technology

[0002] In modern hospital management, precise control of surgical costs and optimization of consumable inventory are crucial. Existing methods assess consumable usage efficiency by correlating preoperative medical images with actual consumables used during surgery. However, in practice, complexity and uncertainty exist. Existing methods rely solely on static preoperative images and final consumable inventory results, making it difficult to detect unforeseen events during surgery. When preoperative medical images fail to fully reveal subtle differences in individual patient anatomy or tissue characteristics, leading to discrepancies between the actual intraoperative situation and predictions, and forcing physicians to perform non-standard procedures (such as changing instruments) and consuming additional consumables, existing methods struggle to effectively capture and quantify the real-time status information of these surgical instruments during operation (such as forces and energy output), and also find it difficult to correlate this information with intraoperative events and physician decisions, resulting in low accuracy in consumable usage analysis.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this invention is to propose a method and system for analyzing the use of consumables in the operating room. This method can analyze the target anatomical location through key mechanical events and generate an event evidence chain, thereby achieving consumables usage analysis and improving accuracy.

[0005] On one hand, embodiments of the present invention provide a method for analyzing the use of operating room consumables, including the following steps:

[0006] Acquire surgical videos and force data during the operation of surgical instruments;

[0007] Based on the force data, an operation reference is generated;

[0008] Based on the force data and the operational reference, identify key mechanical events and target timestamps;

[0009] Based on the target timestamp, extract the video frames corresponding to the key mechanical events from the surgical video;

[0010] On the video frame, determine the two-dimensional position of the tip of the surgical instrument;

[0011] Based on the two-dimensional position of the tip of the surgical instrument, the target anatomical position is generated in the three-dimensional medical imaging model;

[0012] A chain of event evidence is generated based on the target timestamp, the force data, and the target anatomical location.

[0013] In some embodiments, generating an operational reference based on the force data includes:

[0014] Based on the force data, generate a force variation curve;

[0015] Mechanical features are extracted from the force variation curve, including the peak value of the force, the duration of the force value, or the smoothness of the force value variation process;

[0016] Based on the mechanical characteristics, calculate the average value and trend of the characteristics;

[0017] The average value of the features and the trend of change are used as references for the operation.

[0018] In some embodiments, identifying key mechanical events and target timestamps based on the force data and the operational reference includes:

[0019] According to the preset comparison dimension, the first comparison feature on the preset comparison dimension is extracted from the force data, and the second comparison feature on the preset comparison dimension is extracted from the operation reference. The preset comparison dimension includes the threshold comparison dimension or the change curve jitter comparison dimension.

[0020] Calculate the degree of feature deviation based on the first comparison feature and the second comparison feature;

[0021] Identify the key mechanical events based on the degree of deviation of the aforementioned features;

[0022] The timestamp of the key mechanical event is used as the target timestamp.

[0023] In some embodiments, identifying the key mechanical event based on the degree of deviation of the feature includes:

[0024] Construct a multi-dimensional deviation pattern for the key mechanical event, wherein the multi-dimensional deviation pattern is used to characterize a feature set including the magnitude, direction and rate of change of force;

[0025] The degree of feature deviation is matched with the multi-dimensional deviation pattern to obtain the matching result;

[0026] Based on the matching results, the key mechanical events are identified.

[0027] In some embodiments, constructing the multi-dimensional deviation pattern of the key mechanical event includes:

[0028] Tissue structure analysis is performed on the target anatomical region in preoperative medical images to obtain tissue structure information, including tissue density, tissue stiffness, degree of tissue calcification, or degree of tissue fibrosis.

[0029] Based on the tissue structure information, the mechanical response of the target anatomical region under the operation of surgical instruments is calculated. The mechanical response includes the resistance range, deformation characteristics, or force value change trend.

[0030] Based on the mechanical response, the multidimensional deviation mode is constructed.

[0031] In some embodiments, determining the two-dimensional position of the surgical instrument tip on the video frame includes:

[0032] On the video frame, identify the visible range of the surgical instrument;

[0033] Extract instrument feature points from the visible area of ​​the surgical instrument;

[0034] The posture of the surgical instrument in the video frame is determined based on the pre-stored geometric feature information of the surgical instrument and the instrument feature points. The geometric feature information of the surgical instrument includes the relative position information of the instrument tip.

[0035] The two-dimensional position of the surgical instrument tip is calculated based on the posture of the surgical instrument in the video frame and the relative position information of the instrument tip.

[0036] In some embodiments, generating the target anatomical location in a three-dimensional medical imaging model based on the two-dimensional position of the surgical instrument tip includes:

[0037] Acquire endoscopic video, wherein the time series of the endoscopic video is the same as the time series of the surgical video;

[0038] Based on the endoscopic video, determine whether a tip contact event exists, and obtain the tip contact determination result;

[0039] If the tip contact determination result indicates that a tip contact event exists, then the three-dimensional position of the surgical instrument tip in the body surface coordinate system is calculated based on the two-dimensional position of the surgical instrument tip.

[0040] The three-dimensional position is compared with the initial spatial coordinates of the preset anatomical landmark in the three-dimensional medical image model to calculate the local spatial deviation;

[0041] Based on the local spatial deviation, a local deformation compensation vector field is generated;

[0042] The three-dimensional medical image model is updated based on the local deformation compensation vector field.

[0043] The two-dimensional position of the tip of the surgical instrument is projected onto the updated three-dimensional medical image model to obtain the target anatomical position.

[0044] In some embodiments, determining whether a tip contact event exists based on the endoscopic video and obtaining a tip contact determination result includes:

[0045] When the tip of the surgical instrument visually coincides with a preset anatomical landmark in the endoscopic video, a visual coincidence event is triggered, and the first timestamp of the visual coincidence event is obtained.

[0046] When the force sensor detects a force value jump signal, it triggers a force value jump event and obtains the second timestamp of the force value jump event;

[0047] If the first timestamp and the second timestamp are within a preset time window, then the tip contact determination result is determined to be that a tip contact event exists.

[0048] In some embodiments, after generating an event evidence chain based on the target timestamp, the force data, and the target anatomical location, the method further includes:

[0049] Construct a risk event feature set, which includes a sequence of mechanical events, the target anatomical location, and temporal relationships;

[0050] Extract the target event sequence from the event evidence chain;

[0051] The target event sequence is compared with the risk event features to obtain the comparison results;

[0052] Based on the comparison results, non-standard operation information and abnormal consumption information of consumables are identified;

[0053] A feedback report is generated based on the non-standard operation information and the abnormal consumption information of consumables.

[0054] On the other hand, embodiments of the present invention provide an operating room consumables usage analysis system, comprising:

[0055] The data acquisition module is used to acquire surgical videos and force data during the operation of surgical instruments;

[0056] The operation reference generation module is used to generate an operation reference based on the force data.

[0057] The event recognition module is used to identify key mechanical events and target timestamps based on the force data and the operation reference.

[0058] The video frame extraction module is used to extract the video frame corresponding to the key mechanical event from the surgical video based on the target timestamp;

[0059] The instrument two-dimensional position positioning module is used to determine the two-dimensional position of the tip of the surgical instrument on the video frame;

[0060] The anatomical location generation module is used to generate a target anatomical location in a three-dimensional medical image model based on the two-dimensional position of the tip of the surgical instrument.

[0061] The event evidence chain generation module is used to generate an event evidence chain based on the target timestamp, the force data, and the target anatomical location.

[0062] The embodiments of this application include at least the following beneficial effects: First, the embodiments of this application acquire surgical videos and force data during the operation of surgical instruments, generate an operation reference, then identify key mechanical events and target timestamps based on the force data and operation references, extract video frames corresponding to key mechanical events from the surgical videos, determine the two-dimensional position of the tip of the surgical instrument on the video frames, generate the target anatomical position in the three-dimensional medical image model, and finally generate an event evidence chain based on the target timestamp, force data, and target anatomical position. This enables the analysis of the target anatomical position through key mechanical events and the generation of an event evidence chain, thereby realizing consumable usage analysis and improving accuracy.

[0063] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart illustrating an operating room consumables usage analysis method according to an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the structure of an operating room consumables usage analysis system according to an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0068] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0069] Surgical instruments: These are medical devices used in clinical surgery, divided into routine instruments and specialized instruments, covering multiple departments. Specialized instruments, such as ring removal forceps, nerve dissectors, and bone shears, are designed for specific surgical needs and have different functions.

[0070] In modern hospital management, precise control of surgical costs and optimization of consumable inventory are core tasks. This requires correlation analysis between preoperative medical images and the consumables actually used during surgery. Analysis systems typically target a specific type of surgery, such as arthroscopic meniscus repair, and first meticulously analyze the patient's MRI images. Image processing techniques identify information such as the type, size, and location of the meniscus tear. Based on these objective features extracted from the images, the system references past surgical records to generate a standard, recommended list of consumables, such as two sutures of a specific specification, one anchor of a specific type, and a certain amount of gauze. After the surgery, the hospital's Management Information System (HIS) records all items actually consumed. The analysis system then compares this actual usage list with the preoperative image-based recommended list to assess the efficiency of consumable usage, providing data support for subsequent surgical cost budgeting and standardized procedures.

[0071] In routine surgeries without unexpected complications, the system performed quite well, accurately predicting the usage of most consumables. However, in actual clinical practice, complexity and uncertainty are the norm. Soon, a problem arose. For example, in a meniscus repair surgery, the system, based on preoperative MRI images, determined it to be a standard horizontal tear and recommended using two sutures. However, during the surgery, the surgeon discovered that the patient's meniscus tissue was unusually tough and accompanied by calcification in some areas, which was difficult to discern precisely on the MRI images. When the surgeon attempted to suture with the first suture, the needle encountered significant resistance while penetrating the tough tissue, causing a slight deformation of the needle tip. Although this did not affect the completion of the first suture, the surgeon, for safety and surgical quality reasons, decided to discard the damaged needle and use a new one for subsequent suturing. Ultimately, this surgery consumed a total of three sutures, one more than the system predicted.

[0072] After the surgery, the analysis system immediately flagged it as "abnormal" during data comparison. The report showed that the surgery used an extra suture, resulting in below-average material usage efficiency. For hospital administrators, this report provided only partial information; it merely pointed out that "an extra suture was used" without explaining "why an extra suture was used." Administrators might attribute this to the doctor's operational error or unnecessary waste, thus questioning the doctor's adherence to standard operating procedures. For the doctor, however, this was a necessary and correct clinical decision based on the actual intraoperative situation, aimed at ensuring surgical quality and patient safety. However, the crucial information in this decision-making process—"abnormally tough tissue"—was not recorded or understood by the system.

[0073] The root of this problem lies in the fact that the system analysis is based solely on static preoperative images and the final inventory of consumables, lacking the ability to perceive key events during the surgical "process." The surgical process is a dynamic and complex, interactive process, and many factors affecting consumable usage occur precisely during these processes. To address this, the state information of the surgical instruments themselves during use needs to be incorporated into the analysis model. For example, miniature sensors can be integrated into key surgical instruments such as needle holders, scalpels, or electrosurgical scalpels. Taking needle holders as an example, force sensors can be installed on their jaws or operating handles. When the surgeon uses the needle holder to clamp the suture needle and penetrate the tissue, the sensor can record the magnitude and changes in force in real time. Based on this, the system not only analyzes preoperative MRI images but also simultaneously receives and records real-time operational information from the surgical instruments. When reviewing that "abnormal" surgery, the data retrieved by the system was no longer limited to images and consumable lists. During the first suturing operation, the force sensor on the needle holder recorded a peak value far exceeding the normal tissue suturing force. By aligning this force information with the surgeon's suture-changing actions in the surgical video, the system can construct a complete chain of evidence: preoperative images show a standard tear -> intraoperative force sensors detect abnormal tissue resistance -> the surgeon changes the suture instrument -> ultimately, consumable usage increases. Therefore, the system-generated analysis report is no longer a simple "excess" warning, but a well-reasoned explanation of the event: "Due to abnormally tough meniscus tissue (based on: intraoperative instrument force exceeding the threshold), an additional suture was used, which was a necessary consumption within a reasonable range."

[0074] In specific scenarios of operating room consumable usage analysis, when preoperative medical images cannot fully reveal subtle differences in the individual anatomy or tissue characteristics of patients, leading to deviations between the actual situation and the prediction during surgery, and forcing doctors to take non-standard procedures (such as changing instruments) and consuming additional consumables, existing technologies struggle to effectively capture and quantify the real-time status information of surgical instruments during operation (such as force, energy output, etc.), and it is also difficult to correlate this information with intraoperative events and doctor decisions, resulting in low accuracy in consumable usage analysis.

[0075] In view of this, embodiments of this application acquire surgical videos and force data during the operation of surgical instruments, and generate operational references based on this data to identify key mechanical events and target timestamps. Subsequently, video frames corresponding to the key mechanical events are extracted from the surgical videos, and the two-dimensional position of the surgical instrument tip is determined on the video frames. Next, based on the two-dimensional position of the surgical instrument tip, a target anatomical position is generated in a three-dimensional medical imaging model. Finally, by combining the target timestamp, force data, and target anatomical position, an event evidence chain is generated, enabling in-depth analysis of dynamic information during the surgical process, providing a more accurate and objective assessment of the rationality of consumable use, and improving the accuracy of the analysis.

[0076] The embodiments of this application will be explained in detail below with reference to the accompanying drawings:

[0077] Figure 1 This is an optional flowchart of an operating room consumables usage analysis method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0078] Step S101: Obtain surgical video and force data during the operation of surgical instruments;

[0079] Step S102: Generate an operation reference based on the force data;

[0080] Step S103: Identify key mechanical events and target timestamps based on force data and operational references;

[0081] Step S104: Extract the video frames corresponding to key mechanical events from the surgical video based on the target timestamp;

[0082] Step S105: Determine the two-dimensional position of the tip of the surgical instrument on the video frame;

[0083] Step S106: Generate the target anatomical position in the three-dimensional medical imaging model based on the two-dimensional position of the tip of the surgical instrument;

[0084] Step S107: Generate an event evidence chain based on the target timestamp, force data, and target anatomical location.

[0085] Steps S101 to S107 as shown in the embodiments of this application can analyze the target anatomical location through key mechanical events and generate an event evidence chain, thereby realizing consumable usage analysis and improving accuracy.

[0086] In some embodiments, steps S101-S107 can first acquire surgical video and force data during the operation of surgical instruments. For example, the surgical video can be recorded in real-time using an endoscope system or a high-definition camera in the operating room to ensure the continuity and clarity of the video stream. Force data can be acquired by integrating miniature force sensors onto the surgical instruments. These sensors can measure the three-dimensional force, torque, and other information when the instrument tip contacts the tissue in real time and transmit the data to the processing unit. For example, in laparoscopic surgery, piezoelectric force sensors can be installed at the ends of instruments such as forceps and scissors to capture the mechanical feedback when grasping or cutting tissue. Based on the force data, an operational reference is generated. The generation of the operational reference can be accomplished by statistically analyzing the force data of standard operations in a large number of historical surgeries. For example, the average characteristics of a specific operation (such as suturing or cutting) under normal conditions, such as the force range, rate of force change, and duration, can be calculated, and a mechanical model representing a "standard" or "desired" operation can be constructed. This model can be a multi-dimensional feature vector or a preset mechanical curve template.

[0087] Then, based on the force data and operational references, key mechanical events and target timestamps are identified. This can be achieved by comparing the real-time acquired force data with pre-generated operational references. For example, when the real-time force data deviates significantly from the preset range of the operational reference in a certain dimension (such as force magnitude or rate of change), it can be identified as a key mechanical event. Simultaneously, the time point of this event is recorded as the target timestamp. For instance, in orthopedic surgery, when the drill penetrates the bone, the force will have a clear peak and a subsequent stabilization period. If the force suddenly increases or decreases abnormally, it may be identified as a key mechanical event. Based on the target timestamp, video frames corresponding to the key mechanical events are extracted from the surgical video, allowing subsequent visual analysis to focus on visual information directly related to the mechanical events. For example, if an abnormal force is detected when the suture needle penetrates the tissue, the system will extract the video frame at that moment to observe the visual details of the needle tip contacting the tissue.

[0088] Next, the two-dimensional position of the surgical instrument tip is determined on the video frame. Image processing algorithms can be used to identify the contour of the surgical instrument and further locate its pixel coordinates on the image plane. This can be achieved using a pre-trained deep learning model (such as an object detection model based on convolutional neural networks) capable of recognizing the tip features of different types of surgical instruments. For example, for surgical scissors in an endoscopic image, the model can identify the tip position of its blade. Based on the two-dimensional position of the surgical instrument tip, the target anatomical position is generated in the three-dimensional medical image model. For example, a spatial correspondence between the surgical video and the three-dimensional medical image model can be established, which can be achieved through preoperative registration or an intraoperative navigation system. Then, the two-dimensional position of the instrument tip determined on the video frame, combined with the camera parameters and pose information of the endoscope, is back-projected onto the patient's three-dimensional medical image model to determine the precise three-dimensional anatomical position of the instrument tip within the patient's body. For example, if the instrument tip is located in the liver region in the video frame, the corresponding part of the liver can also be accurately located in the three-dimensional model.

[0089] Finally, based on the target timestamp, force data, and target anatomical location, an event evidence chain is generated, integrating all relevant information to form structured data. It correlates the time of the key mechanical event, the corresponding force data (such as force value and force curve), and the precise anatomical location of the instrument tip in the 3D medical imaging model. For example, an event evidence chain could be recorded as: "At time T1, the surgical instrument tip is in region S1 of the liver, and force data F1 shows an abnormally high force value, which may indicate excessive tissue resistance." This evidence chain provides comprehensive and objective evidence for subsequent analysis. This embodiment, by integrating multimodal data, enables in-depth and objective analysis of consumable usage during surgery, effectively solving the problem in existing technologies that cannot capture key intraoperative events and physician decision-making information.

[0090] As can be understood, force data refers to the mechanical feedback data generated when surgical instruments come into contact with tissue during operation. This data is typically acquired through force sensors integrated into the surgical instruments and can include information such as the magnitude, direction, and torque of the force. Force data objectively reflects the interaction between the instrument and tissue and is a key indicator for assessing the precision of the operation, tissue characteristics, and potential risks. Operational reference refers to a benchmark model constructed based on extensive historical surgical data or expert experience to assess the standardization of surgical procedures. It can be a typical variation curve of force data, the average value of mechanical characteristics, or a trend of change. Establishing an operational reference helps identify key mechanical events that deviate from standard procedures. Key mechanical events refer to specific moments or stages during surgery when the force data shows significant changes or deviates from the operational reference during the interaction between the surgical instrument and tissue. For example, in operations such as puncture, cutting, suturing, and dissection, force values ​​may peak, drop abruptly, or remain consistently high; these can all constitute key mechanical events. Identifying these key mechanical events helps locate important operational steps or potential risk points during surgery. An event evidence chain refers to the integration of information such as key mechanical events, target timestamps, force data, and target anatomical locations to form a complete and traceable event record. This evidence chain can comprehensively and objectively reflect the background, mechanical characteristics, spatial location, and time sequence of specific events during surgery, providing a reliable basis for subsequent risk assessment, consumable analysis, and quality control.

[0091] Through the aforementioned technical solutions, this embodiment significantly enhances the understanding and analysis of the surgical process by integrating multimodal data and constructing event evidence chains, providing strong technical support for the refined management of surgical consumables and the continuous improvement of surgical quality. Simultaneously, this embodiment can delve into the surgical process, capturing and quantifying the dynamic interaction between instruments and tissues in real time, thereby identifying key intraoperative events. The effective fusion of force data, surgical videos, and 3D medical imaging models provides more comprehensive and multi-dimensional information, overcoming the limitations of single data sources. By generating event evidence chains, it can provide objective and detailed explanations for abnormal consumption of consumables, distinguishing between reasonable consumption due to operational errors or complex intraoperative situations, thus avoiding misjudgments in physician clinical decisions. Furthermore, it provides hospital administrators with more precise data support, helping to optimize consumable management strategies and providing physicians with personalized operational feedback, promoting continuous improvement in surgical quality.

[0092] In some embodiments, step S102, generating an operational reference based on the force data, may include, but is not limited to, the following steps:

[0093] Generate force variation curves based on the force data;

[0094] Mechanical features are extracted from the force change curve, including the peak value of the force, the duration of the force value, or the smoothness of the force change process.

[0095] Based on the mechanical characteristics, calculate the average value and trend of the characteristics;

[0096] Use the average value and trend of the characteristics as operational references.

[0097] In some embodiments, a force variation curve can be generated based on the force data. For example, force data collected during the operation of surgical instruments, such as the sequence data of force values ​​changing over time obtained by force sensors or tactile feedback devices, can be converted into a visualized curve. This curve can intuitively reflect the magnitude and direction of the force and its dynamic process of change over time.

[0098] Then, mechanical features are extracted from the force variation curve. These features include the peak force, the duration of the force value, and the smoothness of the force variation process. The peak force refers to the maximum instantaneous force value reached during a specific operational phase, reflecting the intensity of the operation. The duration of the force value refers to the length of time the force value remains within a specific range or exceeds a specific threshold, characterizing the continuity of the operation. The smoothness of the force variation process reflects the stability and precision of the operation; for example, a smooth curve may indicate a stable operation, while a sharply fluctuating curve may indicate instability or tremors. The extraction of these mechanical features aims to quantify key mechanical performance characteristics in surgical procedures.

[0099] Next, based on the mechanical characteristics, the average value and trend of change are calculated. The average value can be the average statistical value of the same mechanical characteristic over multiple surgical cycles or a specific time period, such as average peak force or average duration. The trend of change can be a regular description of the evolution of these mechanical characteristics over time or operational stages, such as whether the force value gradually increases, decreases, or remains stable. These calculation results together constitute a quantitative description of the standard or desired operational pattern.

[0100] Finally, the characteristic average value and trend are used as operational references. This operational reference can be regarded as a benchmark model or template, used to compare the force data in actual surgical operations, thereby identifying deviations from the standard operating procedure and determining whether there are any critical mechanical events.

[0101] This embodiment refines the raw force data, transforming it into a force variation curve, thus clearly presenting the dynamic process of force change. Subsequently, representative mechanical features, such as peak force, duration, and smoothness, are extracted from this curve. These features quantify the mechanical performance of the surgical procedure from different dimensions. By further calculating the average value and trend of these mechanical features, a stable and representative operational reference is constructed. This operational reference, as a standardized mechanical behavior pattern, provides a reliable benchmark for subsequent identification of key mechanical events, making the evaluation of actual operations more objective and accurate.

[0102] Through the above technical solution, this embodiment can perform multi-dimensional and refined analysis of force data, thereby generating a more comprehensive and accurate operational reference. This operational reference not only includes information on the intensity and duration of the force but also considers the stability of the operation, making the assessment of the mechanical behavior of actual surgical procedures more precise. Therefore, it can effectively improve the accuracy and reliability of identifying key mechanical events, providing a more solid data foundation for surgical quality assessment and consumable usage analysis.

[0103] In some embodiments, step S103, identifying key mechanical events and target timestamps based on force data and operational references, may include, but is not limited to, the following steps:

[0104] Step S201: According to the preset comparison dimension, extract the first comparison feature on the preset comparison dimension from the force data, and extract the second comparison feature on the preset comparison dimension from the operation reference. The preset comparison dimension includes the threshold comparison dimension or the change curve jitter comparison dimension.

[0105] Step S202: Calculate the degree of feature deviation based on the first and second comparison features;

[0106] Step S203: Identify key mechanical events based on the degree of feature deviation;

[0107] Step S204: Use the timestamps of key mechanical events as the target timestamps.

[0108] In some embodiments, a first comparison feature on the preset comparison dimension can be extracted from the force data according to the preset comparison dimension, and a second comparison feature on the preset comparison dimension can be extracted from the operation reference. The preset comparison dimension includes a threshold comparison dimension or a variation curve jitter comparison dimension. The preset comparison dimension can be used as a reference standard to measure the difference between the actual surgical operation and the standard operation. It is understood that the threshold comparison dimension refers to comparing the magnitude of the force data, for example, setting an upper or lower limit for the force value; when the actual force data exceeds this threshold, it is considered that there is a deviation. The variation curve jitter comparison dimension focuses on the stability or volatility of the force data over time, for example, by calculating the derivative or variance of the force variation curve to measure its jitter degree.

[0109] Then, based on the first and second comparison features, the degree of feature deviation is calculated. The degree of feature deviation can be calculated using various mathematical methods, such as calculating the Euclidean distance, correlation coefficient, mean square error, or percentage deviation between the first and second comparison features, to quantify the difference between actual operation and standard operation.

[0110] Based on the degree of feature deviation, key mechanical events are identified, specifically those where the force data deviates significantly from the operational reference during the surgical procedure. Such events are identified when the calculated feature deviation exceeds a preset threshold, or when the deviation pattern matches a predefined abnormal pattern. Examples include excessive force, abnormal force fluctuations, or force occurring in locations where it should not be applied.

[0111] Finally, the timestamps of the key mechanical events are used as the target timestamps, which represent the exact times when the identified key mechanical events occurred. These timestamps are obtained with millisecond-level accuracy to facilitate subsequent synchronization with the surgical video and extraction of the corresponding video frames.

[0112] This embodiment introduces a preset comparison dimension and extracts corresponding comparison features based on force data and operational references, enabling the quantification of the differences between actual operations and standard operations. By calculating the degree of feature deviation, the standardization of the operation can be objectively assessed. Through quantitative evaluation, when the degree of deviation reaches a certain threshold or conforms to a specific pattern, potential non-standard operations or abnormal mechanical behaviors can be accurately identified, and their timestamps can be obtained simultaneously. This provides precise time positioning and event evidence for subsequent video frame extraction and event evidence chain generation.

[0113] Through the above technical solution, this embodiment can achieve refined identification and accurate positioning of key mechanical events. Compared with relying solely on experience or rough judgment, this method significantly improves the accuracy and objectivity of identification through quantitative comparison and deviation calculation, which helps to promptly identify potential risks in surgical procedures and provides a reliable data foundation for subsequent analysis and feedback.

[0114] In some embodiments, step S203, identifying key mechanical events based on the degree of feature deviation, may include, but is not limited to, the following steps:

[0115] Step S301: Construct a multi-dimensional deviation pattern for key mechanical events. The multi-dimensional deviation pattern is used to characterize a set of features that include the magnitude, direction, and rate of change of force.

[0116] Step S302: Match the feature deviation degree with the multi-dimensional deviation pattern to obtain the matching result;

[0117] Based on the matching results, identify key mechanical events.

[0118] In some embodiments, relying solely on the degree of deviation of a single feature may not be sufficient to capture the subtle differences in complex mechanical events during surgical procedures, resulting in insufficient accuracy and specificity in identifying key events. For example, different types of surgical procedures (such as cutting, blunt dissection, puncture, etc.) may exhibit similar degrees of deviation mechanically, but their inherent mechanical characteristics (such as the direction of force, rate of change, etc.) are quite different. Failure to distinguish between them may affect the accuracy of subsequent analysis.

[0119] To this end, a multi-dimensional deviation pattern for key mechanical events can be constructed first. This allows for the pre-definition of typical mechanical manifestations in various key mechanical events that may occur during surgery. The multi-dimensional deviation pattern characterizes a set of features including the magnitude, direction, and rate of change of force. Understandably, the magnitude of force can refer to its absolute value or its component in a specific direction; the direction of force can refer to the vector direction of the force relative to the surgical instrument or target anatomical position; and the rate of change of force reflects how quickly the force changes over time, such as the slope of its rise or fall. The combination of these multi-dimensional features can more comprehensively and precisely describe the nature of a mechanical event. For example, a "cutting" event might manifest as a continuous, moderate force directed along the cutting path, with a relatively stable rate of change; while a "puncture" event might manifest as an instantaneous peak force, directed perpendicular to the tissue surface, with an extremely rapid rate of change.

[0120] The feature deviation degree is then matched with multi-dimensional deviation patterns to obtain the matching result. For example, the feature deviation degree calculated from the real-time detected force data can be compared with pre-constructed multi-dimensional deviation patterns of various key mechanical events. This matching can be achieved through various algorithms, such as distance-based similarity calculation, pattern recognition algorithms (e.g., support vector machines, neural networks), or rule-based expert systems. The matching result will indicate which pre-defined key mechanical event pattern the currently observed mechanical event is most similar to.

[0121] This embodiment introduces multi-dimensional deviation patterns, enabling the identification of key mechanical events to go beyond a single mechanical feature deviation. Instead, it comprehensively considers multiple dimensions such as the magnitude, direction, and rate of change of force. Therefore, when force data is acquired and the degree of feature deviation is calculated, this deviation is no longer merely a numerical value, but a feature vector containing multi-dimensional information. This feature vector is then precisely matched with pre-constructed "multi-dimensional deviation patterns" representing different key mechanical events. This matching mechanism effectively distinguishes between mechanical events of different natures, such as differentiating between mechanical fluctuations during normal operation and mechanical impacts during abnormal operation, thereby significantly improving the accuracy and specificity of key mechanical event identification.

[0122] To illustrate this technical solution more clearly, a specific example is used below. Suppose that during a laparoscopic surgery, it is necessary to identify two key mechanical events: "tissue cutting" and "vascular clamping." First, multi-dimensional deviation patterns for these two events are constructed: In the "tissue cutting" pattern, the force magnitude is moderate, the direction is along the cutting path, and the force value changes steadily and continuously; in the "vascular clamping" pattern, the force magnitude is small to moderate, the direction is squeezing or closing, the force value changes abruptly at the moment of clamping, and then tends to stabilize. During the surgery, the system continuously acquires the force data of the surgical instruments and calculates the degree of characteristic deviation. For example, when the force data is detected to show that the force value rapidly rises from zero to a moderate level in a short period of time, moves stably along a certain plane, and the force value remains relatively constant, then this degree of characteristic deviation highly matches the preset multi-dimensional deviation pattern for "tissue cutting," and the system identifies that a "tissue cutting" event has occurred. Conversely, if the detected force data shows that the force value jumps instantaneously from zero to a relatively small level, with an inward squeezing direction, and the force value stabilizes rapidly after reaching its peak, then the degree of deviation of this feature matches the "vascular clamp" pattern, and the system identifies the "vascular clamp" event. In this way, even if the degree of force value deviation of the two events may be similar in some single dimensions, the combined differences in their multi-dimensional features allow the system to accurately distinguish them, thereby improving the accuracy and practicality of key event identification.

[0123] Through the above technical solution, this embodiment can more accurately identify and distinguish various key mechanical events occurring during surgery, such as differentiating normal tissue separation from accidental tissue damage, or distinguishing expected instrument contact from unexpected instrument collision. This refined identification capability enables subsequent surgical analysis to obtain more insightful information, thereby providing a more reliable basis for surgical quality assessment, risk warning, and identification of abnormal consumable usage.

[0124] In some embodiments, the construction of the multi-dimensional deviation pattern of the key mechanical event in step S301 may include, but is not limited to, the following steps:

[0125] Tissue structure analysis is performed on the target anatomical region in preoperative medical images to obtain tissue structure information, including tissue density, tissue stiffness, and degree of tissue calcification or fibrosis.

[0126] Based on tissue structure information, calculate the mechanical response of the target anatomical region under surgical instrument operation. The mechanical response includes resistance range, deformation characteristics, or force value change trend.

[0127] Based on the mechanical response, a multi-dimensional deviation mode is constructed.

[0128] In some embodiments, if the multidimensional deviation pattern fails to adequately consider the specific mechanical properties of different biological tissues or individual patient anatomy, it may lead to insufficient accuracy in identifying key mechanical events. For example, tissues of different densities, hardness, or pathological states exhibit significant differences in their mechanical responses under surgical instrument manipulation. If a uniform deviation pattern is used for comparison, it may be impossible to accurately distinguish between normal operations and abnormal events.

[0129] Therefore, a tissue structure analysis can be performed on the target anatomical region in preoperative medical images to obtain tissue structure information. For example, CT, MRI, and other medical imaging data acquired by the patient before surgery can be processed and analyzed to obtain detailed anatomical and histological features of the surgical target region. This tissue structure information includes tissue density, tissue stiffness, and the degree of tissue calcification or fibrosis. Tissue density reflects the compactness of the tissue, tissue stiffness characterizes the tissue's resistance to deformation, and the degree of tissue calcification or fibrosis indicates pathological changes in the tissue. This information can be extracted from medical images using image processing algorithms to provide accurate input for subsequent mechanical response calculations.

[0130] Then, based on the tissue structure information, the mechanical response of the target anatomical region under surgical instrument operation is calculated. For example, based on the obtained tissue structure information, biomechanical models or finite element analysis can be used to simulate the mechanical response that may occur when surgical instruments interact with the target anatomical region. The mechanical response includes resistance range, deformation characteristics, or force value variation trend. The resistance range represents the range of force values ​​that the instrument may encounter under different operations; the deformation characteristics represent the deformation pattern and degree of the tissue under force; and the force value variation trend represents the law of force change with time or instrument displacement. By using tissue structure information, the expected mechanical response under normal operation conditions can be predicted.

[0131] Based on the mechanical response, a multi-dimensional deviation pattern is constructed. The mechanical response calculated above can be used as a benchmark to define the expected set of force magnitude, direction, and rate of change within the normal operating range. This multi-dimensional deviation pattern can be constructed as a series of thresholds, curve templates, or statistical distributions to characterize the normal fluctuation range of mechanical characteristics under specific tissue types and operating conditions, providing a dynamic and adaptive comparison standard for subsequent calculation of characteristic deviation and identification of key mechanical events.

[0132] To illustrate this technical solution more clearly, a specific example is used below. Suppose that during a liver resection surgery, it is necessary to identify whether the surgical instruments applied excessive force or caused improper tearing during the resection process. Preoperatively, histological analysis is performed on the patient's liver CT or MRI images. Through image segmentation and density analysis, the average density, stiffness, and structural information of the liver tissue, as well as the presence of liver fibrosis or tumor areas, can be obtained. For example, normal liver tissue may have low density and moderate stiffness, while fibrotic areas may exhibit higher density and stiffness. Based on this histological information, the expected mechanical response of the surgical instruments when operating in normal and fibrotic liver tissue can be calculated. For example, in normal liver tissue, the resistance range of the instrument may be small, the deformation characteristics may exhibit elastic deformation, and the force value change trend may be relatively smooth; while in fibrotic areas, the resistance range may increase, the deformation characteristics may be more inclined towards plastic deformation, and the force value change trend may show sharp peaks.

[0133] Based on these calculated mechanical responses, a multi-dimensional deviation pattern for liver resection procedures is constructed. This pattern includes sets of mechanical characteristics for both normal and fibrotic liver tissue. For example, deviation is defined as a force exceeding a certain threshold or an abnormal rate of force change in normal liver tissue, while the threshold and rate of change are adjusted accordingly in fibrotic regions. During actual surgery, when force data from surgical instruments is acquired, the system matches it against this customized multi-dimensional deviation pattern. If the actual mechanical data deviates significantly from the pattern for the current operating area—for example, detecting high resistance or drastic force fluctuations similar to those in fibrotic tissue in normal liver tissue—it can more accurately identify a critical mechanical event, such as potential over-force application or improper operation, thus issuing timely warnings or recording relevant information.

[0134] Through the above technical solution, this embodiment can significantly improve the accuracy and specificity of key mechanical event identification. Since the multi-dimensional deviation pattern is constructed based on the actual tissue structure information of the target anatomical region in the patient's preoperative medical images and the calculated mechanical response, this pattern can more accurately reflect the true mechanical behavior of a specific tissue under surgical manipulation. This enables the system to effectively distinguish between normal mechanical fluctuations caused by tissue characteristics and mechanical events that truly indicate non-standard operations or potential risks, thereby reducing false alarms and false negatives, improving the reliability of surgical process monitoring, and providing a more solid foundation for the subsequent identification of non-standard operation information and abnormal consumption of consumables.

[0135] In some embodiments, in step S105, determining the two-dimensional position of the surgical instrument tip on the video frame may include, but is not limited to, the following steps:

[0136] Identify the visible range of surgical instruments within video frames;

[0137] Extract instrument feature points from the visible area of ​​the surgical instruments;

[0138] Based on the pre-stored geometric feature information and feature points of the surgical instruments, the posture of the surgical instruments in the video frame is determined. The geometric feature information of the surgical instruments includes the relative position information of the instrument tips.

[0139] The two-dimensional position of the surgical instrument tip is calculated based on the posture of the surgical instrument in the video frame and the relative position information of the instrument tip.

[0140] In some embodiments, the visible area of ​​the surgical instruments can be identified first on the video frames. For example, image processing techniques, such as edge detection, color segmentation, or deep learning models, can be used to analyze the video frames to accurately define the pixel region of the surgical instruments in the image. Identification of this visible area is a prerequisite for subsequent extraction of instrument feature points. Then, instrument feature points are extracted from the visible area of ​​the surgical instruments. Representative, easily trackable, and locatable geometric points can be selected within the identified surgical instrument region, such as prominent features near joints, bends, specific markers, or tips of the instruments. These feature points can be predefined or dynamically identified using feature extraction algorithms such as SIFT, SURF, or ORB. Next, based on the pre-stored geometric feature information and feature points of the surgical instruments, the posture of the surgical instruments in the video frame is determined. This can be achieved using the known 3D model or geometric parameters of the instruments (i.e., the pre-stored geometric feature information, including the relative position information of the instrument tips), combined with the 2D feature points extracted from the video frame. The rotation and translation information of the surgical instruments in the current video frame, i.e., their posture on the 2D image plane, is calculated using the Perspective-n-Point (PnP) algorithm or other pose estimation algorithms. Finally, based on the posture of the surgical instruments in the video frame and the relative position information of the instrument tips, the 2D position of the instrument tips is calculated. After determining the overall posture of the surgical instruments, combined with the pre-known geometric positional relationship between the instrument tips and the instrument body (relative position information of the instrument tips), the pixel coordinates of the instrument tips in the current video frame, i.e., their 2D position, are accurately calculated through coordinate transformation or geometric projection.

[0141] Through the above technical solution, this embodiment enables precise and automated determination of the two-dimensional position of the surgical instrument tip within a surgical video frame. This refined positioning capability provides high-precision input for generating the target anatomical position in a subsequent three-dimensional medical image model, thereby significantly improving the accuracy and reliability of surgical analysis. Compared to methods relying solely on coarse identification or manual marking, this embodiment, by combining instrument geometric features and pose estimation, effectively overcomes the impact of complex surgical environment factors such as visual occlusion and lighting changes on positioning accuracy, ensuring the stability and consistency of the surgical instrument tip position information, and laying a solid foundation for quantitative analysis and risk assessment of the surgical process.

[0142] In some embodiments, in step S106, generating the target anatomical location in the three-dimensional medical image model based on the two-dimensional position of the surgical instrument tip may include, but is not limited to, the following steps:

[0143] Step S401: Acquire endoscopic video. The time series of the endoscopic video is the same as the time series of the surgical video.

[0144] Step S402: Based on the endoscopic video, determine whether there is a tip contact event and obtain the tip contact determination result;

[0145] Step S403: If the tip contact judgment result is that a tip contact event exists, then calculate the three-dimensional position of the surgical instrument tip in the body surface coordinate system based on the two-dimensional position of the surgical instrument tip.

[0146] Step S404: Compare the three-dimensional position with the initial spatial coordinates of the preset anatomical landmarks in the three-dimensional medical imaging model, and calculate the local spatial deviation;

[0147] Step S405: Generate a local deformation compensation vector field based on the local spatial deviation;

[0148] Step S406: Update the three-dimensional medical image model based on the local deformation compensation vector field;

[0149] Step S407: Project the two-dimensional position of the tip of the surgical instrument onto the updated three-dimensional medical image model to obtain the target anatomical position.

[0150] In some embodiments, due to factors such as patient breathing, heartbeat, tissue traction, or instrument compression, the target anatomical region undergoes real-time deformation and displacement, leading to a deviation between the pre-constructed static three-dimensional medical imaging model and the actual anatomical structure. This may result in inaccurate positioning of the surgical instrument tip in the three-dimensional medical imaging model, thereby affecting the correlation between key mechanical events and the actual anatomical location, and reducing the accuracy and reliability of surgical analysis. Therefore, this embodiment introduces endoscopic video and the determination of tip contact events to achieve dynamic updating and local deformation compensation of the three-dimensional medical imaging model, thereby improving positioning accuracy.

[0151] Endoscopic video can be acquired first; for example, video images inside the surgical cavity can be captured in real time using an endoscopic device. The time series of the endoscopic video is identical to that of the surgical video to ensure temporal synchronization between the two video data sets, facilitating subsequent data fusion and event correlation. For instance, a unified timestamp system or synchronization triggering mechanism can be used to guarantee the synchronization of the video streams.

[0152] Then, based on the endoscopic video, it is determined whether a tip contact event exists, thus obtaining a tip contact determination result. A tip contact event refers to an event in which the tip of a surgical instrument makes physical contact with the target anatomical area. This determination can be achieved by analyzing visual information in the endoscopic video, such as the relative movement, deformation, or color change of the instrument tip and the tissue surface. The purpose is to identify key moments of interaction between the instrument and the tissue, providing important reference points for subsequent localization and model updates. If the tip contact determination result indicates the existence of a tip contact event, the three-dimensional position of the surgical instrument tip in the body surface coordinate system is calculated based on its two-dimensional position. The two-dimensional position information of the instrument tip obtained from the surgical video, combined with the internal perspective provided by the endoscopic video, can be converted into three-dimensional spatial coordinates in the patient's body surface coordinate system using techniques such as stereoscopic vision reconstruction, monocular depth estimation, or triangulation based on known camera parameters. This yields the precise position of the instrument tip in real three-dimensional space, laying the foundation for registration with the three-dimensional medical imaging model.

[0153] The 3D position is then compared with the initial spatial coordinates of preset anatomical landmarks in the 3D medical imaging model to calculate local spatial deviation. Preset anatomical landmarks can be specific anatomical points identified in preoperative images, such as vascular branch points, organ edges, or bony landmarks. By comparing the real-time calculated 3D position of the instrument tip with the corresponding coordinates of these preset landmarks in the model, the spatial inconsistency between the actual and model caused by tissue deformation or patient displacement—i.e., local spatial deviation—can be quantified. Based on the local spatial deviation, a local deformation compensation vector field is generated. This field is a collection of spatial vectors, each indicating the direction and magnitude of deformation compensation required for the corresponding region in the 3D medical imaging model. This vector field can be constructed using interpolation, finite element analysis, or machine learning-based deformation models, with the aim of accurately describing how the model needs to be locally adjusted to align with the actual anatomical structures.

[0154] Finally, the 3D medical imaging model is updated based on the local deformation compensation vector field. This update process involves non-rigid deformation of the 3D medical imaging model, causing its local areas to adjust according to the instructions of the compensation vector field, thereby more accurately reflecting the actual anatomical morphology during the surgical procedure. This helps eliminate the deviation between the model and the actual situation, improving the accuracy of subsequent positioning. The two-dimensional position of the surgical instrument tip is projected onto the updated 3D medical imaging model to obtain the target anatomical position. The two-dimensional position information of the surgical instrument tip (combined with camera parameters) can be reprojected onto this more accurate model to obtain the target anatomical position of the instrument tip within the current actual anatomical structure.

[0155] To illustrate this technical solution more clearly, a specific example is used below. Suppose that during a laparoscopic cholecystectomy, the patient's liver tissue undergoes slight deformation due to respiratory movement and instrument traction. During the procedure, when the tip of a surgical instrument (e.g., an electrocautery hook) contacts the gallbladder wall, the system simultaneously acquires surgical and endoscopic videos. By analyzing the endoscopic video, the system identifies the visual contact event between the instrument tip and the gallbladder wall, and combines this with force sensor data to confirm a sudden change in force, thus determining the presence of a tip contact event. At this point, based on the two-dimensional position of the instrument tip in the surgical video and the endoscopic perspective information, the precise three-dimensional position of the instrument tip in the patient's body surface coordinate system is calculated. This three-dimensional position is then compared with the initial spatial coordinates of pre-defined gallbladder anatomical landmarks (e.g., the gallbladder neck) in the three-dimensional liver model constructed from a preoperative CT scan. If a small spatial deviation is found (e.g., displacement due to liver respiratory movement), the system generates a local deformation compensation vector field based on this deviation to describe how the liver model needs to be locally adjusted to match the current actual morphology. Based on this vector field, the 3D liver model is updated in real time to ensure its deformation matches that of the actual liver. Ultimately, the 2D position of the surgical instrument tip is projected onto this updated, more realistic 3D liver model, thus obtaining its precise target anatomical location on the gallbladder wall. In this way, even under dynamic tissue deformation, the precise correspondence between the surgical instrument and the anatomical structure is ensured, providing high-precision spatial information for subsequent consumable usage analysis.

[0156] Through the above technical solution, this embodiment overcomes the problem of mismatch between the 3D medical imaging model and the actual anatomical structure caused by tissue deformation and patient displacement in traditional methods. By introducing endoscopic video and the judgment of tip contact events, real-time, local deformation compensation and updating of the 3D medical imaging model are achieved, significantly improving the accuracy and reliability of generating the target anatomical position of the surgical instrument tip in the 3D medical imaging model. This makes the correlation between subsequent key mechanical events and the target anatomical position more precise, thus providing a more reliable data foundation for the analysis of operating room consumable usage and enhancing the clinical guidance value of the analysis results.

[0157] In some embodiments, step S402, determining whether a tip contact event exists based on the endoscopic video and obtaining a tip contact determination result, may include, but is not limited to, the following steps:

[0158] When the tip of a surgical instrument visually coincides with a preset anatomical landmark in the endoscopic video, a visual coincidence event is triggered, and the first timestamp of the visual coincidence event is obtained.

[0159] When the force sensor detects a force value jump signal, it triggers a force value jump event and obtains the second timestamp of the force value jump event;

[0160] If the first timestamp and the second timestamp are within a preset time window, then the tip contact determination result is that a tip contact event exists.

[0161] In some embodiments, relying solely on visual or mechanical information to determine a tip contact event may lead to misjudgment or omission. For example, visual overlap may not necessarily represent actual physical contact, or fluctuations in the mechanical signal may not be caused by effective tissue contact, potentially resulting in inaccurate generation of the target anatomical location and affecting the reliability of the event's evidence chain. To address this, a visual overlap event can be triggered when the surgical instrument tip visually overlaps with a preset anatomical landmark in the endoscopic video, and a first timestamp of the event is obtained. Furthermore, a force value jump event is triggered when a force sensor detects a force value jump signal, and a second timestamp of the force value jump event is obtained. The force sensor is typically integrated into or connected to the surgical instrument to monitor the mechanical information experienced by the instrument during operation in real time. A force value jump signal refers to a signal where the force value detected by the force sensor changes significantly and rapidly within a short period of time; this change typically indicates physical contact or interaction between the surgical instrument tip and tissue or anatomical landmark. If the first and second timestamps fall within a preset time window, it indicates that the visual contact and the physical mechanical action occur synchronously in time, confirming the presence of a tip contact event. The preset time window is a pre-defined time interval used to determine whether the visual overlap event and the force jump event are temporally correlated. For example, this time window can be set to tens to hundreds of milliseconds to allow for delays in sensor response and data processing.

[0162] To illustrate this technical solution more clearly, a specific example is used below. Suppose that during a laparoscopic surgery, a surgeon uses surgical forceps equipped with a force sensor to manipulate liver tissue. When the tip of the forceps gradually approaches in the endoscopic video and eventually visually overlaps with a pre-defined tumor marker on the liver, the system immediately records the time of this visual overlap event, for example, the first timestamp is T1. Almost simultaneously, or within a very short time after T1 (e.g., within a pre-defined time window of 50 milliseconds), the force sensor integrated on the forceps detects a rapid increase in force value from zero, indicating that the tip of the forceps has made physical contact with the liver tissue. The system then records the time of this force value jump event, for example, the second timestamp is T2. Since T1 and T2 are within the pre-defined time window, the system determines this to be a genuine tip contact event. If there is only visual overlap without a force value jump, or if the time of the force value jump is too far from the time of visual overlap, it will not be considered a tip contact event, thus avoiding misjudgments caused by non-contact factors such as instrument movement or obstruction of view. In this way, the system can accurately capture every effective interaction between surgical instruments and the target anatomical region, providing high-precision input for subsequent anatomical location generation and event evidence chain construction.

[0163] Through the above technical solution, this embodiment can more accurately and reliably determine the actual contact event between the tip of the surgical instrument and the target anatomical location. This dual verification mechanism, combining visual coincidence and mechanical abrupt change signals and limiting it to a strict time window, effectively eliminates the uncertainty that may be caused by single-modal judgment, significantly reducing the false alarm rate and false negative rate. Therefore, it ensures the accuracy of the target anatomical location generated in the subsequent 3D medical imaging model, thereby improving the accuracy and credibility of the event evidence chain in the entire operating room consumables usage analysis method, and providing a more solid data foundation for surgical quality assessment and risk management.

[0164] In some embodiments, after generating an event evidence chain based on the target timestamp, force data, and target anatomical location, the method further includes:

[0165] Construct a risk event feature set, which includes the sequence of mechanical events, the target's anatomical location, and temporal relationships;

[0166] Extract the target event sequence from the event evidence chain;

[0167] The target event sequence is compared with the combined characteristics of risk events to obtain the comparison results;

[0168] Based on the comparison results, identify non-standard operation information and abnormal consumption information of consumables;

[0169] A feedback report is generated based on non-standard operation information and abnormal consumption information of consumables.

[0170] In some embodiments, simply generating a chain of event evidence may not be sufficient to provide actionable feedback to identify potential non-standard procedures or abnormal consumable consumption, thus failing to directly guide surgical quality improvement and consumable management. Therefore, a risk event feature set can be constructed first. This risk event feature set includes a sequence of mechanical events, a target anatomical location, and a temporal relationship. The risk event feature set can be a predefined set of patterns, or a set trained through machine learning, used to characterize specific risk events (e.g., tissue damage, instrument slippage, overcutting, etc.). The mechanical event sequence can include patterns of mechanical characteristics changing over time, such as peak force, duration, and rate of change. The target anatomical location represents the specific anatomical area where the instrument tip is located when the mechanical event occurs. The temporal relationship represents the temporal sequence and duration of different mechanical events or between mechanical events and anatomical locations.

[0171] Then, the target event sequence is extracted from the event evidence chain. For example, from a generated event evidence chain containing force data, target anatomical location, and timestamps, a series of continuous or related mechanical events and their corresponding anatomical locations and time information can be identified and extracted according to preset rules or algorithms, forming a sequence for comparison. For instance, a sequence of all mechanical events within a specific time period, or a sequence of mechanical events related to a specific anatomical region, can be extracted. The target event sequence is compared with a combination of risk event features to obtain the comparison results. The similarity or conformity between the extracted target event sequence and the preset risk event feature combination can be evaluated using methods such as pattern matching, similarity calculation, or rule engines. For example, Euclidean distance, dynamic time warping (DTW) distance, or whether preset logical rules are met can be calculated between sequences. The comparison result can be a similarity score, a matching percentage, or a Boolean value (match / no match).

[0172] Based on the comparison results, non-standard operation information and abnormal consumable consumption information are identified. Non-standard operation information refers to operational behaviors that do not conform to standard operating procedures or expected mechanical responses, such as applying excessive force to specific tissues, detecting force in locations where it should not be applied, or abnormal operation duration. Abnormal consumable consumption information refers to the use of consumables that does not conform to expected consumption, such as using more hemostatic clips, sutures, or energy devices than usual in a specific operation, which may be related to complications caused by non-standard operations. The identification of this information is based on the comparison results. When the comparison results show that the target event sequence highly matches a combination of characteristics of a certain risk event, the corresponding non-standard operation or abnormal consumable consumption can be inferred.

[0173] Finally, a feedback report is generated based on information about non-standard procedures and abnormal consumption of consumables. This report may include a detailed description of the identified non-standard procedures (e.g., applying continuous high pressure to liver tissue), related abnormal consumption of consumables (e.g., using three additional hemostatic clips), the timestamp of the occurrence, the corresponding surgical video clip, and potential impacts or recommendations. This report can be used for postoperative review, physician training, surgical procedure optimization, or adjustments to consumable procurement strategies.

[0174] To illustrate this technical solution more clearly, a specific example is used below. Assume that during a laparoscopic cholecystectomy, the system has generated an event evidence chain containing surgical instrument force data, target anatomical location (e.g., gallbladder bed), and timestamps. To identify potential risks, the system pre-constructs a risk event feature set for "gallbladder bed tear risk." This set defines a specific sequence of mechanical events and their temporal relationships in the gallbladder bed region where the force applied by the instrument tip consistently exceeds a certain threshold and the force change curve exhibits severe fluctuations (indicating tearing). During the surgery, the system extracts the target event sequence from the real-time updated event evidence chain. For example, within a certain time period, the system detects that the force applied by the surgical instrument tip in the gallbladder bed region consistently exceeds the preset threshold, and the force change curve shows significant fluctuations, which highly matches the feature set for "gallbladder bed tear risk." Based on the comparison results, the system identifies non-standard operational information such as "excessive tearing operation in the gallbladder bed region," and may further identify abnormal consumable consumption information such as "abnormally increased use of hemostatic clips." Finally, the system generates a feedback report that details the time of the incident, its duration, the anatomical areas involved, biomechanical data, and video clips for suggested review. It also points out that targeted training on the doctor's technique may be needed, or further analysis of the consumables used in this type of surgery may be required.

[0175] Through the above technical solution, this embodiment can transform objective data during surgery into understandable and analyzable risk information, overcoming the shortcomings of merely generating a chain of event evidence without subsequent analysis. This allows the operating room to more proactively identify and correct potential problems, such as detecting whether the surgeon has used excessive force, performed improper operations, or wasted consumables during specific procedures. Therefore, this not only helps improve the safety and efficiency of surgery and reduce the risk of complications, but also optimizes the efficiency of consumable usage, achieving refined management, thereby significantly improving the level of surgical quality management and resource utilization efficiency.

[0176] The beneficial effects of implementing the embodiments of the present invention include: First, the embodiments of this application acquire surgical videos and force data during the operation of surgical instruments, generate an operation reference, then identify key mechanical events and target timestamps based on the force data and operation references, extract video frames corresponding to key mechanical events from the surgical videos, determine the two-dimensional position of the tip of the surgical instrument on the video frames, generate the target anatomical position in the three-dimensional medical image model, and finally generate an event evidence chain based on the target timestamp, force data, and target anatomical position. This enables the analysis of the target anatomical position through key mechanical events and the generation of an event evidence chain, thereby realizing consumable usage analysis and improving accuracy.

[0177] like Figure 2 As shown, this embodiment of the invention also provides an operating room consumables usage analysis system, including:

[0178] Data acquisition module 501 is used to acquire surgical videos and force data during the operation of surgical instruments;

[0179] Operation reference generation module 502 is used to generate operation references based on force data;

[0180] The event recognition module 503 is used to identify key mechanical events and target timestamps based on force data and operational references.

[0181] The video frame extraction module 504 is used to extract video frames corresponding to key mechanical events from the surgical video based on the target timestamp;

[0182] The instrument two-dimensional position positioning module 505 is used to determine the two-dimensional position of the tip of the surgical instrument on a video frame;

[0183] Anatomical position generation module 506 is used to generate target anatomical positions in a three-dimensional medical image model based on the two-dimensional position of the tip of the surgical instrument.

[0184] The event evidence chain generation module 507 is used to generate an event evidence chain based on the target timestamp, force data, and target anatomical location.

[0185] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0186] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

Claims

1. A method for analyzing the use of operating room consumables, characterized in that, Includes the following steps: Acquire surgical videos and force data during the operation of surgical instruments; Based on the force data, an operation reference is generated; Based on the force data and the operational reference, identify key mechanical events and target timestamps; Based on the target timestamp, extract the video frames corresponding to the key mechanical events from the surgical video; On the video frame, determine the two-dimensional position of the tip of the surgical instrument; Based on the two-dimensional position of the tip of the surgical instrument, the target anatomical position is generated in the three-dimensional medical imaging model; A chain of event evidence is generated based on the target timestamp, the force data, and the target anatomical location.

2. The method according to claim 1, characterized in that, The step of generating an operation reference based on the force data includes: Based on the force data, generate a force variation curve; Mechanical features are extracted from the force variation curve, including the peak value of the force, the duration of the force value, or the smoothness of the force value variation process; Based on the mechanical characteristics, calculate the average value and trend of the characteristics; The average value of the features and the trend of change are used as references for the operation.

3. The method according to claim 1, characterized in that, The step of identifying key mechanical events and target timestamps based on the force data and the operation reference includes: According to the preset comparison dimension, the first comparison feature on the preset comparison dimension is extracted from the force data, and the second comparison feature on the preset comparison dimension is extracted from the operation reference. The preset comparison dimension includes the threshold comparison dimension or the change curve jitter comparison dimension. Calculate the degree of feature deviation based on the first comparison feature and the second comparison feature; Identify the key mechanical events based on the degree of deviation of the aforementioned features; The timestamp of the key mechanical event is used as the target timestamp.

4. The method according to claim 3, characterized in that, The process of identifying the key mechanical events based on the degree of deviation of the features includes: Construct a multi-dimensional deviation pattern for the key mechanical event, wherein the multi-dimensional deviation pattern is used to characterize a feature set including the magnitude, direction and rate of change of force; The degree of feature deviation is matched with the multi-dimensional deviation pattern to obtain the matching result; Based on the matching results, the key mechanical events are identified.

5. The method according to claim 4, characterized in that, The construction of the multi-dimensional deviation pattern of the key mechanical event includes: Tissue structure analysis is performed on the target anatomical region in preoperative medical images to obtain tissue structure information, including tissue density, tissue stiffness, degree of tissue calcification, or degree of tissue fibrosis. Based on the tissue structure information, the mechanical response of the target anatomical region under the operation of surgical instruments is calculated. The mechanical response includes the resistance range, deformation characteristics, or force value change trend. Based on the mechanical response, the multidimensional deviation mode is constructed.

6. The method according to claim 1, characterized in that, Determining the two-dimensional position of the surgical instrument tip on the video frame includes: On the video frame, identify the visible range of the surgical instrument; Extract instrument feature points from the visible area of ​​the surgical instrument; The posture of the surgical instrument in the video frame is determined based on the pre-stored geometric feature information of the surgical instrument and the instrument feature points. The geometric feature information of the surgical instrument includes the relative position information of the instrument tip. The two-dimensional position of the surgical instrument tip is calculated based on the posture of the surgical instrument in the video frame and the relative position information of the instrument tip.

7. The method according to claim 1, characterized in that, The step of generating the target anatomical location in the three-dimensional medical imaging model based on the two-dimensional position of the tip of the surgical instrument includes: Acquire endoscopic video, wherein the time series of the endoscopic video is the same as the time series of the surgical video; Based on the endoscopic video, determine whether a tip contact event exists, and obtain the tip contact determination result; If the tip contact determination result indicates that a tip contact event exists, then the three-dimensional position of the surgical instrument tip in the body surface coordinate system is calculated based on the two-dimensional position of the surgical instrument tip. The three-dimensional position is compared with the initial spatial coordinates of the preset anatomical landmark in the three-dimensional medical image model to calculate the local spatial deviation; Based on the local spatial deviation, a local deformation compensation vector field is generated; The three-dimensional medical image model is updated based on the local deformation compensation vector field. The two-dimensional position of the tip of the surgical instrument is projected onto the updated three-dimensional medical image model to obtain the target anatomical position.

8. The method according to claim 7, characterized in that, The step of determining whether a tip contact event exists based on the endoscopic video, and obtaining a tip contact determination result, includes: When the tip of the surgical instrument visually coincides with a preset anatomical landmark in the endoscopic video, a visual coincidence event is triggered, and the first timestamp of the visual coincidence event is obtained. When the force sensor detects a force value jump signal, it triggers a force value jump event and obtains the second timestamp of the force value jump event; If the first timestamp and the second timestamp are within a preset time window, then the tip contact determination result is determined to be that a tip contact event exists.

9. The method according to claim 1, characterized in that, After generating the event evidence chain based on the target timestamp, the force data, and the target anatomical location, the method further includes: Construct a risk event feature set, which includes a sequence of mechanical events, the target anatomical location, and temporal relationships; Extract the target event sequence from the event evidence chain; The target event sequence is compared with the risk event features to obtain the comparison results; Based on the comparison results, non-standard operation information and abnormal consumption information of consumables are identified; A feedback report is generated based on the non-standard operation information and the abnormal consumption information of consumables.

10. An operating room consumables usage analysis system, characterized in that, include: The data acquisition module is used to acquire surgical videos and force data during the operation of surgical instruments; The operation reference generation module is used to generate an operation reference based on the force data. The event recognition module is used to identify key mechanical events and target timestamps based on the force data and the operation reference. The video frame extraction module is used to extract the video frame corresponding to the key mechanical event from the surgical video based on the target timestamp; The instrument two-dimensional position positioning module is used to determine the two-dimensional position of the tip of the surgical instrument on the video frame; The anatomical location generation module is used to generate a target anatomical location in a three-dimensional medical image model based on the two-dimensional position of the tip of the surgical instrument. The event evidence chain generation module is used to generate an event evidence chain based on the target timestamp, the force data, and the target anatomical location.

Images