An AI vision-based microsurgical suture needle real-time management and early warning simulation platform

Abstract

This invention relates to the field of medical information and discloses a real-time management and early warning simulation platform for microsurgical sutures based on AI vision. The platform includes: an intelligent counting pad for real-time acquisition of image and weight data of microsurgical sutures; an AI recognition module for processing the image data and outputting a recognition result of the number of microsurgical sutures; a simulation workflow engine for simulating and managing the counting status of microsurgical sutures throughout the entire surgical process in a digital space based on the recognition result and the weight data; and an early warning and display module for displaying the counting result in real-time according to the output of the simulation workflow engine and triggering an early warning signal when an abnormality is detected. This application designs an intelligent counting pad as a data acquisition terminal, integrating a deep learning model combining YOLOv8 and SAM algorithms to achieve sub-millimeter-level recognition and high-precision counting of microsurgical sutures.

Description

Technical Field

[0001] This invention relates to the field of medical information, and in particular to a simulation platform for real-time management and early warning of microsurgical sutures based on AI vision. Background Technology

[0002] Surgery instrument counting is crucial to prevent surgical instruments from being left inside the patient's body. Due to the fast pace of work, rapid technological and equipment updates, and interdisciplinary collaboration inherent in operating rooms, nurses often perform multiple tasks simultaneously, increasing the risk of interruptions in the counting process. Surgical sutures, especially 7-0, 8-0, and even 10-0 microsurgical sutures and finer models, are extremely difficult to count during surgery (as thin as a hair, with a diameter of only 30-70 micrometers) and difficult to visualize in vivo. Their absence poses a significant risk of serious medical accidents. Traditional methods relying on scrub nurses' visual and manual counting are highly prone to errors in the stressful, multi-tasking surgical environment, posing a substantial threat to patient safety. Traditional manual counting methods can no longer meet the extreme demands of modern precision surgery for both safety and efficiency. Summary of the Invention

[0003] The main objective of this invention is to solve the technical problem that existing technologies cannot meet the extreme requirements of safety and efficiency in modern precision surgery. A real-time management and early warning simulation platform for microsurgical sutures based on AI vision is provided, comprising:

[0004] Intelligent counting pads are used to acquire image and weight data of microsurgical sutures in real time;

[0005] The AI ​​recognition module is configured as follows:

[0006] The number of preliminary detection boxes obtained based on the target detection unit is used as the first quantity value, and the number of instance segmentation masks obtained based on the image segmentation unit is used as the second quantity value;

[0007] The first quantity value, the second quantity value, and the weight data are respectively converted into a first probability distribution, a second probability distribution, and a weight likelihood probability model regarding the number of microsurgical sutures.

[0008] Based on Bayes' theorem, the first probability distribution, the second probability distribution, and the weight likelihood probability model are integrated to obtain the final posterior probability distribution of the number of microsurgical sutures.

[0009] If the maximum probability value in the final posterior probability distribution is lower than the confidence threshold, the perception enhancement mechanism is triggered to dynamically adjust the imaging parameters of the smart counting pad to obtain new image data and recalculate the final posterior probability distribution.

[0010] The final number of microsurgical sutures is output based on the updated final posterior probability distribution as the identification result.

[0011] A simulation workflow engine is used to simulate and manage the microsurgical suture counting status throughout the entire surgical process in digital space based on the identification results and the weight data; and

[0012] The early warning and display module is used to display the inventory results in real time based on the output of the simulation workflow engine, and to trigger an early warning signal when an inventory abnormality is detected.

[0013] The platform employs a dual verification mechanism of image recognition and weight verification. The dual verification mechanism includes: calculating the theoretical total weight based on the recognition results and comparing it with the actual total weight measured by the smart counting pad; triggering an early warning signal when the difference obtained from the comparison exceeds a preset threshold.

[0014] The present invention has the following beneficial effects:

[0015] This invention designs an intelligent counting pad as a data acquisition terminal, and combines it with a deep learning model that integrates YOLOv8 and SAM algorithms to achieve sub-millimeter-level identification and high-precision counting of microscopic suture needles.

[0016] This invention utilizes a simulation engine to construct a digital workflow encompassing preoperative, intraoperative, and postoperative procedures, achieving near-zero intraoperative counting time and instant triggering of risk warnings. This project fills a gap in the international field of intelligent counting of microsurgical sutures. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a simulation platform for real-time management and early warning of microsurgical sutures based on AI vision, provided in an embodiment of the present invention. Detailed Implementation

[0018] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] The platform, based on the core concept of AI visual recognition and simulation workflow reconstruction, has built an intelligent surgical suture counting platform that integrates hardware perception, algorithm recognition, process control, and data traceability. Guided by clinical nursing pain points and using simulation technology as the central hub, the system achieves a leap from manual experience-based operation to digital twin verification, truly introducing visibility, measurability, and controllability into the surgical counting scenario.

[0020] This invention discloses a real-time management and early warning simulation platform for microsurgical sutures based on AI vision, comprising:

[0021] Intelligent counting pads are used to acquire image and weight data of microsurgical sutures in real time;

[0022] The AI ​​recognition module is used to process the image data and output the recognition result of the number of microsurgical sutures;

[0023] A simulation workflow engine is used to simulate and manage the microsurgical suture counting status throughout the entire surgical process in digital space based on the identification results and the weight data; and

[0024] The early warning and display module is used to display the inventory results in real time based on the output of the simulation workflow engine, and to trigger an early warning signal when an inventory abnormality is detected.

[0025] The platform employs a dual verification mechanism of image recognition and weight verification. The dual verification mechanism includes: calculating the theoretical total weight based on the recognition results and comparing it with the actual total weight measured by the smart counting pad; triggering an early warning signal when the difference obtained from the comparison exceeds a preset threshold.

[0026] In this invention, the AI ​​recognition module includes a target detection unit and an image segmentation unit:

[0027] The target detection unit processes the image data based on the YOLOv8 algorithm and outputs a preliminary detection box for microsurgical sutures. The image segmentation unit uses the preliminary detection box as input prompts and performs pixel-level instance segmentation of the microsurgical sutures based on the Segment Anything Model algorithm to obtain instance segmentation results. Based on the instance segmentation results, the number and type of microsurgical sutures are output.

[0028] In this invention, the target detection unit further includes:

[0029] A training image dataset containing different types of microsurgical sutures was collected. The bounding boxes of the microsurgical sutures were identified and their dimensions were labeled. The width and height features of the bounding boxes were extracted to form a size dataset. The size dataset was clustered using the K-means clustering algorithm to obtain optimized anchor box dimensions. The optimized anchor box dimensions were integrated into the YOLOv8 model to output the preliminary detection boxes of microsurgical sutures.

[0030] In this invention, the K-means clustering algorithm is applied to perform cluster analysis on the size dataset to obtain optimized anchor frame sizes, including:

[0031] From the size dataset, extract the width w and height h of all bounding boxes to form an i×2 size dataset D = {(w1, h1), (w2, h2), ..., (wi, hi)}, where i is the number of bounding boxes in the size dataset; determine the number of clusters K; randomly select K points from the size dataset D as the initial cluster centers C = {C1, C2, ...,CK};

[0032] The iterative process is as follows:

[0033] Assignment Steps: For each bounding box (wi, hi) in the size dataset, calculate the intersection-union distance (IU) between the bounding box (wi, hi) and all K cluster centers; assign the bounding box to the cluster center with the smallest distance. Update Steps: After all bounding boxes have been assigned, for each cluster j, calculate the median width and height of all bounding boxes in that cluster, and use it as the new cluster center Cj_new; Convergence Criterion: Repeat the assignment and update steps until one of the following conditions is met: the change in cluster centers is less than a preset threshold.

[0034] The maximum number of iterations was reached; after convergence, the final K cluster centers C_final = {(cw1,ch1), (cw2, ch2), ..., (cw_K, ch_K)} were obtained; that is, the optimal anchor frame size was obtained.

[0035] In this invention, the image segmentation unit includes:

[0036] Each preliminary detection box is converted into input to the SAM model. For each preliminary detection box, the SAM model performs forward propagation to generate a corresponding pixel-level instance segmentation mask. The pixel-level instance segmentation mask accurately marks the outline of the microsurgical suture and distinguishes between the background and the microsurgical suture pixels. Based on the SAM encoder-decoder architecture, the encoder extracts image features, and the decoder combines the prompt information to generate the mask. The instance segmentation result for each microsurgical suture is obtained, including the instance segmentation mask, confidence score, and bounding box. The number of microsurgical sutures is calculated based on the instance segmentation mask.

[0037] In this invention, the number of microsurgical sutures is obtained through the following steps:

[0038] Based on the preliminary detection boxes of the target detection unit, the number of preliminary detection boxes is counted to obtain a first quantity value; based on the instance segmentation mask of the image segmentation unit, the number of preliminary detection boxes is counted to obtain a second quantity value; based on the first quantity value and the second quantity value, the number of microsurgical sutures is obtained.

[0039] In this invention, the number of microsurgical sutures is obtained based on a first quantity value and a second quantity value, including:

[0040] The first quantity value, the second quantity value, and the actual weight obtained from the smart counting pad are respectively converted into the first probability distribution, the second probability distribution, and the weight likelihood probability model regarding the number of microsurgical sutures.

[0041] Based on Bayes' theorem, the first probability distribution, the second probability distribution and the weight likelihood probability model are integrated to obtain the final posterior probability distribution of the number of microsurgical sutures.

[0042] If the maximum probability value in the final posterior probability distribution is lower than the confidence threshold, the perception enhancement mechanism is triggered. The instruction control module dynamically adjusts the imaging parameters to obtain new visual data and repeats the aforementioned steps to update the final posterior probability distribution.

[0043] The final number of microsurgical sutures is output based on the updated final posterior probability distribution.

[0044] In this invention, the first probability distribution is obtained through the following steps: obtaining the confidence score of the preliminary detection box; and obtaining the first probability distribution based on the confidence score.

[0045] In this invention, the second probability distribution is obtained through the following steps: obtaining the quality score of the instance segmentation mask; and obtaining the second probability distribution based on the quality score.

[0046] In this invention, the weight likelihood probability model is obtained through the following steps:

[0047] A weight likelihood model was obtained based on the accuracy of the weight sensor and the weight range of a single microsurgical suture.

[0048] In this invention, the dual verification mechanism includes:

[0049] Based on the number and type of microsurgical sutures obtained, and according to the preset standard weight of a single microsurgical suture, the theoretical total weight is calculated. The weighing module of the intelligent counting pad measures the total weight of all items on the pad in real time to obtain the actual total weight. The difference ΔW between the theoretical and actual total weights is calculated. If the difference ΔW is less than the preset tolerance threshold, the system determines that the image recognition and weight verification are consistent, and the counting result is confirmed as reliable. If the difference ΔW is greater than the preset tolerance threshold, a conflict is determined, and different strategies are automatically executed according to the type of conflict: a) Theoretical total weight is less than actual total weight: if suspected foreign object is found, a warning pops up on the screen, and the foreign object is highlighted on the screen; b) Theoretical total weight is greater than actual total weight: if suspected omission is found, a warning pops up on the screen, prompting manual identification.

[0050] Hardware system design, such as Figure 1 :

[0051] 1) Intelligent Counting Pad: Made of medical-grade silicone, it is waterproof and antibacterial. The pad contains an embedded ring-shaped shadowless light source and a macro high-definition camera module. The light source angle and brightness can be automatically adjusted to overcome interference from needle reflections and shadows. It also features a miniature weighing module, implementing a dual verification mechanism of image recognition and weight check, further enhancing recognition robustness and reliability.

[0052] 2) Operating Room Terminal: Utilizes a wall-mounted touchscreen to display real-time count results, AI-recognized images, and risk warnings. The interface is simple and intuitive, allowing nurses to view quantity changes and count status with a single click.

[0053] 3) Server / Cloud: Responsible for AI model inference calculations and data storage. Supports local deployment or cloud connection to meet the information security requirements of different hospitals.

[0054] Software system design:

[0055] The system is trained on a self-acquired high-resolution microscopic needle suture image database, covering multiple angles, different lighting conditions, and overlapping states. It employs an improved fusion of the YOLOv8 target detection algorithm and the Segment Anything Model (SAM) image segmentation algorithm for fusion modeling, achieving dual-channel recognition of target detection and semantic segmentation. By modeling the geometric features of the needle tip, needle arc, and needle tail at multiple scales, the model can accurately segment and count needles even in severely overlapping, tangled, and reflective environments by recognizing key feature points such as the needle tip and needle tail.

[0056] Simulation workflow engine:

[0057] Using the entire process of "pre-operative, intra-operative, and post-operative" as its logical framework, the system incorporates surgical inventory rules and intelligent judgment logic. It can simulate and map the dynamic state of sutures in real-time in a digital space, achieving virtual-real synchronization, automatic updates, and closed-loop management. Once an abnormal quantity is detected, the system will immediately trigger an audible and visual alarm and generate a digital traceability report.

[0058] System Overall Architecture

[0059] The platform adopts a three-tier distributed architecture:

[0060] 1) Perception layer (physical acquisition): The smart counting pad, as an IoT terminal, is responsible for real-time acquisition of image and weight information.

[0061] 2) Simulation Layer (Digital Twin Core): Deployed on local servers or edge computing nodes, this layer contains AI recognition models and business logic systems. It receives data from the perception layer and simulates and reconstructs the distribution and quantity of the physical world's stitches in the digital space, achieving a virtual-real mapping.

[0062] 3) Application layer (interaction and control): Operating room terminal display screen and back-end management system, providing human-computer interaction interface, real-time data display, early warning push and electronic record generation functions.

[0063] This architecture enables "closed-loop control" from physical operation to digital twin, laying a technological foundation for future expansion into robot-assisted surgery and intelligent material management.

[0064] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the AI ​​vision-based microsurgical suture real-time management and early warning simulation platform of the present invention includes:

[0065] Phase 1: Preoperative Preparation

[0066] 1) The scrub nurse opens the sterile packaging and pours a whole pack of new sutures onto the smart counting pad.

[0067] 2) The camera on the counting pad takes a picture automatically, and the AI ​​model completes the recognition within 1 to 2 seconds. The recognition result is displayed on the screen next to it: Recognition successful: 7-0 needles, quantity: 24 needles, and each needle is highlighted with a green box.

[0068] 3) After the nurse confirms that the number on the screen matches the packaging label, she clicks the "Confirm Base Number" button on the screen. The system records this point as the needle base number being confirmed and records the handwashing nurse's employee number.

[0069] 4) The nurse puts away the sutures and places them in a designated area on the instrument table.

[0070] Phase Two: Intraoperative Handover

[0071] 1) The surgeon places the used sutures in the designated area, preferably directly on the smart counting pad.

[0072] 2) Key Steps: Nurses can immediately turn around and retrieve other instruments needed by the doctor without stopping. The system platform automatically completes the following tasks within seconds:

[0073] ① The counting pad detects an object being placed inside and automatically triggers a photo capture and AI recognition.

[0074] ②The AI ​​model analyzes the image and accurately counts the number of stitches: 8.

[0075] ③ The system automatically subtracts these 8 needles from the current "remaining needles on the table" and records a log: "At XX time, 8 needles were recycled and reused."

[0076] 3) When the nurse retrieves other instruments, she can quickly glance at the screen. The screen will clearly indicate: Latest retrieved: 8 instruments. Current remaining on the table: 16 instruments.

[0077] 4) If the AI's confidence level is low, a yellow warning will appear on the screen asking you to confirm the quantity, along with images of suspicious targets it has identified. The nurse only needs to spend one second checking and clicking to confirm or modify.

[0078] Phase 3: Postoperative inventory

[0079] 1) After the surgical suturing is completed, the pre-closure count begins.

[0080] 2) Collect all used and unused sewing needles on the table and place them on the smart counting mat.

[0081] 3) The AI ​​performs a final count to identify the total number.

[0082] 4) The system compares the final quantity with the "base number":

[0083] Match successful: A large green checkmark appears on the screen indicating that the count is correct! An electronic record is generated, which is then signed by the scrub nurse and the circulating nurse for confirmation.

[0084] Quantity discrepancy: A red alert will appear on the screen indicating an inventory error! X units are missing!, along with an audible warning. The system will retrieve all handover records to help trace any potentially missing steps. For example, the record may show that 5 units were collected at a specific time, but the current total is 1 unit short.

[0085] 5) Based on the alarm, the team can immediately focus their search in the surgical field, on and off the operating table, greatly increasing the probability of finding the missing sutures in time.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI vision-based microsurgical suture needle real-time management and early warning simulation platform, characterized in that, The platform includes: Intelligent counting pads are used to acquire image and weight data of microsurgical sutures in real time; The AI ​​recognition module, comprising a target detection unit and an image segmentation unit, is configured as follows: The initial number of detection frames obtained based on the target detection unit is used as the first quantity value; The number of instance segmentation masks obtained based on the image segmentation unit is used as the second quantity value; The first quantity value, the second quantity value, and the weight data are respectively converted into a first probability distribution, a second probability distribution, and a weight likelihood probability model regarding the number of microsurgical sutures. The first probability distribution is obtained through the following steps: obtaining the confidence score of the preliminary detection box; obtaining the first probability distribution based on the confidence score. The second probability distribution is obtained through the following steps: obtaining the quality score of the instance segmentation mask; obtaining the second probability distribution based on the quality score. The weight likelihood probability model is obtained through the following steps: obtaining the weight likelihood model based on the accuracy of the weight sensor and the weight range of a single microsurgical suture. Based on Bayes' theorem, the first probability distribution, the second probability distribution, and the weight likelihood probability model are integrated to obtain the final posterior probability distribution of the number of microsurgical sutures. If the maximum probability value in the final posterior probability distribution is lower than the confidence threshold, the perception enhancement mechanism is triggered to dynamically adjust the imaging parameters of the smart counting pad to obtain new image data and recalculate the final posterior probability distribution. The final number of microsurgical sutures is output as the recognition result based on the updated final posterior probability distribution. A simulation workflow engine is used to simulate and manage the microsurgical suture counting status throughout the entire surgical process in digital space based on the identification results and the weight data; and The early warning and display module is used to display the inventory results in real time based on the output of the simulation workflow engine, and to trigger an early warning signal when an inventory abnormality is detected. The platform employs a dual verification mechanism of image recognition and weight verification. The dual verification mechanism includes: calculating the theoretical total weight based on the recognition results and comparing it with the actual total weight measured by the smart counting pad; triggering an early warning signal when the difference obtained from the comparison exceeds a preset threshold.

2. The AI vision-based microsurgical suture needle real-time management and early warning simulation platform according to claim 1, characterized in that, The AI ​​recognition module includes a target detection unit and an image segmentation unit: The target detection unit processes the image data based on the YOLOv8 algorithm and outputs a preliminary detection box for microsurgical sutures. The image segmentation unit is based on the Segment Anything Model algorithm. Using the preliminary detection box as input prompt, it performs pixel-level instance segmentation on the microsurgical suture to obtain instance segmentation results. Based on the instance segmentation results, the number and type of microsurgical sutures are output.

3. The AI ​​vision-based real-time management and early warning simulation platform for microsurgical sutures according to claim 2, characterized in that, The target detection unit further includes: Collect a training image dataset containing different types of microsurgical sutures, identify the bounding boxes of the microsurgical sutures, and annotate the bounding box dimensions; Extract the width and height features of the bounding box to form a size dataset; The K-means clustering algorithm was applied to perform cluster analysis on the size dataset to obtain the optimized anchor frame size; The optimized anchor frame size is integrated into the YOLOv8 model to output the initial detection frame for microsurgical sutures.

4. The AI vision-based microsurgical suture needle real-time management and early warning simulation platform according to claim 3, characterized in that, The application of the K-means clustering algorithm to perform cluster analysis on the size dataset yields optimized anchor frame sizes, including: From the size dataset, extract the width w and height h of all bounding boxes to form an i×2 size dataset D = {(w1, h1), (w2, h2), ..., (wi, hi)}, where i is the number of all bounding boxes in the size dataset; Determine the number of clusters K; K points are randomly selected from the size dataset D as the initial cluster centers C = {C1, C2, ..., CK}; The iterative process is as follows: Assignment steps: For each bounding box (wi, hi) in the size dataset, calculate the intersection-union distance (IU) between the bounding box (wi, hi) and all K cluster centers; assign the bounding box to the cluster center with the smallest distance. Update steps: After all bounding boxes have been assigned, for each cluster j, calculate the median width and height of all bounding boxes within the cluster, and use it as the new cluster center Cj_new; Convergence check: Repeat the assignment and update steps until one of the following conditions is met: The change in cluster centers is less than a preset threshold; The maximum number of iterations has been reached; After convergence, the final K cluster centers C_final = {(cw1, ch1), (cw2, ch2), ..., (cw_K, ch_K)} are obtained; that is, the optimal anchor frame size is obtained.

5. The AI ​​vision-based real-time management and early warning simulation platform for microsurgical sutures according to claim 4, characterized in that, The image segmentation unit includes: Each preliminary detection box is converted into an input to SAM; For each initial detection box, the SAM model performs forward propagation to generate a corresponding pixel-level instance segmentation mask. The pixel-level instance segmentation mask accurately marks the outline of the microsurgical suture and distinguishes the background and microsurgical suture pixels. The encoder-decoder architecture based on SAM involves the encoder extracting image features and the decoder combining the prompt information to generate a mask. The instance segmentation results for each microsurgical suture are obtained, including instance segmentation mask, confidence score and bounding box; Calculate the number of microsurgical sutures based on instance segmentation masks.

6. The AI vision-based real-time microsurgical needle management and early warning simulation platform according to claim 1, wherein, The dual verification mechanism includes: Based on the number and type of microsurgical sutures obtained; The theoretical total weight is calculated based on the preset standard weight of a single microsurgical suture needle. The weighing module of the smart counting mat measures the total weight of all items on the mat in real time to obtain the actual total weight; Calculate the difference ΔW between the theoretical total weight and the actual total weight; If the difference ΔW is less than the preset tolerance threshold, the system determines that the image recognition and weight verification are consistent, and the counting result is confirmed as reliable. If the difference ΔW is greater than the preset tolerance threshold, a conflict is determined to have occurred, and different strategies are automatically executed according to the type of conflict: a. The theoretical total weight is less than the actual total weight: If it is determined to be a suspected foreign object, a warning will pop up on the screen and the foreign object will be highlighted on the screen; b. The theoretical total weight is greater than the actual total weight: If it is determined to be a suspected omission, a warning will pop up on the screen, prompting manual identification.

Images