An AI vision-based medical operation standardization automatic evaluation method and system

Abstract

The application provides an AI vision-based medical operation standardization automatic evaluation method and system, and relates to the technical field of intelligent medical assistance. Among them, the application first acquires continuous multiple frames of images, and analyzes to obtain the spatial coordinate sequence of the surgical instrument entity and the medical staff limb entity, to generate the interaction trajectory sequence between the two; at the same time, a three-dimensional sterile barrier envelope surface is constructed in the clinical operation space based on the edge contour sequence of the surgical drape; then the spatial topology of the interaction trajectory sequence and the three-dimensional sterile barrier envelope surface is determined, the direction attribute of the penetration and the spatial depth sequence are determined, and the event sequence composed of multiple entity transfer events is disassembled accordingly; finally, the event sequence and the sterile operation timing logic constraint set are compared in the graph structure topology, and the sterile violation node record is output. The application realizes the whole-process automation, objectivity and traceability evaluation of the sterile operation process, and improves the accuracy and efficiency of the medical operation standardization evaluation.

Description

Technical Field

[0001] This application relates to the field of intelligent medical assistance technology, and in particular to an automatic evaluation method and system for the standardization of medical operations based on AI vision. Background Technology

[0002] In clinical surgery, invasive procedures, and interventional treatments, strict adherence to aseptic techniques is crucial for ensuring patient safety and preventing hospital-acquired infections. Aseptic technique requires healthcare professionals to strictly follow a unidirectional and irreversible temporal logic constraint, including the movement of their hands, surgical instruments, and the trajectory of their movements during the procedure. This constraint is based on the principles of "from top to bottom," "from inside to outside," and "contaminated areas must not be re-entered into sterile areas." Any violation of the sterile barrier can directly lead to contamination of the surgical field.

[0003] Existing medical procedure standardization assessment schemes typically employ manual supervision and scoring, along with simple sensor-assisted methods. For example, some hospitals rely on infection control specialists or instructors to conduct on-site visual supervision in the operating room, manually recording and retrospectively tracking the actions of medical staff and their movements in and out of sterile areas. Other schemes involve placing radio frequency identification tags or simple proximity switches on the wrists of medical staff or on instruments, triggering a prompt only when the operator or instrument crosses a pre-set fixed boundary line.

[0004] However, existing solutions have significant limitations in clinical application. First, manual visual supervision is highly dependent on the clinical experience and concentration of the supervisors. Long-term supervision can easily lead to fatigue and omissions, and the judgment criteria of different supervisors vary, making it difficult to guarantee the objectivity and repeatability of the assessment results. Second, solutions based on radio frequency identification or proximity switches can only provide binary boundary crossing prompts, and cannot identify the direction of movement, spatial depth, and residence time distribution on both sides of the sterile barrier when the boundary crossing occurs, let alone determine the temporal logical relationship between multiple crossing behaviors. Third, the sterile barrier is not a rigid plane in actual surgery. Surgical drapes will dynamically deform due to instrument placement, touch by medical staff, and their own gravity. Fixed boundary line solutions cannot adapt to this dynamic geometric change, and are prone to missed or false alarms. Fourth, existing solutions lack the ability to break down violations into events and perform graph topological comparisons, making it impossible to trace violations in a structured way afterward, which is not conducive to continuous quality improvement. Summary of the Invention

[0005] To address the problems existing in the prior art, the purpose of this application is to provide an AI vision-based automatic assessment method and system for the standardization of medical operations. This method obtains the interaction trajectory sequence of surgical instruments and medical staff limbs by performing AI visual analysis on multiple consecutive frames of images in the clinical operation space. Based on the edge contour sequence of the surgical drape, a three-dimensional sterile barrier envelope is dynamically constructed. Then, the interaction trajectory sequence and the three-dimensional sterile barrier envelope are spatially topologically determined and decomposed into event sequences. Finally, the violation of irreversible directional rules is determined by graph structure topological comparison, thereby achieving an objective, quantitative, and traceable assessment of the standardization of sterile operations throughout the entire process.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides an AI-based vision-based automatic assessment method for the standardization of medical procedures, including: Acquire consecutive multi-frame images of the clinical operating space and parse the consecutive multi-frame images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities; Generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence; The edge contour sequence of the surgical drape is obtained by parsing the continuous multi-frame images, and a three-dimensional sterile barrier envelope is constructed in the clinical operating space based on the edge contour sequence of the surgical drape. The interaction trajectory sequence is compared with the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence through the sterile barrier envelope surface. Based on the penetration and exit direction attributes and the spatial depth sequence, the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope is determined. The residence timeline is then decomposed into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo position transfer in the regions on both sides of the sterile barrier envelope. The event sequence is compared with a pre-configured set of aseptic operation timing logic constraints using a graph structure topology. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated.

[0007] Optionally, multiple consecutive frames of images of the clinical operating space are acquired and parsed to obtain a spatial coordinate sequence of surgical instrument entities and medical staff limb entities, including: The apparent regions of human limbs and the apparent regions of instrument materials are extracted from the consecutive multi-frame images to locate the initial spatial detection boxes of the apparent regions of human limbs and the apparent regions of instrument materials. Skeletal joints are extracted within the apparent area of ​​the human limb to construct a dynamic physical linkage model of the medical staff's limb entity, and geometric structural lines are extracted within the apparent area of ​​the instrument material to determine the gripping force points of the surgical instrument entity. Spatial interference determination is performed between the end node of the dynamic physical link model and the gripping force point. When the physical distance between the end node of the dynamic physical link model and the gripping force point in three-dimensional space is less than a preset contact constraint threshold, it is determined that a physical gripping state is established. Based on the chronological order of the image frames, the absolute spatial coordinates of the gripping force points during the period of maintaining the entity's gripping state are extracted sequentially to generate a time-distributed sequence of coordinate nodes. The absolute spatial coordinates of adjacent moments in the coordinate node sequence are physically connected in three dimensions to construct a local spatial offset vector, and the local spatial offset vectors are sequentially spliced ​​together to generate an interactive trajectory sequence.

[0008] Optionally, the edge contour sequence of the surgical drape is obtained by parsing the continuous multi-frame images, and a three-dimensional sterile barrier envelope is constructed in the clinical operating space based on the edge contour sequence of the surgical drape, including: Extract the visual boundary points that separate the specified fabric color space from the background environment color space from the consecutive multi-frame images; Connect the multiple visual boundary points end to end to form a closed planar polygon representing the surgical drape; Using the closed planar polygon as a two-dimensional reference base, and combining the visual depth information in the consecutive multi-frame images, the deformation displacement value along the vertical spatial axis of the actual physical towel surface mapped inside the two-dimensional reference base is extracted. The closed planar polygon is physically stretched along the vertical spatial axis using the deformation displacement value to generate a three-dimensional geometric outer box; The physical surface of the three-dimensional geometric outer box facing the designated external surface is defined as the three-dimensional sterile barrier envelope.

[0009] Optionally, spatial topological determination is performed between the interaction trajectory sequence and the three-dimensional sterile barrier envelope to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence penetrating the sterile barrier envelope, including: Extract the normal vector protruding toward the external space along the surface of the three-dimensional sterile barrier envelope; Extract the intersection points where the interaction trajectory sequence physically intersects with the envelope surface of the three-dimensional sterile barrier, and calculate the velocity vector of the interaction trajectory sequence at the intersection points; The direction angle between the motion velocity vector and the normal vector is compared, and the entry and exit direction attributes of the interaction trajectory sequence at the crossing intersection are confirmed according to the comparison result. The entry and exit direction attributes include entry direction attributes and exit direction attributes. After determining the penetration and exit direction attributes, extract the dwell position points of the interaction trajectory sequence when it enters the internal space of the three-dimensional sterile barrier envelope surface; Calculate the physical perpendicular distance from the dwelling location point to the surface of the three-dimensional sterile barrier envelope along the normal vector in the opposite direction to form a spatial depth sequence.

[0010] Optionally, based on the penetration / exit direction attributes and the spatial depth sequence, the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope is determined. The residence timeline is then decomposed into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo positional transfer in the regions on both sides of the sterile barrier envelope, including: When the penetration and exit direction attributes are obtained, the space intrusion start timestamp is recorded, and when the exit direction attributes are obtained, the space withdrawal end timestamp is recorded. The spatial intrusion start time stamp, the spatial withdrawal end time stamp, and the spatial coordinate sequence between the spatial intrusion start time stamp and the spatial withdrawal end time stamp are bound together to form a continuous dwelling segment. The continuous dwelling segments are spliced ​​together end to end according to the absolute physical timeline of occurrence to construct a dwelling timeline; Extract the key turning point where the spatial depth sequence reverses its trend and crosses zero on the dwell timeline; At the critical turning point, the dwell timeline is physically truncated to separate single physical action segments with independent directions, and each single physical action segment is defined as an entity transfer event. Integrate all entity transfer events to obtain an event sequence.

[0011] Optionally, the event sequence is compared with a pre-configured set of aseptic operation timing logic constraints using a graph structure topology. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated, including: Extract the sequential relationship between the actions of the entity transfer events in the event sequence to construct an action directed graph; Call the pre-established set of standard constraint pointer edges from the set of aseptic operation timing logic constraints; Traverse the directed graph of the action along the time-increasing direction and extract the actual occurrence pointing edge connecting any two adjacent entity transition events; The actual occurrence of the pointing edge is used to retrieve the mapped edge with the same starting trigger node from the standard constraint pointing edge set; When the actual occurrence of the pointing edge indicates that the entity returns from the external sterile state node to the internal sterile state node, and violates the one-way isolation restriction defined by the mapping edge, it is determined that the directional irreversibility rule is triggered, and an operation blocking signal is output to generate a sterile violation node record.

[0012] Optionally, after constructing the three-dimensional sterile barrier envelope, the method further includes a dynamic adaptation process for the three-dimensional sterile barrier envelope: After generating the three-dimensional sterile barrier envelope, the local geometric changes of the closed planar polygon in the newly added image frame are tracked in real time. Locate the local deformation boundary segment on the closed planar polygon whose actual physical coordinate offset distance exceeds the threshold constraint parameter; Re-extract the latest deformation displacement value corresponding to the local deformation boundary segment at the current moment; The latest deformation displacement value is used to re-physically stretch the affected local boundary of the solid geometric outer box, and locally adjust the three-dimensional external contour of the solid geometric outer box. The original three-dimensional sterile barrier envelope area is replaced with the corresponding physical surface of the adjusted three-dimensional geometric outer box.

[0013] Secondly, this application provides an AI vision-based automated assessment system for medical operation standardization, comprising: The acquisition module is used to acquire multiple consecutive frames of images of the clinical operating space and parse the multiple consecutive frames of images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities. The generation module is used to generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence. The construction module is used to parse the continuous multi-frame images to obtain the edge contour sequence of the surgical drape, and to construct a three-dimensional sterile barrier envelope in the clinical operating space based on the edge contour sequence of the surgical drape; The determination module is used to perform spatial topology determination on the interaction trajectory sequence and the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence penetrating the sterile barrier envelope surface. The disassembly module is used to determine the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope surface based on the penetration and exit direction attributes and the spatial depth sequence. The residence timeline is then disassembled into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo position transfer in the regions on both sides of the sterile barrier envelope surface. The comparison module is used to perform graph structure topology comparison between the event sequence and a pre-configured set of aseptic operation timing logic constraints. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated.

[0014] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement an AI vision-based automatic assessment method for medical operation standardization as described in the first aspect above.

[0015] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements an AI vision-based automatic assessment method for medical operation standardization as described in the first aspect.

[0016] This application uses AI visual entity recognition and skeletal joint tracking on multiple consecutive frames of images in the clinical operating space to simultaneously acquire the spatial coordinate sequence of surgical instruments and medical staff limbs and synthesize the interaction trajectory sequence between the two. This allows the gripping action of medical staff and the actual displacement of instruments to be recorded simultaneously under a unified reference system, fundamentally solving the bottleneck of existing solutions being unable to depict the coordinated motion information of instruments and limbs. Furthermore, this application extracts the edge contour of the surgical drape visually and stretches it into a three-dimensional geometric outer box by combining visual depth information. Then, its upward-facing physical surface is used as the envelope of the three-dimensional sterile barrier, so that the sterile barrier is upgraded from the traditional fixed boundary line to a three-dimensional curved surface that dynamically adapts to the deformation of the drape. This avoids the incompatibility of rigid boundaries with actual clinical scenarios and supports local updates as the drape deforms. Furthermore, this application extracts the penetration and exit direction attributes and spatial depth sequence by performing spatial topology determination on the interaction trajectory sequence and the envelope surface of the three-dimensional sterile barrier, and decomposes the dwell timeline into an event sequence composed of multiple entity transfer events, so that continuous motion behavior is structured into discrete events, which facilitates subsequent logical tracing. Finally, this application performs a graph topology comparison between the event sequence and the pre-configured set of aseptic operation timing logic constraints, and determines the violation situation based on the rule of irreversible direction. This makes the evaluation result no longer a simple binary prompt for going out of bounds, but a structured record containing the violation node, violation direction, and violation depth. Overall, it improves the objectivity, quantification, and fine identification ability of irreversible direction violations in the evaluation of medical operation standardization, which facilitates post-event traceability and continuous quality improvement.

[0017] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an AI vision-based automatic assessment method for medical operation standardization, provided as an embodiment of this application; Figure 2 A schematic diagram of the overall hardware deployment structure of the clinical operating space provided in the embodiments of this application; Figure 3 A schematic diagram illustrating the spatial interference determination between the dynamic physical linkage model of a medical worker's limbs and the gripping force point of a surgical instrument, provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the process of constructing a three-dimensional geometric outer box and a three-dimensional sterile barrier envelope surface by stretching the edge contour of the surgical drape provided in this embodiment of the application. Figure 5 A schematic diagram illustrating the interaction trajectory sequence and spatial topology determination of the three-dimensional sterile barrier envelope surface, as well as the confirmation of the penetration and exit direction attributes, provided in an embodiment of this application. Figure 6 This is a schematic diagram illustrating the principle of comparing the event sequence with the aseptic operation timing logic constraint set graph structure topology in the embodiments of this application. Figure 7 This is a schematic diagram of the structure of an AI vision-based automatic assessment system for medical operation standardization, provided as an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] To facilitate understanding of the embodiments of this application, some non-publicly known terms involved in this application will be briefly explained first.

[0022] Clinical operating space refers to the physical three-dimensional space area for performing surgery, invasive procedures, or interventional diagnosis and treatment. In this application, it specifically refers to the physical volume area that is defined by the operating table, surgical drapes, and the range of movement accessible to medical staff, and which requires strict adherence to aseptic operating procedures.

[0023] Surgical instruments refer to tools that are held by medical staff and directly involved in surgical procedures during clinical operations, such as scalpels, hemostatic forceps, tweezers, scissors, and suture needle holders, which are physical objects with obvious metallic or alloy material appearance characteristics.

[0024] The physical body of medical staff refers to the visible arms, hands and their extended segments of medical staff participating in clinical operations, which are abstractly represented in the form of human skeletal joints in this application.

[0025] Surgical drapes are disposable or sterilizable fabrics that cover the operating table or the patient's body surface to delineate sterile and sterile areas. In this application, it is assumed that the surgical drapes have a specified color that is significantly different from the background environment in order to extract the edge contours through color space separation.

[0026] The three-dimensional sterile barrier envelope refers to a three-dimensional curved surface constructed in the clinical operating space based on the edge contour and deformation displacement value of the surgical drape, used to characterize the geometric boundary between sterile and sterile areas. In this application, it is instantiated in the form of a physical surface of a three-dimensional geometric outer box facing outward and upward.

[0027] Interactive trajectory sequence refers to the discrete representation of the three-dimensional spatial trajectory formed by the change of the gripping force point of a surgical instrument in the global coordinate system of the limb over time during the period when the instrument is held by medical staff. It is formed by sequentially splicing together local spatial offset vectors at multiple adjacent moments.

[0028] A physical transfer event refers to a discrete event unit corresponding to a single positional transfer of a surgical instrument entity or a medical staff limb entity between the regions on both sides of the three-dimensional sterile barrier envelope surface with an independent direction. In this application, a single physical action segment obtained after the dwell timeline is truncated at the critical turning point where the spatial depth sequence trend reverses and crosses the zero value is defined as a physical transfer event.

[0029] The set of aseptic operation timing logic constraints refers to a set of standard constraint pointing edges that are pre-established according to the clinical guidelines for aseptic operation and are used to constrain the order of occurrence between entity transfer events. Each constraint pointing edge represents a one-way isolation restriction from the starting trigger node to the ending target node. For example, "instruments that have been passed out of the aseptic area shall not be passed into the aseptic area again" corresponds to a one-way irreversible constraint pointing edge from the internal aseptic state node to the external sterile state node.

[0030] The irreversible direction rule refers to a type of logical constraint that is centrally limited by the timing logic constraints of aseptic operations, requiring that entity transfer events can only occur in one direction and not be allowed to flow back in the opposite direction. It is the core constraint in aseptic operations to prevent contaminated objects from re-contaminating the aseptic area.

[0031] Aseptic violation node records refer to structured data records output when the order of entity transfer events violates the rule of irreversible direction. They include fields such as the trigger timestamp of the violation event, the direction of the violation, the depth of the violation space, the type of associated entity, and the index of the edge pointing to the violation constraint, which facilitates post-event traceability and quality improvement.

[0032] Figure 1 This is a flowchart illustrating an AI vision-based automatic assessment method for the standardization of medical procedures, provided as an embodiment of this application. Figure 1 As shown in the figure, the AI ​​vision-based automatic assessment method for medical operation standardization provided in this application includes steps S100 to S600. The steps are described in detail below with reference to the accompanying drawings.

[0033] Step S100: Acquire a series of multiple frames of images of the clinical operating space and parse the series of multiple frames of images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities.

[0034] Figure 2 This is a schematic diagram of the overall hardware deployment structure of the clinical operating space provided in the embodiments of this application. Figure 2As shown, the overall hardware deployment structure of the clinical operation space includes an operating table 201, surgical drapes 202, a patient 203, medical staff 204, surgical instruments 205, a multi-view image acquisition device 206, an image preprocessing unit 207, an AI visual analysis unit 208, a spatial topology determination unit 209, an event decomposition unit 210, a constraint comparison unit 211, and a monitoring and warning terminal 212. The multi-view image acquisition device 206 consists of multiple RGB-D cameras arranged above the ceiling and on the side walls of the operating room at different locations. These cameras are used to synchronously acquire color and depth images of the clinical operating space at a preset sampling frequency (e.g., 30 frames / second to 60 frames / second, which can be set according to the rhythm of the surgery). The image preprocessing unit 207 is connected to the multi-view image acquisition device 206 and is used to perform distortion correction, white balance correction, and multi-view alignment on the acquired continuous multi-frame images. The AI ​​visual analysis unit 208 is connected to the image preprocessing unit 207 and internally deploys a target detection network for entity recognition and a key point detection network for human skeletal joint extraction. The spatial topology determination unit 209, the event decomposition unit 210, and the constraint comparison unit 211 are connected in series to form the core computing pipeline. The monitoring and warning terminal 212 is connected to the output of the constraint comparison unit 211 and is used to present aseptic violation records to the infection control specialist in real time.

[0035] Specifically, the process of obtaining the spatial coordinate sequence in this step includes: First, the apparent regions of human limbs and the apparent regions of surgical instruments are extracted from the consecutive multi-frame images to locate the initial spatial detection boxes for these regions. The apparent regions of human limbs refer to visually saliently connected regions in the image frame formed by human skin texture, surgical gown cuffs, and gloves; the apparent regions of surgical instruments refer to visually saliently connected regions in the image frame formed by the high reflectivity of metals or alloys. The extraction of the initial spatial detection boxes can be achieved, for example, using a target detection algorithm based on a convolutional neural network, such as the YOLO series detection network (e.g., YOLOv8) or the Faster R-CNN detection network, depending on computational resources and real-time requirements. The target detection algorithm takes consecutive multi-frame images as input and outputs the category label (human limb or surgical instrument), two-dimensional detection box coordinates, and confidence score for each detected target in each frame. Then, combined with the calibration parameters of the multi-view image acquisition device, the two-dimensional detection boxes are back-projected onto the three-dimensional coordinate system of the clinical operating space to obtain the three-dimensional initial spatial detection boxes.

[0036] Secondly, skeletal joints are extracted from the apparent area of ​​the human limb to construct a dynamic physical linkage model of the medical staff's limb entity, and geometric structural lines are extracted from the apparent area of ​​the instrument material to determine the gripping force point of the surgical instrument entity. The extraction of skeletal joints can be achieved, for example, using well-known human pose estimation networks such as OpenPose or HRNet, outputting the three-dimensional coordinates of key anatomical joints such as the shoulder joint, elbow joint, wrist joint, metacarpophalangeal joint, and fingertips; the dynamic physical linkage model refers to a graphical model constructed with the above-mentioned skeletal joints as nodes and the anatomical skeletal connections as edges, which can reflect the instantaneous posture of the medical staff's limb entity; the extraction of geometric structural lines can be achieved, for example, using a combination of Canny edge detection and Hough line transform, extracting the main axis centerline and handle outline of the instrument from the apparent area of ​​the instrument material; the gripping force point refers to the point with the maximum geometric curvature on the handle outline of the instrument, that is, the intersection of the handle and the main axis of the instrument, which conforms to the mechanical force center of human grip.

[0037] Next, spatial interference is determined between the end node of the dynamic physical linkage model and the gripping point. When the physical distance between the end node of the dynamic physical linkage model and the gripping point in three-dimensional space is less than a preset contact constraint threshold, a physical gripping state is established. The end node refers to the fingertip joint or metacarpophalangeal joint in the dynamic physical linkage model, representing the farthest point of the medical staff's limb entity that can perform a grasping action; the physical distance refers to the Euclidean distance between the end node and the gripping point in three-dimensional space; the preset contact constraint threshold can be set to, for example, 3 cm to 5 cm, specifically based on the image acquisition resolution and the size of the human hand. When the physical distance is less than this threshold, it is considered that the medical staff's hand has made physical contact with the instrument handle and formed a grip. It should be explained that the spatial interference determination is introduced to exclude the situation where the medical staff only approaches but does not actually grasp the instrument, avoiding false interaction records in the subsequent trajectory sequence. The specific spatial interference determination relationship can be combined with... Figure 3 illustrate.

[0038] Figure 3 This is a schematic diagram illustrating the spatial interference determination between the dynamic physical linkage model of a medical worker's limbs and the gripping force point of surgical instruments, provided in an embodiment of this application. Figure 3As shown, the dynamic physical linkage model of medical personnel's limbs consists of shoulder joint node 301, elbow joint node 302, wrist joint node 303, metacarpophalangeal joint node 304, and fingertip node 305 connected sequentially. The nodes are connected by solid lines to represent skeletal linkage constraints. Surgical instruments are represented by the instrument's main axis centerline 306 and the instrument handle outline 307. The gripping force point 308 is located at the point of maximum geometric curvature on the instrument handle outline. The spatial interference determination region 309 is a sphere with the gripping force point 308 as its center and a preset contact constraint threshold as its radius. When the fingertip node 305 or the metacarpophalangeal joint node 304 falls into the spatial interference determination region 309, a physical gripping state is determined.

[0039] Next, according to the chronological order of the image frames, the absolute spatial coordinates of the gripping force point during the period of maintaining the entity's gripping state are extracted sequentially, generating a time-distributed sequence of coordinate nodes. The absolute spatial coordinates refer to the three-dimensional position coordinates of the gripping force point in the global coordinate system of the clinical operating space; the coordinate node sequence can be expressed, for example, as follows: ,in, For at any time Below, the absolute spatial coordinates (unit: millimeter or centimeter) of the gripping force point in the global three-dimensional coordinate system of the clinical operation space represent the spatial position of the entity; This is a time variable (unit: seconds), representing the absolute physical time when an image frame occurs; The start timestamp (in seconds) indicates the moment when the entity's holding state is first established or the event sequence begins; The image sampling time interval can be set to, for example, 33 milliseconds (corresponding to a sampling frequency of 30 frames per second).

[0040] Finally, the absolute spatial coordinates of adjacent time points in the coordinate node sequence are physically connected in three dimensions to construct local spatial offset vectors. These local spatial offset vectors are then sequentially concatenated to generate an interactive trajectory sequence. The local spatial offset vector refers to the directed line segment between two adjacent time point coordinate nodes; its direction reflects the instantaneous movement direction of the gripping force point, and its length reflects the instantaneous displacement amplitude. The interactive trajectory sequence is a polygonal line formed by connecting all local spatial offset vectors sequentially, serving as input for subsequent spatial topology determination with the three-dimensional sterile barrier envelope. The reason for using adjacent time point connections instead of spline smoothing to construct the trajectory is that subsequent crossing determination focuses on whether a cross-boundary event actually occurs at discrete time points; excessive smoothing would mask the instantaneous crossing details.

[0041] Step S200: Generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence.

[0042] Step S100 has already obtained the coordinate node sequence of the gripping force point and the interaction trajectory sequence formed by splicing them during the maintenance of the entity's gripping state. This step is a process of clarifying and organizing the results of the transformation from the coordinate node sequence to the interaction trajectory sequence in step S100. Specifically, this step attributes the motion trajectory represented by the gripping force point to both the surgical instrument entity and the medical staff's limb entity, because during the maintenance of the entity's gripping state, the two move synchronously in physical motion and share the same force point, so their interaction trajectory sequence is treated as a unified spatial motion representation object. After the gripping state is released, for example, when the medical staff puts the instrument back on the instrument table, the interaction trajectory sequence terminates at that moment, and a new interaction trajectory sequence is started when the next gripping begins. This step provides a trajectory input that can be processed consistently for subsequent spatial topology determination with the three-dimensional sterile barrier envelope surface. The reason for treating the instrument-limb interaction trajectory sequence as a unified object is that in clinical aseptic operation scenarios, the gripped instrument is regarded as an extension of the medical staff's hand, and violations by both should be evaluated together.

[0043] Step S300: Analyze the continuous multi-frame images to obtain the edge contour sequence of the surgical drape, and construct a three-dimensional sterile barrier envelope in the clinical operating space based on the edge contour sequence of the surgical drape.

[0044] Since the interaction trajectory sequences obtained in steps S100 and S200 are only motion trajectories in a spatial geometric sense, a clear geometric reference surface for the sterile barrier must be established before it can be further determined whether the trajectory has crossed the boundary. This step is used to construct this geometric reference surface, which will be discussed below. Figure 4 Please provide a detailed explanation.

[0045] Figure 4 This is a schematic diagram illustrating the process of constructing a three-dimensional geometric outer box and a three-dimensional sterile barrier envelope by stretching the edge contour of the surgical drape provided in this embodiment of the application. Figure 4 As shown, the visual boundary points 401 of the surgical drape are distributed on the actual physical edge of the surgical drape. Connecting multiple visual boundary points 401 end to end forms a closed planar polygon 402. Using the closed planar polygon 402 as a two-dimensional reference base, the deformation displacement value 403 along the vertical spatial axis is extracted by combining visual depth information. The closed planar polygon 402 is stretched along the vertical spatial axis to the height corresponding to the deformation displacement value 403 to obtain a three-dimensional geometric outer box 404. The physical surface of the three-dimensional geometric outer box 404 facing the specified external upper surface is defined as a three-dimensional sterile barrier envelope surface 405.

[0046] Specifically, the process of constructing the three-dimensional sterile barrier envelope in this step includes: First, visual boundary points separating the specified drape color space from the background environment color space are extracted from the consecutive multi-frame images. The specified drape color space refers to the hue-saturation range occupied by the surgical drape itself in the HSV color space. For example, the hue range corresponding to the commonly used blue-green drape can be set to 150 to 200 degrees, and the saturation range can be set to 0.4 to 0.9, depending on the actual color value of the drape model used. The background environment color space refers to the color range occupied by other visual objects besides the drape (such as surgical gowns, skin, metal instruments, and stainless steel instrument tables). The visual boundary points are the pixel locations where the drape color pixels switch with the non-drape color pixels, which can be extracted, for example, using the Sobel operator combined with threshold segmentation.

[0047] Secondly, multiple visual boundary points are connected end-to-end to form a closed planar polygon representing the surgical drape. Specifically, all visual boundary points are sorted according to their polar angles in the image plane, and redundant points are removed using the Douglas-Peucker polyline simplification algorithm. The remaining points are then connected end-to-end to obtain a closed planar polygon representing the projected edge of the surgical drape. It should be noted that the closed planar polygon is the same as the "closed planar polygon" mentioned in claim 3; both represent the same geometric object, and the term "closed planar polygon" will continue to be used below.

[0048] Next, using the closed planar polygon as a two-dimensional reference base, and combining the visual depth information from the consecutive multi-frame images, the deformation displacement value along the vertical spatial axis of the actual physical draped surface mapped inside the two-dimensional reference base is extracted. The visual depth information comes from the depth channel of the RGB-D camera, and its principle is to obtain the depth value of the physical point corresponding to each pixel through structured light or time-of-flight ranging technology. The deformation displacement value refers to the height difference along the vertical spatial axis of the actual physical draped surface corresponding to all pixels inside the closed planar polygon relative to the horizontal reference plane represented by that plane. For example, it can be obtained by subtracting the average depth of the four corners from the average depth within the region, or by directly using the maximum bulge height within the region as the deformation displacement value, depending on the patient's body shape and the tightness of the draped. The vertical spatial axis refers to the spatial direction perpendicular to the horizontal ground, denoted as the z-axis.

[0049] Next, the closed planar polygon is physically stretched along the vertical spatial axis using the deformation displacement value to generate a solid geometric outer box. Specifically, the closed planar polygon is stretched upwards along the vertical spatial axis with its in-situ bottom surface as the height corresponding to the deformation displacement value, resulting in a prism-shaped solid with both its top and bottom surfaces being closed planar polygons and its sides being quadrilateral bands, denoted as the solid geometric outer box. The solid geometric outer box is physically equivalent to a three-dimensional box-shaped space that encloses the sterile operating area covered above the surgical drape.

[0050] Finally, the physical surface of the three-dimensional geometric outer casing facing the designated external upper surface is defined as the three-dimensional sterile barrier envelope surface. The designated external upper surface refers to the location of the top surface of the three-dimensional geometric outer casing facing the ceiling; this physical surface is the geometric interface through which medical personnel need to enter or withdraw from when holding instruments. Only the top surface is defined as the three-dimensional sterile barrier envelope surface because the bottom surface fits against the patient's body surface, and the sides are usually covered by drapes in clinical practice; only the top surface is the area that medical personnel actually cross.

[0051] Furthermore, considering that surgical drapes may dynamically deform during surgery due to instrument placement, touching by medical staff, etc., this application further provides a dynamic adaptation process for the three-dimensional sterile barrier envelope. Specifically, after generating the three-dimensional sterile barrier envelope, the local geometric changes of the closed planar polygon in the newly added image frame are tracked in real time; the local deformation boundary segment on the closed planar polygon whose actual physical coordinate offset distance exceeds the threshold constraint parameter is located. The threshold constraint parameter can be set to 2 cm to 3 cm, for example, and can be set according to the softness of the surgical drape material; the latest deformation displacement value corresponding to the local deformation boundary segment at the current moment is re-extracted; the latest deformation displacement value is used to re-physically stretch the affected local boundary of the three-dimensional geometric outer box, and the three-dimensional external contour of the three-dimensional geometric outer box is locally adjusted; the corresponding physical surface of the adjusted three-dimensional geometric outer box is used to replace the original three-dimensional sterile barrier envelope area. This dynamic adaptation process updates only the localized areas where deformation occurs, avoiding the waste of computational resources caused by global reconstruction, while ensuring real-time synchronization between the three-dimensional sterile barrier envelope and the actual surgical drape geometry.

[0052] Step S400: Perform spatial topology determination on the interaction trajectory sequence and the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence through the sterile barrier envelope surface.

[0053] Steps S200 and S300 yielded the interactive trajectory sequence and the three-dimensional sterile barrier envelope, respectively. This step requires spatial topological intersection of the two to determine whether and how the trajectory traverses the barrier. The following section will combine... Figure 5 Please provide a detailed explanation.

[0054] Figure 5 This diagram illustrates the interaction trajectory sequence and the spatial topology determination and penetration / exit direction attribute confirmation of the three-dimensional sterile barrier envelope surface provided in the embodiments of this application. Figure 5 As shown, the three-dimensional sterile barrier envelope surface 501 is located in the center of the clinical operating space, and several normal vectors 502 attached to its upper surface protrude towards the external space; the interactive trajectory sequence 503 crosses the barrier from the external space in the form of a broken line and returns again. The interactive trajectory sequence 503 intersects the three-dimensional sterile barrier envelope surface 501 at the crossing intersection point 504 and the crossing intersection point 505; the angle between the motion velocity vector 506 at the crossing intersection point 504 and the direction of the normal vector 502 is θ1, and the angle between the motion velocity vector 507 at the crossing intersection point 505 and the direction of the normal vector 502 is θ2; when the angle θ is greater than 90 degrees, it is determined to be the penetration direction attribute, and when the angle θ is less than 90 degrees, it is determined to be the exit direction attribute; the dwelling position point 508 is located in the internal space of the barrier, and the physical perpendicular distance 509 extending in the opposite direction along the normal vector 502 to the surface of the three-dimensional sterile barrier envelope surface 501 is the spatial depth value corresponding to the dwelling position point.

[0055] Specifically, the process of solving the penetration-out direction attributes and spatial depth sequence in this step includes: First, extract the normal vector protruding towards the external space along the surface of the three-dimensional sterile barrier envelope. The normal vector is a unit vector that is perpendicular to the local tangent plane of the three-dimensional sterile barrier envelope at that point and points upwards into the external space; specifically, it can be obtained by triangulating the three-dimensional sterile barrier envelope, calculating the normal vector of each mesh triangle according to the right-hand rule, and normalizing it to a unit vector.

[0056] Next, the intersection points where the interactive trajectory sequence physically intersects with the envelope surface of the three-dimensional sterile barrier are extracted, and the velocity vector of the interactive trajectory sequence at these intersection points is calculated. The intersection point refers to a discrete spatial point where the polyline segment of the interactive trajectory sequence intersects with the geometric surface represented by the envelope surface of the three-dimensional sterile barrier. For example, the Möller–Trumbore ray-triangle intersection algorithm can be used to solve this problem, intersecting each polyline segment on the surface to obtain the three-dimensional coordinates of all intersection points. The velocity vector refers to the instantaneous direction vector of the interactive trajectory sequence at the intersection point, which can be obtained by dividing the direction vector of the polyline segment containing the intersection point by the corresponding time interval.

[0057] Next, the angle between the motion velocity vector and the normal vector is compared. Based on the comparison result, the entry and exit direction attributes of the interaction trajectory sequence at the crossing intersection are confirmed. The entry and exit direction attributes include both entry and exit direction attributes. Specifically, the angle between the motion velocity vector and the normal vector is calculated using the dot product formula, for example, it can be expressed as... ,in, The velocity vector (in meters per second) of the interaction trajectory sequence at the intersection point represents the direction and speed of the entity's movement at the instant it crosses the sterile barrier. Let be the normal vector (unit: dimensionless direction vector) of the envelope surface of the three-dimensional sterile barrier at the intersection point, with its direction perpendicular to the tangent plane of the envelope surface and pointing towards the external space. The magnitude of the velocity vector (unit: meters per second) represents the instantaneous velocity of the entity. The magnitude of the normal vector (dimensionless, value 1) is given, because the normal vector is usually a unit vector. when When <0, that is, the included angle When the angle is greater than 90 degrees, the components of the motion velocity vector are in the opposite direction to the normal vector, indicating that the entity is moving from the outer space to the inner space, and this is determined to be a penetration direction attribute; when When the angle is greater than 0, that is, the included angle is 0. When the angle is less than 90 degrees, the components of the motion velocity vector are in the same direction as the normal vector, indicating that the entity is moving from the internal space to the external space, and is determined to be a passing-out direction attribute.

[0058] Next, after determining the penetration and exit direction attributes, the dwelling points of the interaction trajectory sequence when it enters the internal space of the three-dimensional sterile barrier envelope are extracted. The dwelling points refer to all trajectory sampling points located on one side of the internal space of the three-dimensional sterile barrier envelope after the intersection point determined to be the penetration direction attribute and before the intersection point determined to be the exit direction attribute.

[0059] Finally, the physical perpendicular distance from the dwelling position point to the surface of the three-dimensional sterile barrier envelope is calculated by extending the normal vector backward along the dwelling position point, thus forming a spatial depth sequence. The physical perpendicular distance refers to the shortest Euclidean distance from the dwelling position point to the three-dimensional sterile barrier envelope surface, which can be obtained by extending the normal direction along the surface position corresponding to that point backward. Arranging the physical perpendicular distances corresponding to each dwelling position point in temporal order yields the spatial depth sequence. The spatial depth sequence physically reflects the instantaneous "depth" of an entity within the sterile barrier's interior space.

[0060] Step S500: Determine the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope surface based on the penetration and exit direction attributes and the spatial depth sequence. Decompose the residence timeline into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo position transfer in the regions on both sides of the sterile barrier envelope surface.

[0061] The penetration and exit direction attributes and spatial depth sequence obtained in step S400 are only instantaneous judgment results at discrete moments. They cannot directly depict the movement pattern of the entity on both sides of the barrier during the entire surgical process. It is necessary to further integrate the instantaneous judgment results into continuous dwell information in the time dimension, and then break them down into discrete events that can be compared by rules.

[0062] Specifically, the process of generating the event sequence in this step includes: First, upon acquiring the intrusion / exit direction attribute, a spatial intrusion start timestamp is recorded; upon acquiring the exit direction attribute, a spatial withdrawal end timestamp is recorded. The spatial intrusion start timestamp refers to the absolute physical time value corresponding to the instant when the intrusion direction attribute is first determined; the spatial withdrawal end timestamp refers to the absolute physical time value corresponding to the instant when the exit direction attribute is first determined; both are synchronized with the global acquisition time of the corresponding image frame.

[0063] Secondly, the spatial intrusion start timestamp, the spatial withdrawal end timestamp, and the spatial coordinate sequence between the spatial intrusion start timestamp and the spatial withdrawal end timestamp are bounded together to form a continuous dwelling segment. The continuous dwelling segment refers to the continuous spatiotemporal segment experienced by an entity from its complete entry into the barrier's interior space to its re-exit from the barrier, which is composed of the start and end timestamps and all spatial coordinates of the intermediate process. The reason for using boundary binding instead of simple time segmentation is to ensure that each dwelling segment physically corresponds to a complete entry and exit action.

[0064] Next, the continuous residence segments are spliced ​​together end-to-end according to the absolute physical timeline of their occurrence to construct a residence timeline. The residence timeline refers to the entire time sequence spliced ​​together by all continuous residence segments and the blank intervals between segments (the time periods when the entity is located in the space outside the barrier) in absolute physical time order, covering the entire time interval of the entire surgical procedure from start to finish.

[0065] Next, the key inflection points where the spatial depth sequence experiences a trend reversal and crosses zero are extracted from the dwell timeline. The trend reversal refers to the moment when the first derivative of the spatial depth sequence changes sign, i.e., from increasing to decreasing or from decreasing to increasing; crossing zero refers to the spatial depth sequence changing from a positive value to zero or from zero to a positive value. The moment that simultaneously satisfies both trend reversal and crossing zero is the key inflection point, physically corresponding to the instant when the entity changes its direction of movement and just passes through the barrier surface. It should be noted that the purpose of extracting key inflection points is to divide a continuous motion into several uniformly oriented segments according to changes in physical direction, ensuring that each segment corresponds to a single simple entry or exit action.

[0066] Next, at the key turning points, the dwell timeline is physically truncated to separate single physical action segments with independent directions, and each single physical action segment is defined as an entity transfer event. A single physical action segment refers to the movement period between two adjacent key turning points, during which the entity's movement direction remains unidirectional (either continuously entering or continuously exiting). The entity transfer event includes start and end timestamps, direction attributes, maximum spatial depth, associated entity type (surgical instruments or medical personnel limbs), and the corresponding spatial coordinate sequence.

[0067] Finally, all entity transfer events are integrated to obtain an event sequence. This event sequence is a set of entity transfer events arranged in chronological order, which serves as the input for subsequent graph structure topology comparison with the aseptic operation temporal logic constraint set.

[0068] Step S600: Perform a graph structure topology comparison between the event sequence and the pre-configured aseptic operation timing logic constraint set. When the occurrence order of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the aseptic operation timing logic constraint set, generate an aseptic violation node record.

[0069] The event sequence obtained in step S500 is only a structured record of actions. It needs to be compared with pre-configured aseptic operation rules to identify which events constitute violations. The following section will combine... Figure 6 Please provide a detailed explanation.

[0070] Figure 6 This is a schematic diagram illustrating the principle of comparing the event sequence with the aseptic operation timing logic constraint set graph structure topology provided in the embodiments of this application. Figure 6As shown, the event sequence side consists of entity transfer events 601, 602, and 603 arranged in chronological order. Nodes are connected by actual occurrence pointing edges 604 and 605, forming an action directed graph. The aseptic operation timing logic constraint set side includes an external sterile state node 606, an internal sterile state node 607, and a standard constraint pointing edge 608 connecting the two. The direction of the standard constraint pointing edge 608 points from the internal sterile state node 607 to the external sterile state node 606, representing the irreversible rule of "cannot return after being withdrawn from the sterile area". The actual occurrence pointing edge in the action directed graph is searched in the standard constraint pointing edge set for mapping edges with the same action start trigger node. The directional pointing relationship between the two is compared. When there is a directional conflict, i.e., the directional irreversible rule is triggered, a sterile violation node record 609 is generated.

[0071] Specifically, the process of generating the sterile violation node record in this step includes: First, the sequential relationship between the actions of the entity transfer events in the event sequence is extracted to construct an action directed graph. The action directed graph is a graph structure with entity transfer events as nodes and the temporal connection between adjacent entity transfer events as directed edges. Each directed edge points from the end state of the previous event to the start state of the next event. The nodes of the action directed graph have directional attributes (entering or exiting) and associated entity type attributes, while the directed edges have time interval attributes.

[0072] Secondly, a pre-established set of standard constraint pointing edges is invoked from the set of aseptic operation timing logic constraints. The set of aseptic operation timing logic constraints is a rule base pre-established according to national and industry aseptic operation standards, consisting of several standard constraint pointing edges. Each standard constraint pointing edge points from a starting trigger node (e.g., the "internal aseptic state" node) to an ending target node (e.g., the "external sterile state" node) and is accompanied by a direction irreversibility flag. This rule base can be reviewed and confirmed by infection control experts during system deployment and supports subsequent expansion and updates based on new clinical standards.

[0073] Next, traverse the directed graph of actions along the time-increasing direction and extract the actual occurrence pointing edge connecting any two adjacent entity transfer events. The actual occurrence pointing edge refers to the directed edge in the directed graph of actions from the entity transfer event with a earlier time to the entity transfer event with a later time, and its direction reflects the actual transfer direction of the entity between the regions on both sides of the barrier.

[0074] Next, the actual pointing edge is used to retrieve a mapped edge with the same starting trigger node from the set of standard constraint pointing edges. The mapped edge refers to a standard constraint pointing edge in the set of standard constraint pointing edges whose starting trigger node state attribute is consistent with the starting node state attribute of the actual pointing edge. The retrieval process can be implemented, for example, using a state key-value lookup based on a hash table to ensure real-time performance.

[0075] Finally, when the actual occurrence of the pointing edge indicating an entity returning from an external sterile state node to an internal sterile state node, violating the unidirectional isolation constraint defined by the mapping edge, the directional irreversibility rule is triggered, and an operation blocking signal is output to generate a sterile violation node record. The unidirectional isolation constraint refers to the specific content of the directional irreversibility rule represented by the mapping edge, that is, only unidirectional transfer of entities from the internal sterile state node to the external sterile state node is allowed, and reverse transfer is considered a violation; the operation blocking signal is presented in real time through the monitoring and warning terminal, triggering an audible and visual warning and prompting on-site medical staff to suspend the current operation; the sterile violation node record contains structured fields such as the trigger timestamp of the violation event, the violation direction, the violation spatial depth, the associated entity type, and the index of the violation constraint pointing edge, which are persistently stored in the violation record database of the evaluation system for easy retrospective and quality improvement.

[0076] In summary, the embodiments of this application achieve objectivity, quantification, and traceability of the standardization assessment of medical operations by performing AI visual analysis, spatial topology determination, event decomposition, and graph structure topology comparison on multiple consecutive frames of images of the clinical operation space, without relying on human visual supervision.

[0077] The following is combined Figure 7 The AI ​​vision-based automatic evaluation system for medical operation standardization provided in the embodiments of this application will be described. Figure 7 This is a schematic diagram of the structure of an AI vision-based automated assessment system for medical operation standardization, provided as an embodiment of this application. Figure 7 As shown, the AI ​​vision-based automatic evaluation system for medical operation standardization 700 includes an acquisition module 701, a generation module 702, a construction module 703, a judgment module 704, a disassembly module 705, and a comparison module 706.

[0078] The acquisition module 701 is used to acquire multiple consecutive frames of images of the clinical operating space and parse the multiple consecutive frames of images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities. The acquisition module 701 is communicatively connected to the multi-view image acquisition device, and its specific implementation is the same as the aforementioned method step S100, which will not be repeated here.

[0079] The generation module 702 is used to generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence. The input end of the generation module 702 is connected to the output end of the acquisition module 701, and its specific implementation is the same as the aforementioned method step S200.

[0080] The construction module 703 is used to parse the continuous multi-frame images to obtain the edge contour sequence of the surgical drape, and to construct a three-dimensional sterile barrier envelope surface in the clinical operating space based on the edge contour sequence of the surgical drape. The input end of the construction module 703 is connected to the output end of the acquisition module 701, and its specific implementation is the same as the aforementioned method step S300.

[0081] The determination module 704 is used to perform spatial topology determination on the interaction trajectory sequence and the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence through the sterile barrier envelope surface. The input terminal of the determination module 704 is connected to the output terminal of the generation module 702 and the construction module 703 respectively, and its specific implementation is the same as the aforementioned method step S400.

[0082] The disassembly module 705 is used to determine the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope surface based on the penetration and exit direction attributes and the spatial depth sequence. The residence timeline is then disassembled into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo positional transfer in the regions on both sides of the sterile barrier envelope surface. The input end of the disassembly module 705 is connected to the output end of the determination module 704, and its specific implementation is the same as the aforementioned method step S500.

[0083] The comparison module 706 is used to perform a graph structure topological comparison between the event sequence and a pre-configured set of aseptic operation timing logic constraints. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated. The input of the comparison module 706 is connected to the output of the disassembly module 705, and the output of the comparison module 706 is connected to the monitoring and warning terminal. Its specific implementation is the same as the aforementioned method step S600.

[0084] This application also provides a computing device, including a processing component and a storage component. The processing component may be implemented using a general-purpose microprocessor, digital signal processor, or field-programmable gate array, for example; the storage component may be implemented using non-volatile memory, for example. The storage component stores one or more computer instructions, which are invoked and executed by the processing component to implement the AI ​​vision-based automatic assessment method for medical operation standardization described in any of the above embodiments of this application.

[0085] This application also provides a computer storage medium storing a computer program. When executed by a computer, the computer program implements the AI ​​vision-based automatic assessment method for medical operation standardization described in any of the above embodiments of this application. The computer storage medium can be any physical medium capable of storing a computer program, such as a read-only memory, random access memory, disk, or optical disk.

[0086] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An AI vision-based automatic evaluation method for medical operation standardization, characterized in that, include: Acquire consecutive multi-frame images of the clinical operating space and parse the consecutive multi-frame images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities; Generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence; The edge contour sequence of the surgical drape is obtained by parsing the continuous multi-frame images, and a three-dimensional sterile barrier envelope is constructed in the clinical operating space based on the edge contour sequence of the surgical drape. The interaction trajectory sequence is compared with the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence through the sterile barrier envelope surface. Based on the penetration and exit direction attributes and the spatial depth sequence, the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope is determined. The residence timeline is then decomposed into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo position transfer in the regions on both sides of the sterile barrier envelope. The event sequence is compared with a pre-configured set of aseptic operation timing logic constraints using a graph structure topology. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated.

2. The method of claim 1, wherein, Acquire a series of consecutive frames of images of the clinical operating space, and parse the consecutive frames to obtain a spatial coordinate sequence of surgical instruments and medical personnel limbs, including: The apparent regions of human limbs and the apparent regions of instrument materials are extracted from the consecutive multi-frame images to locate the initial spatial detection boxes of the apparent regions of human limbs and the apparent regions of instrument materials. Bone joints are extracted within the apparent area of ​​the human limb to construct a dynamic physical linkage model of the medical staff's limb entity, and geometric structural lines are extracted within the apparent area of ​​the instrument material to determine the gripping force points of the surgical instrument entity. Spatial interference determination is performed between the end node of the dynamic physical link model and the gripping force point. When the physical distance between the end node of the dynamic physical link model and the gripping force point in three-dimensional space is less than the preset contact constraint threshold, it is determined that a physical gripping state is established. Based on the chronological order of the image frames, the absolute spatial coordinates of the gripping force points during the period of maintaining the entity's gripping state are extracted sequentially to generate a time-distributed sequence of coordinate nodes. The absolute spatial coordinates of adjacent moments in the coordinate node sequence are physically connected in three dimensions to construct a local spatial offset vector, and the local spatial offset vectors are sequentially spliced ​​together to generate an interactive trajectory sequence.

3. The method of claim 1, wherein, The edge contour sequence of the surgical drape is obtained by parsing the continuous multi-frame images. Based on the edge contour sequence of the surgical drape, a three-dimensional sterile barrier envelope is constructed in the clinical operating space, including: Extract the visual boundary points that separate the specified fabric color space from the background environment color space from the consecutive multi-frame images; Connect the multiple visual boundary points end to end to form a closed planar polygon representing the surgical drape; Using the closed planar polygon as a two-dimensional reference base, and combining the visual depth information in the consecutive multi-frame images, the deformation displacement value along the vertical spatial axis of the actual physical towel surface mapped inside the two-dimensional reference base is extracted. The closed planar polygon is physically stretched along the vertical spatial axis using the deformation displacement value to generate a three-dimensional geometric outer box; The physical surface of the three-dimensional geometric outer box facing the designated external surface is defined as the three-dimensional sterile barrier envelope.

4. The method of claim 1, wherein, The interaction trajectory sequence is compared with the three-dimensional sterile barrier envelope surface using spatial topology determination to determine the penetration direction attributes and spatial depth sequence of the interaction trajectory sequence through the sterile barrier envelope surface, including: Extract the normal vector protruding toward the external space along the surface of the three-dimensional sterile barrier envelope; Extract the intersection points where the interaction trajectory sequence physically intersects with the envelope surface of the three-dimensional sterile barrier, and calculate the velocity vector of the interaction trajectory sequence at the intersection points; The direction angle between the motion velocity vector and the normal vector is compared, and the entry and exit direction attributes of the interaction trajectory sequence at the crossing intersection are confirmed according to the comparison result. The entry and exit direction attributes include entry direction attributes and exit direction attributes. After determining the penetration and exit direction attributes, extract the dwell position points of the interaction trajectory sequence when it enters the internal space of the three-dimensional sterile barrier envelope surface; Calculate the physical perpendicular distance from the dwelling location point to the surface of the three-dimensional sterile barrier envelope along the normal vector in the opposite direction to form a spatial depth sequence.

5. The method of claim 1, wherein, Based on the penetration and exit direction attributes and the spatial depth sequence, the residence timelines of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope are determined. These residence timelines are then decomposed into an event sequence composed of multiple entity transfer events, according to the order in which the surgical instrument entity and the medical staff limb entity undergo positional transfers in the regions on both sides of the sterile barrier envelope, including: When the penetration and exit direction attributes are obtained, the space intrusion start timestamp is recorded, and when the exit direction attributes are obtained, the space withdrawal end timestamp is recorded. The spatial intrusion start time stamp, the spatial withdrawal end time stamp, and the spatial coordinate sequence between the spatial intrusion start time stamp and the spatial withdrawal end time stamp are bound together to form a continuous dwelling segment. The continuous dwelling segments are spliced ​​together end to end according to the absolute physical timeline of occurrence to construct a dwelling timeline; Extract the key turning point where the spatial depth sequence reverses its trend and crosses zero on the dwell timeline; At the critical turning point, the dwell timeline is physically truncated to separate single physical action segments with independent directions, and each single physical action segment is defined as an entity transfer event. Integrate all entity transfer events to obtain an event sequence.

6. The method of claim 1, wherein, The event sequence is compared with a pre-configured set of aseptic operation timing logic constraints using a graph structure topology. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated, including: Extract the sequential relationship between the actions of the entity transfer events in the event sequence to construct an action directed graph; Call the pre-established set of standard constraint pointer edges from the set of aseptic operation timing logic constraints; Traverse the directed graph of the action along the time-increasing direction and extract the actual occurrence pointing edge connecting any two adjacent entity transition events; The actual occurrence of the pointing edge is used to retrieve the mapped edge with the same starting trigger node from the standard constraint pointing edge set; When the actual occurrence of the pointing edge indicates that the entity returns from the external sterile state node to the internal sterile state node, and violates the one-way isolation restriction defined by the mapping edge, it is determined that the directional irreversibility rule is triggered, and an operation blocking signal is output to generate a sterile violation node record.

7. The AI vision-based medical operation standardization automatic evaluation method according to claim 3, characterized in that, After constructing the three-dimensional sterile barrier envelope, the process also includes a dynamic adaptation process for the three-dimensional sterile barrier envelope: After generating the three-dimensional sterile barrier envelope, the local geometric changes of the closed planar polygon in the newly added image frame are tracked in real time. Locate the local deformation boundary segment on the closed planar polygon whose actual physical coordinate offset distance exceeds the threshold constraint parameter; Re-extract the latest deformation displacement value corresponding to the local deformation boundary segment at the current moment; The latest deformation displacement value is used to re-physically stretch the affected local boundary of the solid geometric outer box, and locally adjust the three-dimensional external contour of the solid geometric outer box. The original three-dimensional sterile barrier envelope area is replaced with the corresponding physical surface of the adjusted three-dimensional geometric outer box.

8. An AI vision-based automatic evaluation system for medical operation standardization, characterized in that, include: The acquisition module is used to acquire multiple consecutive frames of images of the clinical operating space and parse the multiple consecutive frames of images to obtain the spatial coordinate sequence of surgical instrument entities and medical staff limb entities. The generation module is used to generate an interaction trajectory sequence between the surgical instrument entity and the medical staff limb entity based on the spatial coordinate sequence. The construction module is used to parse the continuous multi-frame images to obtain the edge contour sequence of the surgical drape, and to construct a three-dimensional sterile barrier envelope in the clinical operating space based on the edge contour sequence of the surgical drape; The determination module is used to perform spatial topology determination on the interaction trajectory sequence and the three-dimensional sterile barrier envelope surface to determine the penetration direction attribute and spatial depth sequence of the interaction trajectory sequence penetrating the sterile barrier envelope surface. The disassembly module is used to determine the residence timeline of the surgical instrument entity and the medical staff limb entity in the regions on both sides of the three-dimensional sterile barrier envelope surface based on the penetration and exit direction attributes and the spatial depth sequence. The residence timeline is then disassembled into an event sequence composed of multiple entity transfer events according to the order in which the surgical instrument entity and the medical staff limb entity undergo position transfer in the regions on both sides of the sterile barrier envelope surface. The comparison module is used to perform graph structure topology comparison between the event sequence and a pre-configured set of aseptic operation timing logic constraints. When the order of occurrence of multiple entity transfer events in the event sequence violates the directional irreversibility rule defined in the set of aseptic operation timing logic constraints, an aseptic violation node record is generated.

9. A computing device, comprising: It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement an AI vision-based automatic assessment method for medical operation standardization as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements an AI vision-based automatic assessment method for medical operation standardization as described in any one of claims 1 to 7.

Images