An image recognition method and system for execution state of operating room aseptic operation specification

Abstract

The application discloses a kind of operating room aseptic operation standard execution state image recognition method and system, it is related to image processing technical field, including obtaining the video image data of washing hand area, and hand region recognition is generated hand region recognition data;Right thumb wrap parameter and left thumb wrap parameter are determined based on hand region recognition data, and get heterolateral contact matrix;Based on heterolateral contact matrix, left thumb effective kneading frame and right thumb effective kneading frame are obtained by continuous kneading determination, frame time mark is obtained and continuous segment analysis is carried out, left thumb kneading parameter and right thumb kneading parameter are generated;Based on left thumb kneading parameter and right thumb kneading parameter, compliance evaluation analysis is carried out, and standard execution state recognition value is obtained, so as to obtain corresponding standard determination result.The application can improve the identification accuracy of thumb kneading step.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an image recognition method and system for the execution status of aseptic operating procedures in an operating room. Background Technology

[0002] Operating room aseptic techniques are crucial for reducing the risk of surgical infection. Preoperative surgical handwashing typically requires following a prescribed sequence, site, and duration. With the application of video capture equipment and image recognition technology in medical management, current technologies can capture images of the handwashing process using cameras and automatically identify handwashing actions using human posture recognition, hand detection, or motion classification models. This helps determine whether the operator has followed the preset handwashing procedures.

[0003] However, during preoperative surgical handwashing, the thumb-rubbing motion in the seven-step handwashing technique typically occurs when both hands are close together or overlapping, resulting in a small visible area of ​​the thumb. Furthermore, different operators exhibit variations in hand shape, rubbing amplitude, and rhythm. In this context, existing image recognition methods based on overall hand posture or movement classification easily confuse ordinary hand-rubbing with the thumb-rubbing motion, making it difficult to reliably distinguish whether a single thumb is sufficiently rubbed, and to accurately determine whether the corresponding action meets the effective continuity requirement. This affects the accuracy of identifying the execution status of aseptic procedures in the operating room. Therefore, the thumb-rubbing motion is obscured by overlapping hands, has a small local amplitude, and is affected by interference between the left and right hand movements, making it difficult to accurately identify the effective contact state, continuous execution state, and bilateral completion state in the thumb-rubbing step, thus hindering the accurate determination of the execution status of aseptic procedures in the operating room. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an image recognition method and system for the execution status of aseptic operation procedures in the operating room.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, this invention discloses an image recognition method for the execution status of aseptic operation procedures in an operating room, comprising the following steps: acquiring video image data of a handwashing area and performing hand region recognition to generate hand region recognition data, the hand region recognition data including the left palm region, right palm region, left thumb region, right thumb region, key points of the left wrist, and key points of the right wrist; based on the hand region recognition data, determining the right thumb wrapping parameter between the left palm region and the right thumb region, and the left thumb wrapping parameter between the right palm region and the left thumb region, and combining the right thumb wrapping parameter and the left thumb wrapping parameter to obtain an opposite contact matrix; based on the opposite contact matrix, performing continuous rubbing judgment to obtain effective rubbing frames of the left and right thumbs, obtaining frame time identifiers, and performing continuous segment analysis on the effective rubbing frames of the left and right thumbs respectively to generate left thumb rubbing parameters and right thumb rubbing parameters respectively; performing compliance evaluation analysis based on the left thumb rubbing parameters and the right thumb rubbing parameters to obtain the standard execution status recognition value, thereby obtaining the corresponding standard judgment result.

[0007] Secondly, this invention discloses an image recognition system for the execution of aseptic operating procedures in an operating room, comprising the following modules: an image recognition module, used to acquire video image data of a handwashing area and perform hand region recognition to generate hand region recognition data, the hand region recognition data including the left palm region, right palm region, left thumb region, right thumb region, key points of the left wrist, and key points of the right wrist; and a contact analysis module, used to determine the right thumb wrapping parameters between the left palm region and the right thumb region, and the contact parameters between the right palm region and the left hand, based on the hand region recognition data. The left thumb wrapping parameters between the thumb regions are combined with the right thumb wrapping parameters to obtain the contralateral contact matrix. The rubbing judgment module is used to perform continuous rubbing judgment based on the contralateral contact matrix to obtain the effective rubbing frames of the left and right thumbs, obtain the frame time identifier, and perform continuous segment analysis on the effective rubbing frames of the left and right thumbs respectively to generate the left thumb rubbing parameters and the right thumb rubbing parameters respectively. The standardization judgment module is used to perform compliance evaluation analysis based on the left thumb rubbing parameters and the right thumb rubbing parameters to obtain the standardization execution status identification value, thereby obtaining the corresponding standardization judgment result.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0009] 1. This invention acquires video image data of the handwashing area and performs hand area recognition to generate hand area recognition data, thereby constructing a side contact matrix. This allows the thumb rubbing process to be characterized by the temporal wrapping contact state between the opposite palm area and the thumb area. This achieves the technical effect of accurately identifying the effective contact state, continuous execution state, and bilateral completion state in the thumb rubbing step. It effectively solves the problem in the prior art that it is difficult to accurately obtain the execution state of the aseptic operation procedure in the operating room due to the difference in the amplitude of local rubbing movements and the mutual interference of the left and right hand movements.

[0010] 2. This invention generates hand region recognition data by performing hand region recognition on video image data, thereby determining the relationship between the left and right hands and the positional relationship between the palm region and the thumb region. This enables accurate positioning of the local thumb rubbing area during preoperative surgical handwashing, reducing the impact of misidentification caused by relying solely on overall hand movement recognition.

[0011] 3. By determining the effective rubbing frames of the left and right thumbs based on the opposite contact matrix, and performing continuous segment analysis, the effective rubbing state of a single frame can be converted into continuous rubbing segments of the left and right thumbs with temporal continuity, thereby improving the stability of the left and right thumb rubbing parameters. Attached Figure Description

[0012] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0013] Figure 1 This is the overall flowchart of the present invention;

[0014] Figure 2 This is a flowchart illustrating the generation of hand region recognition data and opposite-side contact matrix according to the present invention.

[0015] Figure 3 A flowchart for generating continuous kneading determination and standardized determination results in this invention;

[0016] Figure 4 This is a system architecture diagram of the present invention. Detailed Implementation

[0017] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0018] Preoperative surgical handwashing is a crucial step in the aseptic procedure of the operating room. Existing image recognition methods typically determine whether the operator has completed the handwashing action through hand target detection, posture recognition, or action classification. However, in the thumb-rubbing step, the hands are often overlapping, the visible area of ​​the thumb is small, the contact relationship between the palm and thumb areas is not obvious, and the left and right hands need to complete the rubbing action separately. This makes it difficult for existing methods to accurately identify the effective contact state, continuous execution state, and bilateral completion state in the thumb-rubbing step. To address the aforementioned issues, this application acquires video image data of the handwashing area and performs hand region recognition, generating hand region recognition data including the left palm region, right palm region, left thumb region, right thumb region, key points of the left wrist, and key points of the right wrist. Based on the hand region recognition data, it determines the right thumb wrapping parameter between the left palm region and the right thumb region, and the left thumb wrapping parameter between the right palm region and the left thumb region, and combines the right thumb wrapping parameter and the left thumb wrapping parameter to obtain the opposite-side contact matrix. Based on the opposite-side contact matrix, it performs continuous rubbing judgment to obtain the effective rubbing frames of the left and right thumbs, obtains the frame time identifier, and performs continuous segment analysis on the effective rubbing frames of the left and right thumbs respectively to generate the left thumb rubbing parameter and the right thumb rubbing parameter. Based on the left thumb rubbing parameter and the right thumb rubbing parameter, it performs compliance evaluation analysis to obtain the standard execution status identification value, thereby obtaining the corresponding standard judgment result. This application can improve the accuracy of identifying the effective contact state, continuous execution state, and bilateral completion state in the thumb rubbing step, and reduce the impact of brief contact, hand occlusion, and single-frame detection fluctuations on the standard judgment results, thereby improving the reliability of the identification of the execution state of aseptic operation standards in the operating room.

[0019] This application provides an image recognition method for the execution status of aseptic operation procedures in an operating room. The method includes the following steps: acquiring video image data of a handwashing area and performing hand region recognition to generate hand region recognition data, which includes the left palm region, right palm region, left thumb region, right thumb region, key points of the left wrist, and key points of the right wrist; based on the hand region recognition data, determining the right thumb wrapping parameter between the left palm region and the right thumb region, and the left thumb wrapping parameter between the right palm region and the left thumb region, and combining the right thumb wrapping parameter and the left thumb wrapping parameter to obtain an opposite contact matrix; based on the opposite contact matrix, performing continuous rubbing judgment to obtain effective rubbing frames of the left and right thumbs, obtaining frame time identifiers, and performing continuous segment analysis on the effective rubbing frames of the left and right thumbs respectively to generate left thumb rubbing parameters and right thumb rubbing parameters respectively; performing compliance evaluation analysis based on the left thumb rubbing parameters and right thumb rubbing parameters to obtain the standard execution status recognition value, thereby obtaining the corresponding standard judgment result.

[0020] In this embodiment, as Figure 1 As shown, Figure 1 The overall flowchart of this invention is as follows: Hand region recognition is performed and hand region recognition data is generated; based on the hand region recognition data, the right thumb wrapping parameters and left thumb wrapping parameters are determined, and the opposite-side contact matrix is ​​obtained; based on the opposite-side contact matrix, continuous rubbing is determined to obtain the left thumb effective rubbing frame and the right thumb effective rubbing frame; frame time identifiers are obtained and continuous segment analysis is performed to generate left thumb rubbing parameters and right thumb rubbing parameters; based on the left thumb rubbing parameters and the right thumb rubbing parameters, compliance evaluation analysis is performed to obtain the standard execution status identification value and the corresponding standard judgment result.

[0021] Hand region recognition data can be obtained by reading handwashing image frames frame by frame from video image data, then inputting these frames into a hand object detection model to obtain hand candidate bounding box data; inputting the image regions within the hand candidate bounding boxes into a semantic segmentation model to obtain palm region segmentation data and thumb region segmentation data; obtaining wrist keypoint localization data through a hand keypoint localization model; and matching the left and right hands based on the wrist keypoint localization data to obtain hand region recognition data. The hand object detection model is the YOLO model, and the semantic segmentation model is MediaPipe Hands (hand keypoint recognition model).

[0022] Left palm region, right palm region, left thumb region, and right thumb region: These all represent image regions composed of pixels in the same handwashing image frame. The acquisition method is as follows: the semantic segmentation model outputs the category to which each pixel belongs, and the pixels belonging to the category of left palm, right palm, left thumb, or right thumb are respectively collected to form the corresponding regions.

[0023] Left and right wrist key points: These represent key points in the image coordinate system representing the wrist position, used to help determine whether the hand region belongs to the left or right hand. The acquisition method is as follows: hand image data is input into the hand key point localization model, which outputs the image coordinates of the wrist key points; the processor matches the left and right hands based on the spatial relationship between the wrist key points and the corresponding palm and thumb areas.

[0024] Right thumb wrapping parameters and left thumb wrapping parameters: These represent a set of parameters used to describe the wrapping and kneading relationship between the palm and thumb regions on opposite sides. The right thumb wrapping parameter is obtained from data between the left palm region and the right thumb region, and the left thumb wrapping parameter is obtained from data between the right palm region and the left thumb region. The acquisition method is as follows: the processor performs boundary extraction, area overlap analysis, and relative position change analysis on the hand region recognition data to obtain the contact distance value, coverage ratio, and relative displacement, respectively. Then, the three parameters corresponding to the same thumb are combined to form the corresponding wrapping parameters.

[0025] The opposite-side contact matrix represents a set of wrapping parameters arranged according to frame time identifiers across multiple frames. It characterizes the temporal wrapping contact states between the left palm region and the right thumb region, as well as between the right palm region and the left thumb region. Specifically, the right thumb wrapping parameters and left thumb wrapping parameters are read from each image frame and arranged from earliest to latest according to the frame time identifiers to form the opposite-side contact matrix.

[0026] Left thumb effective rubbing frame and right thumb effective rubbing frame: These represent image frames that simultaneously meet the conditions of contact distance, coverage ratio, and relative displacement within a single image frame. They are used to characterize whether the left or right thumb is in an effective rubbing state within that frame. The acquisition method is as follows: The corresponding parameters are read frame by frame from the opposite contact matrix. The contact distance value is compared with a preset contact distance threshold, the coverage ratio is compared with a preset coverage ratio threshold, and the relative displacement is compared with a preset displacement threshold. When all the above conditions are met simultaneously, the corresponding image frame is marked as a left thumb effective rubbing frame or a right thumb effective rubbing frame.

[0027] Left thumb rubbing parameters and right thumb rubbing parameters: These represent the set of parameters used to evaluate the standardization of thumb rubbing after continuous segment analysis, including continuous rubbing duration, coverage compliance rate, and completion status indicator. The acquisition method is as follows: Based on frame time identifiers, continuous segment analysis is performed on effective left thumb rubbing frames and effective right thumb rubbing frames respectively. The continuous rubbing duration value is counted, the percentage of image frames meeting the coverage compliance rate is counted, and corresponding completion status indicators are generated based on the continuous rubbing duration value and coverage compliance rate.

[0028] **Standard Execution Status Identification Value:** This represents the numerical identification result obtained after evaluating the compliance of the left and right thumb rubbing parameters. It characterizes the degree of compliance between the current thumb rubbing step and the preset aseptic operation standard requirements. **Standard Judgment Result:** This represents the final judgment output generated based on the standard execution status identification value, including the pass / fail judgment result, the fail / unqualified judgment result, and their corresponding reason types. **Acquisition Method:** The processor compares the standard execution status identification value with a preset pass / fail threshold and combines this with the non-compliant parameters in the left and right thumb rubbing parameters to generate the corresponding judgment result.

[0029] By first identifying the hand region, then constructing right thumb wrapping parameters, left thumb wrapping parameters, and contralateral contact matrices, and finally generating left and right thumb rubbing parameters based on continuous rubbing judgment, the solution no longer relies on coarse-grained action classification of the entire hand. Instead, it identifies the contralateral contact relationship, coverage state, and continuous execution state within the thumb rubbing step. This step can form a standardized recognition basis with temporal continuity, even in cases where the hands overlap and obscure each other, the visible area of ​​the thumb is small, and the action states of the left and right hands interfere with each other. This improves the overall accuracy of identifying the standardized execution state corresponding to the thumb rubbing step and provides a data foundation for subsequent judgments of insufficient output duration, insufficient coverage, or incomplete unilateral action.

[0030] Furthermore, generating hand region recognition data specifically includes: extracting image frame data from video image data, writing frame time identifiers for each handwashing image frame, and then performing hand target detection to generate hand candidate box data; based on the candidate box coordinates in the hand candidate box data, extracting image regions located within the hand candidate boxes from the image frame data, thereby generating hand image data; performing palm region segmentation, thumb region segmentation, and wrist key point localization processing on the hand image data to generate palm region segmentation data, thumb region segmentation data, and wrist key point localization data; and based on the wrist key point localization data, performing left and right hand matching on the palm region segmentation data and thumb region segmentation data to generate hand region recognition data.

[0031] In this embodiment, image frame data refers to single or multiple frames of image data extracted from video image data in chronological order.

[0032] Hand candidate bounding box data: This represents the rectangular bounding box data obtained after hand target detection. The acquisition method is as follows: input the handwashing image frame into the trained hand target detection model, read the coordinates of the candidate bounding boxes output by the model; filter out candidate bounding boxes with low confidence according to a preset confidence threshold, and retain the hand candidate bounding box data for subsequent cropping.

[0033] Hand image data: This represents the image region cropped from the image frame data according to the coordinates of the hand candidate box. It is obtained by reading the coordinates of the top-left and bottom-right corners of the hand candidate box, and then calling the image cropping function to extract the image region within the candidate box from the handwashing image frame.

[0034] Palm region segmentation data and thumb region segmentation data: These represent the pixel classification results corresponding to the palm and thumb regions in hand image data. The acquisition method is as follows: input the hand image data into a trained semantic segmentation model, which outputs the classification result of each pixel belonging to the palm region, thumb region, or other regions; extract the pixels belonging to the palm region and thumb region respectively to form palm region segmentation data and thumb region segmentation data.

[0035] The wrist key point localization data is obtained by inputting hand image data into the hand key point localization model and reading the wrist key point coordinates output by the model. If the model outputs multiple hand key points, the key point corresponding to the wrist position is selected as the wrist key point localization data.

[0036] By extracting video image data frame by frame, detecting hand targets, cropping candidate boxes, segmenting the palm region, segmenting the thumb region, and locating wrist key points, a more granular region recognition is achieved within the hand candidate box. This reduces the impact of non-hand areas such as sinks, background objects, sleeves, and instrument reflections on the recognition results. Furthermore, by unifying the left and right hand attribution through wrist key point localization data, it is easier to determine the right thumb wrapping parameters between the left palm region and the right thumb region, as well as the left thumb wrapping parameters between the right palm region and the left thumb region.

[0037] Furthermore, the parameters for the right thumb wrapping between the left palm region and the right thumb region, and the parameters for the left thumb wrapping between the right palm region and the left thumb region, are determined. Specifically, this includes: performing connected component analysis based on hand region recognition data, extracting boundary pixels at the outer edges of each region, and sequentially connecting adjacent boundary pixels to obtain the boundaries of each region, including the boundaries of the left palm region, right palm region, left thumb region, and right thumb region; performing boundary point distance traversal processing on the boundaries of the left palm region and right thumb region to obtain the minimum boundary point distance, which is used as the right thumb contact distance value; performing area overlap analysis on the left palm region and right thumb region, counting the number of overlapping pixels between the image projection areas of the left palm region and the right thumb region, and counting the total number of pixels in the image projection area of ​​the right thumb region, dividing the number of overlapping pixels by the total number of pixels to obtain the right thumb coverage ratio; and comparing the center point of the left palm region with the right thumb region in consecutive image frames. The relative position change analysis of the center point yields the positional change of the left palm region relative to the right thumb region, which is used as the relative displacement of the right thumb. The right thumb contact distance, right thumb coverage ratio, and right thumb relative displacement are combined as the right thumb wrapping parameter. The boundary point distance traversal processing of the boundary between the right palm region and the left thumb region yields the minimum boundary point distance, which is used as the left thumb contact distance. The area overlap analysis of the right palm region and the left thumb region is performed to count the number of overlapping pixels between the image projection areas of the right palm region and the left thumb region, and the total number of pixels in the image projection area of ​​the left thumb region is also counted. The left thumb coverage ratio is obtained by dividing the number of overlapping pixels by the total number of pixels. The relative position change analysis of the center point of the right palm region and the center point of the left thumb region in consecutive image frames yields the positional change of the right palm region relative to the left thumb region, which is used as the relative displacement of the left thumb. The left thumb contact distance, left thumb coverage ratio, and left thumb relative displacement are combined as the left thumb wrapping parameter.

[0038] In this embodiment, the boundaries of each region are obtained by performing connected component analysis on the palm region segmentation data and thumb region segmentation data, retaining the target connected component with the largest area or the target connected component that is continuous with the position of the previous and next frames, and then calling the OpenCV contour extraction function to extract the boundary, obtaining the boundary pixels of the outer edge of each region, and connecting them according to the adjacent relationship to form the boundary of each region.

[0039] Right thumb contact distance value: This represents the minimum boundary distance between the boundaries of the left palm region and the right thumb region in the same image frame, used to characterize whether the left palm region and the right thumb region are close to contact. It is obtained as follows: The processor iterates through the boundary pixels on the boundaries of the left palm region and the right thumb region, calculates the distance between any two boundary pixels using the Euclidean distance formula, and then takes the minimum value. To eliminate the dimensional differences caused by shooting distance and hand size, this minimum distance is divided by the width of the candidate hand bounding box in the same frame to obtain the right thumb contact distance value. The method for obtaining the left thumb contact distance value is the same as that for the right thumb contact distance value.

[0040] The right thumb coverage ratio is used to characterize the degree to which the right thumb region is surrounded and covered by the left palm region. It is obtained by converting the left palm region and the right thumb region into pixel sets in the same image coordinate system, counting the number of pixels in the intersection of the two pixel sets, and dividing this number by the number of pixels in the right thumb region to obtain the right thumb coverage ratio. The left thumb coverage ratio is obtained in the same way.

[0041] The relative displacement of the right thumb represents the degree of change in the relative positional relationship between the center points of the left palm region and the right thumb region in consecutive image frames. It is used to characterize whether there is a rubbing motion between the left palm region and the right thumb region. The method of obtaining it is as follows: Calculate the relative vector changes between the center points of the left palm region and the right thumb region in two consecutive frames, and normalize the vectors using the width of the candidate bounding box of the hand in the same frame to obtain the relative displacement of the right thumb. The relative displacement of the left thumb is obtained in the same way.

[0042] The center point of the region can be obtained using the formula for calculating the centroid of the region:

[0043] ;

[0044] in, Indicates the first Frame Image Frame Region The regional center point, Indicates the first Any region to be calculated within a frame of an image. Indicates the region area, Represents the x-coordinate of a pixel. Represents the ordinate of a pixel.

[0045] The smaller the contact distance value of the right thumb or left thumb, the closer the corresponding palm area and thumb area are, and the more it conforms to the wrapping contact state. The larger the coverage ratio of the right thumb or left thumb, the higher the degree to which the corresponding thumb area is covered by the opposite palm area, and the more it conforms to the thumb wrapping and kneading requirements. The larger the relative displacement of the right thumb or left thumb, the more obvious the relative movement between the opposite palm area and the corresponding thumb area in consecutive image frames, and the more it can eliminate misjudgments caused by static adhesion. However, the relative displacement needs to be used in conjunction with the contact distance value and coverage ratio to avoid misjudging long-distance swings as effective kneading.

[0046] By decomposing the wrapping and rubbing relationship into three calculable parameters—contact distance, coverage ratio, and relative displacement—the image recognition process can simultaneously reflect whether there is near contact, whether a wrapping or covering effect has formed, and whether relative rubbing motion exists. The contact distance reduces misjudgments of long-distance non-contact actions; the coverage ratio reflects the actual degree of wrapping of the palm area over the thumb area; and the relative displacement eliminates static states with only contact but no rubbing motion. These three parameters work together to ensure that the subsequent opposite-side contact matrix retains spatial contact and motion change information of the thumb rubbing step, thereby improving the stability of thumb rubbing motion recognition.

[0047] Furthermore, the heterodyne contact matrix is ​​obtained, specifically including: obtaining the frame time identifier corresponding to each image frame in the video image data; according to the frame time identifier, forming a single-frame heterodyne contact vector by combining the right thumb contact distance value, right thumb coverage ratio, right thumb relative displacement, left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement corresponding to the same image frame; arranging multiple single-frame heterodyne contact vectors in chronological order to obtain the heterodyne contact matrix, which is used to characterize the temporal wrapping contact state between the left palm region and the right thumb region, and between the right palm region and the left thumb region.

[0048] In this embodiment, as Figure 2 As shown, Figure 2 The flowchart for generating hand region recognition data and contralateral contact matrix according to the present invention includes the following steps: extracting handwashing image frame data from video image data and writing a frame time identifier for each handwashing image frame; performing hand target detection on the handwashing image frame data to generate hand candidate box data and cropping hand image data; performing palm region segmentation, thumb region segmentation, and wrist key point localization processing on the hand image data, and performing left and right hand matching to generate hand region recognition data; determining the right thumb wrapping parameters and left thumb wrapping parameters based on the hand region recognition data to form a single-frame contralateral contact vector, and arranging them in chronological order to obtain the contralateral contact matrix.

[0049] The single-frame opposite-side contact vector is obtained by the processor using a data vector composed of the right thumb contact distance value, right thumb coverage ratio, right thumb relative displacement, left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement in a fixed order under the same frame time identifier. This data vector is then arranged according to a preset field order to form the single-frame opposite-side contact vector.

[0050] The opposite contact matrix represents a matrix formed by arranging multiple single-frame opposite contact vectors from earliest to latest according to their frame time identifiers. It can be obtained by first sorting the image frames by their frame time identifiers, and then using the single-frame opposite contact vectors corresponding to each image frame as a row in the matrix.

[0051] Temporal wrap-around contact state: This represents the state of contact, coverage, and relative movement between the palm and thumb regions on the opposite side in multiple consecutive image frames as time changes. It is characterized by the data of each row in the opposite contact matrix in chronological order. Continuous rubbing is determined by reading the continuous rows of data in the opposite contact matrix.

[0052] The single-frame cross-side contact vector is represented as:

[0053] ;

[0054] in, Indicates the first The single-frame opposite-side contact vector corresponding to the frame image frame. Indicates the first The right thumb contact distance value in the frame image. Indicates the first The proportion of the right thumb covering the image frame. Indicates the first The relative displacement of the right thumb in a frame of an image. Indicates the first The left thumb contact distance value in the frame image. Indicates the first The proportion of the left thumb covering the image frame. Indicates the first The relative displacement of the left thumb in the frame of the image.

[0055] The more image frames there are, the more comprehensive the temporal information contained in the contralateral contact matrix, which is more beneficial for subsequent continuous rubbing judgment. In the contralateral contact matrix, the columns containing the right thumb contact distance value and the left thumb contact distance value are used to determine the degree of contact proximity; the columns containing the right thumb coverage ratio and the left thumb coverage ratio are used to determine the coverage state; and the columns containing the right thumb relative displacement and the left thumb relative displacement are used to determine the relative rubbing motion state. Aligning the parameters in each column according to the same frame time marker avoids misjudgments caused by asynchrony between the left and right hands or parameter misalignment.

[0056] By organizing the right thumb contact distance, right thumb coverage ratio, right thumb relative displacement, left thumb contact distance, left thumb coverage ratio, and left thumb relative displacement within the same image frame into a single-frame opposite-side contact vector, and arranging them into an opposite-side contact matrix according to frame time identifiers, the contact, coverage, and motion information originally scattered across multiple image frames and regions is unified into a data structure. This step ensures that subsequent continuous rubbing judgments read the corresponding parameters of the left and right thumbs under the same time reference, reducing the risk of misjudgment caused by frame order disorder, parameter field misalignment, or asynchronous states of the left and right hands. The opposite-side contact matrix plays a role in connecting spatial recognition results and temporal action judgment in the overall scheme, facilitating the identification of valid rubbing frames, continuous rubbing segments, and standardized execution state recognition values.

[0057] Further, continuous segment analysis is performed to generate left thumb rubbing parameters and right thumb rubbing parameters, specifically including: reading the right thumb contact distance value, right thumb coverage ratio, and right thumb relative displacement from the contralateral contact matrix; when the right thumb contact distance value in a certain image frame is below a preset contact distance threshold, the right thumb coverage ratio is above a preset coverage ratio threshold, and the right thumb relative displacement is above a preset displacement threshold, then the image frame is marked as a valid right thumb rubbing frame; reading the left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement from the contralateral contact matrix; when the left thumb contact distance value in a certain image frame is below a preset contact distance threshold, the left thumb coverage ratio is above a preset coverage ratio threshold, and the left thumb relative displacement is above a preset displacement threshold... When the preset displacement threshold is exceeded, the image frame is marked as a valid left thumb rubbing frame. Continuous segment analysis is performed on the valid right thumb rubbing frames to obtain continuous right thumb rubbing segments, and continuous segment analysis is also performed on the valid left thumb rubbing frames to obtain continuous left thumb rubbing segments. Right thumb rubbing parameters are generated based on the continuous right thumb rubbing segments, and left thumb rubbing parameters are generated based on the continuous left thumb rubbing segments. The right thumb rubbing parameters include the continuous right thumb rubbing duration, the right thumb coverage ratio, and the right thumb completion status indicator. The left thumb rubbing parameters include the continuous left thumb rubbing duration, the left thumb coverage ratio, and the left thumb completion status indicator. The right thumb completion status indicator includes a right thumb completion indicator and a right thumb incomplete indicator, and the left thumb completion status indicator includes a left thumb completion indicator and a left thumb incomplete indicator.

[0058] In this embodiment, as Figure 3 As shown, Figure 3The flowchart for generating continuous rubbing judgment and standard judgment results of this invention includes the following steps: Reading the opposite-side contact matrix to form a right thumb processing branch and a left thumb processing branch. The right thumb processing branch reads the right thumb contact distance value, right thumb coverage ratio, and right thumb relative displacement, marks the effective right thumb rubbing frames, and performs continuous segment analysis to obtain the right thumb continuous rubbing segments and right thumb rubbing parameters. The left thumb processing branch reads the left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement, marks the left thumb effective rubbing frames, and performs continuous segment analysis to obtain the left thumb continuous rubbing segments and left thumb rubbing parameters. The right thumb rubbing parameters and left thumb rubbing parameters are jointly entered into a compliance evaluation analysis to obtain a duration evaluation value and a coverage evaluation value, and are jointly analyzed to obtain a standard execution status identification value. Based on the comparison result between the standard execution status identification value and a preset qualified threshold, a qualified judgment result is generated; alternatively, non-compliant parameters are read and a judgment result for insufficient duration, insufficient coverage, or unilateral incompleteness is generated.

[0059] Continuous segment analysis is performed on the effective right thumb rubbing frames to obtain continuous right thumb rubbing segments. The specific method is as follows: Arrange the image frames in the video image data in chronological order according to the frame time identifier, and mark the effective right thumb rubbing frames and non-effective right thumb rubbing frames in the arranged image frames; traverse sequentially starting from the earliest image frame in the chronological order. When the first effective right thumb rubbing frame is encountered, it is taken as the starting frame of the current continuous right thumb rubbing segment; continue traversing the subsequent image frames in chronological order. If the subsequent image frame is also marked as an effective right thumb rubbing frame, it is merged into the current continuous right thumb rubbing segment; if the subsequent image frame is not marked as an effective right thumb rubbing frame, the current continuous right thumb rubbing segment terminates at the last effective right thumb rubbing frame before the image frame not marked as an effective right thumb rubbing frame. After the current right thumb rubbing segment ends, the remaining image frames continue to be traversed in chronological order. When a valid right thumb rubbing frame is encountered again, it is used as the starting frame of a new right thumb rubbing segment, and new right thumb rubbing segments are determined in the same way until all image frames in the video image data have been traversed. This results in one or more right thumb rubbing segments. Each right thumb rubbing segment consists of temporally consecutive image frames, all marked as valid right thumb rubbing frames. Each right thumb rubbing segment corresponds to a segment start time and a segment end time. The segment start time is the frame time identifier corresponding to the earliest valid right thumb rubbing frame in the segment, and the segment end time is the frame time identifier corresponding to the latest valid right thumb rubbing frame in the segment. Based on the segment start time and segment end time, the duration of the corresponding right thumb rubbing segment can be further calculated. The method for analyzing consecutive segments of effective left thumb rubbing frames to obtain consecutive left thumb rubbing segments is the same as that for right thumb effective rubbing frames, except that right thumb effective rubbing frames are replaced with left thumb effective rubbing frames, and consecutive right thumb rubbing segments are replaced with consecutive left thumb rubbing segments. Specifically, after arranging each image frame in chronological order according to the frame time identifier, consecutive left thumb effective rubbing frames are sequentially traversed and extracted. Consecutive left thumb effective rubbing frames are grouped into the same consecutive left thumb rubbing segment. When an image frame that is not marked as a left thumb effective rubbing frame appears, the current consecutive left thumb rubbing segment is terminated, and a new consecutive left thumb rubbing segment is established when a new left thumb effective rubbing frame appears again.

[0060] By performing continuous segment analysis on the effective rubbing frames of the right and left thumbs, the scattered single-frame rubbing judgment results can be converted into continuous motion data with start time, end time, and duration. A single effective rubbing frame can only indicate that the contact distance, coverage ratio, and relative displacement meet the preset conditions at a certain instant, but it cannot indicate whether the operator has continuously performed the thumb rubbing action. If the judgment is made directly based on the number of effective rubbing frames, it is easy to mistake brief contact, accidental occlusion, instantaneous overlap, or fluctuations in single-frame detection as effective rubbing processes, resulting in unstable identification values ​​for the standard execution status.

[0061] By analyzing consecutive segments, it is possible to distinguish between scattered image frames that occasionally meet the rubbing condition and actual rubbing actions that continuously meet the rubbing condition. Continuous rubbing segments of the right and left thumbs provide clear data ranges for subsequent statistical analysis of the duration of continuous rubbing of the right and left thumbs, as well as the coverage rates of the right and left thumbs. This ensures that the generated right and left thumb rubbing parameters no longer depend on isolated frames but on the continuous action process. This reduces the impact of single-frame misidentification, action pauses, and brief contact on the evaluation results, improving the accuracy of identifying continuous execution and bilateral completion states in the thumb rubbing step.

[0062] Continuous segment analysis can transform the temporal sequence surrounding contact states into an evaluable rubbing process. The contralateral contact matrix addresses whether there is effective contact and relative motion between the contralateral palm and thumb regions in each image frame, while continuous segment analysis further addresses whether this effective contact and relative motion constitute a continuous rubbing action. Therefore, it can not only determine whether an effective rubbing frame has occurred, but also whether a continuous and effective rubbing process has been formed, thus providing a more reliable temporal basis for generating the final standardized judgment result.

[0063] Furthermore, the continuous rubbing duration of the right thumb, the coverage rate of the right thumb, and the completion status indicator are obtained as follows: The effective rubbing frames of the right thumb are divided into continuous segments according to the frame time identifier to obtain effective rubbing segments of the right thumb; when there is only one effective rubbing segment of the right thumb, this segment is taken as the target effective rubbing segment; otherwise, the duration of each effective rubbing segment is counted, and the segment with the longest duration is taken as the target effective rubbing segment; based on the frame time identifier corresponding to the target effective rubbing segment, the continuous rubbing duration of the right thumb is counted; and so on. The number of image frames in the target right thumb effective rubbing segment with a right thumb coverage ratio exceeding a preset coverage ratio threshold is recorded as the right thumb coverage compliance frame number; the proportion of the right thumb coverage compliance frame number to the total number of image frames in the target right thumb effective rubbing segment is obtained to obtain the right thumb coverage compliance ratio; when the right thumb continuous rubbing duration is above a preset rubbing duration threshold and the right thumb coverage compliance ratio is above a preset coverage compliance threshold, a right thumb completion status flag is generated; otherwise, a right thumb incomplete status flag is generated; the right thumb continuous rubbing duration, right thumb coverage compliance ratio, and right thumb completion status flag are combined as the right thumb rubbing parameters.

[0064] In this embodiment, the effective right thumb rubbing segment refers to a set of continuous image frames formed by dividing the effective right thumb rubbing frames according to their frame time identifiers. The acquisition method is as follows: The marked effective right thumb rubbing frames are arranged from earliest to latest according to their frame time identifiers; when there is no break between two adjacent effective right thumb rubbing frames exceeding a preset continuous interval frame number threshold (i.e., the continuous interval frame number is less than the preset continuous interval frame number threshold), they are grouped into the same effective right thumb rubbing segment; when the break between two adjacent effective right thumb rubbing frames reaches the preset continuous interval frame number threshold, they are divided into different effective right thumb rubbing segments. This process can be performed by the video analysis processor by calling the image frame timestamp table and the effective rubbing frame label table. The acquisition method for the effective left thumb rubbing segment is the same as that for the effective right thumb rubbing segment.

[0065] The target effective rubbing segment of the right thumb refers to the effective rubbing segment of the right thumb used to statistically analyze the duration of continuous rubbing, the coverage rate of the right thumb, and the completion status of the right thumb. If there are multiple effective rubbing segments of the right thumb, the duration of each segment is calculated separately, and the segment with the longest duration is selected as the target effective rubbing segment of the right thumb. The purpose is to prioritize retaining the main segment that best represents the actual rubbing process of the right thumb in cases of partial occlusion, short-term frame loss, or interruption of action.

[0066] The duration of continuous rubbing with the right thumb refers to the duration of the effective rubbing segment of the target right thumb on the timeline. It can be obtained by reading the frame time markers of the first and last frames of the effective rubbing segment, and then reading the average time interval between adjacent image frames within that segment. The average time interval is then added to the frame time marker of the last frame after subtracting the frame time marker of the first frame from the frame time marker of the last frame. Including the average time interval avoids underestimating the acquisition period corresponding to the last frame when only the time difference between the first and last frames is used.

[0067] The number of right thumb coverage frames meeting the standard and the right thumb coverage ratio meeting the standard were both obtained from the same set of data. The right thumb coverage ratio is determined by the right thumb coverage ratio corresponding to each image frame within the target effective right thumb rubbing segment. Image frames with a right thumb coverage ratio above a preset coverage ratio threshold are counted as right thumb coverage meeting the standard. The number of right thumb coverage meeting the standard is then counted to obtain the total number of right thumb coverage meeting frames. The right thumb coverage ratio is a dimensionless ratio value, ranging from 0 to 1.

[0068] The right thumb completion status flag indicates whether the right thumb rubbing action has met the preset completion requirements. This flag includes a right thumb completion flag and a right thumb incomplete flag. The continuous rubbing duration of the right thumb is compared with a preset rubbing duration threshold, and the right thumb coverage ratio is compared with a preset coverage ratio threshold. A right thumb completion flag is generated when both the continuous rubbing duration and the coverage ratio are above the preset coverage ratio threshold; otherwise, a right thumb incomplete flag is generated. It should be noted that the right thumb completion status flag is a status parameter, not the final specification execution status identification value.

[0069] The methods for obtaining the continuous rubbing duration of the left thumb, the coverage ratio of the left thumb, and the completion status of the left thumb are the same as those for obtaining the continuous rubbing duration of the right thumb, the coverage ratio of the right thumb, and the completion status of the right thumb. The only difference is that the data objects read are replaced by the effective rubbing frames and coverage ratio of the right thumb with the effective rubbing frames and coverage ratio of the left thumb.

[0070] By dividing the effective rubbing frame into effective rubbing segments and selecting the segment with the longest duration from multiple segments as the target effective rubbing segment, the interference of short-term occlusion, image recognition frame loss, and discontinuous slight contact on the statistical results of rubbing parameters can be reduced. By simultaneously calculating the continuous rubbing duration and the coverage compliance rate, and then generating a completion status identifier, the completion judgment of the unilateral thumb rubbing action can be constrained by both duration and coverage stability, avoiding misjudgment of completion based solely on instantaneous contact or a single duration condition, thereby improving the reliability of the standard execution status identification value.

[0071] Furthermore, the continuous rubbing determination also includes: performing interval splicing processing on the effective rubbing frames of the right thumb and the left thumb respectively; sorting the effective rubbing frames of the right thumb according to the frame time identifier, and sequentially determining the preceding effective frame of the right thumb and the following effective frame of the right thumb; when there is an image frame that is not marked as a valid rubbing frame of the right thumb between the preceding and following effective frames of the right thumb, then the image frame is used as a candidate interval frame of the right thumb; counting the number of consecutive candidate interval frames of the right thumb to obtain the number of consecutive interval frames of each right thumb; if the number of consecutive interval frames of the right thumb is small... If the number of consecutive interval frames is within a preset threshold, the preceding valid frame of the right thumb, the consecutive candidate interval frames of the right thumb, and the following valid frame of the right thumb are jointly stitched together to obtain a continuous rubbing segment of the right thumb. If the number of consecutive interval frames of the right thumb is above the preset threshold, joint stitching is not performed, and the preceding valid frame of the right thumb and the following valid frame of the right thumb are divided into different continuous rubbing segments of the right thumb. The valid rubbing frames of the left thumb are analyzed to obtain a continuous rubbing segment of the left thumb. Right thumb rubbing parameters are generated based on the continuous rubbing segments of the right thumb, and left thumb rubbing parameters are generated based on the continuous rubbing segments of the left thumb.

[0072] In this embodiment, the interval splicing process refers to determining whether a small number of image frames not marked as valid splicing frames exist between adjacent valid splicing frames. This involves identifying whether the interval is caused by short-term frame loss or short-term occlusion, and merging the preceding and following valid splicing frames and the intermediate candidate interval frames into a single continuous splicing segment if the conditions are met. Specifically, this involves reading the valid splicing frame label table, the image frame sequence number table, and the frame time identifier table; calculating the number of consecutive candidate interval frames between preceding and following valid splicing frames according to the temporal position of the valid splicing frames; comparing this number with a preset threshold for the number of consecutive interval frames; and then performing splicing or disconnection.

[0073] The preceding and following valid frames of the right thumb refer to the two adjacent valid right thumb rubbing frames that are used for judgment after the valid right thumb rubbing frames are sorted by frame time identifier. The valid right thumb rubbing frames are traversed in ascending order according to the frame time identifier. The currently read valid right thumb rubbing frame is taken as the preceding valid frame, and the next valid right thumb rubbing frame is taken as the following valid frame.

[0074] A right thumb candidate interval frame refers to an image frame located between the preceding and following valid right thumb frames that has not been marked as a valid right thumb rubbing frame. By reading the image frame sequence table, the image frames between the preceding and following valid right thumb frames are searched frame by frame; if an intermediate image frame is not marked as a valid right thumb rubbing frame, it is recorded as a right thumb candidate interval frame.

[0075] The number of consecutive right thumb interval frames refers to the number of consecutive right thumb candidate interval frames. This is obtained by reading the frame numbers of the preceding and following valid right thumb frames, subtracting the frame number of the preceding valid right thumb frame from the frame number of the following valid right thumb frame, and then subtracting 1.

[0076] The methods for obtaining the preceding valid frames, following valid frames, candidate interval frames, and number of consecutive interval frames of the left thumb are the same as those for the corresponding parameters of the right thumb, except that the data object read is replaced by the valid rubbing frames of the left thumb instead of the valid rubbing frames of the right thumb.

[0077] By setting an interval splicing process, the generation of continuous rubbing segments no longer relies entirely on every frame being continuously marked as a valid rubbing frame. This provides greater tolerance for short-term occlusion, hand overlap, local boundary recognition jitter, and short-term missed detections by the model. By merging candidate interval frames smaller than a preset continuous interval frame number threshold into the same continuous rubbing segment, the integrity of the actual continuous rubbing action on the timeline can be maintained. By not splicing disconnections that reach the preset continuous interval frame number threshold, the erroneous merging of paused actions or actions at different stages can be avoided. This step improves the statistical stability of the continuous rubbing duration value in the overall scheme and indirectly improves the accuracy of the right thumb completion status identifier, left thumb completion status identifier, and standardized execution status identification value.

[0078] Furthermore, the standard execution status identification value is obtained, specifically including: based on the analysis of the continuous rubbing duration values ​​of the left and right thumbs, the duration compliance ratio is obtained, which is used to characterize the degree of common compliance of the continuous rubbing duration values ​​of the left and right thumbs relative to the preset rubbing duration threshold; based on the analysis of the coverage compliance ratio of the left and right thumbs, the comprehensive coverage ratio is obtained, which is used to characterize the degree of common compliance of the coverage compliance ratio of the left and right thumbs relative to the preset coverage compliance threshold; based on the joint analysis of the duration compliance ratio and the comprehensive coverage ratio, the standard execution status identification value is obtained.

[0079] In this embodiment, the duration target ratio refers to the degree to which the continuous rubbing time values ​​of the left and right thumbs meet the preset rubbing time threshold. It is obtained by calculating the ratio of the continuous rubbing time value of the left thumb to the preset rubbing time threshold, and the ratio of the continuous rubbing time value of the right thumb to the preset rubbing time threshold, thus obtaining the left thumb duration target ratio and the right thumb duration target ratio. The smaller of these two values ​​is then taken as the duration target ratio. If the duration target ratio is greater than 1, then the value of the duration target ratio is set to 1. The purpose of using a smaller value is to reflect the principle of the weakest link, that is, insufficient rubbing time on either thumb will reduce the overall duration evaluation result.

[0080] The overall coverage ratio refers to the degree to which both the left and right thumb coverage ratios meet the target coverage thresholds. It is obtained by calculating the ratios of the left thumb coverage ratio to the preset threshold and the right thumb coverage ratio to the preset threshold, respectively. The smaller of these two ratios is then taken as the overall coverage ratio. If the overall coverage ratio is greater than 1, it is set to 1. Using a smaller value aims to prevent sufficient coverage on one side from masking insufficient coverage on the other.

[0081] The standard execution status identification value is a numerical value used to comprehensively represent whether the thumb rubbing step meets the requirements of the aseptic operation standard in the operating room. It is obtained by weighting and summing the duration-to-standard ratio and the overall coverage ratio according to their respective duration and coverage influence weights.

[0082] The duration influence weight and coverage influence weight are weighted coefficients used to determine the degree of influence of the duration-to-standard ratio and the overall coverage ratio on the identification value of the standard implementation status. They are obtained as follows: historical handwashing videos and corresponding manual annotation results are read; the duration-to-standard ratio and overall coverage ratio of each sample are calculated; candidate duration influence weights are iterated between 0 and 1 with a preset step size; and the coverage influence weight is set to 1 minus the candidate duration influence weight. For each group of candidate weights, the consistency rate between the identification result and the manual annotation result is calculated; the group of candidate weights with the highest consistency rate is taken as the duration influence weight and coverage influence weight.

[0083] By generating duration-to-standard ratio and overall coverage ratio based on continuous rubbing duration and coverage compliance ratio, respectively, and employing the left-right thumb short plank principle, the system avoids situations where a better performance on one thumb masks insufficient rubbing on the other, ensuring the evaluation results better align with the standard requirement that both thumbs should be rubbed during preoperative surgical handwashing. Weighted joint analysis of duration-to-standard ratio and overall coverage ratio allows adjustment of the influence of duration and coverage adequacy on the standard compliance status identification value according to different operating room quality control requirements. This step transforms the previously obtained left and right thumb rubbing parameters into directly usable standard compliance status identification values, enhancing the correlation between image recognition results and aseptic technique standard evaluation.

[0084] Furthermore, the corresponding standard judgment results are obtained, specifically including: when the standard execution status identification value is above the preset qualified threshold, the standard judgment result is that the operation is in compliance; otherwise, it is that the operation is not in compliance. When the standard execution status identification value is less than the preset qualified threshold, the non-compliant parameters in the left thumb rubbing parameters and right thumb rubbing parameters are read. The non-compliant parameters include the continuous rubbing duration value of the left thumb or the continuous rubbing duration value of the right thumb which is less than the preset rubbing duration threshold, the coverage compliance ratio of the left thumb or the coverage compliance ratio of the right thumb which is less than the preset coverage compliance threshold, and the left thumb incomplete indicator or the right thumb incomplete indicator. When the non-compliant parameter includes the continuous rubbing duration value of the left thumb or the continuous rubbing duration value of the right thumb, an insufficient duration judgment result is generated. When the non-compliant parameter includes the coverage compliance ratio of the left thumb or the coverage compliance ratio of the right thumb, an insufficient coverage judgment result is generated. When the non-compliant parameter includes the left thumb completion status indicator or the right thumb completion status indicator, a unilateral incomplete judgment result is generated.

[0085] In this embodiment, the "non-compliant parameters" refer to the set of parameters used to locate the cause of non-compliance when the standard execution status identification value is less than a preset qualified threshold. The acquisition method is as follows: read the left thumb rubbing parameters and the right thumb rubbing parameters, and add the following to the non-compliant parameters: the continuous rubbing duration value of the left thumb or right thumb that is less than a preset rubbing duration threshold, the coverage compliance ratio of the left thumb or right thumb that is less than a preset coverage compliance threshold, and the left thumb incomplete indicator or right thumb incomplete indicator.

[0086] By comparing the standard execution status identification value with a preset pass threshold, the continuous quantity evaluation results obtained from previous calculations can be converted into standard judgment results that are easy to use for quality control management. By reading the non-compliant parameters and outputting the judgment results for insufficient duration, insufficient coverage, and unilateral incompleteness, the cause of non-compliance can be located while giving a pass or fail conclusion. A closed-loop output is achieved from image recognition data, kneading parameters, standard execution status identification values ​​to the final standard judgment result, enabling the execution status of aseptic operation standards in the operating room to not only be identified but also interpreted and traced, thereby improving the usability of image recognition results in operating room quality control scenarios.

[0087] like Figure 4As shown in the embodiment of this application, an image recognition system for the execution status of aseptic operation procedures in an operating room includes the following modules: an image recognition module, used to acquire video image data of the handwashing area, and perform hand region recognition to generate hand region recognition data, which includes the left palm region, right palm region, left thumb region, right thumb region, key points of the left wrist, and key points of the right wrist; and a contact analysis module, used to determine the right thumb wrapping parameters between the left palm region and the right thumb region, and the contact parameters between the right palm region and the left palm region, based on the hand region recognition data. The left thumb wrapping parameters between the thumb regions are combined with the right thumb wrapping parameters to obtain the contralateral contact matrix. The rubbing judgment module is used to perform continuous rubbing judgment based on the contralateral contact matrix to obtain the effective rubbing frames of the left and right thumbs, obtain the frame time identifier, and perform continuous segment analysis on the effective rubbing frames of the left and right thumbs respectively to generate the left thumb rubbing parameters and the right thumb rubbing parameters respectively. The standardization judgment module is used to perform compliance evaluation analysis based on the left thumb rubbing parameters and the right thumb rubbing parameters to obtain the standardization execution status identification value, thereby obtaining the corresponding standardization judgment result.

[0088] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. An image recognition method for the execution status of aseptic operation procedures in an operating room, characterized in that: Includes the following steps: Acquire video image data of the handwashing area, perform hand area recognition, and generate hand area recognition data. The hand area recognition data includes the left palm area, the right palm area, the left thumb area, the right thumb area, key points of the left wrist, and key points of the right wrist. Based on hand region recognition data, the right thumb wrapping parameter between the left palm region and the right thumb region, and the left thumb wrapping parameter between the right palm region and the left thumb region are determined. The opposite contact matrix is ​​obtained by combining the right thumb wrapping parameter and the left thumb wrapping parameter. Based on the contralateral contact matrix, the effective rubbing frames of the left and right thumbs are determined by continuous rubbing. The frame time identifier is obtained, and continuous segment analysis is performed on the effective rubbing frames of the left and right thumbs respectively to generate the rubbing parameters of the left and right thumbs respectively. Based on the left thumb rubbing parameters and the right thumb rubbing parameters, a compliance evaluation analysis is performed to obtain the standard execution status identification value, thereby obtaining the corresponding standard judgment result.

2. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 1, characterized in that: The generation of hand region recognition data specifically includes: Image frame data is extracted from video image data, and a frame time identifier is written for each handwashing image frame before hand target detection is performed to generate hand candidate box data; Based on the candidate box coordinates in the hand candidate box data, the image region located within the hand candidate box is extracted from the image frame data, thereby generating hand image data; Hand image data is processed by palm region segmentation, thumb region segmentation, and wrist key point localization to generate palm region segmentation data, thumb region segmentation data, and wrist key point localization data. Based on wrist key point localization data, left and right hand matching is performed on palm region segmentation data and thumb region segmentation data to generate hand region recognition data.

3. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 1, characterized in that: The determination of the right thumb wrapping parameters between the left palm area and the right thumb area, and the left thumb wrapping parameters between the right palm area and the left thumb area, specifically includes: Connectivity analysis is performed based on hand region recognition data. Boundary pixels located at the outer edge of each region are extracted and adjacent boundary pixels are connected sequentially to obtain the boundaries of each region. The boundaries of each region include the boundary of the left palm region, the boundary of the right palm region, the boundary of the left thumb region, and the boundary of the right thumb region. The boundary point distance between the left palm area and the right thumb area is processed by traversing the boundary point distance to obtain the minimum boundary point distance, which is used as the right thumb contact distance value. By analyzing the overlap of the left palm area and the right thumb area, the coverage ratio of the right thumb is obtained. The relative positional change of the center point of the left palm region and the center point of the right thumb region in consecutive image frames is analyzed to obtain the relative displacement of the right thumb. The right thumb contact distance, right thumb coverage ratio, and right thumb relative displacement are combined as the right thumb wrapping parameters. The boundary point distance between the right palm area and the left thumb area is processed by iterating through the boundary points to obtain the minimum boundary point distance, which is used as the contact distance value of the left thumb. By performing an area overlap analysis between the right palm region and the left thumb region, the ratio of the image projection area corresponding to the right palm region to the image projection area corresponding to the left thumb region is obtained, which is used as the left thumb coverage ratio. The relative positional change of the center point of the right palm region and the center point of the left thumb region in consecutive image frames is analyzed to obtain the positional change of the right palm region relative to the left thumb region, which is used as the relative displacement of the left thumb. The left thumb contact distance, left thumb coverage ratio, and left thumb relative displacement are combined as the left thumb wrapping parameters.

4. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 1, characterized in that: The process of obtaining the opposite-side contact matrix specifically includes: Obtain the frame time identifier corresponding to each image frame in the video image data; According to the frame time identifier, the right thumb contact distance value, right thumb coverage ratio, right thumb relative displacement, left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement corresponding to the same image frame are used to form a single frame opposite-side contact vector; Arrange multiple single-frame opposite-side contact vectors in chronological order to obtain an opposite-side contact matrix. The opposite-side contact matrix is ​​used to characterize the temporal wrap-around contact state between the left palm region and the right thumb region, and between the right palm region and the left thumb region.

5. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 1, characterized in that: The step of performing continuous segment analysis to generate left thumb rubbing parameters and right thumb rubbing parameters specifically includes: Read the right thumb contact distance value, right thumb coverage ratio, and right thumb relative displacement from the opposite contact matrix; When the right thumb contact distance value in a certain image frame is below the preset contact distance threshold, the right thumb coverage ratio is above the preset coverage ratio threshold, and the right thumb relative displacement is above the preset displacement threshold, the image frame is marked as a valid right thumb rubbing frame. Read the left thumb contact distance value, left thumb coverage ratio, and left thumb relative displacement from the opposite contact matrix; When the left thumb contact distance value in a certain image frame is below the preset contact distance threshold, the left thumb coverage ratio is above the preset coverage ratio threshold, and the left thumb relative displacement is above the preset displacement threshold, then the image frame is marked as a valid left thumb rubbing frame. Continuous segment analysis is performed on the effective rubbing frames of the right thumb to obtain continuous rubbing segments of the right thumb, and continuous segment analysis is performed on the effective rubbing frames of the left thumb to obtain continuous rubbing segments of the left thumb; Right thumb rubbing parameters are generated based on continuous rubbing segments of the right thumb, and left thumb rubbing parameters are generated based on continuous rubbing segments of the left thumb. The right thumb rubbing parameters include the continuous rubbing duration of the right thumb, the coverage ratio of the right thumb, and the completion status indicator of the right thumb; the left thumb rubbing parameters include the continuous rubbing duration of the left thumb, the coverage ratio of the left thumb, and the completion status indicator of the left thumb. The right thumb completion status indicator includes a right thumb completion indicator and a right thumb incomplete indicator, and the left thumb completion status indicator includes a left thumb completion indicator and a left thumb incomplete indicator.

6. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 5, characterized in that: The specific methods for obtaining the continuous rubbing duration of the right thumb, the coverage ratio of the right thumb, and the completion status indicator of the right thumb are as follows: The effective rubbing frames of the right thumb are divided into continuous segments according to the frame time identifier to obtain the effective rubbing segments of the right thumb. When there is only one effective kneading segment of the right thumb, then the effective kneading segment of the right thumb is taken as the target effective kneading segment of the right thumb; otherwise, the duration of each effective kneading segment of the right thumb is counted, and the effective kneading segment of the right thumb with the longest duration is taken as the target effective kneading segment of the right thumb. Based on the frame time identifier corresponding to the effective rubbing segment of the target right thumb, the continuous rubbing duration of the right thumb is calculated. The number of image frames in the effective rubbing segment of the target right thumb with a right thumb coverage ratio above the preset coverage ratio threshold is counted and recorded as the number of right thumb coverage qualified frames; The proportion of right thumb coverage meeting the standard is obtained by taking the proportion of right thumb coverage meeting the standard out of the total number of image frames in the effective rubbing segment of the target right thumb. When the duration of continuous rubbing with the right thumb is above the preset rubbing duration threshold and the coverage ratio of the right thumb is above the preset coverage threshold, a right thumb completion status indicator is generated; otherwise, a right thumb incomplete status indicator is generated. The duration of continuous rubbing with the right thumb, the coverage rate of the right thumb, and the completion status indicator of the right thumb are combined as parameters for the right thumb rubbing.

7. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 5, characterized in that: The continuous kneading determination also includes: The effective rubbing frames of the right thumb and the effective rubbing frames of the left thumb are spliced ​​at intervals respectively. The effective rubbing frames of the right thumb are sorted in time according to the frame time identifier, and the effective frames preceding the right thumb and the effective frames following the effective frames preceding the right thumb are determined in sequence. When there is an image frame that is not marked as a valid rubbing frame of the right thumb between the preceding valid frame and the following valid frame of the right thumb, then the image frame is regarded as a candidate interval frame of the right thumb. The number of consecutive right thumb candidate interval frames is counted to obtain the number of consecutive right thumb interval frames. If the number of consecutive right thumb interval frames is less than the preset threshold for the number of consecutive interval frames, the preceding valid right thumb frame, the consecutive right thumb candidate interval frames, and the following valid right thumb frame are combined and spliced ​​to obtain a continuous right thumb rubbing segment. If the number of consecutive interval frames of the right thumb exceeds the preset threshold for consecutive interval frames, then joint splicing will not be performed, and the valid frames before and after the right thumb will be divided into different consecutive rubbing segments of the right thumb. Effective frame analysis of the effective rubbing frames of the left thumb yields the continuous rubbing segments of the left thumb; Right thumb rubbing parameters are generated based on continuous rubbing segments of the right thumb, and left thumb rubbing parameters are generated based on continuous rubbing segments of the left thumb.

8. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 5, characterized in that: The obtained standardized execution state identification value specifically includes: Based on the analysis of the continuous rubbing duration values ​​of the left thumb and the right thumb, the duration target ratio is obtained. The duration target ratio is used to characterize the degree to which the continuous rubbing duration values ​​of the left thumb and the right thumb meet the preset rubbing duration threshold. Based on the analysis of the coverage achievement ratio of the left thumb and the coverage achievement ratio of the right thumb, a comprehensive coverage ratio is obtained. The comprehensive coverage ratio is used to characterize the degree of common achievement of the coverage achievement ratio of the left thumb and the coverage achievement ratio of the right thumb relative to the preset coverage achievement threshold. A joint analysis based on the duration of the standard and the comprehensive coverage ratio yields the identification value of the standard execution status.

9. The image recognition method for the execution status of aseptic operation procedures in the operating room according to claim 1, characterized in that: The specific steps to obtain the corresponding standard determination result include: When the standard execution status identification value is above the preset qualified threshold, the standard judgment result is that the operation is in compliance; otherwise, it is that the operation is not in compliance. When the standard execution status identification value is less than the preset qualified threshold, read the non-compliant parameters in the left thumb rubbing parameters and right thumb rubbing parameters. The non-compliant parameters include the left thumb continuous rubbing duration value or the right thumb continuous rubbing duration value which is less than the preset rubbing duration threshold, the left thumb coverage compliance ratio or the right thumb coverage compliance ratio which is less than the preset coverage compliance threshold, and the left thumb incomplete indicator or the right thumb incomplete indicator. When the parameters that do not meet the standard include the duration of continuous rubbing with the left thumb or the duration of continuous rubbing with the right thumb, an insufficient duration judgment result is generated; When the parameters that do not meet the standards include the coverage ratio of the left thumb or the coverage ratio of the right thumb, an insufficient coverage judgment result is generated; When the unmet parameter includes either the left thumb completion status indicator or the right thumb completion status indicator, a unilateral uncompleted judgment result is generated.

10. A system for image recognition of the operating room aseptic operation procedure execution status as described in any one of claims 1-9, characterized in that: Includes the following modules: The image recognition module is used to acquire video image data of the handwashing area, perform hand area recognition, and generate hand area recognition data. The hand area recognition data includes the left palm area, the right palm area, the left thumb area, the right thumb area, key points of the left wrist, and key points of the right wrist. The contact analysis module is used to determine the right thumb wrapping parameter between the left palm area and the right thumb area, and the left thumb wrapping parameter between the right palm area and the left thumb area, based on hand area recognition data. The right thumb wrapping parameter and the left thumb wrapping parameter are combined to obtain the opposite-side contact matrix. The rubbing determination module is used to determine the effective rubbing frames of the left thumb and right thumb based on the opposite contact matrix, obtain the frame time identifier, and perform continuous segment analysis on the effective rubbing frames of the left thumb and right thumb respectively to generate the left thumb rubbing parameters and right thumb rubbing parameters respectively. The standard determination module is used to perform compliance evaluation analysis based on the left thumb rubbing parameters and the right thumb rubbing parameters, obtain the standard execution status identification value, and thus obtain the corresponding standard determination result.

Images