Method and system for analyzing surgical textile and computer-readable medium

Abstract

To provide a system and method for detecting, counting and analyzing ingredients of blood of a surgical textile.SOLUTION: A system for depth imaging 122, for measuring ingredients of a blood, comprises a color camera system 308 for acquiring a color image, an IR camera 304 for acquiring an infrared image, and an internal processor 306 that identifies a shape of a surgical textile using the infrared image, generates shape information from the infrared image, identifies a surgical-textile pixel in the color image using the shape information and extracts information of blood from the surgical textile pixel.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 62 / 387,222, filed on December 23, 2015. This application is also related to U.S. Patent Application No. 13 / 544,664, filed on July 9, 2012. This document is hereby incorporated by reference in its entirety. 13 / 544,664. This document is hereby incorporated by reference in its entirety.

Background Art

[0002] The present invention generally relates to the field of blood loss management, and more specifically, to a method for evaluating the amount of blood components in a surgical fabric in the field of blood loss management, which is novel and useful.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present invention generally relates to the field of blood loss management, and more specifically, to a method for evaluating the amount of blood components in a surgical fabric in the field of blood loss management, which is novel and useful. Systems and methods are provided for detecting, counting, and analyzing the blood content of a surgical fabric, and an infrared camera or a depth camera is utilized with respect to a color image.

Means for Solving the Problems

[0004] In one embodiment, a system for measuring the blood content is provided, including a color imaging system configured to acquire a color image, an infrared imaging system configured to acquire an infrared image, and a processor that uses the infrared image to identify the shape of the surgical fabric, generates shape information from the infrared image, and uses the shape information to identify surgical fabric pixels in the color image. ​​​​​​​A processor configured to identify and extract information about blood from surgical fabric pixels, is provided. The processor converts an infrared image into a coordinate system corresponding to a color image, and generates a depth image from one of the infrared images, identifies a set of flat pixels in the depth image, and / or is further configured to detect the outer perimeter of an image of a surgical fabric in the depth image. The detection of the outer perimeter of the image of the surgical fabric may be performed by looking for a sharp vertical depth drop in the depth image. The processor may further be configured to downsample the depth image or the infrared image, generate a mask image from the downsampled depth image, upsample the mask image, perform filling of missing infrared pixel regions in the infrared image, generate a mask image using the detected outer perimeter, and / or transform the mask image into the coordinate system of the color image. The detected outer perimeter may be the top and bottom edges of the image of the surgical fabric, or the edges on both sides of the image of the surgical fabric, and / or the corners of the image of the surgical fabric. In another example, a method for analyzing a subject containing blood components is provided, including obtaining a depth image and a color image of the subject containing blood components, determining the presence of the subject in the field of view using the depth image, and evaluating the characteristics of the blood components using the color image. The method may further include generating an image mask using the depth image and applying the image mask to the color image to remove background pixels. The method may further include applying at least one of erosion and dilation image processing to the depth image. The processor downsamples the depth image or the infrared image, generates a mask image from the downsampled depth image, upsamples the mask image, performs filling of missing infrared pixel regions in the infrared image, generates a mask image using the detected outer perimeter, and / or transforms the mask image into the coordinate system of the color image. The detected outer perimeter may be the top and bottom edges of the image of the surgical fabric, or the edges on both sides of the image of the surgical fabric, and / or the corners of the image of the surgical fabric. The method may further include generating an image mask using the depth image and applying the image mask to the color image to remove background pixels. The method may further include applying at least one of erosion and dilation image processing to the depth image. The detected outer perimeter may be the top and bottom edges of the image of the surgical fabric, or the edges on both sides of the image of the surgical fabric, and / or the corners of the image of the surgical fabric. In another example, a method for analyzing a subject containing blood components is provided, including obtaining a depth image and a color image of the subject containing blood components, determining the presence of the subject in the field of view using the depth image, and evaluating the characteristics of the blood components using the color image.

[0005] The method may further include generating an image mask using the depth image and applying the image mask to the color image to remove background pixels. The method may further include applying at least one of erosion and dilation image processing to the depth image. The method may further include generating an image mask using the depth image and applying the image mask to the color image to remove background pixels. The method may further include applying at least one of erosion and dilation image processing to the depth image. The method may further include generating an image mask using the depth image and applying the image mask to the color image to remove background pixels. The method may further include applying at least one of erosion and dilation image processing to the depth image. The method may further include applying at least one of erosion and dilation image processing to the depth image.

[0006] In yet another embodiment, a method for evaluating the amount of fluid components in a surgical fabric is provided, including, at a first time, obtaining a first infrared image of the field of view and detecting the shape or surface characteristics of the surgical fabric in the first infrared image, and when the shape or surface characteristics in the first infrared image are detected, indexing a frame counter, and at a second time following the first time, obtaining a second infrared image of the field of view and detecting the surface characteristics of the surgical fabric in the second infrared image, and when the shape or surface characteristics in the second infrared image are detected, indexing the frame counter, and when the shape or surface characteristics of the surgical fabric in the second infrared image are not detected, indexing a non-detection counter. The method may further include clearing the frame counter and the non-detection counter when a pre-specified value is reached. The method may further include comparing the frame counter to a threshold value, and when the frame counter may be above the threshold value, obtaining a color image of the field of view and converting the region of the color image corresponding to the surgical fabric for evaluation of the amount of blood components, and clearing the frame counter. The method may further include generating an image mask using the second infrared image and using the image mask on the color image to identify the region of the color image corresponding to the surgical fabric.

[0007] In another embodiment, a method for performing fluid monitoring is provided, including obtaining an infrared image of the field of view and obtaining a color image of the field of view, generating an image mask using the infrared image, and using the image mask on the color image to identify the region of the color image corresponding to the surgical fabric. including the steps of converting an infrared image into a geometric perspective view of a color image. The method may further include performing a low-resolution process on the infrared image before generating an image mask. The method may further include performing a high-resolution process on the image mask before identifying regions of the color image using the image mask.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0009] Overestimation and underestimation of a patient's blood loss are major factors that increase the costs of surgery and other surgical medical procedures in hospitals, clinics, and other medical facilities. Specifically, overestimation of a patient's blood loss results in wasting blood of a transfusion grade and increasing the surgical costs of medical institutions. Also, blood ​​​ can lead to a shortage. Under - estimating a patient's blood loss is a significant factor in the delay of resuscitation and blood transfusion when significant bleeding is occurring, and is associated with infections, readmissions, and litigation every year for the majority of patients that could have been avoided. For this reason, in the field of surgery, there is a need for new and useful systems and methods for evaluating the volume of blood outside the body within a physical sample.

[0010] The systems and methods described herein utilize a color camera system to acquire images of surgical fabrics or other operating room surfaces and to evaluate the blood content within each image. Additionally, the system can track multiple surgical fabrics used during a surgery to assist surgical or operating room staff in verifying that all surgical fabrics are present and not left inside the patient before surgical closure. Surgical fabrics can include absorbent surgical gauze sponges, surgical clothing, surgical towels, surgical gowns, scrubs or other apparel, tables, beds or bed sheets, walls, floor surfaces, external skin surfaces, surgical gloves, surgical instruments, or any other surface, material, substrate, or item such as a target.

[0011] To facilitate the presentation of an object to a color camera system for analysis and / or counting, an infrared or other type of camera configured to provide depth information can be used to detect the presentation of the object for analysis. Depth information from such a system can more easily distinguish the object located in the foreground from other objects that will be in the background and are not intended to be analyzed. Examples of optical systems capable of providing depth information include range - gated time - of - flight (ToF) cameras, RF - modulated ToF cameras , including pulsed light ToF, and projection light stereo cameras, etc.

[0012] To improve the accuracy of the detection process, other features of the object can also be detected to reduce the risk of false positive results. The reliability of the detection process can be improved by identifying the presence of the object in multiple consecutive frames before analyzing the object and providing a result.

[0013] 1. Use Generally, as shown in FIG. 1, the various methods described herein are executed by a computer device 100 to use an optical sensor system 104 to obtain one or more images 102 and detect a surgical fabric 106a such as a surgical gauze, a surgical sponge, or a surgical towel within a field of view 108 as shown in S110. In S111, the image 102 is analyzed to confirm the presence of the surgical fabric (image) 106b over the seek of the frames output by the optical sensor system in S111. When its presence is confirmed, in S125, a color image 110 of the surgical fabric 106b is automatically obtained or the last color image currently secured in the image 102 is utilized. The computer device 100 then indexes a local surgical fabric counter 112 in S150 and locally converts the color values of the pixels of the color image 110 for evaluation of the amount of blood components (e.g., hemoglobin mass, total blood volume) within the surgical fabric 106b in S140, or the color image 110 can be passed to a distal computer system for such processing.

[0014] ​​​​​​​​​​​​​​​In particular, the computing device 100 may detect, in a first frame (e.g., an infrared image), Detecting the possible surgical fabric images 106 and calculating the frame count with respect to block S112. The data 114 is indexed and the possible surgical fabric 106 is detected again in the next frame. If so, the frame counter 114 is reindexed or the surgical fabric image 106b is next If not detected in the next frame, the frame counter 114 is cleared in block S130. A possible surgical fabric 106b is detected in the next frame. This process continues for as long as and until the frame counter 114 contains a value equal to a certain threshold 116. The process is repeated and the frame counter 114 is indexed. If the color image 116 includes a value equal to the value 116, the computing device 10 and the color image 110 is processed to identify the blood constituents ( For example, the content of hemoglobin is determined, and / or in block S150, the detection With respect to the surgical fabric 106 that has been processed, the fabric counter 112 can be updated. For example, the computing device 100 may determine that a surgical fabric 106 corresponds to 7. Meeting a threshold value of "7" 116 that correlates to 7 consecutive frames that contain the specified region. In addition, the computer device 100 detects that the frame counter 114 has a value of "7". 10, a digital color photographic image 110 of the surgical fabric 106b is obtained, and the image 110 to determine the total mass or volume of hemoglobin or blood components within the surgical fabric 106. In other embodiments, the threshold value can be determined from about 3 to about 100, or from about 5 to about 100. It may be set in the range of about 70, or from about 30 to about 50, or from about 8 to about 20. Tracking the surgical fabric 106 through the sequence of the arm, and by only confirming the presence of the surgical fabric 106b when the surgical fabric 106b is detected within the minimum number of consecutive frames, the computer device 100 may minimize or eliminate the confirmation of false positives of the surgical fabric 106b, thereby minimizing or eliminating false fabric counter events, as well as the automated collection and processing of images without surgical fabric. This tracking process may also improve computational efficiency by reducing the need to perform determinations of blood content for all color images.

[0015] In a further embodiment, the computer device 100 may also be configured to automatically detect the surgical fabric 106a within its field of view 108 (i.e., within the field of view of one or more optical sensors of the optical sensor system 104 incorporated and / or connected to the computer device 100), thereby reducing or eliminating the need for the user to manually trigger the acquisition of a surgical fabric containing blood by depressing a foot pedal, touch control, gesture control, or voice control. Further, the computer system 100 may provide real-time visual feedback, such as a visible indicator that the surgical fabric 106a has been detected within the field of view 108 of the computer device 100, that an image of the surgical fabric has been obtained, and / or a prompt to manipulate the surgical fabric 106a to provide a sufficient or high-quality image, through the incorporated or connected display 118.

[0016] The methods described herein may be used in an operating room, transport room, ambulance, or any other medical or​​​​​​​​​​​​​​​ A computer device installed or arranged within an emergency response space or unit 100 can be locally executed. However, this method can also be executed locally and / or distally by a distal server or the like. The blocks of this method are described herein as the detection of surgical sponges, as well as the acquisition and processing of images of surgical sponges. However, the blocks of this method can additionally or alternatively be implemented to detect surgical gauze, surgical towels, or any other fabric and acquire and process images of such fabric. However, the blocks of this method can additionally or alternatively be implemented to detect surgical gauze, surgical towels, or any other fabric and acquire and process images of such fabric. can be implemented.

[0017] 2. Exemplary Method As shown in FIG. 1, a schematic method S100 for evaluating the amount of blood components in a surgical fabric at a first time point includes, at block S110, using an optical system 104 to acquire a first infrared or depth image 102 of a field of view 108. In this particular embodiment, the optical system 104 includes an embedded display 118 and a computer device 100 having a color camera or color imaging system 120 on the same side as the display 118 of the computer device. However, in other embodiments, the computer device 100 may be configured to use a camera on the opposite side of the display 118. To acquire the depth image 102, an infrared camera or depth imaging system 122 is directly attached to the computer device 100. However, in other embodiments, the depth imaging system 122 may be attached at a position spaced apart from the computer device 100. In some variations, the infrared or depth camera 120 is a specific computer To acquire the depth image 102, an infrared camera or depth imaging system 122 is directly attached to the computer device 100. However, in other embodiments, the depth imaging system 122 may be attached at a position spaced apart from the computer device 100. In some variations, the infrared or depth camera 120 is a specific computer In some variations, the infrared or depth camera 120 is a specific computer configured to be attached to the data device 100, whereby the optical axis of this second camera 121 is vertically and parallel to the optical axis of the color camera 120. However, in other embodiments, the optical axis of the second camera 121 may be aligned with the horizontal axis of the color camera 120, or may not be aligned with any axis of the color camera. The second camera 122 may alternatively be configured to intersect the optical axis of the color camera 120 at a distance within a range of about 15 cm to 200 cm, or about 30 cm to about 100 cm, from the color camera 120.

[0018] The image 102 is then analyzed at S111 to detect surface or shape characteristics corresponding to the surgical fabric 106b of the first infrared image 102. If the presence of the surgical fabric 106b is confirmed, the frame counter 114 is indexed at S112. At a second or subsequent time point following the first time point, a second or subsequent infrared image 102 of the field of view 108 is acquired, and the frame counter 114 is indexed again in response to the detection of surface or shape characteristics of the surgical fabric 106b in the second or subsequent infrared image 102. This detection is repeated until the threshold 116 is reached or until the surface or shape characteristics of the surgical fabric cannot be detected in the second or subsequent infrared image, at which point the frame counter is cleared at S130. At S113, if the frame counter contains a value equal to the threshold, the color image 110 of the field of view 108 may be acquired by the color camera 120. In a variant where the color image is acquired with an infrared ​The last color image 110 obtained therefrom can be used. Surgical fabric 106b The regions of the color image 110 corresponding to are identified, and then these regions are further analyzed to identify portions containing blood These portions are then converted at block S140 to an assessment of the amount of blood components. For this purpose, the system clears the frame counter at S130

[0019] A further variant of the method is also shown in FIG. 1. For example, at a first point in time, when a first image 102 of the field of view 100 is obtained by the optical system 104 at S110 the rendered form of the first image 102 can be provided to the display 118 at S120 This displayed image 102 can correspond to a depth image or a color image. In response to detecting the first surface or shape characteristics of the surgical fabric 106b within the first image 102 the frame counter 114 is indexed at S112 and at the display 118, at block S122, the region of the first image 102 corresponding to the first surface or shape of the surgical fabric 106b is highlighted This variant of the method can also include, at a second or subsequent point in time following the first point in time, obtaining a second or subsequent image 102 of the field of view 108 and rendering the form of the second or subsequent image 102 on the display 118 And in response to detecting the second surface or shape characteristics of the surgical fabric 106b within the second image 102 the frame counter 114 is indexed at S112 and at the display 118, at block S122, based on the value of the frame counter 114, the region of the second image 102 corresponding to the second surface or shape is highlighted. The second or subsequent image surface or shape​​​​​​​ The detection of the second surface or the shape characteristics of the surgical fabric 106b in image 102 was not detected In this case, the frame counter 114 is cleared at S130, or the frame counter 114 contains a value equal to the threshold 116, and in block S150, the fabric counter 112 is indexed and the frame counter 114 is cleared. In various surgeries or surgical treatments using different surgical fabrics, multiple surgical fabric counters may be provided, and the corresponding counter for surgical fabric 1 06 can be indexed at S150.

[0020] 3. System As shown in FIG. 1, the computer device 100 includes an imaging system 104 configured to define a field of view 108 and acquire depth and color data from that field of view 1 08. Specifically, the computer device 100 may include a depth imaging camera or system 122 that outputs depth information or a map 102 of the field of view 108, and a color imaging camera or system 120 that outputs a color image 11 0 of the field of view 108.

[0021] As shown in FIG. 3, in one embodiment, the depth imaging system 122 defines a projection field of view and has a housing 300 with a structure of an infrared ( "IR") light projector 302 that projects infrared light in a dot pattern onto the projection field of view , defines an infrared field of view that is substantially the same as the projection field of view, and is configured to acquire a sequence of infrared images (or "infrared frames") of the infrared field of view. The IR camera 304, and based on, for example, the density or displacement of dots across each infrared image, the distorted dot pattern detected by the IR camera 304 recorded in each infrared image including an internal processor 306 configured to convert it into a depth map 102 . The IR projector may comprise an IR laser or diode. U.S. Patent Application No. 8,3 74,397 is an example of this IR depth technology. This document is incorporated herein by reference in its entirety . Thus, during operation of the computer device 100, the depth imaging system can output a sequence of depth maps 102, each depth map 1 02 including an indication of the distance between the IR camera 122 and the surface within the infrared field of view at a corresponding time . Of course, in addition to IR-based cameras, other types of cameras of similar construction, such as, but not limited to, stereo cameras in which stereoscopic triangulation can be used with a pair of stereo images to determine distance, camera focus depth, and other known types of depth cameras may also be used. In some variations, the depth imaging system 12 2 may include a color camera system 308 that is used in place of the color camera system internal to the computer device . The color camera system may include a color camera 120, such as a CCD camera or a CMOS camera that supports red, green, and blue color channels . The color camera 120 may define a color field of view that is substantially similar (i.e., substantially overlapping) to the infrared field of view (and thus the projection field of view) of the IR camera 122

[0022] . As described below, the computer device 100 performs the method at a rate of 30 Hz, for example, during each sampling period while generating the depth map 102 and capturing the color image 110 . . The color camera 120 may define a color field of view that is substantially similar (i.e., substantially overlapping) to the infrared field of view (and thus the projection field of view) of the IR camera 122 (i.e., substantially overlapping). The computer device 100 may perform the method at a rate of 30 Hz, for example, during each sampling period while generating the depth map 102 and capturing the color image 110, as described below . The computer device 100, during each sampling period while performing the method at a rate of 30 Hz, for example, generates the depth map 102 and captures the color image 110 . such that the depth imaging system 122 and the color imaging system 120 are each triggered, i.e., can be activated. In other embodiments, the sampling rate of the camera(s) may be in the range of from about 0.1 Hz to about 1 kHz, or from about 60 Hz to about 240 kHz. For this reason, the computer device 100 can acquire one depth map 102 and one color image 110 (a "frame pair") during each sampling period, and for each sampling period, process both the depth data and the color data to detect the presence of a surgical sponge or other surgical fabric 106 within the field of view of the computer device 100 at the corresponding time. Alternatively, the computer device 100 can sample the depth imaging system and process the depth map data to detect a surgical sponge or surgical fabric within the field of view 108 of the computer device 100. Also, the computer system 100 can acquire a color image 110 through the color imaging system when the value within the frame counter 114 reaches a threshold 116, as described below. In still other embodiments, the ratio of the depth image 102 to the color image 110 may be fixed or predetermined, and may be from about 3:2 to about 100:1, or from about 2:1 to about 50:1, or from about 30:1 to about 50:1. The computer device 100 may also include a display 118 adjacent to the IR camera 122 and the color camera 120. The display 118 can define a visual area that is substantially perpendicular to the fields of view of the IR camera and the color camera, thereby from about 0.1 Hz to about 1 kHz, or from about 60 Hz to about 240 kHz For this reason, the computer device 100 can, during each sampling period, acquire one depth map 102 and one color image 110 (a "frame pair"), and for each sampling period, process both the depth data and the color data to detect the presence of a surgical sponge or other surgical fabric 106 within the field of view of the computer device 100 at the corresponding time. during each sampling period, acquire one depth map 102 and one color image 110 (a "frame pair") For each sampling period, process both the depth data and the color data to detect the presence of a surgical sponge or other surgical fabric 106 within the field of view of the computer device 100 at the corresponding time. For each sampling period, process both the depth data and the color data to detect the presence of a surgical sponge or other surgical fabric 106 within the field of view of the computer device 100 at the corresponding time. In an alternative embodiment, the computer device 100 can sample the depth imaging system and process the depth map data to detect a surgical sponge or surgical fabric within the field of view 108 of the computer device 100. The computer device 100 can sample the depth imaging system and process the depth map data to detect a surgical sponge or surgical fabric within the field of view 108 of the computer device 100. The computer device 100 can sample the depth imaging system and process the depth map data to detect a surgical sponge or surgical fabric within the field of view 108 of the computer device 100. The computer system 100 can also acquire a color image 110 through the color imaging system when the value within the frame counter 114 reaches a threshold 116, as described below. The computer system 100 can also acquire a color image 110 through the color imaging system when the value within the frame counter 114 reaches a threshold 116, as described below. The computer system 100 can also acquire a color image 110 through the color imaging system when the value within the frame counter 114 reaches a threshold 116, as described below. In still other embodiments, the ratio of the depth image 102 to the color image 110 may be fixed or predetermined, and may be from about 3:2 to about 100:1, or from about 2:1 to about 50:1, or from about 30:1 to about 50:1. In still other embodiments, the ratio of the depth image 102 to the color image 110 may be fixed or predetermined, and may be from about 3:2 to about 100:1, or from about 2:1 to about 50:1, or from about 30:1 to about 50:1.

[0023] The computer device 100 may also include a display 118 adjacent to the IR camera 122 and the color camera 120. The computer device 100 may also include a display 118 adjacent to the IR camera 122 and the color camera 120. The display 118 can define a visual area that is substantially perpendicular to the fields of view of the IR camera and the color camera, thereby A user who holds the ji (e.g., a nurse, a technician, an anesthesiologist) can stand in front of the IR camera 122 and the color camera 120 while viewing the content of the display 118. For example, as described below, the display 118 can render one or more arbitrary things, such as a) an indicator that a sponge has been detected, b) an indicator that an image of the sponge has been recorded, c) the value of the sponge counter, and d) a prompt to handle the sponge before an image is acquired, in substantially real time. In one embodiment, the computer device includes a tablet computer with a front-facing color camera, an integrated display, and an IR projector and an IR camera attached to the tablet and facing outward from the tablet, defining a projection field of view and an infrared field of view that are substantially similar to the color field of view of the color camera. However, the computer device can include any other optical, imaging, and / or processing components that are built-in or connected and configured in any other suitable way. 4. Exemplary user experience In one embodiment, the computer device 100 is installed in the operating room and executes the method shown in FIG. 1 in the form of an original blood loss monitoring application used during surgery, surgical treatment, or a procedure. In this embodiment, the computer device 100 continuously acquires infrared images 102 and / or color images 110 in its field of view 108 and discards these images if no surgical sponge 106a is detected within the image or frame.

[0024] ​​​​​​​​​​​​The computer device 100 may render these frames or images of the display 118 in near real time. The rendered image is usually a color image 1 10, but may also be a depth image 102 or a composite image of a color image and a depth image. During operation, a user (e.g., a nurse, technician, anesthesiologist) may hold the surgical sponge 106a in front of the imaging system 1 04. Here, when the surgical sponge 106a is within the field of view 108 of the imaging system 104, the imaging system 104 acquires a first depth map 102 and / or a first color image 110 (a "first frame pair", or a "frame set" if three or more images or maps are grouped) during a first sampling period, and then, at S110, the computer device 10 0 tracks the presence of the surgical sponge 106a within the field of view 108. The computer device 10 0 then implements the machine vision techniques described herein to associate the region of the first depth map 102 (and the corresponding region of the first color image 110) with the surgical sponge 106a, and, at S112, increments the frame counter 114 from "0" to "1". In response to detecting the image 106b of the surgical sponge within the first depth map 102 (or the first color image 110), the computer device 100 renders the first infrared image 102 (or the first color image 110) on the display 118 with a first-width, colored (e.g., green, red, yellow, or another distinct color) line or shape overlay 400 along the perimeter of the detected surgical sponge, and / or with a colored overlay area of a first opaque portion over the detected area of the surgical sponge. ​

[0025] During successive sampling periods, the imaging system 104 acquires a second depth map 102 and / or a second color image 110 (the "second frame pair" or "second frame set"), and the computer device 100 again implements machine vision techniques to correlate an area of the second depth map 102 (and the corresponding area of the first color image 110 ) with the surgical sponge 106a, incrementing the frame counter 11 4 from "1" to "2". In response to detecting an image 106b of the surgical sponge within the second frame pair, the computer device 100 renders the second infrared image 102 on the display with a wider area color-coded line or shape of a second width or line length along the perimeter of the detected surgical sponge 106b - greater than or different from the first width or line length - and / or an overlaid area color-coded with a second opaque portion color - greater than or different from the first opaque portion - over the detection area of the surgical sponge 106b.

[0026] The computer device 100 acquires frame pairs during successive sampling periods, detects the surgical sponge 106b within the frame pairs, and in response increments the frame counter 114, and can repeat this process to refresh the display 118 with the current depth map 102 (or color image 110), with the line widths 400, 402, and opaque portions of the overlay perimeter and / or overlay area, respectively, indicating the value of the frame counter. The surgical fabric 106 in the images 102, 110 ​​​​​To dynamically demonstrate the process of confirming the presence of b, a larger overlay width, or other changing signs, may be provided. For example, the color of the overlay may change from red to yellow and then to green. When the value on the frame counter 114 reaches a threshold value (e.g., "7" ), the computer device acquires a color image through the color camera, renders a visual indicator that the color image has been acquired on the display, and resets the frame counter to "0" without additional manual input. For example, for an imaging system operating at 3 0 Hz and with the frame counter set to "7", regarding the preset threshold, the computer device implements the above-described methods and techniques to detect, confirm, and acquire an image of the surgical sponge within less than 0.3 seconds after the surgical sponge enters the field of view of the computer device. In other embodiments, the threshold value may be set in a range of about 5 to 100, or about 5 to 50, or about 4 to about 30, or about 20 to about 40. The computer device may be further configured to perform an analysis regarding the movement of the surgical fabric 106a from the field of view 108 as shown in S500 and FIG. 5. There are also cases. The computer device 100 may be configured to implement the methods and techniques described in U.S. Patent Application No. 13 / 544,664 to convert the color values of the pixels in the image into an evaluation amount of the blood components in the surgical sponge imaged in block S140. Depending on the confirmation of the surgical sponge 106b within the field of view 108 of the computer device 100 and the acquisition of the color image 110 of the surgical sponge 106a, the computer device 1

[0027] and the acquisition of the color image 110 of the surgical sponge 106a, the computer device 1 and techniques to convert the color values of the pixels in the image into an evaluation amount of the blood components in the surgical sponge imaged in block S140. Depending on the confirmation of the surgical sponge 106b within the field of view 108 of the computer device 100 and the acquisition of the color image 110 of the surgical sponge 106a, the computer device 1 In response to the confirmation of the surgical sponge 106b within the field of view 108 of the computer device 100 and and the acquisition of the color image 110 of the surgical sponge 106a, the computer device 1 00 maintains the total count of surgical sponges passing through the computer device 100 during surgery or a procedure. In block S150, the counter 112 of the sponge or surgical fabric may also be indexed to maintain the count.

[0028] 5. Identifier Block S110 of the method shows obtaining a first image 102 of the field of view 108 at a first point in time. Generally, during each sampling period, the computer device 100 acts to obtain an infrared image 102 and / or a color image 110 through the imaging system 104 in S110. As described above, the computer device 100 may convert the infrared image 102 into a depth map of its field of view or extract a depth map therefrom. The computer device 100 can repeat this process over time at a sampling rate (or "frame rate") of 30 Hz or other sampling rates described herein during its operation. However, the computer device can obtain depth data and color data of its field of view in any other way and at any other appropriate sampling rate through any other type of sensor in block S110.

[0029] After obtaining the image(s) 102 and / or 110, in S111, the computer device 100 analyzes the image(s) 102 and / or 110 to determine the presence of one or more internal image characteristics therein. Or, the image(s) 102 and / or 110 obtained during the current sampling period are compared with the sponge or surgical fabric 106b. ​​​​​​​​​​​​​​Identify it differently as either including or not including the sponge or surgical fabric 106b. Here Based on this determination, the computer device 100 then, respectively, indexes the frame counter in S112 or clears the frame counter in S130 if there is such a case. Specifically, as shown in FIGS. 1 and 2, the computer device 100 applies one or more discriminators obtained during the sampling period to the depth map 102 and / or color image 110 to determine whether a surgical sponge or fabric 106a is present or not within the field of view 108 of the computer device 100 during the sampling period, and operates the frame counter 114 accordingly. In one exemplary method S200, the computer device 100, as in S110 of FIG. 1, acquires the IR image 102 and / or color image 110, and then, optionally,

[0030] converts the image 102 and / or image 110 into a depth map in S210. In some embodiments, since the IR "image" output from the IR imaging system 122 is already a depth map, the conversion may not be necessary. Or, the depth map may be extracted from the alpha channel (or other channels ) of the infrared image without any conversion or calculation. In S220, the computer device 100 applies a peripheral discriminator to the depth map obtained during the sampling period to correlate the area within the depth map with a possible surgical sponge. In one embodiment, to compensate for a surgical sponge that may not be held in a position substantially perpendicular to the infrared camera, the computer device 100 applies a peripheral discriminator to the depth map obtained during the sampling period to correlate the area within the depth map with a possible surgical sponge. In one embodiment, to compensate for a surgical sponge that may not be held in a position substantially perpendicular to the infrared camera, the computer device 100 The computer device scans the depth map - along both the X - axis and the Y - axis of the depth map - such that, over a line of 10 adjacent pixels, a change in depth is identified along the Z - axis in the depth map that exceeds a threshold of 10%, 20%, 30% or more, or 40% or more. In this example, the computer device 100 can correlate each of the short lines of adjacent pixels. Across this line, the depth values exceed the peripheral conditions by more than 10%, and then, adjacent similar peripheral conditions within the depth map are grouped around the perimeter of distinct surfaces within the field of view of the computer device When these distinct surfaces are thus detected, the computer device can perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device thus scans the depth map with respect to areas bounded by a substantially linear outer perimeter characterized by a sharp increase in depth (from the imaging system) from the inside of the area to the outside of the area When these distinct surfaces are thus detected, the computer device can perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge When these distinct surfaces are thus detected, the computer device can perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device can thus perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device can thus perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device can thus perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device can thus perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge The computer device can thus perform template matching, surface reconciliation, or other machine vision techniques to identify distinct, substantially linear surfaces within the depth map of a potential surgical sponge In this embodiment, the computer device can implement a geometric shape discriminator that compensates for the looseness (e.g., lack of rigidity) of the surgical sponge, including the curvature of one or more sides of the linear sponge when held vertically in front of the imaging system In this embodiment, the computer device can implement a geometric shape discriminator that compensates for the looseness (e.g., lack of rigidity) of the surgical sponge, including the curvature of one or more sides of the linear sponge when held vertically in front of the imaging system In this embodiment, the computer device can implement a geometric shape discriminator that compensates for the looseness (e.g., lack of rigidity) of the surgical sponge, including the curvature of one or more sides of the linear sponge when held vertically in front of the imaging system The looseness of the surgical sponge can vary, for example, with the volume of fluid within the surgical sponge, the tension applied to the surgical sponge by the user In the foregoing embodiment, the computer device can also exclude all surfaces that do not include a single pixel or a cluster of small pixels at the horizontal and longitudinal center of the depth image

[0031] In the foregoing embodiment, the computer device can also exclude all surfaces that do not include a single pixel or a cluster of small pixels at the horizontal and longitudinal center of the depth image detected surfaces that do not match predefined points or regions within the depth map, etc. may all be rejected. In this way, the computer device applies the 2D position discriminator to the depth map and may reject surfaces that are not substantially centered within the field of view of the imaging system. However, the computer device may implement any other geometric discriminator in any other way to detect a nearly linear outer perimeter of adjacent surfaces within the depth map and correlate this surface with a potential surgical sponge.

[0032] In other embodiments, the detection of the outer perimeter may be set to identify a more rapid change or decrease in depth, such as greater than, for example, 15%, 20%, 30%, 40%, 50%, 60%, or 70%. The detection of the outer perimeter may be performed with respect to all four edges of a square or rectangular surgical fabric, but in other embodiments, the computer device 100 performs an edge scan only with respect to the X-axis or Y-axis with respect to the change in depth and does not perform it on other axes. In this way, by connecting each end point of the edge to the estimated edge, the edge may be estimated along the other axis. Thus, in some variations, only the top and bottom edges may be detected, and the lateral edges may be estimated from the ends of the top and bottom edges. Alternatively, the lateral edges may be detected and the top and bottom edges may be estimated.

[0033] In another embodiment, in S230, the computer device 100 optionally applies a flatness discriminator to a selected surface of the depth map - as described above, correlated with a potential surgical sponge or fabric - to determine if the selected surface is a surgical sponge or Determine or confirm that it is flat enough to show the fabric 106a. In this embodiment , the computer device 100 selects a set of pixels (e.g., 3, 4 , 5, 6, or 10) in the depth map, maps a virtual plane to the set of pixels, and then , from the virtual plane, the vertical direction to other pixels in the depth map corresponding to the selected surface can be estimated for the distance histogram. Thus, by the histogram, if the selected surface is shown to remain within the distance threshold of the virtual plane over its height and width (optionally, removing outliers to reduce distortion), the computer device 100 can determine or analyze whether the selected surface has sufficient flatness to show the surgical sponge held at its upper corner . However, if the histogram shows that the area of the selected surface is outside the distance threshold from the virtual plane , the computer device 100 can discard the selected surface as not corresponding to the surgical sponge held at its upper corner . In this embodiment, the computer device can implement a distance threshold, such as ±0.1 inch, ±0.25 inch, ±0.5 inch, ±0.75 inch, or ±1 inch, to compensate for folds, wrinkles, and / or pleats of the surgical sponge . In this embodiment, if the selected surface is rejected under the flatness discriminator, the computer device can generate a prompt in the original and / or diagram to extend the surgical sponge to pass the flatness discriminator . The computer device can render this prompt on the display in near real time . . If the selected surface is rejected under the flatness discriminator, the computer device can generate a prompt in the original and / or diagram to extend the surgical sponge to pass the flatness discriminator . The computer device can render this prompt on the display in near real time . .

[0034] In yet another embodiment, in S240, the computer device applies an identifier of perpendicularity to the depth map to confirm that the selected surface is sufficiently perpendicular to the field of view of the imaging system and is suitable for imaging and processing in block S140 of FIG. 1. For example, the computer device 100 estimates an imaginary line perpendicular to the virtual plane described above, and if the imaginary line forms a maximum angle exceeding a threshold angle with respect to the center of the field of view of the imaging system, the selected surface is rejected, and if the maximum angle formed between the imaginary line and the field of view of the imaging system remains within the threshold angle, the selected surface passes. In the alternative method S600 shown in FIG. 6, after applying the perpendicularity identifier in S240, instead of rejecting an image that is not sufficiently perpendicular to the computer device 100 or the optical system 104, the computer device 100 may, in S602, optionally correct for distortion in the depth map. To correct for distortion in the depth map, the computer device 100 may detect the four corners and / or four edges of the surgical fabric 106b in the image 102 and apply a 3D rotation matrix or a translation matrix to flatten the corners or edges in the same plane within the Z-axis.

[0035] After correcting for distortion, the computer device 100 reapplies the flatness identifier to the surgically distorted fabric 106b and then rejects the selected surface if the maximum change in the distance value for the pixels across the selected surface in the depth image distorted due to folds, wrinkles, and / or creases of the surgical sponge exceeds a threshold distance, and also ​​​​​​​​​​​​​​is the maximum change in the value of the distance for pixels across a selected surface in a distorted depth image. If the maximum change in the distance is at a threshold distance, the computer system 100 may continue with further identification and / or processing using the distorted depth image by passing over the selected surface. The threshold distance may be in the range of about -1 cm to about +1 cm, about -2 cm to about +2 cm, or about -3 cm to about +3 cm, or about -4 cm to about +4 cm. If the distorted image did not pass through the flatness discriminator at S604 after being distorted at S602, the computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. Alternatively, if the selected surface is rejected under the verticality discriminator, the computer device may adjust the surgical sponge and generate a prompt in the original and / or diagram to pass through the verticality discriminator for an additional specific number of frames. And the computer device may render this prompt on the display in near real-time. The number of frames may be, for example, in the range of about 15 frames to 120 frames, or about 30 frames to about 60 frames. When the maximum change in the distance for pixels across a selected surface in a distorted depth image is at a threshold distance, the computer system 100 may continue with further identification and / or processing using the distorted depth image by passing over the selected surface. The threshold distance may be in the range of about -1 cm to about +1 cm, about -2 cm to about +2 cm, or about -3 cm to about +3 cm, or about -4 cm to about +4 cm. cm to about +1 cm, about -2 cm to about +2 cm, or about -3 cm to about +3 cm, or about -4 cm to about +4 cm. If the distorted image did not pass through the flatness discriminator at S604 after being distorted at S602, the computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. If the distorted image did not pass through the flatness discriminator at S604 after being distorted at S602, the computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. The computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. The computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. The computer system 100 may set the frame counter at S130, with or without providing a notification or message on the display 118 of the computer device 100, to reposition the position of the surgical fabric 106a within the field of view 108 to a relatively more vertical orientation. Alternatively, if the selected surface is rejected under the verticality discriminator, the computer device may adjust the surgical sponge and generate a prompt in the original and / or diagram to pass through the verticality discriminator for an additional specific number of frames. The computer device may adjust the surgical sponge and generate a prompt in the original and / or diagram to pass through the verticality discriminator for an additional specific number of frames. The computer device may adjust the surgical sponge and generate a prompt in the original and / or diagram to pass through the verticality discriminator for an additional specific number of frames. And the computer device may render this prompt on the display in near real-time. The computer device may render this prompt on the display in near real-time. The number of frames may be, for example, in the range of about 15 frames to 120 frames, or about 30 frames to about 60 frames. The number of frames may be, for example, in the range of about 15 frames to 120 frames, or about 30 frames to about 60 frames.

[0036] Referring again to the method S200 of FIG. 2, the computer device may optionally apply a distance discriminator to the depth map at S250 to confirm that the selected surface is within the appropriate depth of the imaging system 104. For example, the computer device 100 Referring again to the method S200 of FIG. 2, the computer device may optionally apply a distance discriminator to the depth map at S250 to confirm that the selected surface is within the appropriate depth of the imaging system 104. Referring again to the method S200 of FIG. 2, the computer device may optionally apply a distance discriminator to the depth map at S250 to confirm that the selected surface is within the appropriate depth of the imaging system 104. if, at S230, the selected surface is sufficiently flat and, at S240, is sufficiently perpendicular to the field of view of the imaging system 10, the computer device 10 0 may reject the selected surface if more than the minimum number of pixels (e.g., 1%) of the depth image 102 showing the selected surface of the surgical fabric 106b are outside a predetermined depth range (e.g., from 0.5 meters to 1.25 meters, or from 30 cm to about 100 cm, or from about 4 0 cm to about 75 cm) from the imaging system 104. In this embodiment, if the selected surface is rejected under the distance discriminator at S250 , the computer device 100 may optionally generate prompts on the display 118 in the original and / or diagram to further intend to pass the distance discriminator at S250 and move the surgical sponge 106a closer or farther from the optical system 10 4 of the computer device 100. The computer device 100 can render this prompt on the display 118 in substantially real time. In the above embodiment, the computer device 100 can also convert the length and width of the pixels of the selected surface in the depth image into actual length and actual distance based on the distance value of the pixels in the depth image. The computer device thus rejects the selected surface if the actual length or actual width of the selected surface is outside a preset length range or a preset width range, respectively, or otherwise may confirm that such a surgical fabric is being used. For the surgical fabric

[0037] ​​​​​​​​​The preset dimensions include 2 inches × 2 inches, 3 inches × 3 inches, 4 inches × 4 inches, 4 inches × 8 inches, 4 inches × 12 inches, 4 inches × 16 inches, 4 in ches × 18 inches, 12 inches × 12 inches, 16 inches × 16 inches, 16 inches × 24 inches, 18 inches × 30 inches, 15 inches × 24 inches, 16 inches × 23 inches, 12 inches × 13 inches, 13 inches × 18 inches, 24 inches × 24 inches, 36 in ches × 8 inches, 36 inches × 36 inches, etc. For each type of sponge, the user may input the type and / or size of the sponge at the start of a surgical procedure or treatment, and the computer device 100 may, for example, detect dimensions within 0.5 inches or 0.75 inches along the width or length for dimensions less than 5 inches, 8 inches, or 10 inches, and within 0 .5 inches, 1 inch, 1.5 inches, or 2 inches for dimensions greater than 8 inches, 10 inches, 12 inches, or 15 inches of those surgical fabrics.

[0038] In another embodiment, the computer device, at S260, applies a fabric identifier to the depth map to determine if the actual fabric of the selected surface shown in the depth map sufficiently corresponds to the surgical fabric. In one example, the computer device 100 projects or defines a pixel cluster (e.g., a 25-pixel by 25-pixel cluster) within the depth map corresponding to the selected surface onto a virtual plane, and then applies machine vision fabric analysis methods and techniques to Whether the quantification value of the surface fabric is within a single value range, a dry surgical sponge, moisture A certain surgical sponge, and within a range of values of a plurality of distances indicating a soaked surgical sponge In some cases, the computer device can accept the selected surface. Otherwise If so, the computer device may reject the selected surface. In other embodiments, fast A Fourier transform or a wavelet transform is applied to the fabric 10 6b in the images (s) 102, 110 to evaluate the frequency domain of the fabric in one or more image channels such as, for example, the RGB or depth channel In some cases. In some variants, the detected fabric characteristics match the surgical fabric in the library of the computer device 100 However, if it does not match the surgical fabric (s) selected by the user at the start of the surgical operation or procedure, the computer device 100 may prompt the user to confirm whether a new or different surgical fabric Is being shown as being used, and may also prompt for the initialization or activation of a different surgical fabric counter for that surgical fabric In another embodiment, the computer device 100 applies a color identifier to the color image 110 at S270 to confirm that the selected surface includes one or more colors indicating a dry or blood - containing surgical sponge In one example, the computer device 100 determines that at least a threshold percentage (e.g., 70%) of the pixels in the region of the color image corresponding to the selected surface is within the range of red values (e.g., From [ff 00 00] to [ff 40 00]), or within a range of values close to white, as shown in FIG. 1

[0039] In another embodiment, the computer device 100 checks whether the selected surface includes one or more colors indicating a dry or blood - containing surgical sponge By applying a color identifier to the color image 110 at S270. In one example, the computer device 100 determines that at least a threshold percentage (e.g., 70%) of the pixels in the region of the color image corresponding to the selected surface is within the range of red values (e.g., From [ff 00 00] to [ff 40 00]), or within a range of values close to white, as shown in FIG. 1 A threshold percentage (e.g., 70%) of the pixels in the region of the color image corresponding to the selected surface is within the range of red values (e.g., ff 00 00] to [ff 40 00]), or within a range of values close to white, as shown in FIG. 1 If it includes the color within the surrounding of [ff ff ff], the selected surface is allowed. Otherwise, the computer device may reject the selected surface.

[0040] However, the computer device 100 may apply any other discriminator parameters to one or more selected surfaces within the frame pair and / or the depth map or color image of each frame pair. During operation, the computer device 100 may thus apply one or more discriminator parameters to the current frame pair, and also, if all the applied discriminator parameters pass the selected surface or pass the selected surface with sufficient certainty, the selected surface within the frame pair can be confirmed as the display of the surgical sponge. Alternatively, the computer device 100 may, when at least the threshold number of applied discriminator parameters pass the selected surface or by combining the results of all the applied discriminator parameters to reach at least the threshold certainty, confirm the selected surface within the frame pair as the display of the surgical sponge. The computer device can thus index the frame counter in block S112, such as from "0" to "1" or from "4" to "5". In some further variations, the user may adjust the sensitivity and / or specificity of the computer device 100 via the user interface by adjusting the number or weight of the discriminators required to be applied or to pass through the pair or set of images.

[0041] ​​​​​​​​​​​​Otherwise, if the above conditions are not met, the computer device 100 discards the frame pair and may reset the frame counter 114 to "0" as not including the area showing the surgical sponge 106b. If the surface selected within the current frame pair is significantly different in position, size, texture, depth, etc. from the surface selected within the previous frame pair, the computer device 100 may also discard the frame pair and reset the frame counter to "0". For example, even if a surface showing a surgical sponge is detected within the current frame pair, if the center of the selected surface within the current frame pair is offset by more than 0.5 inches in any direction from the center of the selected surface within the previous frame pair, the current frame pair can be rejected and the frame counter reset. In this way, the computer device can reject the surgical sponge and other surfaces moving through the field of view of the imaging system and only confirm the sponge sample of any image that remains substantially static within the field of view of the imaging system. discards the frame pair as not including the area showing the surgical sponge 106b and may reset the frame counter 114 to "0" at S130. The computer device 100 may discard the frame pair and reset the frame counter to "0" if the surface selected within the current frame pair is significantly different in position, size, texture, depth, etc. from the surface selected within the previous frame pair. from the surface selected within the previous frame pair, the computer device 100 may also discard the frame pair and reset the frame counter to "0". For example, even if a surface showing a surgical sponge is detected within the current frame pair, if the center of the selected surface within the current frame pair is offset by more than 0.5 inches in any direction from the center of the selected surface within the previous frame pair, the current frame pair can be rejected and the frame counter reset. In some cases, even if a surface showing a surgical sponge is detected within the current frame pair, the computer device may reject the current frame pair and reset the frame counter if the center of the selected surface within the current frame pair is offset by more than 0.5 inches in any direction from the center of the selected surface within the previous frame pair. In this way, the computer device can reject the surgical sponge and other surfaces moving through the field of view of the imaging system and only confirm the sponge sample of any image that remains substantially static within the field of view of the imaging system. The computer device can thus reject the surgical sponge and other surfaces moving through the field of view of the imaging system and only confirm the sponge sample of any image that remains substantially static within the field of view of the imaging system. In this way, the computer device can reject the surgical sponge and other surfaces moving through the field of view of the imaging system and only confirm the sponge sample of any image that remains substantially static within the field of view of the imaging system. In this way, the computer device can reject the surgical sponge and other surfaces moving through the field of view of the imaging system and only confirm the sponge sample of any image that remains substantially static within the field of view of the imaging system.

[0042] Although the discriminators S230, S240, S250, S260, S270 are shown as being performed in a specific order at S200, those skilled in the art will understand that different orders of these and other discriminators may be used, or that one or more discriminators may be performed in parallel in a plurality of cores and / or processors of the computer device 100, such as multiple streams. or that one or more discriminators may be performed in parallel in a plurality of cores and / or processors of the computer device 100, such as multiple streams. or that one or more discriminators may be performed in parallel in a plurality of cores and / or processors of the computer device 100, such as multiple streams. will understand that different orders of these and other discriminators may be used, or that one or more discriminators may be performed in parallel in a plurality of cores and / or processors of the computer device 100, such as multiple streams.

[0043] 6. Confirmation of Sponge Next, in response to the frame counter 114 encompassing a value equal to the threshold 116, the computer device further processes the image 110 before or after recording the color image 110 of the field of view 108 to evaluate the contents of the blood and clears the frame counter 114 in block S130. Generally, for each frame pair within the sequence of frame pairs, the computer device 100 analyzes the frame pair independently of the previous frame pair, as shown in FIGS. 1 and 2, to identify the surgical sponge or fabric 106b within the frame pair or to confirm that the surgical sponge or fabric 106b is not present within the frame pair and, accordingly, indexes or clears the frame counter. If the frame counter 114 encompasses a value equal to the threshold 116, the total number of consecutive frame pairs equal to the threshold is determined to include the surgical sponge or fabric 106b, and the computer device 100, for this reason, confirms that the surgical sponge or fabric 106a is present within the field of view 108 of the imaging system 104 and that the area within the field of view is of an appropriate degree of certainty when the frame counter 114 encompasses a value equal to the threshold 116. The computer device may implement a preset threshold, such as one of "7", "10", or "15", or alternatively, it may be within a range of about 5 to 100, or about 10 to 70, or about 30 to 50. Alternatively, the computer system may allow the user to implement a custom threshold setting. For example, the user may lower the threshold to shorten the time for viewing and processing images of the surgical sponge.

[0044] can be reduced, and the user can increase the threshold to reduce the false positive detection of the surgical sponge within the field of view of the imaging system. In yet other variations, the computer device may include a discontinuity counter. The discontinuity counter is for each instance where a specific minimum number of consecutive frames are satisfied during a surgical procedure or treatment, but the frame number threshold is not satisfied. Over time, depending on the value of the discontinuity counter, the computer device may adjust the increase or decrease of the threshold to change the sensitivity of the detection method. This can adapt to the position of the light or other environmental conditions that may affect the accuracy of method S100.

[0045] When the computer device thus confirms the presence of the surgical sponge 106a within its field of view 108, the computer device 100 may record the last depth map 102 and / or color image 110 acquired by the imaging system, or may serve as a trigger for the imaging system to acquire a new depth map 102 and / or color image 110. The computer device 100 thus processes this pair of images as described below.

[0046] In some variations, to reduce the risk of inadvertently recounting or re - analyzing the surgical fabric 106a provided to the optical system 104 just before resetting the frame counter 114 in S130 or before restarting the scan of a new surgical fabric 106a in S 110, the computer device, as shown in S500 of FIG. 5, before acquiring an image to identify the next surgical fabric 1 06a, ensures that the surgical fabric 106a is removed from the field of view 108. The blood sample may be further configured to detect that the blood sample has been After determining the liquid content and / or incrementing the fabric counter 112 in S150, S5 At 10, the computing device 100 generates a new depth map 102 and / or a A laser image 110 is acquired, and in S520, the surgical sponge 106a is removed from the field of view 108. Determine whether the object has been removed or is otherwise not detected in the image 102 or the image 110. If the surgical fabric 106b is still present in the image 102 or image 110, The user has not yet removed the previous surgical fabric or has not yet captured a sufficient number of images or frames. For this reason, the surgical fabric is not removed, and the computer device displays the "no textil e" counter to zero.

[0047] If the surgical fabric 106b is not identified in the frame or image, then in S540, The "no textile" counter is incremented, and then the counter changes to "no textil e threshold, and a sufficient number of consecutive "no textile" frames are checked. If the threshold is met, the "no text" message is displayed. The "frame" counter may be reset, and then the computing device may proceed to step S130. In some embodiments, the "no textile" threshold can be reset. The value may be from about 5 to 100, or from about 5 to 50, or from about 4 to about 30, or from about 20 to The "no textile" threshold for the S500 can be in the range of about 40. This may be the same as the threshold 116 in S100 or may be different.

[0048] 7. Dynamic Classifier In one embodiment, the computer system applies the identifier parameters of the same order setting to each frame pair obtained through the operation. Alternatively, the computer system can apply a set of identifier parameters to the frame pair based on the current value within the frame counter. For example, if the frame counter contains a value of "0", the computer device 100 can apply the outer perimeter identifier and the horizontal / longitudinal position identifier to the frame pair, and then, if the frame counter contains a value of "1", apply the outer perimeter identifier and the perpendicularity identifier to the frame pair. The outer perimeter identifier and the distance identifier may be applied to the frame pair when the frame counter contains a value of "2" or more, the outer perimeter identifier and the fabric identifier may be applied to the frame pair when the frame counter contains a value of "6", and the outer perimeter identifier and the color identifier may be applied to the frame pair when the frame counter contains a value of "7". However, the computer device can apply any other single or combined identifier parameters to the frame pair to identify the surgical sponge within the frame pair or discard the frame pair. In some other variations, the outer perimeter and / or position identifier can be applied to all frames, while the remaining identifiers are applied to different frames. In some variations, the different frames for the remaining identifiers are predefined, while in other variations, the different frames may depend on the processing time or the completion of the previous identifier. 8. User Interface Block S120 of the method shows the rendering in the form of a first image on the display.

[0049] Block S122 of the method includes displaying a first pixel corresponding to the first surface on the display. FIG. 1 illustrates a computer device that highlights an area of ​​an image. In step S120, the current frame pair (i.e., the most recent Nearby, a depth map (or color image) from a frame pair acquired by an imaging system The shape of the image is then rendered, and in block S122, a virtual object is generated over the depth map. Rendering the burley, as shown in Figure 1, in some cases against surgical sponges. A visual indication is provided that a corresponding surface was detected within the frame pair.

[0050] In one embodiment, the computing device dewarps the depth map in real time. , and render these undistorted depth maps on the display. Then, as each depth map is processed, the computing device can, in near real-time, A region (e.g., a bounded area) with a depth map that may correspond to a sponge for use The potential surgical site is then identified and the corresponding depth map on the display is then scanned to identify the potential surgical site. For example, you can use the The device identifies possible surgical locations within the depth map rendered on the display. A colored outline can be overlaid around the perimeter of the sponge. In this embodiment, the computing device may generate a frame counter proportional to the current value on the frame counter. The line width of the outline can be set. Similarly, the computing device Red if the current value of the frame counter is "0", and green if the current value of the frame counter is "2 If it is 「」, it is orange. If the current value of the frame counter is 「4」, it is yellow, and If the current value of the frame counter is 「6」, it is green, etc. Based on the current value of the frame counter, the color of the outline can be set. In another embodiment, the computer de vice can overlay a semi - transparent colored area that overlaps across a depth - mapped area - dis played on a rendering corresponding to a possible surgical sponge. In this embodiment, when the current value of the frame counter is 「0」, the opacity of the overlay area is 12%. When the current value of the frame counter is 「1」 it is 24% opacity. When the current value of the frame counter is 「3」, it is 36% opacity,... and when the current value of the frame counter is 「7」, it is 84% opacity etc. The opacity of the overlay area corresponding to the current value of the frame counter can be set. Similarly, the computer device can set the color of the overlay area based on the current value on the frame counter, such as those described above. Therefore, in block S120 and block S122, the computer device can provide a real - time visual feedback regarding the detection of the surgical sponge within the field of view of the imaging system. The computer device can additionally or alternatively

[0051] perform any of the above - mentioned methods and techniques to render a color image and / or overlay on the display in near real - time and provide such feedback to the user. overlay on the display in near real - time and provide such feedback to the user.

[0052] 9. Mask ​​​The presence of surgical sponge or fabric 106b in image 102 and / or 110 may Once identified, the computing device 100 may apply an image mask or perform a depth map. The perimeter of the selected and confirmed area in the frame is then plotted against the corresponding color image in the same frame pair. 2. Select an area of ​​the color image in the image, and perform block S140 and / or block S Process at 150.

[0053] In some variations, to facilitate computational speed, efficiency, and / or accuracy, A method for forming and applying a mask to a color image S700 may optionally be used. For example, in FIG. 7, the depth imaging system 122 captures a depth image in S710 and a color imaging system The depth channel information can be used to obtain the associated image from the camera 120. Thus, in S720, the extracted or acquired, if necessary, from the infrared image In S730, the outer periphery or edge discriminator in S220 is used. The perimeter or edge identifier, which may be the same, is used to identify the perimeter of the surgical fabric. The image can then be undistorted, if necessary, in S740 or reconstructed as described herein. As described elsewhere in this specification, the color image may be transformed into a perspective view of the coordinate system of the color image. The images are also processed in the S750 for color and / or contrast equalization. For example, the histogram distribution or color curves of the RGB channels , and smoothing the color image to different histograms or curves. Images of surgical fabrics is thus applied to the color image as a mask in S760. ,Remove background color pixels from the analysis that are unnecessary.

[0054] In another embodiment of the masking method S800, shown in FIG. 8, the depth image and the color image are obtained from an infrared system and a color system at S810. At S810, a first mask is generated from the depth channel of the infrared image by treating the maximum foreground pixel value as positive and all other pixels as negative. This may be the result of the surgical fabric being provided to the IR camera below or at the minimum depth range of the IR camera. At S8 30 and S840, the first mask undergoes erosion and dilation. These are threshold-based morphological image processing operations that can be used to reduce noise and false negative pixels. For example, to perform erosion, each pixel in the mask is replaced with the minimum value of the surrounding pixels, which may be configured or defined as a pixel zone or window of, for example, 3×3 or 5×5, 7×7, 3×5, or 5×3 pixels. This erosion process can be applied to the first mask as a single pass or through multiple passes, and different pixel windows can be used on different passes to generate a second mask. This erosion process can be used to reduce background noise or other false positive pixels in the first mask. Next, a dilation process can be applied to the first mask to generate a third mask. Here, each pixel in the first mask is replaced with the maximum value within a group of surrounding pixels of a pixel zone or window of, for example, 3×3 or 5×5, 7×7, 3×5, or 5×3 pixels. The grouping may be the same as or different from the grouping used in the erosion process, and again, each pixel window can be used on different passes. This dilation process can be used to reduce false negative pixels in the first mask. At S850, a second mask is generated from the color image by treating the maximum foreground pixel value as positive and all other pixels as negative. This may be the result of the surgical fabric being provided to the color camera below or at the minimum depth range of the color camera. At S860 and S870, the second mask undergoes erosion and dilation, similar to the first mask. These operations can be used to reduce noise and false negative pixels in the second mask. At S880, a final mask is generated by combining the first and second masks. The combination can be performed by logical AND, OR, XOR, or other logical operations. The final mask can be used to identify the surgical fabric in the combined depth and color images. For example, if the final mask is generated by logical AND of the first and second masks, pixels that are positive in both masks are considered part of the surgical fabric. The final mask can be used for further processing, such as segmentation, classification, or tracking of the surgical fabric in the images. In some embodiments, the masking method may also include additional steps, such as filtering, thresholding, or morphological operations on the depth and color images before generating the masks. These additional steps can be used to improve the quality of the masks and the accuracy of the identification of the surgical fabric. For example, a Gaussian filter can be applied to the depth and color images to reduce noise. A threshold can be applied to the filtered images to binarize them before generating the masks. Morphological operations, such as opening and closing, can be applied to the binarized images to further improve the quality of the masks. The masking method can be implemented in software, hardware, or a combination of both. For example, the method can be implemented as a computer program running on a general-purpose computer or a dedicated image processing device. The infrared and color systems can be implemented using cameras, lenses, filters, and other components. The morphological operations can be implemented using libraries or functions provided by image processing software. The combination of the masks can be performed using logical operators provided by programming languages or software libraries. The masking method can be used in various applications, such as surgical navigation, robotic surgery, and image-guided therapy. In surgical navigation, the method can be used to identify the surgical fabric in the images and provide real-time feedback to the surgeon. In robotic surgery, the method can be used to guide the robotic arm to the surgical fabric and perform precise operations. In image-guided therapy, the method can be used to monitor the progress of the therapy and adjust the treatment plan based on the position and shape of the surgical fabric. false negative pixels in the first mask. At S850, a second mask is generated from the color image by treating the maximum foreground pixel value as positive and all other pixels as negative. This may be the result of the surgical fabric being provided to the color camera below or at the minimum depth range of the color camera. At S860 and S870, the second mask undergoes erosion and dilation, similar to the first mask. These operations can be used to reduce noise and false negative pixels in the second mask. At S880, a final mask is generated by combining the first and second masks. The combination can be performed by logical AND, OR, XOR, or other logical operations. The final mask can be used to identify the surgical fabric in the combined depth and color images. For example, if the final mask is generated by logical AND of the first and second masks, pixels that are positive in both masks are considered part of the surgical fabric. The final mask can be used for further processing, such as segmentation, classification, or tracking of the surgical fabric in the images. In some embodiments, the masking method may also include additional steps, such as filtering, thresholding, or morphological operations on the depth and color images before generating the masks. These additional steps can be used to improve the quality of the masks and the accuracy of the identification of the surgical fabric. For example, a Gaussian filter can be applied to the depth and color images to reduce noise. A threshold can be applied to the filtered images to binarize them before generating the masks. Morphological operations, such as opening and closing, can be applied to the binarized images to further improve the quality of the masks. The masking method can be implemented in software, hardware, or a combination of both. For example, the method can be implemented as a computer program running on a general-purpose computer or a dedicated image processing device. The infrared and color systems can be implemented using cameras, lenses, filters, and other components. The morphological operations can be implemented using libraries or functions provided by image processing software. The combination of the masks can be performed using logical operators provided by programming languages or software libraries. The masking method can be used in various applications, such as surgical navigation, robotic surgery, and image-guided therapy. In surgical navigation, the method can be used to identify the surgical fabric in the images and provide real-time feedback to the surgeon. In robotic surgery, the method can be used to guide the robotic arm to the surgical fabric and perform precise operations. In image-guided therapy, the method can be used to monitor the progress of the therapy and adjust the treatment plan based on the position and shape of the surgical fabric. or 5×5, 7×7, 3×5, or 5×3 pixels. The grouping may be the same as or different from the grouping used in the erosion process, and again, each pixel window can be used on different passes. This dilation process can be used to reduce false negative pixels in the first mask. At S850, a second mask is generated from the color image by treating the maximum foreground pixel value as positive and all other pixels as negative. This may be the result of the surgical fabric being provided to the color camera below or at the minimum depth range of the color camera. At S860 and S870, the second mask undergoes erosion and dilation, similar to the first mask. These operations can be used to reduce noise and false negative pixels in the second mask. At S880, a final mask is generated by combining the first and second masks. The combination can be performed by logical AND, OR, XOR, or other logical operations. The final mask can be used to identify the surgical fabric in the combined depth and color images. For example, if the final mask is generated by logical AND of the first and second masks, pixels that are positive in both masks are considered part of the surgical fabric. The final mask can be used for further processing, such as segmentation, classification, or tracking of the surgical fabric in the images. In some embodiments, the masking method may also include additional steps, such as filtering, thresholding, or morphological operations on the depth and color images before generating the masks. These additional steps can be used to improve the quality of the masks and the accuracy of the identification of the surgical fabric. For example, a Gaussian filter can be applied to the depth and color images to reduce noise. A threshold can be applied to the filtered images to binarize them before generating the masks. Morphological operations, such as opening and closing, can be applied to the binarized images to further improve the quality of the masks. The masking method can be implemented in software, hardware, or a combination of both. For example, the method can be implemented as a computer program running on a general-purpose computer or a dedicated image processing device. The infrared and color systems can be implemented using cameras, lenses, filters, and other components. The morphological operations can be implemented using libraries or functions provided by image processing software. The combination of the masks can be performed using logical operators provided by programming languages or software libraries. The masking method can be used in various applications, such as surgical navigation, robotic surgery, and image-guided therapy. In surgical navigation, the method can be used to identify the surgical fabric in the images and provide real-time feedback to the surgeon. In robotic surgery, the method can be used to guide the robotic arm to the surgical fabric and perform precise operations. In image-guided therapy, the method can be used to monitor the progress of the therapy and adjust the treatment plan based on the position and shape of the surgical fabric. The same or different groupings for the paths can be performed with a single path or multiple paths. This enlargement process can be used to potentially return to certain false negative pixels. This Using these three masks, at S850, for each pixel of the mask, a 2-bit, fourth mask is optionally generated, where all pixels from the second mask are flagged as "11", or alternatively, flagged as the exact foreground pixel . Pixels not in the third mask are flagged as "00", or alternatively, flagged as the exact background pixel. Pixels within the first mask but not in the second mask are labeled as "10" as the prospective foreground pixel, and pixels within the third mask but not in the first mask are labeled as "01", that is, prospective background pixels. Depending on the desired sensitivity and / or specificity required, the first, second, or fourth mask

[0055] can be used to identify the corresponding pixels in the color image for further processing. In a mask such as the first mask or the fourth mask, additional processing such as peripheral or edge detection can be applied to further refine the mask before application to the color image.

[0056] 10. Removal of the hand The computer device also identifies the regions of the color image corresponding to the user's hand - holding the surgical sponge - and these regions may be discarded from the processing in blocks S140 and S150. For example, the computer device is a surgical sponge ​​​​​Scan the selected area of the depth map within the frame pair that has been confirmed to indicate and is near the corner of this selected area, indicating the distance from the imaging system reduced by hand correlate the area within the depth image, and then, for the purpose of removal from the processing in block S140 it may be necessary to identify the corresponding area of the color image. The computer device may remove this area from the outline of the surgical sponge shown on the display The computer device may apply the aforementioned methods and techniques to the frame pair specified for processing only after the surgical sponge has been confirmed to be within the field of view of the imaging system There are cases. Alternatively, the computer device may apply the aforementioned methods and techniques to each frame pair and to each depth image or color image rendered on the display and may be applicable .

[0057] 11. Amount of blood components and sponge counter In block S140, when a color image that includes an area showing a surgical sponge at an appropriate distance and appropriately vertically and sufficiently flat within the field of view of the imaging system is acquired and recorded the computer device can perform the methods and techniques described in U.S. Patent Application No. 13 / 544,664 to convert the color values included in the pixels in the color image into an evaluation of the mass, volume, or other quantitative measurement values of the entire amount of blood, red blood cells, hemoglobin, and / or other blood components within the actual surgical sponge shown in the color image The document of U.S. Patent Application No. 13 / 544,664 is hereby incorporated by reference in its entirety into this specification .

[0058] ​​​In some variations, before converting the color values of the pixels into an evaluation of the values of the blood components , the image (with or without any mask described herein) may also undergo compensation for ambient light as described in U.S. Patent Application No. 15 / 1 54,917. This document is hereby incorporated by reference in its entirety .

[0059] In block S150, the computer device may also implement the methods and techniques described in U.S. Patent Application No. 13 / 54 4,664 to index a counter indicating the total number of surgical sponges identified and counted during the operation in which the computer device is operating . In this variation, the computer device may render the value included in the sponge counter on the display and update this rendered value substantially in real time when a surgical sponge is confirmed within the field of view of the imaging system . For example, the computer device may present the updated value of the sponge counter on the display when a surgical sponge is confirmed and counted . Similarly, if a potential surgical sponge is detected within a frame pair, the computer device may overlay the next value of the sponge counter (i.e., the current sponge count + 1) across the depth (or color) image rendered on the display , and the computer device may increase the opacity of the overlaid value of the sponge counter as the value within the frame counter increases for the next frame pair . In another embodiment, after segmenting the surgical fabric, the color image may be applied to the regions of the image containing blood . . . . .

[0060] . Based on this, it is further segmented or masked to exclude the blood-free area of the surgical sponge. This can be achieved by using an algorithm based on the following rules. It can be implemented.

Number

Number

[0061] 12. Counting of Sponges by Voice In one variant, the computer device may acquire and store voice data, along with data on the amount of image and blood components, regarding the surgical sponge that is detected and imaged during operation. Specifically, the computer device can record the user (for example, two users such as a nurse, an anesthesiologist, a technician, etc.) who continuously indicates the count of sponges by voice.

[0062] In one embodiment, when a potential sponge is detected in a frame pair and it is indexed as 1 in the frame counter, the computer device may start recording the voice by sampling the microphone connected to or incorporated in the computer device. Then, when the value of the frame counter reaches the threshold and thus the surgical sponge is confirmed. ​​​​​​​​​​​If so, the computing device will, after 5 seconds, confirm that the sponge has been confirmed and that a new sponge has been created. The audio recording continues until the camera is detected within the field of view of the imaging system, whichever comes first. The computing device can then record this audio in a corresponding external The images can be stored and linked with color images of the cosmetic sponges. The computer device records a color image of the surgical sponge and transmits the color image to the surgical sponge. This color is converted into an estimate of the amount of a blood component in the volume of the blood (e.g., hemoglobin volume). - Evaluation of the images and the amount of blood components was performed according to the time the images were acquired, the time of day the images were taken ... The value of the sponge counter and the sounds of two users reading and checking the sponge count value. It can be stored along with the voice recording.

[0063] In another embodiment, the computing device records a single audio track throughout the session. Then, the audio track is spliced ​​into a number of separate audio segments, and each audio segment is Based on the time of the audio segment and the time the color image was captured, the recorded color - Matching the image and the amount of blood components to the assessment, and then each audio segment is matched to its counterpart The color image may be stored with a tag or other pointer to the corresponding color image or associated data. There may be cases where this occurs.

[0064] 12. Rejection of duplicate sponges – RFID In one variation, the computing device includes a radio frequency identification ("RFID") reader and In this variation, the computing device further includes an FID antenna. The counter is indexed with "1" (or some other value below the threshold) to indicate that the imaging system Shows a possible surgical sponge within the field of view, and the computer device can emit an excitation signal through the RFID antenna and then sample the RFID reader. If the sponge within the field of view of the imaging system contains an RFID tag, this RFID tag can emit to return a data packet to the RFID reader. The computer device applies the reception of this data packet from the surgical sponge to identify the sponge, and then, as described above, immediately when flatness, distance, position, and / or other requirements are met and before the image counter reaches the threshold, it can acquire and process the color image of the sponge. The computer device can additionally or alternatively implement managed machine learning techniques to train an optical sponge detection model based on the surgical sponge and this RFID data packet received from the depth map and / or color image of the surgical sponge. Alternatively, the computer device can emit an excitation signal and can only read the data packet coming from the proximal sponge when the frame counter reaches the threshold and the surgical sponge is thus optically identified. and then sample the RFID reader. If the sponge within the field of view of the imaging system contains an RFID tag, this RFID tag can emit to return a data packet to the RFID reader. The computer device applies the reception of this data packet from the surgical sponge to identify the sponge, and then, as described above, immediately when flatness, distance, position, and / or other requirements are met and before the image counter reaches the threshold, it can acquire and process the color image of the sponge. The computer device can additionally or alternatively implement managed machine learning techniques to train an optical sponge detection model based on the surgical sponge and this RFID data packet received from the depth map and / or color image of the surgical sponge. Alternatively, the computer device can emit an excitation signal and can only read the data packet coming from the proximal sponge when the frame counter reaches the threshold and the surgical sponge is thus optically identified. and the surgical sponge is thus optically identified. only read the data packet coming from the proximal sponge.

[0065] In the above-described embodiments, the computer device may compare the RFID data packet received from the current surgical sponge during the operating period with the data packets previously received from other sponges in order to detect duplicate scans of the same surgical sponge. Specifically, if the RFID data packet received from the surgical sponge is identical to the RFID data packet previously received during the same operating period, the computer device determines a duplicate. In the above-described embodiments, the computer device may compare the RFID data packet received from the current surgical sponge during the operating period with the data packets previously received from other sponges in order to detect duplicate scans of the same surgical sponge. Specifically, if the RFID data packet received from the surgical sponge is identical to the RFID data packet previously received during the same operating period, the computer device determines a duplicate. If the RFID data packet received from the surgical sponge is identical to the RFID data packet previously received during the same operating period, the computer device determines a duplicate. Reject the surgical sponge, discard the corresponding frame pair, and it is possible to discard the evaluation of the amount of blood components related to the surgical sponge.

[0066] Furthermore, in this variant, to prevent the detection of other proximal RFID tags, the computer device can change the power output of the RFID antenna according to the distance between the computer device and the sponge evaluated from the current depth map. Specifically, the computer device can convert the value of the distance within the area of the current depth map corresponding to the surgical sponge whose possibility has been confirmed into the power level of the excitation signal subsequently emitted by the RFID antenna. However, in this variant, if no data packet is received after the conversion of the excitation signal, the computer device can confirm the surgical sponge and proceed according to each block of the above method to process the color image of the sponge. Additionally, if no RFID data packet is received after emitting the excitation signal, the computer device can tag the surgical sponge (such as the value of the sponge counter of the surgical sponge, the color image of the surgical sponge, etc.) assuming that RFID is not enabled.

[0067] In a further embodiment, the computer device may also identify any barcode, QR code, or other visible code on the surgical fabric. For example, the processor may scan the color image with respect to the four corners of the QR code or barcode and then identify the area where the code is attached. In block S410, the processor thus Decode the region with the mark to determine the code string. The code string is used to uniquely identify the detected surgical fabric and prevent double counting if the same surgical fabric is provided to the system again, and may be used to provide a warning to the user or surgeon. In this way, the detected surgical fabric can be uniquely identified, and double counting can be avoided when the same surgical fabric is provided to the system again, and a warning can be provided to the user or surgeon. In another variant, the computer device implements machine vision technology to detect the reflective material on the surgical sponge shown within the frame pair and identify the sponge (uniquely) based on the detected reflective material. For example, the computer device may include a second IR emitter configured to illuminate the field of view of the IR camera with electromagnetic radiation that is printed on the surgical sponge or reflected by a material woven into the surgical sponge and detected by the IR camera. During the sampling period, the computer device thus obtains and processes a second infrared image to detect a standard or substantially unique pattern of the reflective material within the surgical sponge in the field of view of the imaging system. The computer device then implements the methods and techniques described above to confirm the presence of the surgical sponge in the field of view of the imaging system, uniquely identify the image based on the pattern of the reflective material, reject duplicate surgical sponges, trigger the early acquisition and processing of the color image of the surgical sponge, and / or improve the optical surgical sponge detection model related to the managed machine learning technology. In this way, the detected surgical fabric can be uniquely identified, and double counting can be avoided when the same surgical fabric is provided to the system again, and a warning can be provided to the user or surgeon.

[0068] 13. Rejection of Duplicate Sponges - Reflective IR Coating In another variant, the computer device implements machine vision technology to detect the reflective material on the surgical sponge shown within the frame pair and identify the sponge (uniquely) based on the detected reflective material. For example, the computer device may include a second IR emitter configured to illuminate the field of view of the IR camera with electromagnetic radiation that is printed on the surgical sponge or reflected by a material woven into the surgical sponge and detected by the IR camera. During the sampling period, the computer device thus obtains and processes a second infrared image to detect a standard or substantially unique pattern of the reflective material within the surgical sponge in the field of view of the imaging system. The computer device then implements the methods and techniques described above to confirm the presence of the surgical sponge in the field of view of the imaging system, uniquely identify the image based on the pattern of the reflective material, reject duplicate surgical sponges, trigger the early acquisition and processing of the color image of the surgical sponge, and / or improve the optical surgical sponge detection model related to the managed machine learning technology. For example, the computer device may include a second IR emitter configured to illuminate the field of view of the IR camera with electromagnetic radiation that is printed on the surgical sponge or reflected by a material woven into the surgical sponge and detected by the IR camera. During the sampling period, the computer device thus obtains and processes a second infrared image to detect a standard or substantially unique pattern of the reflective material within the surgical sponge in the field of view of the imaging system. The computer device then implements the methods and techniques described above to confirm the presence of the surgical sponge in the field of view of the imaging system, uniquely identify the image based on the pattern of the reflective material, reject duplicate surgical sponges, trigger the early acquisition and processing of the color image of the surgical sponge, and / or improve the optical surgical sponge detection model related to the managed machine learning technology. For example, the computer device may include a second IR emitter configured to illuminate the field of view of the IR camera with electromagnetic radiation that is printed on the surgical sponge or reflected by a material woven into the surgical sponge and detected by the IR camera. During the sampling period, the computer device thus obtains and processes a second infrared image to detect a standard or substantially unique pattern of the reflective material within the surgical sponge in the field of view of the imaging system. The computer device then implements the methods and techniques described above to confirm the presence of the surgical sponge in the field of view of the imaging system, uniquely identify the image based on the pattern of the reflective material, reject duplicate surgical sponges, trigger the early acquisition and processing of the color image of the surgical sponge, and / or improve the optical surgical sponge detection model related to the managed machine learning technology. For example, the computer device may include a second IR emitter configured to illuminate the field of view of the IR camera with electromagnetic radiation that is printed on the surgical sponge or reflected by a material woven into the surgical sponge and detected by the IR camera. During the sampling period, the computer device thus obtains and processes a second infrared image to detect a standard or substantially unique pattern of the reflective material within the surgical sponge in the field of view of the imaging system. The computer device then implements the methods and techniques described above to confirm the presence of the surgical sponge in the field of view of the imaging system, uniquely identify the image based on the pattern of the reflective material, reject duplicate surgical sponges, trigger the early acquisition and processing of the color image of the surgical sponge, and / or improve the optical surgical sponge detection model related to the managed machine learning technology.

[0069] The systems and methods described herein store computer-readable instructions in a computer. Implemented at least in part as a machine configured to receive a computer-readable medium. The instructions may be implemented and / or executed by a user computer or mobile device, an application on a mobile device, a smart phone, or any suitable combination thereof; , applet, host, server, network, website, communication server, communication interface interface, hardware / firmware / software elements combined with Other systems and methods of embodiments may be implemented by computer executable components. A computer-readable medium having computer-readable instructions stored thereon. The instructions may be implemented and / or executed at least in part as a computer-implemented machine. The present invention is implemented by computer-executable components in combination with devices and networks of the above type. The program may be executed by computer-executable components embedded in the program. Possible media are RAM, ROM, flash memory, EEPROM, optical disk (CD or DVD), hard drive, floppy drive, or any suitable device. The computer-executable program may be stored on any suitable computer-readable medium. The component may be a processor, but may also be any suitable dedicated hardware device. The device may (alternatively or additionally) execute the instructions.

[0070] In this description, terms such as components, modules, devices, etc. may be implemented in various ways. When referring to any type of logical or functional circuit, block, and / or process, For example, the functions of the various circuits and / or blocks may be combined with each other to It can be any other number of devices. Or, this device can be a multipurpose computer or a processing / graphic hardware, and can be provided with programming instructions transmitted via a transmission carrier wave. Also, this device can be implemented as a hardware logic circuit that implements the functions included by the novel things in this specification. Finally, this device can be implemented using specific-purpose instructions (SIMD instructions), field-programmable logic arrays, or a mixture thereof that provide the desired level of performance and cost. As for the aspects of the methods and systems described in this specification, such as logic, they can be functionally programmed into any of various circuits, including field-programmable gate arrays ("FPGAs"), programmable array logic ("PAL") devices, electrically programmable logic and memory devices, and standard cell-based

[0071] devices, as well as programmable logic devices ("PLDs") such as application-specific integrated circuits. Some other deformation forms for implementing each aspect include memory devices, microcontrollers with memory (such as EEPROM), embedded microprocessors, firmware, software, etc. Further, each aspect can be implemented in a microprocessor having software-based circuit emulation, separate logic (sequential and concatenated), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above-mentioned device types. The basic device technology is of various component types, such as complementary metal-oxide-semiconductor ("CMOS") ). There are also cases where it is implemented as being functionally programmed into any of various circuits. Some other deformation forms for implementing each aspect include memory devices, microcontrollers with memory (such as EEPROM), embedded microprocessors, firmware, software, etc. Further, each aspect can be implemented in a microprocessor having software-based circuit emulation, separate logic (sequential and concatenated), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above-mentioned device types. The basic device technology is of various component types, such as complementary metal-oxide-semiconductor ("CMOS") circuit emulation, separate logic (sequential and concatenated), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above-mentioned device types. The basic device technology is of various component types, such as complementary metal-oxide-semiconductor ("CMOS") or the like. is of various component types, such as complementary metal-oxide-semiconductor ("CMOS") to metal-oxide semiconductor field effect transistor (MOSFET) technology, emitter-coupled logic (ECL), bipolar technologies such as, polymer technologies (e.g., silicon conjugated polymers and metal conjugated polymer metal structures), and mixtures of analog and digital, among others.

[0072] Those skilled in the art will understand from the foregoing detailed description, as well as from the drawings and the claims, that changes and modifications may be made to the embodiments of the present invention without departing from the scope of the invention as defined in the appended claims. The foregoing description of the embodiments of the present invention is not intended to limit the present invention to these embodiments, but rather is intended to enable those skilled in the art to practice and use the present invention.

[0073] The foregoing description of the embodiments of the present invention is not intended to limit the present invention to these embodiments, but rather is intended to enable those skilled in the art to practice and use the present invention. Rather, it is intended to enable those skilled in the art to practice and use the present invention. The variations, configurations, embodiments, exemplary embodiments, and examples described herein are optional and not exclusive of those they describe. The invention described herein can include any and all substitutions of these variations, configurations, embodiments, exemplary embodiments, and examples. ​

Claims

1. a color imaging system configured to capture a color image; an infrared imaging system configured to acquire an infrared image; 1. A processor comprising: Using the infrared image to identify a shape of the surgical fabric; generating shape information from the infrared image; using said shape information to identify surgical fabric pixels within said color image; the processor configured to extract blood information from the surgical fabric pixels. A system for measuring blood content comprising:

2. The processor is adapted to transform the infrared image into a coordinate system corresponding to the color image. The system of claim 1 further comprising:

3. The processor is further configured to generate a depth image from one of the infrared images. The system of claim 1 .

4. The processor further comprises: The system of claim 3 .

5. The processor is further configured to detect a perimeter of an image of a surgical fabric within the depth image. The system according to claim 4 .

6. The detection of the periphery of the image of the surgical fabric is based on a sudden vertical depth drop in the depth image. The system of claim 5 , wherein the system is implemented by searching for:

7. The processor is further adapted to perform low resolution processing of the depth image or the infrared image. The system of claim 3 .

8. The processor is adapted to generate a mask image from the reduced resolution depth image. The system of claim 7 further comprising:

9. The processor is further configured to perform high resolution processing of the mask image. Item 9. The system according to item 8.

10. The processor is adapted to perform filling of missing infrared pixel regions in the infrared image. The system of claim 1 further configured to:

11. The processor is further configured to generate a mask image using the detected perimeter. The system of claim 5 .

12. The processor further comprises: transforming the mask image into a coordinate system of the color image. The system of claim 11 configured to:

13. 12. The method of claim 11, wherein the detected perimeter is the top and bottom edges of an image of a surgical fabric. system.

14. The method of claim 11, wherein the detected perimeter comprises opposite edges of an image of a surgical fabric. system.

15. The system of claim 11, wherein the detected perimeter comprises a corner of an image of a surgical fabric. Hmm.

16. 1. A method for analyzing a subject containing a blood constituent, comprising: acquiring depth and color images of an object including a blood component; using the depth image to determine the presence of the object within a field of view; and evaluating a characteristic of a blood component using the color image. method.

17. generating an image mask using the depth image; applying the image mask to the color image to remove background pixels; and The method of claim 16, further comprising:

18. and applying at least one of the following image processes to the depth image: erosion and dilation. The method of claim 17 .

19. 1. A method for assessing the amount of a fluid component in a surgical fabric, comprising: acquiring a first infrared image of a field of view at a first time; Detecting a shape or surface characteristic of a surgical fabric in the first infrared image; When the shape or surface feature is detected in the first infrared image, a frame counter is Indexing; and acquiring a second infrared image of the field of view at a second time subsequent to the first time; 、 Detecting a surface characteristic of the surgical fabric in the second infrared image; When the shape or surface feature is detected in the second infrared image, the frame counter and assigning an index to the data. a non-detection when the shape or surface characteristic of the surgical fabric in the second infrared image is not detected; indexing a counter.

20. When a preset value is reached, the frame counter and the non-detection counter are cleared.

20. The method of claim 19, further comprising:

21. comparing the frame counter to a threshold; If the frame counter is greater than or equal to the threshold, a color image of the field of view is acquired. And, converting an area of ​​said color image corresponding to the surgical fabric into an estimate of the amount of blood constituent; 、 20. The method of claim 19, further comprising: clearing the frame counter.

22. generating an image mask using the second infrared image; and A color image corresponding to the surgical fabric is obtained by applying the image mask to the color image. and identifying the region of:

23. 1. A method for performing fluid monitoring, comprising: acquiring an infrared image of a field of view; acquiring a color image of the field of view; generating an image mask using the infrared image; The image mask is applied to the color image to identify areas of the color image that correspond to surgical fabrics. and identifying a region.

24. 2. The method of claim 1 further comprising: converting the infrared image into a geometric perspective view of the color image.

24. The method according to claim 23.

25. and down-resolving the infrared image prior to generating the image mask.

24. The method of claim 23.

26. Prior to using the image mask to identify the region of the color image, the image mask 26. The method of claim 25, further comprising high resolution processing of

Images