System and method for tracking surgical instrument

Abstract

To provide a method and system for tracking a surgical instrument.SOLUTION: The method for tracking a surgical instrument executed by at least one processor includes: receiving at least one image including at least one surgical instrument; determining a first collection of surgical instrument; determining a second collection of surgical instruments on the basis of at least one received image at least partially; and providing a notification on the basis of the first collection and the second collection of the surgical instruments.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Priority claim The application was filed on June 3, 2019 and is related to a "system for tracking surgical instruments and This application claims priority to U.S. Provisional Patent Application No. 62 / 856,701, entitled "Method and Apparatus for Producing and Composing a Novel Microorganism," which is hereby incorporated by reference in its entirety. is incorporated herein by reference in its entirety.

[0002] The subject matter disclosed herein relates generally to the medical arts, and more specifically to procedures. The present invention relates to a novel and useful system and method for tracking surgical instruments. The system includes specialized machines that facilitate tracking of surgical instruments, and the use of such specialized machines Software-configured computerized transformations and improvements to such transformations; and Such dedicated machines facilitate tracking of surgical instruments, an improvement over other dedicated machines. This includes improvements in technology that are being implemented. [Background technology]

[0003] Tracking surgical instruments during a medical procedure reduces the risk of the instruments being inadvertently retained within the patient. It can be important to ensure that surgical instruments are not retained within the patient. Current methods for tracking and counting surgical instruments revolve around the manual tracking and counting of surgical instruments. Staff fatigue, complex procedures that make it difficult to track surgical instruments, poor accounting systems, and Manual collection may be difficult due to instruments sticking together or otherwise obscuring the Moreover, compliance with established protocols is often difficult, especially in the In long or complex procedures, the surgical instruments may be incomplete. remains a significant problem during medical procedures. Instruments left inside the patient during surgery can cause injury to the patient. poses significant health risks to many and increases the overall time and cost of healthcare. Thus, a new and improved method and system for tracking surgical instruments during a medical procedure is provided. It may be desirable. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 is a schematic block diagram illustrating a method for tracking a surgical instrument, according to some exemplary embodiments. [Diagram 2] FIG. 1 is a schematic block diagram illustrating a method for tracking a surgical instrument, according to some exemplary embodiments. [Diagram 3] FIG. 13 illustrates a graphical user interface showing information related to a first tally of surgical instruments, according to some exemplary embodiments. [Figure 4] FIG. 1 is a schematic diagram illustrating a container configured to receive surgical instruments for counting, according to some exemplary embodiments. [Diagram 5] 1A-1C are schematic diagrams illustrating mounting structures configured to hold imaging devices, according to some exemplary embodiments. [Figure 6] FIG. 1 is a schematic diagram illustrating a protocol for counting surgical instruments, according to some exemplary embodiments. [Figure 7] 1 is a schematic diagram illustrating a user operating a mobile device to capture an image of a container configured to receive a surgical instrument, according to some exemplary embodiments. [Figure 8] FIG. 1 is a schematic block diagram illustrating components of a method for verifying the location of a surgical instrument, according to some exemplary embodiments. [Figure 9] 1 illustrates a display of information generated by a method for tracking surgical instruments, according to some exemplary embodiments. [Figure 10]1 illustrates a display of information generated by a method for tracking surgical instruments, according to some exemplary embodiments. [Figure 11] 13A-13C illustrate graphical user interfaces prompting user verification of surgical fiber location, according to some exemplary embodiments. [Figure 12] 1 is a schematic diagram illustrating a method for analyzing surgical fabric, according to some exemplary embodiments. [Figure 13] FIG. 1 illustrates a graphical user interface showing information related to tracking of a surgical instrument, according to some exemplary embodiments. [Figure 14] 1A-1C are diagrams illustrating graphical user interfaces showing information related to tracking of surgical instruments at intermediate points during a surgical procedure, in accordance with some exemplary embodiments. [Figure 15] FIG. 13 illustrates a graphical user interface including a surgical instrument location summary report, according to some exemplary embodiments. [Figure 16] FIG. 1 illustrates a summary report, according to some exemplary embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Exemplary methods (e.g., algorithms) facilitate tracking of surgical instruments and A system (e.g., a dedicated machine configured with dedicated software) can track surgical instruments. The examples are merely representative of possible variations. Unless otherwise specified, structures (such as modules and other structural components) are optional and may be combined. Operations (procedures, algorithms, other functions, etc.) can be They can be reordered, combined, or subdivided. In the following description, for illustrative purposes only. Numerous specific details are set forth in order to provide a thorough understanding of various exemplary embodiments. However, it will be understood by those skilled in the art that the present subject matter may be practiced without these specific details. It will be clear.

[0006] Using any one or more of the exemplary methods and systems described herein To track (e.g., identify, locate, monitor, etc.) one or more surgical instruments For example, such methods and systems can be used to measure the position of surgical instruments at various times. The number of surgical instruments can be compared to determine if all surgical instruments are accounted for. Such methods and systems may incorporate computer vision, machine learning, deep learning, or any suitable combination thereof to determine the accuracy of the surgical instrument. Using advanced computational techniques, for example, manual Improve accuracy of instrument counting, improve compliance with counting protocols, and Streamline the process of identifying where surgical instruments are located and improve overall risk assessment associated with retaining surgical instruments. The medical procedure may be performed in a time-saving manner, at reduced cost, or at any suitable combination thereof. Accurate identification of introduced surgical instruments is essential to prevent inadvertent storage of surgical instruments on patients after medical procedures. Surgical instrument tracking can also help to monitor the use, accuracy, and timeliness of surgical instruments during a procedure. provide useful information regarding compliance with established protocols, or both , which may help the user trigger various responses when a particular indication is present.

[0007] The methods and systems described herein may be used in a hospital or clinic setting (e.g., surgery or clinical setting), military setting (e.g., battlefield), or other appropriate treatment setting; The methods and systems described herein can be used in a variety of settings. Additional surgical equipment such as surgical fabric (e.g., surgical sponges or surgical gauze pads) Although described with respect to traces, in other variations, such methods and systems may further include or alternatively, may be used to track other instruments (e.g., surgical equipment or instruments). obtain.

[0008] Although exemplary methods are described herein primarily with respect to tracking surgical fibers, Such methods can be applied to any instrument used in a medical procedure. For example, Such methods may involve the use of scalpels, scissors, sutures, needles, gauze, bandages, or any other suitable instruments. The present invention may include identifying, locating, or tracking other surgical instruments, such as surgical instruments. The location of surgical instruments is confirmed using a multi-section system used to count surgical instruments such as surgical textiles. This may also include the use of containers with multiple compartments. Such multi-compartment containers are sometimes called "tally bags." However, in some exemplary embodiments, the method for tracking a surgical instrument includes: A counting bag may not be used. The calculation technique involves counting the equipment without the use of a counting bag. The present invention can be applied to collect instruments grouped or organized in any suitable manner. can be used to measure

[0009] Overview of the Example Described herein, among other things, is an exemplary method for tracking a surgical instrument. These methods may be performed by one or more processors. In an exemplary embodiment, a method for tracking surgical instruments includes a first tally of the surgical instruments. receiving one or more images, each image including one or more surgical instruments; and determining, based at least in part on the one or more received images, a field of view that includes the surgical and determining a second tally of the device. Embodiments also include a first collection of surgical instruments, a second collection of surgical instruments, or both. For example, the notification may include providing a notification based on a first count of the surgical instruments and the procedure. The second compilation of surgical instruments may be provided based on a comparison between the first compilation of surgical instruments and the second compilation of surgical instruments. In this embodiment, the first tally may generally be determined prior to the medical procedure and the second tally may generally be determined after the medical procedure. The determination may be made during or after treatment.

[0010] Receiving the first tally of the surgical instruments identifies one or more surgical instruments depicted in the image. For example, the identification of one or more surgical instruments may include computer vision. including the use of computer-aided design, machine learning, other computational techniques, or any suitable combination thereof. In some exemplary embodiments, receiving the first tally of surgical instruments comprises at least The method may be based, in part, on detecting the weight of one or more surgical instruments. In certain exemplary embodiments, the first tally of surgical instruments is determined at least in part by user input. The determination of the second count of the surgical instruments may be based on one or more received images. The method may include identifying one or more surgical instruments depicted in one of the received images. The one or more may be, for example, a depth image (e.g., an infrared depth image), a color image, or other In some exemplary embodiments, one or more receivers may be used. The identification of the one or more surgical instruments depicted in the received image includes at least one compartment. and identifying and characterizing the container depicted in the one or more images. Characterization may include determining one or more parameters of the container. The determination of the parameter or parameters may be used, for example, to quantify one or more compartments in a vessel, to measure the size and shape of a vessel, Determining the geometric arrangement of one or more compartments in the Determining the presence or absence of several (e.g., each) surgical instruments, or any suitable combination thereof. In various exemplary embodiments, determining a second count of the surgical instruments. may additionally or alternatively be configured to at least partially hold one or more surgical instruments. The method may be based on detecting a change in weight of a container configured to receive the change in weight.

[0011] Determining the second count of the surgical instruments may include updating a current index value of the surgical instruments. Some exemplary embodiments of the methods described herein may include displaying a For example, on a graphical user interface of a mobile computing device and displaying a current index value of the surgical instrument. The form may be a first collection of surgical instruments, a second collection of surgical instruments, or both. Various exemplary methods include prompting the user to verify the accuracy of the tabulation. The embodiment includes a first tally of surgical instruments, a second tally of surgical instruments, a notification, or any combination thereof. Any suitable combination may be displayed on a display (e.g., a display screen). For example, some exemplary embodiments of the methods described herein may include tracking surgical instruments. In certain exemplary embodiments, the method includes providing a summary report of the ingredients. providing a notification based on a comparison between the first tally and a second tally, The method may include generating an alert in response to a mismatch between the two tabulations. The alert may include generating an alert in response to a mismatch between the two tabulations. For example, the alert may display a given protocol or may be sent via a protocol. The device can guide the user through the process, or both.

[0012] The present specification provides, among other things, a method for identifying, tracking, and locating surgical instruments. Exemplary systems are also described. A system for tracking surgical instruments includes one or more For example, one or more processors may be A first compilation is received, one or more images are received, each image being one or more surgical images. Depicting a field of view including the instrument, based at least in part on the one or more received images. , the one or more processors may be configured to determine a second tally of the surgical instruments. may also be based on a first tally of surgical instruments, a second tally of surgical instruments, or both. For example, the notification may include a first count of surgical instruments and a second count of surgical instruments. The one or more processors may also provide a second summary based on the comparison to the one or more or configured to identify one or more surgical instruments depicted in the plurality of received images. The one or more surgical instruments may be made of a surgical fabric (e.g., a surgical sponge or At least one of the received images may be a depth image. In some exemplary embodiments, the depth image is an infrared image. The one or more processors may also include at least one and adapted to identify and characterize a container depicted in one or more images. In some exemplary embodiments, the one or more processors may be configured to The method is configured to determine one or more parameters of the device. Example parameters include capacity, The geometric arrangement of one or more compartments within the device, the surgical use of each of the one or more compartments, The container may be fitted with or without fixtures, or both. and a plurality of pockets arranged in a grid of the container. The container may be lined with opaque material, colored material, or any of the above. The present invention may be implemented in any suitable combination.

[0013] Certain exemplary embodiments of the systems described herein include one or more processes. The optical sensor further includes an optical sensor configured to generate an image received by the optical sensor. In some exemplary embodiments, the system displays a summary report of the tracked surgical instruments. The display screen may also display, for example, the first tally and the second tally. The system may be configured to display a notification based on a comparison between the measurement and the actual measurement. According to an embodiment, the notification is based on a predefined protocol if the first tally does not match the second tally. According to certain exemplary embodiments, one or more processors At least one of these is included in a mobile computing device. The computing device can be detached and mounted on a stand or other suitable mounting device. It can be attached to a Noh.

[0014] Certain exemplary embodiments of the methods described herein include one or more surgical instruments. determining, by one or more processors, the presence of For example, tracking of a surgical instrument may include receiving an image of a field of view, includes one or more surgical instruments and detects the presence of the one or more surgical instruments in the image. and providing an indication of the determined presence of one or more surgical instruments in the image. Certain exemplary methods include detecting a feature in an image based on at least one machine learning model. The method further includes quantifying one or more of the surgical instruments. In some embodiments, the machine learning model may incorporate one or more deep learning techniques. Other computational techniques may additionally or alternatively be used to identify the location of one or more surgical instruments in the image. An exemplary method includes: Determining the presence of one or more surgical instruments may include detecting the instruments in the image. The method may include detecting the presence of a container, the container including a plurality of compartments, each compartment including at least The surgical instrument is configured to receive at least one respective surgical instrument. An instrument may consist of one or more surgical fabrics, one or more surgical instruments, or any combination thereof. According to some exemplary embodiments, the present invention may include any suitable combination of the above-described features. The disclosed method includes sorting each compartment of the container based on the presence or absence of a surgical instrument. Providing an indication of the determined presence of the surgical instrument includes detecting at least one of the surgical instruments depicted in the image. Providing an indication of the determined presence of the surgical instrument.

[0015] Example of a method for tracking surgical instruments Described herein is a method for tracking (e.g., identifying) surgical instruments during a medical procedure or the like. 1 is an exemplary method for detecting, locating, detecting an object over time, etc. Such a method for tracking may include a processor, an optical detection system, a computer vision system, application, machine learning, deep learning algorithms, or any suitable combination thereof. Various computational components are used to facilitate the process of identifying, tracking and locating surgical instruments. Figure 1 shows a system for tracking surgical instruments during a medical procedure. A schematic diagram of some exemplary embodiments of the method 100 is provided. The method 100 includes one or determines a first count of the plurality of surgical instruments S110 and determines one or more surgical instruments S12 determining a second tally of 0 and a first tally of the one or more surgical instruments and one or more providing a notification S130 based on at least one of the second counts of the surgical instruments. For example, a notification may be provided based on a comparison between the first tally and the second tally ( For example, check for a match between the first and second aggregates. The sum of the first and second aggregates, tracking progress toward incrementally increasing the aggregate until If the match fails (e.g., there is no match), warn the user or A perfect combination.

[0016] An additional exemplary embodiment of a method 200 for tracking a surgical instrument is shown in FIG. The method 200 includes, with one or more processors, generating a first count of the surgical instruments S220. Receive one or more images S230, each image showing one or more surgical instruments. and based at least in part on the one or more received images, determining whether or not a surgical instrument is located. The method 200 may include determining a second count of the surgical instruments S240. providing a notice based on a comparison between the tally and a second tally of surgical instruments; It can be seen.

[0017] In some exemplary embodiments, the first collection of surgical instruments is associated with a first time. while a second count of surgical instruments may be associated with a second time after the first time. For example, the first count of surgical instruments may be the "initial count" and may be the instruments that are introduced into the operating room and packaged. corresponds to the number of surgical instruments removed from the equipment or prepared for use during a medical procedure. The second tally of surgical instruments may correspond to the number of surgical instruments used during a medical procedure. As used herein, the term "used" refers to equipment intended for disposal. Used instruments may be soiled or unsoiled (e.g., blood, other liquids, other The second tally may be calculated at intermediate times. Measured at a point in time or at the end of a procedure (e.g., after a procedure is completed, before the end of surgery) The "final" point in time can be measured as the "final tally."

[0018] Maintain (e.g., update) and compare the number of surgical instruments at various points in a medical procedure. This can facilitate accurate location of surgical instruments. Some exemplary implementations may provide the advantage of preventing the device from being retained within the patient. In terms of form, the method of comparing the counts is to match the initial count of surgical instruments with the final count of surgical instruments. This may include having the system check whether the initial tally matches the final tally (e.g., checking that the initial tally matches the final tally). ). This match ensures that all surgical instruments are accounted for before the end of the procedure. According to certain exemplary embodiments, the present invention provides an advantage in determining The method provides a notification to the user based on whether the first tally matches the second tally. If the first count and the second count do not match, such method may further include: notify or otherwise alert the user to take one or more predetermined actions; This may include performing a procedure to locate a surgical instrument (e.g., searching for missing surgical instruments).

[0019] As described above, for tracking (e.g., identifying and / or detecting) surgical instruments, Any one or more of the methods described herein may employ one or more computational techniques. In some exemplary embodiments, the surgical instrument tracking system may be implemented as a Such methods may involve computer vision, machine learning, deep learning algorithms, or other For example, computer vision can involve the use of images, a sequence of The method may include one or more computational techniques for identifying objects within an image or frame of a video. As another example, machine learning or deep learning techniques can include detecting associated surgical instruments in images. By identifying complex patterns, it is possible to train and plan to perform specific tasks. This may involve the use of mathematical models (e.g., a still image, a series of images, or a video frame). In some machine learning techniques, a computer algorithm Contains labeled images (e.g., labeled with features associated with surgical instruments) By being trained on a variety of datasets, it is possible for a machine to perform complex tasks. It "trains" the machine to recognize certain patterns using a labeled dataset. You can train a machine learning model to perform a programmed task. can be performed with a high level of accuracy. In some implementations, if a model is used But over time, we update our algorithms based on the actual data we receive, and The learning curve can be improved by continually updating the sensor to improve object tracking, detection, or both. For example, machine learning can be used to identify one or more instruments, Deep learning can generate models that measure the accuracy of measurements, aggregate measurements across multiple instruments, or both. It is a type of machine learning that uses multiple layers or analyses to extract relevant features from input data. Based on artificial neural networks that extract signatures and detect, identify, or track surgical instruments. As used herein, the term "computational techniques" refers to the techniques described herein. The term should not be limited to describing any one or more of the methods used to Additionally, some implementations may combine aspects of multiple computational techniques. For example, one or more computer vision techniques may include machine learning, deep learning, For example, a code used to identify a surgical instrument may be implemented through a Computer vision can be achieved by using machine learning algorithms, deep learning algorithms, or both. can be utilized to perform identification of the surgical instrument.

[0020] According to various exemplary embodiments, a mobile computer, such as a mobile phone or tablet, The tracking device may be configured to use one or more of the methods described herein for tracking a surgical instrument. or a plurality of different aspects of the present invention. For example, one on a mobile device or using a software application executable by multiple processors, Determining a first tally, determining a second tally, comparing the first tally with the second tally, and Based on the received message, one or more actions may be performed, including notifying the user. The one or more processors on the mobile device can be configured to analyze the images. The one or more processors of the device may perform any one of the methods described herein or In relation to multiple, machine learning algorithms, computer vision technology, deep learning algorithms The system may be configured to execute any of the following algorithms: In an exemplary embodiment, the mobile application includes a method for tracking surgical instruments. Execute all steps of the method and generate a result based on one or more operations (e.g., steps) of the method that were executed. The mobile device is configured to display information to one or more users via a Implementing one or more of these methods of tracking surgical instruments can facilitate accurate and timely tracking of surgical instruments. It not only automates and streamlines the certification process, but also makes it easy to locate surgical instruments accurately. Provides additional virtual presence that guides users through the proper location verification protocol to Moreover, implementing the computing techniques on mobile devices provides the advantage of This provides an improved way to track surgical instruments in a highly accessible and intuitive way. In other exemplary embodiments, some of the operations of the method or All of this may be performed using one or more external processors. using a mobile device to capture one or more images, receive information, and display information and use an external processor to perform one or more machine learning algorithms. The analysis of one or more images can then be performed.

[0021] Example of determining the first aggregate Some exemplary embodiments of a method for tracking a surgical instrument include: In some implementations, the first tally includes receiving a tally of the medical procedure use. Intended or prepared for use (e.g., introduced into an operating room, removed from packaging, or both) and a corresponding number of one or more surgical instruments may be established. The tally can be established once at any point during the procedure. In some implementations, the first tally is , is or includes the number of surgical instruments introduced before or at the start of a surgical procedure. In some implementations, the first tally is then updated throughout the course of treatment (e.g., For example, when a surgical instrument is introduced into an operating room, removed from its packaging, or For example, one package of surgical instruments is introduced into the operating room and at the start of the procedure ( For example, the patient may be removed from the packaging (e.g., before the surgical phase of the procedure begins). At some point during the medical procedure, such as while the patient is being examined, a second package of surgical instruments is collected and delivered to the operating room. The first count is then introduced and the surgical instrument can be removed from the package. The number of surgical instruments in the package may be updated (e.g., incremented) at any time during the procedure. number of surgical instruments can be introduced, and each time a new set of surgical instruments is introduced The first tally can be updated.

[0022] Establishing a first tally of instruments allows for the accounting of all surgical instruments at the end of the procedure. For example, the methods described herein may provide the benefit of facilitating the delivery of The tally of instruments present in the first tally is compared to the first tally to confirm the location of all surgical instruments. This may include determining whether the patient is in a suitable environment (e.g., as opposed to being retained within the patient). Establish a first collection of surgical instruments, as described below, using any suitable method. In some implementations, establishing the first tally of the devices can be performed manually. For example, a first tally of surgical instruments may be received as a manual or other user input. As another example, the first collection of surgical instruments may additionally or alternatively include: by one or more processors (e.g., one or more The computational technique for establishing the first tally may be, for example, Compared to manual methods, it improves accuracy and reduces user error to establish first tally and This may provide the advantage of streamlining the process of determining the aggregate.

[0023] In some exemplary embodiments, as described above, the first tally of the instruments is based on user input and For example, establishing a first tally of surgical instruments may be received using manual tally techniques. The user may manually count the plurality of surgical instruments to determine a first count, A first tally can be provided via the user interface. For example, a user can Receiving a package containing one or more surgical instruments and retrieving the instruments from the package, e.g. When the user separates the fibers from each other, such as while placing them on a surface such as a table In some implementations, the manual counting can be done by a user. entered into a computing system by (e.g., manual entry on a keypad or touch screen) For example, a user may use a program executable on a mobile computing device to Manual tallies can be entered into the calculation system via the mobile application. In some implementations, the mobile application may prompt the user to enter a manual tally. do.

[0024] In some exemplary embodiments, the determination of the first count of surgical instruments may include additional or alternative Alternatively, computer vision, machine learning, deep learning, other computational techniques, or their equivalent. For example, establishing the first tally may include using one or more Segment the image to identify one or more surgical instruments and aggregate one or more surgical instruments The method may include analyzing one or more of the data items counted in the first count, or both. The image of the surgical instrument may be acquired using any suitable imaging device. For example, on a mobile computing device (e.g., a mobile phone, a tablet, etc.) and capturing images of one or more surgical instruments using an imaging system such as The images can be still images or frames from a video feed. In some implementations, multiple images (e.g., multiple frames from a video feed) are processed. In some implementations, the image is processed to determine a first count of the surgical instruments. In some implementations, the image may be or include a color image, but in other implementations, the image may be a depth image (e.g., a red X-ray, stereoscopic, ultrasound, etc.), hyperspectral imagery, or other suitable types of images. Additionally, the multiple images of the one or more surgical instruments may be simultaneously The images may be acquired simultaneously or sequentially and may be combined with different types (e.g., depth images). In general, the one or more processors may include one or more to receive one or more images, or to analyze one or more images, or both. Additionally, the one or more processors may be configured with computing technology (e.g., (using machine learning, deep learning, or any suitable combination thereof) Implement a method to identify one or more surgical instruments and aggregate one or more surgical instruments. One or more processors may perform the following operations on the received image: The present invention relates to a processor for a surgical instrument, the processor comprising: It can be implemented in a computing device such as a In some implementations, a mobile application executable on a computing device The application may be used in combination with any of the modifications described below to provide a first 1. The present invention is capable of acquiring, receiving, and processing data relating to the aggregation of

[0025] In some exemplary embodiments, image analysis is used to determine the first count of surgical instruments. The act of performing the procedure includes the following operations: The user applies one or more surgical instruments, such as surgical fabrics, to the The user then obtains a package containing the equipment and brings the package to the operating room. The user may then Remove surgical instruments from their packaging, separate the surgical instruments, and place them on a table or other surface. Next, the user can capture the image using a camera or other device on a mobile computing device. and acquiring one or more images of the surgical instruments on the table using the imaging system. The processor of the mobile computing device then analyzes the image to ,Identify individual surgical instruments in an image, tally up the number of surgical instruments in an image, or perform other For example, the first collection of surgical instruments is computer vision, machine learning, deep learning techniques, or any suitable combination thereof. In determining the second tally of surgical instruments, which can be determined using computational techniques With respect to the process, any one or more of those described in more detail below may be included. The sensor can store information regarding the detection and counting of surgical instruments in a memory. In some implementations, the first counting determination includes determining a processor count of surgical instruments in the image. updating the first tally by adding the number to the number of surgical instruments previously tallied; (e.g., surgical instruments depicted in images taken at a prior time point). Alternatively, the first tally determination may include tallying the number of surgical instruments in images taken at a given time. This may include counting the number of surgical instruments (e.g., if no surgical instruments have been previously counted).

[0026] In some exemplary embodiments, the one or more computational techniques include: Detecting a characteristic arm movement of the user to determine the tally. The movements can be characterized in any suitable manner, for example by detecting characteristic arm movements. When removing the surgical fabric from the packaging, the user must The user separates surgical instruments by "distributing" the fibers on the surface and then counting them. The imaging system may include identifying "countermeasure" actions to be taken when The system may be positioned to view the user's arm movements (e.g., front view, side view, overhead view, etc.). , or any suitable combination thereof), the processor may The user may remove the surgical fabric from the package and place it on a surface. The sweeping (e.g., distributing) arm movement when placing the surgical fiber is a good indicator of the imaging system. and can be detected using computational analysis (e.g., performed by a processor). Thus, each arm movement detected by the processor is converted into an aggregated surgical instrument. and may contribute to a first tally of surgical instruments.

[0027] In certain exemplary embodiments, the one or more computational techniques utilize the weight information to For example, surgical instruments can be placed on the scale in sequence, and the total number of surgical instruments is determined. The incremental weight changes are then counted by one or more processors to determine a first tally of the ingredients. It is possible to detect the presence of additional surgical instruments on the scale. Based on the incremental weight change, the processor can detect the presence of additional surgical instruments. Each time the processor detects an incremental weight change, the processor You can increase the total.

[0028] Additionally or alternatively, according to some exemplary embodiments, a plurality of surgical instruments may be The weight of each individual surgical instrument can be used to determine the surgical instrument weight. A first count of the instruments can be determined. For example, the user can select the first count of the instruments by selecting the user interface. The user may be prompted to enter the type of surgical instrument to be weighed via the The type can be searched for, for example, by searching a database of existing surgical instrument types. by navigating the hierarchical menu of surgical instrument types or by selecting a surgical instrument type. The type of surgical instrument can be entered by, for example, entering the function (e.g. fabric, surgical instrument type, etc.) or instrument-specific information such as brand, size, material, etc. They may be classified or otherwise distinguished based on their characteristics. The surgical instrument type may be determined based on other identifications (e.g., (by scanning a bar code or radio frequency identification (RFID) tag attached to the scale). The receiving processor may then assign weights to a single surgical instrument based on the type of surgical instrument received. Thus, information regarding the weight of the surgical instrument can be received. , either manually entered by a user or stored in memory in a look-up table; or by scanning electronic tags, such as bar codes or RFID tags, on surgical instruments. Based on the information regarding the weight of a single surgical instrument, the processor may The total weight of the instruments is calculated by dividing the total weight of the instruments by the known weight of a single instrument. The number of surgical instruments can be determined.

[0029] As described above, the mobile application may include one or more It can be used in conjunction with weight-based methods. Figure 3 shows how weight can be measured through the use of a scale. A graphical user interface 3 showing information relevant to determining the first tally 3 illustrates an exemplary embodiment of the graphical user interface of FIG. The display shows the type of surgical instrument 310 being used and the current determination 320 of the first tally. The graphical user interface 300 allows the user to place a new surgical instrument on the scale 330. In addition, the graphical user interface 300 The user may manually override (340a) the first tally to provide a manual tally and then recalculate the first tally. Selectable icons 340a and 340b allow the user to exit the b) (e.g., all surgical instruments of a selected type are, at least for now, , indicating that it is being aggregated into the first tally).

[0030] In some exemplary embodiments, one or more of the following may additionally or alternatively determine the first tally: One or more computational technologies may be used to identify a scannable tag (e.g., RFID, barcode, or similar). For example, a surgical instrument may be used in a surgical room. when they are introduced into the body, when they are removed from the packaging, or at some other time prior to use. In other situations, the packaging of the surgical instruments may include individual RFID tags that can be scanned. The present invention is configured to allow the location of multiple surgical instruments with a single scan of the available tags. For example, packaging for surgical instruments may include a scannable tag for inclusion in the first tally. Additionally, the package may include one scannable tag that communicates to the processor the number of devices in the package. Thus, the method of determining the first tally includes using a surgical instrument or hand to determine the first tally. This may include scanning a tag on the packaging of the surgical instrument.

[0031] Determining the first tally may include one or more of the methods described above. In a typical embodiment, multiple methods for determining the first tally are used to examine the accuracy of the first tally. The methods for determining the first tally may be combined in any suitable manner. For example, detecting incremental weight changes on a scale or detecting known weights for a single surgical instrument. We use weight-based methods such as the use of weight-based information to identify and count surgical instruments in images. It can be combined with image-based methods such as using one or more computational techniques. As another example, image-based methods may include one or more methods that involve recognition of characteristic movements of a user. It can be combined with multiple calculation techniques. It can use one or more manual counting techniques. and verifying any one or more of the methods described herein for determining the first count. In some exemplary embodiments, the mobile application may The user is prompted to verify the first aggregate determination generated using the technique. In an exemplary embodiment, as described above with respect to FIG. 3, the method includes: If the user determines that the first tally is inaccurate, the user may choose to overrule the first tally automatically or by calculation. This includes the ability to override the settings.

[0032] In some exemplary embodiments, determining the first count of surgical instruments comprises dividing the surgical instruments into groups. The determination of the first count includes the distinction between different surgical instruments, the classification of the same surgical instruments, and Different types of determinations, or both, may be included. For example, different types of surgical fabrics may be distinguished from one another (e.g., by material, function, size, brand, or any of these) Another example would be to use two different types of surgical instruments in combination with each other. (e.g., a scalpel from a pair of scissors). 1. Using any suitable method, including any one or more of computational techniques, to obtain one or more The type of surgical instrument or instruments may be determined, such as the type of surgical fabric. In some implementations, the first aggregation decision is to determine all aggregated items, regardless of type. In addition to the first tally of all surgical instruments, a separate category of first tally may be presented. For example, the first collection may include a type-specific first collection of 18" x 18" surgical fabrics. and another type-specific first tally of 4 inch by 4 inch surgical fabric. do.

[0033] Any of the methods described herein for determining a first count may be used to determine a second count. Similarly, to determine the first tally, Any of the methods described herein for the treatment of a surgical instrument may additionally or alternatively be used to Any other suitable analysis may be performed (e.g., using used (tracking equipment, tracking unused equipment, or both).

[0034] Example of determining the second aggregate Some exemplary embodiments of a method for tracking a surgical instrument include: As described above, the second count of surgical instruments may further include determining a count of the number of surgical instruments. The second tally can be compared to the first tally of the procedure (e.g., surgical procedure, etc.). The second tally may be determined at any appropriate time during the medical procedure. For example, the second tally may be determined at any appropriate time during the procedure. may be an "interim summary" determined at an intermediate point during treatment between the start and end of Interim determinations may be made during (e.g., throughout) the course of a procedure (e.g., when a surgical instrument is in use). Updates the surgical instrument index counters when the surgical instrument is used or designated for disposal Comparing the interim tally to the first tally to determine whether the remaining The number of surgical instruments that have been "used" can be shown. The second tally is the final collection at the end of the procedure. The final count can be compared to the first count to ensure that all surgical instruments have been accounted for. Determine whether surgical instruments are retained in the body of a patient undergoing a procedure and reduce the risk of retaining the surgical instrument in the body. It may be desirable to limit the amount of time that a treatment can be stopped. The end of treatment may be indicated in any suitable manner. For example, For example, the end of the procedure may occur when the patient is ready to be removed from the operating room (e.g., the surgical site is after the surgical site has been closed), or before the patient's surgical site has been closed but after the surgical procedure has been completed. could be.

[0035] Additionally or alternatively, any of the methods described herein for determining the second tally. can be used to determine the first tally. Similarly, and / or using any of the methods described herein for producing a surgical instrument. Any other suitable analysis of the above may be performed (e.g., using used equipment tracking, unused equipment tracking, or both).

[0036] Example of using tally bags In some exemplary embodiments, counting the surgical instruments includes determining the location of the surgical instruments. The use of a tally bag to check the accuracy of the tally bag allows for accurate measurement of the surgical fabric. Systematic counting may be facilitated and the risk of surgical instruments being retained inside the patient may be reduced. One or more users may choose to have their data discarded in order to facilitate efficient and accurate counting. The surgical fabric designated for may be placed in the pocket of the tally bag. Disposal of fibers or other equipment may, in some exemplary embodiments, be accomplished using a multi-stage equipment collection process. For example, in some operating room procedures, surgical fabrics may be used in the procedure and then They are stored in one or more collection containers (e.g., near the operating table or elsewhere in the operating room). "Kick Bucket" in the form of an example of a metal bucket lined with a plastic liner placed During or after the procedure, one or more healthcare professionals may then 2. Remove the surgical fibers from the collection container or containers and collect the surgical instruments for counting. In another exemplary embodiment, the surgical fabric can be placed in a separate collection bag. It can be placed directly into the tally bag without the use of a collection container. The location of surgical fibers can be confirmed using a tally bag. The term "used in" refers to surgical fabrics that have been used to absorb blood or other fluids. are not limited to those introduced into the operating room and then considered disposable or otherwise It may refer to any surgical textile designated for disposal at the time of disposal. According to an embodiment, manual counting of surgical fibers is performed according to a specific protocol as described below. and placing a surgical instrument, such as a surgical cloth, in a compartment of the tally bag in accordance with the method of the present invention. or one or more computational techniques to locate surgical instruments. Errors related to manual tallying using one or more tally bags using the method can be automated and reduced.

[0037] In some situations, healthcare professionals may use one or more tally bags to hold surgical instruments. A standardized procedure or protocol can be used when counting ingredients. One such protocol for use in the operating room is shown. The pockets of the tally bag 600 are numbered in the order shown, from top to bottom and right to left. 610. Thus, if the procedure is performed correctly, the first filled pocket The first pocket filled is the bottom right corner pocket (pocket 1), and the last pocket filled is the top left corner According to another protocol, the tally bag 600 is It may be filled from top to bottom and left to right, but any other suitable protocol may be followed. can be done.

[0038] FIG. 4 shows a schematic of a tally bag 400 for use in a medical procedure. 4. The pouch 400 includes a lining 402 and a number of pockets 410 or other compartments. Thus, the tally bag 400 includes ten pockets arranged in a rectangular array. The ten pockets 410 are arranged in five rows, each row being one of the pockets 410. Each row may include, for example, a larger gap formed between the backing 402 material and the front material. The larger pocket 412 may include a larger pocket 412 The pocket 410 may be divided into two smaller pockets 410 with a seal 414 forming a seam therebetween. However, the tally bag 400 may include any suitable number of pockets 410. The pocket 410 may be configured in any suitable manner. For example, the pocket Some or all of 410 may be separate compartments, and some or all of pockets 410 may be separate compartments. All are formed from a single larger pocket (e.g., For example, a larger pocket 412), or any suitable combination thereof. The tally bag 400 is attached to a stamp such as a pole 430 (e.g., an intravenous (IV) pole). The pole 430 may be attached to one or more (e.g., two) hooks. The tally bag 400 may include a lateral mounting rod 431 including a tally bag 432. One or more (e.g., two) hooks 432 that allow the bag 400 to be hung from a hook 432. The tally bag 400 shown in FIG. 4 is rectangular and may include one or more openings 404. 10 rectangular pockets 410, and the tally bag 400 and its pockets 410 are It may be of any suitable configuration.

[0039] The tally bag 400 may include (e.g., may be made from) any suitable material. For example, the lining 402 of the tally bag 400 may be made of a polymer, plastic, silicone, or the like. cellulose, nylon, rayon, any other suitable material, or any suitable combination thereof. As another example, the lining 40 of the tally bag 400 may be made of a flexible material such as polyester. 2 may be made of rigid plastic, polymer, metal, any other suitable material, or any of the foregoing. The lining of the tally bag 400 may be made of a more rigid material such as a suitable combination of The light source 402 may be transparent, translucent, opaque, colored, or any suitable combination thereof. For example, the backing 402 can be opaque blue. 402 (e.g., opaque blue) is used to individually select the tally bag 400 from among multiple tally bags. This allows for easier identification (e.g., multiple tally bags can be stacked on the same pole 430, In the case of layered in front of each other or multiple poles (e.g., pole 430) in front of each other If the fabric is layered, it may provide a contrasting background against which the surgical fabric stands out. The pocket 410 can be made of the same or different material as the backing 402. The support 410 can be made of a flexible plastic, silicone, or polymeric material. The pocket 410 made of a flexible material allows for easy placement of surgical fabric therein. The pocket 410 may also be transparent or opaque. In some exemplary embodiments, one or more pockets 410 may be used to hold surgical instruments. It is transparent to allow easy visualization and tabulation of

[0040] The tally bag 400 shown in FIG. 4 is rectangular and has ten rectangular pockets 410. The tally bag 400 may be of any suitable configuration. For example, the tally bag 400 The backing 402 may be rectangular, substantially square in shape, oval, triangular, or any suitable shape. The tally bag 400 may have ten pockets 41 as shown in FIG. It is not necessary to have 0, but may include any suitable number of pockets 410. For example, The tally bags are 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15. , 16, 17, 18, 19, 20, or any suitable number of pockets 410. FIG. 4 shows a tally bag 400 configured to be suspended from a pole 430. The tally bag 400 may be attached in any suitable manner. For example, the tally bag 40 0 may have one or more openings 404 or any other suitable mounting location. For example, the tally bag may be configured to be attached to a wall hook. A frame configured to be placed on top of or on a door or cabinet. As another example, the tally bag 400 may be attached to a mounting rod or The tally bag 400 may be attached to a wall with adhesive or other suitable fasteners. The bag does not need to be sized to fit the dimensions of a 40 0, its pocket 410, or both, may be in contact with other tally bags, their pockets, or Both can be of different sizes, and there are various sizes of surgical fabrics, It houses a variety of surgical instruments.

[0041] In some exemplary embodiments, the pole 430 also mounts an imaging device. The one or more imaging devices may include a mount for imaging the first aggregate. A method for tracking surgical instruments, including a first tally determination, a second tally determination, or both. Generate one or more images for use in various components (e.g., operations) In particular, the tally bag 400 is attached to a pole 430. The imaging device or devices determine a second count of the surgical instruments in the count bag 400. For example, a mount for an imaging device can be used to provide a summary 400, so that the user can easily take (e.g., capture) an image of the group 400. The user does not need to hold the imaging device while taking the image. The mount may be fixedly attached to the pole 430 or may be removably attached to the pole 430. Additionally, the mount may be adjustable along the pole 430 (e.g., For example, the pole 430 is configured to slide vertically up and down, and rotates around the pole 430. (e.g., adjustable to the left or right side of the pole 430), or both. FIG. 5 illustrates a mobile device 550 (e.g., a tablet or smartphone). a mount 540 attached to a pole 530 configured to hold a Schematically illustrates an exemplary embodiment of a tally bag stand 500. In this position, the mount 540 is adapted to securely and releasably hold the mobile device 550. The grip 542 is configured to be attached to the mount 540 and the mobile phone. The friction fit between the mobile device 550, the adjustable clamp, the mount 540 and the mobile device 550 A snap fit with a mating part on the roller device 550, one or more magnets, a mount a removable adhesive layer attached to the mobile device 540 or the mobile device 550; The mobile device may be connected to the network using any suitable mechanism, such as through any suitable combination thereof. In the exemplary embodiment shown, the mount 540 can hold a vice 550. The mounting mechanism 544 is fixed to the pole 530. , clamps, clasps, hinges, threads, one or more fasteners (e.g., mechanical fasteners Any suitable mounting method, such as a screw, adhesive, or both, or any suitable combination thereof. For example, the attachment mechanism 544 may be sized to fit the pole 530. and an adjustable clamp (e.g., an adjustable jaw) configured to attach and adjust to the ) or expandable sleeves or other inserts (e.g., inflatable foam, etc.) In some exemplary embodiments, the attachment mechanism 544 may include a The pole 530 may include one or more fasteners configured to couple to the pole 530 (e.g., 540 and pole 530 are inserted through the openings to attach the nut, other threaded backing, The mount 540 in FIG. 5 is Although depicted as extending horizontally from the pole 530, the mount 540 is mobile. The camera of the device 550 captures a tally bag (e.g., The camera may be positioned to capture an image of the camera 400 at any suitable angle. Additionally, the mount 540 may be attached to any suitable mounting surface of the tally bag stand 500. For example, the mount 540 may be secured to a tally bag stand 500. The pole 530, the mounting rod 531, or the hook 532 may be fixed to the The mount 540 can have any suitable configuration. The mount 540 may be substantially straight, or the mount 540 may include a curved configuration. Mount 540 is adjustable in length and angle using a gooseneck structure. (e.g., adjustable in angle between mount 540 and pole 530), or any of the above. This can be any suitable combination according to the particular needs of the user or the procedure at hand. This may provide the advantage of allowing a user to adjust the position of the mount 540. do.

[0042] Examples of computational techniques for tracking surgical instruments In some exemplary embodiments, one or more computational techniques are used to Surgical instruments that may be placed in multiple tally bags (e.g., tally bag 400) Identify, count, or both. Automate and streamline the process of locating surgical instruments. In order to achieve this, one or more such counts may be combined with one or more count bags. It may be desirable to use computational techniques to improve the accuracy of one or more aggregates; Therefore, compliance with the protocol for using one or more tally bags is required. For example, one or more images of the tally bag 400 may be taken. 400 , using one or more computational techniques to Surgical fibers can be identified and counted (e.g., as described elsewhere in this document). (Use computer vision, machine learning, or deep learning to make the The user may view a screen of a mobile device 550 (e.g., a mobile computing device). taking an image of the tally bag 400 using an imaging system such as a FIG. 7 illustrates a mobile device 720 being used to capture an image of a tally bag 730. As shown in FIG. 7, the user 710 is The camera 722 is positioned so that the tally bag 730 is within the field of view of the camera 722, as shown by the dashed line. 7. Arranging the imaging system in the exemplary form of a mobile device 720 with 722 One or more images generated by the camera 722 of the mobile device 720 can be The images can be analyzed using one or more of the computational techniques described herein. FIG. 7 shows a mobile device 720 held in front of a tally bag 730 to generate an image. 7 shows a user 710 using a mouse, while the imaging system (e.g., a mobile device 720) 5, or via a mount (e.g., mount 540 shown in FIG. 5) in any suitable manner. It may be located adjacent to the tally bag 730 or or angled toward a pocket (e.g., pocket 410) of tally bag 730. The tally bag 730 is directly on the tally bag 730.

[0043] Tracking algorithm model example FIG. 8 illustrates a method according to some exemplary embodiments, which includes various training The model is placed in one or more tally bags (e.g., tally bag 400). In addition to the method 800 (e.g., tracking algorithm) used to track the surgical instrument, 8 illustrates exemplary relationships and data flows between possible components of the method 800. Each model can use machine learning, deep learning, or both to achieve a sub-goal. . The algorithms of each model of method 800 can be combined (e.g., in a unique way) to create one Or arrive at the exact location of surgical instruments in multiple tally bags and provide additional information to the user In some exemplary embodiments, a method for tracking a surgical instrument may include: , relying on two or more independent aggregation approaches, as well as one or more error mitigation techniques To arrive at a robust method for tabulating surgical instruments, the method 800 described with reference to FIG. (e.g., considered as a global tracking algorithm) is described in terms of tally bags. However, method 800 may be implemented using an instrument (e.g., a surgical instrument) that is configured or included in any suitable manner. ) can be used to aggregate them. are not required to be performed in the order described or depicted in the flow chart shown in FIG. This may be performed in any suitable order.

[0044] As shown in FIG. 8, the input of the method 800 is one or more aggregate bags (810). The image of the one or more tally bags 810 may be captured by any suitable imaging device. The information may be obtained in any suitable manner using the data. In this example, a user may enter a card number on a mobile device 720 (e.g., a mobile phone or tablet). A digital camera 722 is used to capture an image of the tally bag 730. Among the various imaging systems such as cameras, computer-mounted cameras, video cameras, etc. Any one or more of the following can be used to obtain an input image. The image is then processed using the The image may be received by a processor 820. The image may be pre-processed 820 for subsequent analysis. Image pre-processing may involve, for example, brightness normalization, contrast correction, and one or more deep learning techniques. Reducing the image size to dimensions suitable for processing by learning or computer vision models, or or any suitable combination thereof. A count of the surgical instruments in the bag 400 or 730 (e.g., first count, second count, , further aggregation, or any suitable combination thereof) may be determined by determining the aggregate bag in the image. A tally bag detection model 840 for detecting the presence of a tally bag, using geometric features (e.g., tally bags) and the dimensions and other characteristics of the pocket or pockets in the tally bag a tally bag shape model 830 for detecting the presence of any implements in the tally bag; Model 850 for detecting surgical instruments in each pocket (e.g., general or An empty / full pocket classifier model 860 for determining the presence or absence of a particular type of pocket, or In order to synthesize information about the surgical instruments in the imaged count bag, Instrument aggregation model to provide an aggregation of surgical instruments within a group, or both 870, analyzing the image using one or more of a variety of computational techniques. In some exemplary embodiments, after any pre-processing operations, the method 800 includes For example, one path may be used to directly carry surgical instruments in a collection bag. One path may be based on detecting pockets in a tally bag and detecting pockets. The method 800 may then be based on classifying the multiple counting paths as empty or full. The results of the tests can be combined and then used to detect errors using one or more error detection algorithms. , final location confirmation of the surgical instruments can be reached.

[0045] The tally bag detection model 840 may be configured to pre-process one or more tally bags for analysis. The system can receive processed or raw images. 40 is based on at least one trained machine learning or deep learning model The presence of at least one tally bag in the image may be identified based in part on the An example of an object detection model is a faster region-based convolutional neural network (R- CNN), Single Short Detector (SSD), and Region-Based Fully Convolutional Networks (R-FCN) and can achieve high performance in object detection tasks. 840 also utilizes one or more of such trained models to generate a and identifying a location of each tally bag within the tally bag and detecting one or more edges, corners, or Both, or any suitable combination thereof, may be identified or tallied 844 For example, the exemplary graphical user interfaces shown in FIGS. As shown, the count bag detection model 840 detects and indicates the presence of the count bag 910. In the image (e.g., shown with a corresponding bounding box), The location, one or more boundaries of the tally bag 910, or both may be indicated. In some exemplary embodiments, the count bag detection model 840 performs a count bag detection process according to the identification of the count bag. 3. Crop the tally bag image (e.g., the portion of the image that corresponds to the identified tally bag) (to effectively separate the bags) and a cropped tally bag image 84 for later use. It can output 2.

[0046] The tally bag shape model 830 includes geometric information about one or more tally bags, For example, the vertical and horizontal dimensions of the tally bag (e.g., tally bag 400 or 730); The pocket's location (e.g., pocket 410), or both, may be encoded and identified. The tally bag shape model helps to identify specific geometric features of the tally bag. 0, with or without other geometric information. For example, the make (e.g., brand or manufacturer) of the tally bag, model, or It may depend on the specific manufacturer or manufacturer of the tally bag, or both, and may be pre-coded (e.g. or manually enter the values ​​associated with the model), learned from training data In some exemplary embodiments, the aggregate bag shape model 830 may be Any other suitable information that may be useful in identifying and / or analyzing the shape of the tally bag. With or without, one or more of the following may be located at the corners 814 or edges of the tally bag: In some exemplary embodiments, the shape of the tally bag can be , identified based at least in part on knowledge of the manufacturer of the tally bag, which may be determined by user input. Force users (e.g., via the user interface of a mobile application) The identification code (e.g., bar code, The bag can be scanned using a QR code or directly from the bag image. By recognizing the manufacturer of the product (e.g., by its label, distinctive color feature, distinctive geometry, based on optical character recognition of the For example, a user can input the manufacturer of the tally bag, and the shape model can be Known information can be used to help determine the geometric information of the tally bag. As another example, the user may apply a reference marker or a tally bag, its pockets or both by changing the size of one or more virtual contours that define Therefore, you can manually define the boundaries of the tally bag, its pockets, or both on your mobile device. The aggregate bag shape model can be identified by a homography or one or more Other perspective transformations can be used to automatically identify the locations of pockets in the tally. Based on these positions, the aggregate bag shape model 830 generates an empty / full pocket classifier model Crop the image of each tally bag pocket for acceptance by 860. Thus, the tally bag shape model 830 includes a pocket image 832 (e.g., a tally bag pocket corresponding to a segment, a cropped image segment, or any suitable combination thereof. This can also provide information about pockets, the identification of the parts of the image that are 9, the position of the corner 834, or both. As shown in the exemplary graphical user interface, the aggregate bag shape model 830 detects the location of the tally bag, the shape of the pockets of the tally bag, or both; In the displayed image, the pockets of the tally bag (e.g., pockets 920A-920C) , with pockets indicated by corresponding bounding boxes.

[0047] As described above, a method 80 for tracking (e.g., by counting) surgical instruments is provided. There are several exemplary embodiments (e.g., variations) of the present invention. In the present invention, the surgical instrument detection model 850 is implemented using a deep learning algorithm or other suitable artificial intelligence. Using the model, each surgical instrument, as generated by the tally bag detection model 840, The surgical instruments (e.g., each surgical fabric) and the surgical instruments in the cropped tally bag image 842 The deep learning algorithm used in the surgical tool detection model 850 is A dataset representing a diverse array of aggregate bag images (e.g., a reference dataset or other training dataset), each for each surgical instrument in the image. The ID of the surgical tool is labeled. Therefore, we use deep learning to model surgical tools detection 850. The algorithm identifies surgical instruments (e.g., surgical tissue) within each tally bag in the image. The surgical instrument detection model 850 can perform aggregation basing, localization, or both. Data can be generated about the boundaries of each surgical instrument in the group. , this can be done, for example, by identifying a corner 852 of the fiber. The Surgical Instrument Detection Model 850 detects all detected items in one or more tally bags. Provides the location of surgical instruments and tallying of surgical instruments in one or more tally bags (e.g. , a first aggregate, a second aggregate, a further aggregate, or any suitable combination thereof) of It can enable decisions.

[0048] In certain exemplary embodiments, the empty / full pocket classifier model 860 is Analyze the image of one or more tally bags identified by the tally bag shape model 830. By doing so, it is possible to identify whether each pocket is full or empty. The resulting cropped images of each pocket are then fed into an empty / full pocket classifier model. 860. The empty / full pocket classifier model 860 determines which pockets are empty and labeled with an empty / full pocket identifier to identify which pockets are full. A deep learning algorithm trained on a dataset of tagged tally bag images. Examples of suitable object classification networks include Inception, ResNet, MobileNet, MnasNet, and EfficientNet These include: Empty / full pocket classification, which can achieve high performance in object classification tasks. The output 862 of the container model 860 is a count bag or bags that indicates which pockets are empty. It also contains information about which pockets are full, thus making it possible to classify empty / full pockets. The instrument model 860 may include a count of surgical instruments in one or more count bags (e.g., a first aggregating, second aggregating, further aggregating, or any suitable combination thereof) 9 and 10 show the results of the empty / full pocket classifier model 860 output 862. The following shows an example of a display that corresponds to the information (e.g., surgical instrument detection model 850, empty / full pocket Based on a combination of the 860 metric classifiers, or both, the aggregate bag is , a full tally bag (e.g., as shown in Figs. 9 and 10), a partially full tally bag, 9 and 10. As shown, each pocket in the image of the tally bag is filled with a surgical sponge (e.g., pocket 9). 920A and 920B) or as detected surgical instruments. The pocket 920C may be separately identified as not containing any surgical instruments (e.g., pocket 920C). The instrument aggregation model 870 synthesizes the information generated by one or more of the aforementioned models. For example, the instrument aggregation model 870 The method relies on a priori information (e.g., the determined shape of the tally bag, the detection of surgical instruments, or both). The method can be used to perform post-processing, and one or more surgical instruments (surgical fibers) can be used. ), one or more aggregate bags, or both. and providing a final surgical instrument count, tally bag location, one or more surgical instrument counts, position, one or more empty pocket positions, and one or more full pocket positions. As shown in FIG. 8, the tool count model 870 is A total appliance tally can be determined in 872. For example, the appliance tally model 870 can A surgical instrument detection model 8 may utilize information from one or more of the aforementioned models. 50, an empty / full pocket classifier model 860, or both, and a tally bag 8 72. Further, in some exemplary embodiments, the number of surgical instruments per The tool aggregation model 870 may incorporate information from one or more of the other models described above in various ways. Combining methods to determine the instrumental summary using a combination of methods and Compare the tally arrived at using the Determine.

[0049] For example, the instrument aggregation model 870 is generated by the surgical instrument detection model 850 as described above. The first estimate of the surgical instruments generated by the endoscopic method and the empty / full pocket classifier model, as described above, are used. The second estimate of surgical instruments produced by the 860 model was compared with the surgical instrument tally (e.g. For example, a first tally of surgical instruments or a second tally of surgical instruments) may be determined. Then, the instrument aggregation model 870 can be obtained using two different types of deep learning models. In addition, the location of the detected surgical instruments in the collection bag can be compared (e.g., The results of object detection by the tool detection model 850 are compared with the empty / full pocket classifier model 860. In some exemplary embodiments, the instrument The aggregation model 870 may be configured to use the surgical instrument detection and / or analysis to determine a first estimate of the surgical instrument aggregation. Regarding surgical instruments (e.g., surgical fabrics) identified and detected by the Model 850 The appliance aggregation model 870 receives the empty / full pockets classifier model 860. The information gathered can be used to generate a second estimate of the surgical instrument tally. If the surgical instrument tabulations generated using the method of are consistent, the instrument tabulation model 870 may be Use matching tally as output in providing tally of fixtures per tally bag 872 If the two counts do not match, the instrument count model 870 may select the surgical instrument detection A unique approach is to combine the outputs of Model 850 and the empty / full pocket classifier Model 860. You can respond by using novel approaches, including: To revise the aggregation and improve the accuracy of tracking (e.g., aggregation), such as through approaches that This may involve utilizing knowledge of the shape of the metering bag.

[0050] The instrument aggregation model 870 also includes one or more decisions for one or more instrument aggregation estimates. One or more methods may be implemented to verify the configuration (e.g., a surgical instrument detection model, (When the tabulation based on the model 850 and the empty / full pocket classifier model 860 does not match. Example For example, the tool count model 870 is identified or counted by the count bag shape model 830. The total number of pockets identified or collected by the empty / full pocket classifier model 860. Take the total number of full or empty pockets counted and determine whether surgical instruments (e.g., surgical tissue) are present. , in the pockets of the tally bag (e.g., if there is only one surgical instrument in each full pocket, For example, a number of bags may be determined based on the assumption that the number of bags is 1. There are 10 pockets identified by the or 9 pockets are labeled as full, then the instrument count model 8 Using this information, 70 can determine that the surgical instrument tally is nine.

[0051] As shown in Figure 8, additional useful information may be obtained by determining one or more aggregates of surgical instruments. In addition to setting the number of samples (e.g., first or second collection of surgical fabric in a collection bag), various models Based on information obtained from one or more of the For example, the instrument count model 870 can provide count bag locations 874 within the image. This is useful to automatically highlight all aggregate bags for visualization. As another example, the instrument count model 870 may determine which pockets of the count bag contain surgical instruments. The location of identified or counted surgical instruments within the count bag, including an indication of whether the instrument contains This information may be used, for example, to provide one or more error corrections, as described below. This information can be used in instrument counting protocols (e.g., surgical instruments). It can also be useful for determining compliance with protocols for counting data. For example, If there is surgical fabric in the top pocket of the collection bag but the bottom pocket of the collection bag is empty, In this case, this is an approved method that requires the user to fill the pockets of the tally bag from the bottom up. Another example of useful information is the instrument counting model. The Dell 870 can provide the location of empty and full pockets within an image of a tally bag. This also helps determine compliance with the counting protocol and allows for the Verification of position or number of surgical instruments, tally bag for easy evaluation by end user The pockets may serve as highlighting the location of empty pockets within the pockets, or both.

[0052] Example of combining models to reduce aggregation errors Method 800 (e.g., may be considered as an overall tracking algorithm for tracking a surgical instrument) The models include the surgical instrument detection model 850 and the empty / full pocket classifier model 860. By combining multiple (e.g., several) models (e.g., deep learning models) of and knowledge of the tally bag shape (e.g., from the tally bag shape model 830) to Improve the accuracy of tracking (e.g., counting) by eliminating or mitigating various types of errors For illustrative purposes, multiple models are combined. An example of the two combined methods is described below.

[0053] One example of combining multiple models to improve overall tracking accuracy is Combined analysis can correct or address common errors in instrumental aggregation models. For example, the first type of error is the false detection of the presence of a surgical instrument (i.e., (failure to detect absence of instruments), leading to reduced accuracy and incorrect counting of surgical instruments The second type of error is the false detection of the absence of a surgical instrument (i.e., Failure to detect the presence of surgical instruments) which can lead to reduced recovery rates and This could lead to an incorrect count of the equipment used.

[0054] In some exemplary embodiments of the method 800, one or more of these types of errors may occur. Both are solved by algorithmic matching (e.g., algorithmic error correction). For example, a false detection of the presence or absence of a surgical instrument may be determined by, for example, The assumption is that the locations should closely match the locations of the detected pockets on the tally bag. Therefore, the error correction model or models may be used to correct the error. The rule states that the determined location of the tally bag pocket does not match the determined location of the tally bag pocket. "The error correction model can identify and reject surgical instruments. The center of the surgical instrument (e.g., center of gravity) and the closest metric obtained from the shape of the metric bag Reject identified surgical instruments based on the Euclidean distance between the center of the bag pocket. In other words, the center of the detected surgical instrument and the center of the nearest tally bag pocket are not aligned. If the Euclidean distance between,,,,exceeds a predefined threshold, then the error correction model,is identified. may reject the surgical instrument as an error (e.g., as misidentified) and therefore , and do not include this identification in the surgical instrument tally. Correspondence between the tools and the tally bag pockets is established based on the shortest Euclidean distance. That is, the error correction model determines the identified pocket that is closest to each instrument. by verifying that each surgical instrument corresponds to a pocket; Surgical instruments may be matched to specific pockets.

[0055] Once a correspondence is established, a comparison can be performed between the instrument detection results and the empty / full pocket classification results. For example, the surgical instrument detection model 850 can detect a surgical instrument that actually exists (see above). If the second type of error (instance of the error) cannot be identified, the empty / full pocket classifier model This error occurs because the Dell 860 identifies the corresponding pocket as full (e.g. , by the error correction model). Similarly, the surgical tool detection model 8 50 are surgeries in pockets that are determined to be empty based on the empty / full classifier model 860 If the instrument is misidentified, this detection instance is may be rejected.

[0056] One advantage of this algorithmic matching approach is that it requires very little training data to implement. Requires less data and improves tracking of surgical instruments compared to using a single model alone. (e.g., using only the surgical instrument detection model 850 or or using only the empty / full pocket classifier model 860).

[0057] In another exemplary embodiment, the present invention aims to improve overall tracking accuracy and reduce aggregation errors. The goal is to combine multiple models to form a tracking ensemble network. For example, one or more training sessions as described in more detail below. In addition to using the technique, the ensemble of surgical instrument detection networks is trained with multiple appropriate ensemble learning techniques (e.g., snapshot ensemble) It can be scanned.

[0058] As an illustrative example, a penalty term may be added to the training of the surgical instrument detection model 850. Specifically, a penalty term can be added to ensure that the detected position is closer to the corresponding of surgical instruments that are not close (e.g., within a threshold distance) to the (e.g., closest) pocket location By automatically penalizing detections, we refine the loss function of the surgical instrument detection model850 The penalty term can be modified, for example, by detecting the presence of a surgical instrument (e.g., The corners of the fiber (for example, the corners of the tally bag pockets) and the corners of the tally bag pockets (for example, using the tally bag shape model 830) This penalty term can be related to the distance between the surgical instrument and the target object (determined using the surgical instrument detection model). Adding directly to your training with the 850 is a tally bag (e.g. a sponge bag or Incorporates prior knowledge of the shape of the surgical instrument (or other tally bag) and therefore The output of the output model 850 is used to calculate the location of pockets (e.g., a grid of pockets) in a tally bag. In addition, the end-to-end detection of the Model 850 The training is performed to obtain the geometric information of the tally bag that was available during the training phase. This information can be automatically taken into account.

[0059] Furthermore, in this example, the ensemble of multiple instrument detection networks of different types is A training dataset (e.g., see below) can be generated randomly or in a K-fold manner. modifying one or more hyperparameters of the network; parameters (e.g., depth, regularization, penalty weight, or any suitable combination of them) Change the hyperparameters and create ensembles such as snapshot ensembles. It can be trained by implementing learning techniques. Similarly, multiple vacancies of different types can be Ensemble of full-pocket classification networks with modified network architecture Either by changing the hyperparameters of the network (e.g., as shown above), or Or any suitable combination thereof.

[0060] Finally, all the different instrument detection networks and all the empty / full pocket classification networks were The results from the network are a new stacked network that can be called a tracking ensemble network. The tracking ensemble network can be used in the generalized network. A separate validation data set is used to learn how to best combine predictions from the group members. training it separately on a set of data (e.g., a "hold-out" validation set) This separate training improves the overall accuracy of tracking (e.g., counting), as well as the accuracy of each Accuracy of surgical instrument prediction, empty / full pocket prediction, or any suitable combination thereof For example, a tracking ensemble network can be used to improve the accuracy of a 10-dimensional Evaluate the presence vector (e.g., "l" indicates a full pocket and "O" indicates an empty pocket). for a 10-pocket tally bag), and accordingly, the predicted occupancy vector (e.g., Competitive ensemble instrument detection networks and that of empty / full pocket classification networks We compute a loss function based on the dot product of the vectors (from each vector) and the ground truth occupancy vector. Thus, once trained, the tracking ensemble network can be To improve the overall accuracy (e.g., the overall tracking algorithm), The predictions from each of the detection and object classification networks can be quantified. Therefore, the grid penalty and ensemble approach above is much more difficult to implement. Many training data are available, which may improve the performance of the method 800. can be improved.

[0061] Training data collection example Successful training of one or more of the above models is based on a representative data set of training examples. Collect a dataset and identify the instruments (e.g., surgical instruments) in the dataset for the desired end goal. This can be helped by properly labeling the objects (tools). One such dataset is Or multiple are representative, for example, operating room or trained models It may include variations in the tally bag image that are expected to be seen in other settings where it is used. , a model such as one or more of the models of the method 800 (e.g., a tracking algorithm) described above. The dataset used to train the model is a set of “good” These may include "good" and "bad" images.

[0062] In some exemplary embodiments of the model training described herein, good The image must be a complete, correctly oriented, vertically oriented front view of the tally bag (e.g., side view, top view, or angled view) and no part of the bag may be Not blocked by objects. Good enough resolution to read the text on the bag. It is possible to focus on a specific image. The imaging device may automatically capture images at sufficient resolution by default. The statue's tally bag has intact pocket dividers (e.g., no seams dividing the pockets). (not tearable), and each pocket may contain zero or one surgical instrument. The model or models include images depicting expected deformations during various medical procedures. It can be trained on a set of good images, including ,Changes in the tally bag, image background and / or settings as shown in Table 1 below. One or more of the shapes may be included. [Table 1]

[0063] In some exemplary embodiments of model training described herein, The images include images that are typically analyzed, such as images where features of a tally bag or surgical instruments are difficult to detect. may contain images that are difficult to interpret (e.g., poor lighting, blurry images, or both) (The reason is that bad images are more likely to be submitted by users during practice, so the bad image transformations Train one or more models on the It may be useful to create one or more robust aggregate models that have these bad images. The image shows a "guardrail" algorithm that can warn users about incorrect usage of tally bags. It can also be used to help create a program that alerts the user when a bad image is taken, The user may be notified to provide a better image (eg, a new, clearer image). Examples of bad images include images where parts of the bag are in contact with foreground objects (e.g., the user's hand, a handrail, a tube, etc.). , or any suitable combination thereof), pocket dividers or pocket seals (e.g., between two or more surgical instruments) from angles that make them difficult to see. Tally bags with photographed images, torn pocket dividers or torn pocket seals Images of such tears (e.g., one large pocket where there should be two separate pockets) creating a pocket), images with multiple surgical instruments in one pocket, incorrect lighting conditions, Others that may produce images that are difficult for algorithms to analyze and extract features from. An image taken under the above conditions, an image having any one or more combinations of the above aspects Includes statue.

[0064] Other examples of validating aggregates In some exemplary embodiments, the methods described herein relate to method 800. One or more additional checks to provide further checks (e.g., independent checks) Providing one or more additional check methods may include a verification step of the surgical instrument. Such additional methods may improve the accuracy of the location of surgical instruments in the tally bag. Such additional methods may include prompting the user to manually verify the tally. User search of information generated by one or more of the various sub-models of method 800 For example, one or more of such additional methods may include: 800 correctly identifies pockets, correctly identifies tally bags, correctly identifies full pockets correctly identify the surgical fabric in the pocket, or any suitable combination thereof For example, FIG. 11 illustrates a mobile device 8 shows an example embodiment of a display on a device, the display being displayed according to method 800. Verify that the number of surgical instruments (e.g., surgical sponges) identified and counted is correct. As shown by the box surrounding the surgical instrument in Figure 11, Thus, one or more of such additional methods may be identified by method 800 as a surgical instrument. The method may include displaying information to the user indicating what was done.

[0065] In certain exemplary embodiments, the methods described herein include using the weight information to determine the surgical procedure. For example, verifying the tally of the port to which the tally bag is attached. The pole can be attached to a scale. The scale attached to the pole allows the surgical fibers to be weighed on the tally bar. Each time the weight is placed on the scale, the change in weight is sensed, thus changing the weight sensed by the scale. The surgical instruments may be tallied based on the number of volume increments. The scale may be attached to the pole in any suitable manner. For example, the mounting pole, the tally bag, or both can be attached to the scale. (e.g., the mounting pole, tally bag, or both can be placed on top of the scale) (e.g., by placing a weight on a bag and measuring the change in weight as the item is added to the bag) or The scale is an exemplary form of strain gauge coupled to a mounting pole and coupled to a tally bag. The strain gauge can be mounted in a fixed position (e.g., one end of the strain gauge is suspended from a mounting pole). (The strain gauge is attached to the other end of the bag, and the sizing bag is hung from the other end of the strain gauge.) Some examples In an exemplary embodiment, the weight information may be generated by one or more other methods. Instead of, or in addition to, the information generated from one or more surgical instrument collection containers. For example, the collection container may include a weight measurement system. When the instrument is removed, a weight measuring system detects the weight loss and transfers the fibers from the surgical instrument to the collection volume. The processor records the number of weight losses and determines that the item has been removed from the collection container. A count of surgical instruments removed from the collection container and therefore the surgical instruments previously present in the collection container Therefore, the use of one or more collection containers allows the determination of the total number of items. ,An additional check on the counting of surgical instruments (e.g., surgical textiles) may be provided.

[0066] In various exemplary embodiments, the methods described herein include removing surgical tissue from a collection container. This may include using time data from removal of surgical instruments and The placement of the surgical instruments may provide additional validation of the surgical instrument count. For example, the processor may A timestamp is recorded each time the quantity measurement system detects the removal of a surgical instrument from the collection container. Further, the processor may generate a weight measurement system handover to the tally bag. A timestamp can be generated each time an additional surgical instrument is sensed. The processor in the collection system may time-stamp the removal of the surgical instrument from the collection container (e.g., timestamp of the surgical instrument being added to the tally bag (e.g. , additional timestamps) and verify the surgical instrument tally (e.g., additional verification) The retrieve timestamp and the add timestamp can be substantially If they match, it provides information that indicates the surgical fabric count is correct.

[0067] Example of determining interim totals In some exemplary embodiments, the determination of the second aggregate is based on a medical procedure (e.g., a surgical procedure). ) determining an intermediate tally of the surgical instruments during the process. The second tally determination is to determine the final tally of the surgical instruments after the medical procedure is completed. For example, the methods discussed herein for tracking surgical instruments (e.g., method 8) 00) uses a second collection system to locate surgical instruments at any time during the course of a medical procedure. The method may include comparing a total (e.g., as an interim or final tally) to the first tally. Various exemplary embodiments of such methods may include, but are not limited to, the following: In this case, the second tally may be determined by using any one or more of the above algorithms. It may include.

[0068] Determining the interim tally of the surgical instruments may include updating index counters of the surgical instruments. In some exemplary embodiments, the update of the index counter is performed by updating one or more of the tally bags. A tally bag containing one or more surgical instruments (e.g., surgical tissue) in a number of pockets. The tally bag is filled with surgical fabric by the user. , one or more images of the tally bag may be generated. One or more images of the tally bag can be received, using one or more of the techniques described above. One or more images are processed using one or more computational techniques, such as The surgical fibers in the bag or multiple counting bags can be identified and counted.

[0069] In some exemplary embodiments, instead of analyzing an image of a collection bag containing surgical instruments, In addition to detecting or analyzing the object, the methods described herein may also include detecting a handheld object within the field of view of the camera. For example, the user may hold a surgical instrument (e.g., a surgical sponge) The user can hold a surgical instrument (such as a surgical fabric) up to the camera, which then displays an image depicting the surgical instrument. The images can be used to detect the presence of surgical instruments depicted in the images. The surgical instrument may be received and analyzed by one or more processors configured to: The tool images can be used to track surgical tools. For example, index counters The system uses a camera to hold surgical instruments up to the camera and identify them using one or more processors. In some exemplary embodiments, the camera may display the surgical instrument. The index counter updated using image analysis to verify the index count of surgical instruments. Further indexing updated using image analysis of surgical instruments in one or more tally bags The individual surgical procedures can be compared to a target counter (e.g., as described above). Images of instruments (e.g., surgical instruments) are acquired and analyzed to identify collections of such surgical instruments. The figure shows an example of a graphical user interface that updates the user The camera 1210 captures an image (e.g., a depth image, a color image, or both). The surgical fabric 1220 is held in front of a camera that is configured to receive the surgical fabric 1220, which results in one or more processing components The component (e.g., a processor) may identify the surgical fiber 1220 in the image and image the The surgical fabric 1220 may be included in the instrument collection 1230. The system and method are described in U.S. Pat. No. 8,897,523 and U.S. Patent Publication No. 2017 / 013363. No. 0186160, each of which is incorporated herein by reference. The entire contents of which are incorporated herein by reference. In addition, as described below, One or more methods may be implemented using one or more of the methods described herein for tracking surgical instruments. It can be used in combination with multiple methods, thus allowing the user to select one This may include holding one or more surgical instruments in front of the camera.

[0070] The determination of the interim count of surgical instruments is based on one or more of the following: Count the number of surgical fibers in a number of containers (e.g., one or more tally bags) This process may be repeated throughout the medical procedure, The index counter is continuously updated to indicate the number of surgical loads loaded into one or more vessels at any given time. A running tally of instruments (e.g., surgical equipment) may be maintained. In some exemplary embodiments, By comparing the intermediate tally (e.g., the current value of an index counter) with the first tally of surgical instruments, For example, a user may wish to load and locate a set of tally bags. The number of surgical instruments currently in use or missing You may want to know the number of fixtures.

[0071] Certain exemplary embodiments of the methods described herein may be used at any suitable time during a medical procedure. This includes displaying or otherwise making available to users intermediate summaries at any time. This provides useful information about how many surgical fibers are located at a particular time. FIG. 14 is a graphical user interface that displays information about the interim tally. 13 shows an example of a user interface 1300. Box 1310 on the screen of FIG. The number of surgical sponges located compared to the first count of sponges (e.g., "40") The total number of surgical sponges identified (e.g., "17") is also shown. Information relating to type (e.g. brand, size, function type, etc.) is provided in Box 13 20, 1330, and 1340. The process 1300 can also be used to display the first page by the user selecting the “aggregated” icon 1350. 1 tally and determine the final tally by selecting the final tally icon 1360. This provides functionality that allows

[0072] Example of duplicate image detection Some exemplary embodiments of the methods described herein may be used to determine whether a particular tally bag has previously The user may further determine whether the image and analysis are being performed on the plurality of devices. Images of individual bags of surgical instruments, multiple surgical instruments, or both may be captured. Bags, individual surgical instruments, or both may be erroneously imaged multiple times, resulting in the loss of To combat these consequences of inadvertent duplicate images, Analyze one or more images to determine the likelihood that an image depicts the same tally bag as another image Determine if the count bag has already been located. An example of a method for doing this is to have the first and second images each show the same fibers (e.g., the same aggregates). Measure or measure the likelihood of having an image area depicting at least a portion of the The first and second images may each include determining the same fiber in another manner. An exemplary method for measuring the likelihood of having an image region that is at least partially depicted is PCT Application No. PCT / US2017 / 068, which is incorporated herein by reference in its entirety. The details are described in Issue 234. Images of specific count bags have already been received and analyzed. An exemplary method for determining whether a first image region depicts a tally bag is receiving a first image, a second image region depicting a tally bag; a first aspect of the first image, a second aspect of the second image, or both; Such a method may include defining at least one classification function based in part on the An exemplary method includes: the first and second image regions depicting at least a portion of the same tally bag. The method may further include measuring the likelihood that the classification algorithm is trained. Based at least in part on classification functions, such as by using a machine. The method may include comparing the measured likelihood of overlapping images to a predetermined threshold, the threshold being: The cutoff for classifying image pairs as "non-overlapping" or "potentially overlapping". The first image area and the second image area are or include the same surgical fabric or We conducted a similar process to measure the feasibility of depicting at least some of the other surgical instruments. can be executed.

[0073] If the method 800 determines that the likelihood of overlapping images exceeds a predetermined threshold, the method 800 For example, the resulting tally bag or surgical instrument may be used to facilitate uptake of the Each new image of a tool may correspond to an increase in the tally of surgical tools (e.g., a second (Surgical instruments extracted from the patient or the sterile field of the surgical procedure for counting), counting is There is at least a measured likelihood that the bag, surgical instruments, or both were imaged multiple times. The potential for a surgical bag or surgical instrument to be Detection of duplicate images is useful, for example, to detect cases where a tally bag, a surgical instrument, or both are imaged multiple times. As another example, a prompt may be triggered to the user to confirm whether the Detection of potential duplicate images of tally bags or surgical instruments increases tally of surgical instruments Automatically withholds the tally bag from being imaged. An exemplary method for determining whether a bag is in use may include the use of an identification tag. the device, or both, may include a scannable tag (such as an RFID tag or a barcode); A scanner can be used to determine if the tally bag has already been imaged. The scanner may be a separate scanning device or may be integrated into the mobile device. Instead of, or in addition to, one or more techniques involving image analysis, tagging or One or more exemplary methods may be used, including scanning a bar code. .

[0074] Example of determining the final tally The second tally may, in some exemplary embodiments, include a final tally of the surgical instruments. For example, an exemplary embodiment of the method described herein may include determining a final tally of surgical instruments. and comparing the final count to the first count of the surgical instruments. The count may be used, for example, to determine if a surgical instrument is located at a point in the procedure when it is no longer being introduced or used. Based on the final location check, which refers to the total number of surgical instruments identified, distinguish it from the interim total. For example, a final count may be performed at the end of a medical procedure. The end of the medical procedure may be indicated in any suitable manner. For example, the end of the medical procedure may be indicated by closing the surgical site and then pressing the hand. After all surgical procedures have been performed on the patient, except for closing the surgical site, the patient is taken out of surgery. When it is ready to leave the room, or any other time suitable to be used as final instructions. In some exemplary embodiments, the determination of the final tally may be prompted by the user. For example, a mobile application or other computing program may It may be possible to allow the user to indicate when the final tally icon in FIG. 14 is displayed. In some exemplary embodiments, the mobile application may include, among other things, The surgical instrument count should be labeled as the final count based on the instructions provided. For example, the user may be prompted to verify that the metric counter is equal to the value of the first tally. When this is reached, the mobile application will show the current value of the metric counter as the final tally. In another exemplary embodiment, the user may be prompted to confirm whether or not to The mobile application will then determine whether the metric counter is the final tally after a certain period of time. The user may be prompted to indicate whether the item should be displayed.

[0075] Example of providing notifications based on the first aggregate, the second aggregate, or both Various exemplary embodiments of the methods described herein for tracking a surgical instrument include: The method may include comparing the first tally of surgical instruments to the second tally of surgical instruments. For example, Some exemplary embodiments of such methods for tracking surgical instruments, as described above, include: Comparing an intermediate tally (e.g., between one or more intermediate tally) to the first tally Certain exemplary embodiments of such a method include comparing the final tally to a first tally to determine whether the first tally is greater than the final tally. and whether the final tally matches, and therefore all surgical instruments introduced into the operating room. Comparing the final count with the first count This allows medical staff to ensure that all surgical equipment or other surgical instruments are accounted for and Some of these methods can be used to ensure that the drug is not retained by or within the patient. Some example embodiments may determine whether a first tally matches a second tally (e.g., the final This may include informing users of the nature of the methodology (aggregated or interim aggregates). Certain exemplary embodiments include a first tally, a second tally, and a first tally equal to the second tally. A requirement for a surgical instrument collection measurement in which an indication of whether the instrument matches is displayed to one or more users. This includes displaying the About report.

[0076] FIG. 15 shows a graphical user interface displaying a summary of surgical fiber data. The screen in Figure 15 shows an example of the number of fibers that have been "counted" (e.g., those introduced into the operating room). the number of "out-of-scan" fibers (e.g., imaged in a tally bag), and unused fibers Indicates the number of cases (e.g., introduced into the operating room but not yet imaged in the tally bag) In some implementations, unused surgical fabrics are spun in the same manner as used surgical fabrics. In some other implementations, unused surgical The unused fiber is identified as unused in one or more tally bags and displayed accordingly. They are located using different methods, some of which are labeled as such above. In an exemplary embodiment, the method described herein detects whether the compared counts match. This includes notifying the user (e.g., by providing a notification to the user). Such notification may include a short message to the user indicating that the counts match, or The notification can be a symbol. For example, the notification can have a check mark next to the first and final tally or other symbols. Displaying a number or other appropriate symbol and coloring the tally a particular number (e.g., green) when the tally is matched ) and to show that the first tally and the final tally match. audible indication that the final tally is a match, some form of flashing or vibration, or and reproducing the appropriate combination. As will be explained in more detail, summary reports are intended for personal reference, hospital records (for accountability purposes, etc.) may be provided to one or more users for purposes of providing a summary report, The survey included a first and final tally for each of one or more types of surgical instruments. For example, the summary may show a first tally and a second tally, such as the "match" count shown in FIG. It may include or indicate a comparison between the aggregates.

[0077] In certain exemplary embodiments, the methods described herein may include determining whether the compared counts match. The final tally of surgical instruments includes providing a notice to the user that the surgical instruments are In circumstances where the number of users is not consistent with the tally of 1, the notification may include a warning to the user. Example of a display that indicates to the user that one or more surgical instruments are not approved. The display may be in the form of a display (e.g., on a mobile device) showing the compared tallies. A statement that the compared aggregates match, except in a manner that contradicts the statement that they do not match. Some exemplary implementations may include any one or more of the elements described above. The form notifies the user that the first and final tally of surgical instruments does not match. provides a predetermined protocol for proceeding in the event of a non-matching location check. For example, notifications to users may be sent at certain protocol stages (e.g., on mobile devices). The system will display the results on the screen and provide instructions on how to handle cases where the initial and final counts of surgical instruments do not match. The medical personnel may provide an accompanying audio cue to the medical personnel. Become familiar with one or more protocols regarding the preceding methods in the case of no surgical instruments However, there is an additional process that is not affected by the potential uncertainty of the missing surgical instruments. Providing a representation (e.g., additional instances of a protocol) allows users to This can be useful in helping surgical instruments held within a patient to conform to the Fewer instances, better detection of retained surgical instruments when they occur, a more effective response in the case of a retained surgical instrument, or any suitable combination thereof. If the first and final counts do not match, one of the methods described herein may lead to Some exemplary embodiments include automatically sending notifications to alert additional personnel, responding For example, the first non-matching The counting and final tally will be conducted by hospital administrative staff, radiologists (e.g., (to obtain pre-approval for the procedure), x-ray technicians, additional medical personnel to remedy the situation, or may trigger notification to other appropriate personnel.

[0078] If the first and final counts do not match, the document will be used to verify the location of the surgical instruments. Some exemplary embodiments of the method discussed in this application include determining whether a surgical instrument is held. A predetermined profile is used to determine whether the device is being used (e.g., in or retained by the patient). Execution of a given protocol may include communicating with a user through the protocol. One or more computational tools, such as a device and display for guiding the user The protocol begins with the first consultation with all medical personnel in the operating room. This may include informing the patient of any surgical instruments that may be in their possession. Col was tasked with recounting all the surgical instruments in the operating room and identifying any that may have been left behind. A recount of surgical instruments may include checking the room for one or more manual aggregation methods, or any suitable combination thereof, to An example of a manual counting method may include operating a counting system. Recount, recount of the number of surgical instruments in each count bag, recount of the number of surgical instruments in each pocket A check of each pocket to determine whether or not Checking the operating room for surgical instruments includes checking for trash (e.g., checking one or more collection containers; Checking the surroundings for instances of multiple instruments in the same pocket Checking one or more tally bags or any suitable combination thereof. Even after recounting the surgical instruments and checking for misplaced surgical instruments, the first and final If the tallies don't match, the protocol states that x-rays or other internal images of the patient should be taken. , detecting the presence of a surgical instrument held within the patient. This may include alerting the radiologist to the need to take an x-ray image and notifying the radiologist to proceed. Approval from a radiologist may be required. X-ray imaging may be an additional or alternative option. This may include notifying other appropriate medical personnel if a protocol is not met. This may also include notifying hospital administrators of new surgical instrument tallies and other administrative personnel.

[0079] Additional method examples The computational systems and methods (e.g., algorithms) described herein may be used to improve Some of these methods have broad applicability for affecting a variety of improvements in Some exemplary embodiments, as described in more detail below, provide additional user compliance. Tracking and guiding the user through protocols and surgical instruments throughout the procedure Tracking tool usage, receiving patient and / or operating room information, specific instructions Trigger additional actions upon receiving a It may include.

[0080] Certain exemplary embodiments of the methods described herein include one or more protocols Tracking user compliance includes tracking compliance with aggregate Includes tracking of user compliance to fiber counting protocols for bag use As described above, the one or more predetermined treatments are administered to one or more tally bags. The desired method or methods for placing the tool may be controlled by the appropriate program or programs. According to the protocol, the operating room personnel should fill each tally bag pocket from bottom to top and right to left. You may be instructed to fill to the left. Depending on the situation, you may be asked to follow one or more of the methods above. The user can then define the bag filling protocol using one or more algorithms developed in Such an algorithm can be evaluated for compliance with one or more receives a plurality of input images and determines compliance based on one or more of the images. For example, a series of images taken over time can be analyzed to determine how many In other situations, video analysis can be used to determine the order in which items were filled. The algorithm can identify and track the order in which the bags were filled. Or, by analyzing the time series of the video, we can determine the order in which users filled the pockets of the tally bag. assessing user compliance with one or more applicable protocols; This can be done.

[0081] Some exemplary embodiments of the methods described herein include the use of non-regulatory Providing notification upon detecting compliance. For example, such methods The hospital administration staff should be aware of the need to provide warnings to users as a reminder of proper protocols. Sending notice to the staff, other hospital personnel, or both, indicating misuse of fiber counting procedures or any suitable combination thereof. The law could include triggering a warning if other medical personnel intervene with an incorrect filling procedure. For example, misplaced surgical instruments (e.g., contrary to prescribed procedures in the applicable protocol) and detecting surgical instruments placed in the pockets of the tally bag, such a method A supervisor or colleague can be notified to intervene in the bag filling process.

[0082] Certain exemplary embodiments of the methods described herein include aggregation or tracking algorithms. This includes prompting users to enter their name in conjunction with the use of the mobile app. The application operates the application to take images of the tally bags for processing. prompt users to enter their name or other unique identifier before or after For example, a mobile application may require users to provide such identifying information. The user can be prompted to scan their ID badge to receive a compliance report. The data can be combined with identifying information to enhance user accountability. Combining the license information with user identification information allows hospital administrators to implement appropriate instrument counting procedures. This provides the advantage of being able to implement training or disciplinary protocols for users who do not comply. obtain.

[0083] Various exemplary embodiments of the methods described herein use tally bags to tally This includes guiding the user through a protocol (e.g., a fiber counting protocol). or using computational techniques to, for example, detect one or more visual cues, one or By presenting multiple audio cues, or any suitable combination of them, Indicate to the user which pockets of the tally bag should be filled or which the user has forgotten to fill For example, the display screen may identify a pocket that is full of one or more users. A visual indication may be provided (e.g., a button around the pocket) to guide the user to the next pocket to be filled. (providing a circle or a circle, or displaying arrows as directional guidance, or both). Thus, such a method would guide the user through a protocol for filling the tally bags. It can be induced.

[0084] Some exemplary embodiments of the methods described herein include a method for producing a surgical instrument. or a plurality of other types of tracking metrics. It can be used to collect information about medical procedures at various points throughout the facility. For example, such methods include tracking used and unused surgical instruments. Tracking of used and unused instruments can be done by determining whether the user is using the instrument or not. For example, a user may manually separate or sort used vessels based on The equipment may be placed in one container and the unused equipment in another. Various methods, including multiple computational techniques, are used to identify and separate used and unused containers. Such a method allows for the counting of used and unused containers. and capturing an image of a portion of the operating room using one of the image analysis methods described herein. One or more of these may be used to extract information from the images. Identifying and counting the instruments used may provide one or more useful metrics regarding the medical procedure. A single snapshot at a specific point in time during a medical procedure (e.g., at the end of a medical procedure) It may provide information regarding the number of devices removed from the package and the number of devices actually used. A series of images taken throughout the medical procedure are used to identify used and unused instruments. and how the device is used in the medical procedure over time. We can provide information on the above.

[0085] Certain exemplary embodiments of the methods described herein include a method for receiving information related to a medical procedure. Such methods may allow a user to obtain patient demographics or characteristics (e.g., height, weight, sex, age, etc.), the medical procedure to be performed, the expected length of the medical procedure, Whether the procedure is scheduled or emergency, one or more pre-existing conditions, and other relevant information , or any suitable combination thereof. Such methods may include the use of various methods for determining one or more environmental parameters. This may include receiving and / or displaying information such as which operating rooms are in use and which operating rooms are in operation. availability, whether a particular piece of equipment is available in a particular operating room, which medical personnel are currently working or any other suitable parameter, or any suitable combination thereof.

[0086] Some exemplary embodiments of the methods described herein may include event or protocol The method includes receiving information that can trigger a test (e.g., a test device held within the patient's body). or one or more surgical instruments if it is determined that a relatively high risk exists for multiple surgical instruments. Such methods may be used to request or initiate one or more preventive measures during a medical procedure. This may include allowing the user to request additional equipment or devices for use. Such a method may include, upon receipt of information regarding one or more particular instructions, Triggering a specific event during treatment. One or more specific patient parameters indicate a high risk of experiencing a particular event. For example, threshold values ​​for surgical instruments expected to be used during a medical procedure may be indicated. Patients with a BMI in a certain range in combination with the number (e.g., patients with a first count exceeding a given count) (when the surgical instrument is retained) may indicate a high risk of retention. Such a method would combine patient data with the surgical instrument count once a threshold number of surgical instruments have been collected. This may include being used in conjunction to trigger one or more additional protocols. Such protocols may, for example, include providing the appropriate equipment to take x-ray images of patients. Alerts the x-ray technician to take an x-ray image of the patient and provides notifications to the radiologist This may include obtaining prior approval for the transfer, or both.

[0087] Certain exemplary embodiments of the methods described herein include displaying information related to a medical procedure. Such methods may include providing a summary report or otherwise making available to the public The summary report may include, for example, a medical record (MMR) for the patient. Evidence of compliance with one or more surgical instrument tracking protocols as part of A summary report may contain pertinent information. As above, one or more collections of surgical instruments, one or more collections of surgical instruments Information relating to multiple comparisons, or both, may be displayed in a summary report. or alternatively, one or more of individual surgical instruments, individual tally bags, or both. The images may be stored for recordation (e.g., in a summary report), displayed, or both. However, the type of medical procedure, the length of the medical procedure, patient information, doctor information, hospital information, etc. Additional information about the medical procedure may also be displayed, such as Metrics of blood loss may also be displayed. For example, blood loss of a patient may be measured using analysis of surgical fabrics. If blood loss is monitored, an estimate of blood loss can be displayed in the summary report. The display may include one or more symbolic representations that indicate the results of the procedure. For example, the display may can be a particular color within a given range (e.g., blood loss is defined as between certain thresholds) If it is within the range, the display is green, if the blood loss is not within the range, (which may be red). One or more summary reports may be provided at any appropriate time during the medical procedure. One or more summary reports may be generated. The information is displayed on a display screen of a mobile device such as a tablet, and is stored in memory (e.g., as part of the patient or hospital's electronic medical record) and are stored for later viewing and The data may be sent digitally to a medical professional, sent to a printer and printed, or This is a good combination. Figure 16 shows information on compliance with the counting protocol. 1 shows an example of a summary report that includes the medical procedure performed, as well as additional information about the medical procedure performed. The 16 summary reports include information such as case number, case location (e.g., which operating room was used), ), start and end times of medical procedures, duration of medical procedures, number of surgical instruments collected (e.g. For example, surgical fabric labeled "sponge" and the person who performed the final tally of surgical instruments. According to various exemplary embodiments, the summary report includes information about the names of the , displaying any suitable information regarding a medical procedure, such as any information described herein. It is possible.

[0088] Some exemplary embodiments of the methods described herein include a step of entering an "emergency mode" In one or more modes, some functionality of the method or system is For example, the fiber counting algorithm (e.g., via a mobile application) The system (implemented as a system for determining whether a user is eligible to receive a notification) may allow the user to select an emergency mode, which may be implemented in the aggregation process. This could indicate that protocols are no longer being followed, which could lead to the suspension of collection of instrument aggregate data. Suspension, recording of user compliance tracking to aggregation protocols, emergency to hospital staff One or more of a variety of actions may be triggered, such as an alert of the event.

[0089] In certain exemplary embodiments of the methods described herein, one or more images (e.g., For example, one or more surgical instruments, one or more tally bags, or any of these. (showing an appropriate combination of blood loss, blood component loss, etc.) to analyze the patient's condition (e.g., total blood loss, blood component loss, etc.) For example, as shown in FIG. 12, The imaging device 1100 includes an imaging system with one or more cameras 1120. The system 1122 may be used to capture one or more images 1102 of the surgical fabric. The one or more images 1102 may be of surgical gauze, a surgical sponge, or a surgical A field of view 1108 containing a surgical fabric 1106a, such as a piece of surgical towel, is captured and depicted. The surgical fabric 1106a can be held by the user. As such, the captured images, including the image 1106b of the surgical fabric 1106a, The image may be displayed on a display 1118 of a computing device 1100 or the like. Alternatively, the plurality of images 1102 may include blood content, blood component content, or the like in the imaged surgical fabric. , or both. As described in incorporated U.S. Pat. No. 8,897,523, the color of a color image The relevant features (e.g., intensity) can be processed locally or by a remote computer for such processing. The computer system detects blood or blood components (e.g., hemoglobin) in the surgical fabric 1106a. The amount of globin in the blood may be converted to an estimate of the amount of globin in the blood (globin mass, whole blood volume, or both).

[0090] Additionally, one or more images 1102 may include depth images (e.g., infrared data, stereo images, etc.) The depth information from such a depth image may include depth information based on a depth image, a depth image based on a depth map, or both. The degree information allows objects in the foreground to be more easily distinguished from other objects in the background. The depth image is analyzed (e.g., as described in the above-incorporated U.S. Patent Publication No. 2017 / 01861 60), to confirm the presence of the surgical fiber 1106a in the image. The computing device 1100 can then point to a local surgical fiber counter. The local surgical fiber counter can be marked by any one of the above methods or A count of surgical instruments determined by multiple (e.g., using image analysis of the count bag) The present invention may be used to verify the accuracy of the present invention or as a standalone method of counting surgical instruments.

[0091] Example of a system for tracking surgical instruments Also described herein are exemplary systems for tracking surgical instruments. Such a system for tracking surgical instruments includes determining a first tally of the surgical instruments. For example, such a system may include one or more components configured to One or more imaging systems are used to determine a first count of the surgical instruments. a measuring system, one or more weighing systems (e.g., scales), or any suitable thereof. Such a system may include a combination of the first and second counts of surgical instruments. For example, such a system may include one or more components configured in a one or more pockets each configured to contain a plurality of surgical instruments; Such a system may include one or more tally bags equipped with one or more The surgical equipment may include additional equipment used to house and handle multiple surgical instruments. For example: Such systems may include one or more collection containers for holding used surgical instruments. a container; one or more stands for holding one or more tally bags; imaging; One or more mobile computing devices used for image processing, or both Such a system may include one or more mounts for holding a vice. One or more images of one or more tally bags or other containers for surgical instruments The first tally may include one or more imaging systems configured to generate a first tally. The imaging system used to acquire the second tally Such a system may be the same as the imaging system for first counting, second counting, or is used in determining both, and one or more methods for comparing the first tally to the second tally. For example, such one or more processing components may include: Analyzing one or more images to identify surgical instruments appearing in the images or to identify surgical instruments in the images Such a one or The processing components may be computer vision, machine learning, depth perception, or any combination thereof. The surgical instruments may be tracked using one or more computational techniques, such as any suitable combination of Such components (e.g., processing components, image The imaging component, or both, may be implemented on one or more mobile computing devices. Each of the sensors may be implemented via one or more cameras or other image sensors, One or more processors, one or more displays, or any suitable combination thereof. It may include any suitable combination.

[0092] According to some exemplary embodiments of the systems described herein, such The computing device or devices may include, for example, a mobile phone, a smartphone, A tablet computer, a laptop computer, or any suitable combination thereof. Each of the devices may include a camera, a processor, a display, and a mobile device such as a However, certain exemplary embodiments may include, but are not limited to, any suitable combination thereof. In certain embodiments, some or all of the system components described herein may be implemented as individual They may be separated as separate interconnected devices or other components. For example, a camera, The display, or both, shall be located substantially near the tally bag (e.g., in the operating room). while the processor may be located remotely (e.g., the camera, display, or or both), a wired or wireless communications network Such a computing device communicates with the camera and the display via may be used to display information to one or more users. For example, the computing device may receive a summary of one or more surgical instruments, The system analyzes multiple patient parameters, one or more medical procedures, hospitals, operating rooms, patient blood loss, or any suitable combination thereof, or Such a computer may include one or more display screens capable of displaying any of the following information: One or more of the computing devices may be configured to provide audio information to one or more users. It can be configured as follows.

[0093] The one or more processors may be configured to perform any one or more of the methods described herein. For example, one such processor may be configured to execute some or all of the The one or more receive one or more images and process one or more computational algorithms (e.g., For example, computer vision, machine learning, deep learning, or any suitable combination thereof. Performing image matching and analyzing such images, receiving and analyzing data (e.g., from user input or computer memory), triggering one or more responses, or Any one of such processors may be configured to perform any suitable combination thereof. or more images depicting one or more surgical instruments; one or more tally bags, weight information, a tag, for determining a first and second tally of time stamp data (e.g., paired with weight information), user input, or to receive and analyze one or more images depicting any suitable combination of One or more of such processors may be configured, for example, to perform one or more functions. Run a machine learning algorithm to analyze one or more images and One such processor may be configured to determine the number of surgical instruments to be displayed. or more, a middle ranking of surgical instruments based on the number of surgical instruments appearing in one or more images. One or more of such processors may be configured to: The surgical instrument may be configured to determine a final tally of the surgical instruments, such determination of the final tally being One or more of such processors may include accepting manual input from a user. may be configured to compare the first tally of surgical instruments to the second tally of surgical instruments ( (e.g., intermediate or final aggregates). One or more of such processors may: Configured to generate a notification (e.g., an alert) based on a comparison of the first aggregate to the second aggregate If the first and second counts do not match, then among such processors, One or more of the may be configured to generate or display an alert or other notification. Additionally or alternatively, one or more of such processors may be used to control other instruments (e.g. Determine tally of used and unused devices, track compliance information, and accept user input triggering one or more responses to situations that arise during a medical procedure; one or more of the following related to the transmission of information and warnings to the A number of additional tasks can be performed.

[0094] In general, any one or more processors described herein may include a processor that is stored in memory. The device may be configured to execute instructions stored in a memory, such that, when executing the instructions, one or more The processor may perform one or more of the operations of one or more of the methods described herein. The instructions are sent to a user computer, mobile device, wristband, smartphone, etc. or any suitable combination of applications, applets, hosts, servers, Servers, networks, websites, communication services, communication interfaces, hardware A hardware element, a firmware element, a software element, or any suitable combination thereof. The instructions may be executed by a computer executable component integrated with the memory. , or RAM, ROM, flash memory, EEPROM, optical devices (CDs and DVDs) D), hard drive, floppy drive, or any other suitable device. The method can be stored in a computer readable medium.

[0095] Performing some or all of any one or more of the methods described herein One or more processors configured to execute the The computing device may be integrated into the computing device or may be implemented as an external processor. In some exemplary embodiments, one or more of such processors may be cloud-based. computer systems, mainframe computer systems, grid computer systems A computer device or other suitable computer system. One or more of the systems described herein may be integrated into a system. or a plurality of external processors. Alternatively, one or more of such processors may receive an image of a surgical instrument, The remote control system reconstructs or analyzes the images as described above and transmits information relevant to the tracking of surgical instruments. It can be integrated into the server.

[0096] As noted above, one or more exemplary embodiments of the system described herein may include , one or more imaging systems. For example, such a system may be One or more of the instruments (e.g., surgical instruments contained in one or more tally bags) The camera may be operable to generate an image, such as one or more sets of still images or video. The image or images may be a color image, a depth image (e.g. (e.g. infrared, stereoscopic, ultrasound, etc.), hyperspectral imagery, or another suitable type of imagery; or any suitable combination thereof. It may include an image sensor (e.g., CCD, CMOS, etc.) that detects the red, green, blue ( using color components of the RGB (red, green, blue, blue) or one or more other suitable optical components. Capture color optical digital images. For example, a camera with appropriate corresponding optics, Filters (e.g., color filter arrays such as Bayer pattern filters), or As another example, the camera may include a single image sensor paired with a white At least one prism for splitting the light into separate color channels (e.g., RGB) The imaging system may include multiple image sensors paired with appropriate corresponding optics, such as a diffractive surface or a mirror. One or more imaging systems, each of which is detected by a respective image sensor; One or more computing tools, or any suitable combination thereof, may be used on the same device. The imaging system may be configured on a single optical fiber, or the imaging system may transmit image data to one or more other optical fibers. It may be a separate component that communicates with one or more of such imaging systems. The cameras may be part of a mobile device (e.g., a camera on a smartphone). It may or may not be configured to implement one or more of the above processing components. Such imaging systems may also be configured to process the images into one or more processors for analysis. , to a database that stores the images (e.g., cloud storage), or both. The device may be configured to:

[0097] One or more exemplary embodiments of the system described herein may include a user (e.g., For example, to a doctor, nurse, other healthcare professional, or any appropriate combination of these) The device may include a display operable to display or otherwise communicate information to a patient. information, one or more images of the surgical instrument, one or more images related to locating the surgical instrument Multiple quantified metrics, a first tally and a second tally (e.g., final tally) of surgical instruments ) one or more notices regarding a comparison of the first and second counts of surgical instruments Predefined protocols for users to follow if not available; weight of surgical instruments (e.g. surgical fabrics); one or more measurements of the possibility of multi-imaging, the value of the index counter of the surgical instrument, the surgical instrument One or more warnings or prompts to the user indicating potential overlapping imaging of the device; or a plurality of summary reports, or any suitable combination thereof. The image may appear on the screen of a handheld or mobile device, a computer monitor, or a television screen. The display may include a display, a projector screen, or other suitable display. It may be integrated into the same device as one or more components of the system. In addition, the display may be a standalone separate monitor, screen, or other suitable display. It may include play.

[0098] According to some exemplary embodiments, the display allows the user to interact with the displayed information. A user interface (e.g., a graphical user interface) that allows users to interact with the For example, the user interface may be configured to display a The user may manipulate one or more images (e.g., zoom, crop, rotate, or or any suitable combination thereof), or a picture depicting at least a portion of a surgical instrument. It may be possible to manually define the image region. As another example, the user interface may , the user can select one or more display options (e.g., font, color, language, or Select the content (patient information, quantified metrics, or any appropriate combination of these) and or other bodily fluid-related information, warnings, etc.), or both. As yet another example, the user interface may include a user interface for depicting overlapping surgical instruments. Select one or more images to delete for further editing, or tap one or more images on the display. The display may allow the user to rotate, flip, or manipulate the image. Interactive, resistive or capacitive, responding to skin, a stylus, or other user touch The display may include a multi-touch screen. The display may be connected to a mouse, keyboard, or other It may be user interactive via a cursor controlled by another input device, such as For example, the display may allow the user to select a first tally, a second tally, patient information, or any of the above. The system may allow the user to manually enter information indicating any suitable combination of

[0099] One or more exemplary embodiments of the system described herein include a surgical instrument. Location confirmation (e.g., first count or second count), the value of the index counter of the surgical instrument, One or more comparisons of a first count and a second count (e.g., a final count) of surgical instruments. notification of the first count and the second count of surgical instruments, and the prescribed procedure that the user should follow if the first count and the second count of surgical instruments do not match. a protocol for performing a surgical procedure, one or more measurements of the likelihood of overlapping imaging of the surgical instrument, One or more warnings or prompts to the user indicating the possibility of duplicate imaging of or a plurality of summary reports, or any suitable combination thereof, The display may include a speaker or other suitable audio system. The first tally matches the second tally (e.g., the final tally), If the user does not provide a notification or warning, the system may provide one or more notifications or warnings. This can help prompt specific actions in response, such as initiating a specific protocol. As another example, the display, the audio system, or both may be or provide multiple warnings or prompts to the user to prevent two or more images from showing the same surgical instrument. This allows the user to determine whether duplicate imaging has occurred. and the location of counted or uncounted surgical instruments, or It may be useful to reassess any suitable combination of these.

[0100] One or more exemplary embodiments of the system described herein include a surgical instrument collection. For example, such a system may include instruments and devices for use in the measurement method. one or more collection containers, one or more for use in tracking and counting surgical instruments; a tally bag, one or more stands configured to hold the tally bag; or one configured to hold multiple imaging systems (e.g., mobile devices). or multiple mounts, one or more mounted on one or more components of the system; The weight sensing component may include any number of weight sensing components, or any suitable combination thereof.

[0101] One or more exemplary embodiments of the system described herein include surgical fabrics, The surgical instrument may include one or more collection containers for use in collecting any of the surgical instruments. Collection containers are used to hold used equipment before it is counted (e.g. after the fibers have been used). A collection container (e.g., a "kick bucket") may be used by medical personnel. It may be of any suitable size, shape, or material. The collection vessel may have a circular cross section. The container may be or may include a small bucket-like container and may include a liner. The collection container can be made of metal, plastic, or any suitable material. The liner of the collection vessel can be made of any suitable material. For example, the liner can be made of a transparent The clear liner can be a clear plastic liner. The clear liner is convenient for separating dirty surgical instruments from clean hands. The collection container may be of any suitable size. and may accommodate any suitable number of surgical instruments. An exemplary embodiment includes one or more collection containers located at various locations throughout the operating room. For example, one or more collection containers may be placed near a surgical table to allow a physician or nurse to This allows the nurse to easily dispose of surgical instruments after use.

[0102] One or more exemplary embodiments of the system described herein may include a surgical instrument. It may include one or more counting bags for use in containing and counting. For example, As described above, the tally bag is lined and adapted to accommodate one or more surgical instruments. and one or more pockets configured in a manner similar to that described above. The tally bag may be of any suitable size. Such a system may include one or more tally bags of various sizes. For example, to accommodate different types of surgical instruments (e.g., different sized surgical instruments). Different bag sizes can be set to accommodate different types of surgical tissue. The fibers are available in a variety of sizes, and the tally bags may be of any suitable number and of any suitable type. The bag can be appropriately sized to accommodate the surgical fabric. For example, the bag can be The tally bag may be sized to accommodate between 1 and 30 surgical fibers. , may have any suitable number of pockets for accommodating any suitable number of surgical fabrics. Fiber counting protocols generally involve placing one surgical fiber per pocket However, the pockets of the tally bag may be configured to accommodate any suitable number of surgical fibers. In addition, the pockets of the tally bag are large enough to accommodate various types of surgical fabrics. For example, the pockets on the tally bag may be 4 inches by 4 inches or Surgical fabrics in various sizes ranging from 18" x 18" The pocket of the tally bag may also be configured to accommodate laparotomy fibers (e.g., laparotomy fibers). The tubing may be configured to accommodate various types of surgical fabrics, such as surgical fabrics for surgical applications.

[0103] One or more exemplary embodiments of the system described herein for tracking a surgical instrument. Suitable embodiments may include one or more hooks, mounts, or any suitable combination thereof. The stand may include one or more stands, such as a pole with a mounting bracket. one or more for holding multiple imaging devices (e.g., mobile devices) Such a system may include one or more tally bags, Each is configured to include a surgical instrument, such as a surgical device, and is adapted to serve one or more users. Assists the user in implementing one or more aggregation protocols for surgical instruments. (See, for example, FIG. 4 ), the tally bag is lined and has one or more pockets. The backing may interact with one or more hooks of the pole to provide one or more The tally bag may include one or more openings configured to hang the tally bag from a hook. The components of the tally bag (e.g., the liner and pockets) may be made of any suitable material. The lining of the tally bag may be of any suitable color. It may be transparent blue, opaque white, or any other suitable color, and the pocket may be made of a transparent material. The counting bags may be stacked (e.g., stacked on top of each other) so that they are easily identifiable. (When stacked together) one or more of the counting bags may have an opaque, colored lining. It may be beneficial to the environment, resulting in both dirty and clean surgical instruments being placed in it. Contrasting colors (e.g. dirty and clean surgical fabrics). In addition, transparent pockets The surgical instrument may be visualized in the pocket, with one or more markers on the instrument (e.g., For example, it may be beneficial to allow visualization of the endoscopic site (e.g., tags on surgical fabrics), or both. The tally bags can be any shape and size that is convenient for hanging the tally bags from a pole. The exemplary embodiment shown in FIG. A rectangular tally bag with a pocket in the shape of a square is used, but the tally bag and its pocket can be of any suitable shape and size, as described above.

[0104] One or more exemplary embodiments of the system described herein for tracking a surgical instrument. Such embodiments may include one or more weight sensing components. The elements sense the presence of surgical instruments, weigh them, count them, or Any suitable combination may be used. As noted above, one or more weight The detection component includes one or more surgical instruments that are used to determine a first count of the surgical instruments. For example, the one or more weight sensing components may include a scale. configured to receive the tally and communicate the weight information to a processor to determine a first tally; The one or more weight sensing components may also be coupled to one or more collection containers (e.g. For example, a tally bag or "kick bucket" is attached or incorporated into the second The one or more collection containers may be used by a user to determine a tally of one or more A scale or scales were used to detect the weight loss when surgical instruments were removed from the collection container. may be coupled to or include other suitable weight sensing components. Multiple tally bags are fitted with a weight sensor to sense weight changes when surgical instruments are added to the tally bag. FIG. 1 is an exemplary form of one or more strain gauges or scales coupled to a pole for measuring the strain of the The weight sensing component may be coupled to a second tally of the surgical instrument.

[0105] The preceding description, for purposes of explanation, uses specific nomenclature to provide a thorough understanding of the subject matter. However, it will be apparent to one skilled in the art that specific details are not required to practice the present subject matter. It will be apparent, therefore, that the foregoing description of specific exemplary embodiments is given by way of example and not limitation. These are presented for purposes of illustration and description. They are not intended to be exhaustive or to disclose the present subject matter. It is not intended to be limited to the precise form depicted. In light of the above teachings, many Modifications and variations are possible. The illustrative embodiments are illustrative of the principles of the present subject matter and its practical application. These have been selected and described in order to explain some of the features and functions of the present invention and to enable those skilled in the art to easily understand the present subject matter and Various exemplary embodiments may be utilized with various modifications suited to the particular use contemplated. The following claims and their equivalents define the scope of the invention. It is intended.

[0106] The following enumerated descriptions relate to the methods, machine-readable media, and systems described herein. Various examples of (eg, machines, devices, or other apparatus) are described.

[0107] A first example provides a method for tracking a surgical instrument, the method comprising: On one or more processors Receiving a first tally of surgical instruments; receiving one or more images, each image showing one or more surgical instruments; receiving the one or more images of a field of view including: determining a second tally of the surgical instruments based at least in part on the one or more received images; To decide, Providing a notification based on a comparison of a first tally of surgical instruments to a second tally of surgical instruments. Includes and.

[0108] A second example provides a method as in the first example, wherein determining the second tally of surgical instruments comprises: or identifying one or more surgical instruments in the plurality of received images.

[0109] A third example provides a method according to the second example, in which at least one of the received images is a depth image. Provide.

[0110] A fourth example provides a method according to the third example, in which the depth image is an infrared image.

[0111] A fifth example provides a method according to any one of the second to fourth examples, comprising: The other is a color image.

[0112] A sixth example provides a method according to any of the second to fifth examples, wherein the surgical instrument comprises It is a fiber for use.

[0113] A seventh example provides a method according to any of the second to sixth examples, further comprising: Identifying the surgical instrument includes identifying and detecting a vessel having one or more compartments in one or more images. This includes identifying and characterizing the

[0114] An eighth example provides a method according to the seventh example, wherein the container comprises a liner and a rectangular shape on the liner. and a plurality of pockets arranged in a grid.

[0115] A ninth example provides a method according to the eighth example, wherein the plurality of pockets includes a transparent material.

[0116] A tenth example provides the method of the eighth example, wherein the backing comprises an opaque material.

[0117] An eleventh example provides a method according to any one of the seventh to tenth examples, further comprising characterizing the container. includes determining one or more parameters of the vessel.

[0118] A twelfth example provides a method according to the eleventh example, comprising: Determining the parameter includes quantifying one or more compartments of the vessel.

[0119] A thirteenth example provides a method according to the eleventh or twelfth example, comprising: The determination of the one or more parameters determines the geometry of one or more compartments of the container. This includes determining

[0120] A fourteenth example provides a method according to any one of the eleventh to thirteenth examples, comprising: The determination of one or more parameters of the vessel may include determining the one or more parameters of the vessel in each of one or more compartments of the vessel. This includes determining whether or not there are surgical instruments in the area.

[0121] A fifteenth example provides a method according to any one of the first to fourteenth examples, comprising: The determination of the aggregate of the two is, at least in part, to hold one or more surgical instruments. It is further based on detecting a change in weight of the configured container.

[0122] A sixteenth example provides a method according to any one of the first to fifteenth examples, comprising: Receiving the tally includes identifying one or more surgical instruments within the image.

[0123] A seventeenth example provides a method according to any one of the first to sixteenth examples, comprising: The receipt of the tally is based, at least in part, on detecting the weight of the one or more surgical instruments. Made.

[0124] An eighteenth example provides a method according to any one of the first to seventeenth examples, comprising: The tally of 1 is provided via user input.

[0125] A nineteenth example provides a method according to any one of the first to eighteenth examples, comprising: The determination of the second tally includes updating the current index value of the surgical instrument.

[0126] A twentieth example provides a method according to the nineteenth example, comprising displaying a current index value of a surgical instrument. Further including displaying in play.

[0127] A twenty-first example provides a method according to any one of the first to twentieth examples, comprising: and further prompting the user to verify the accuracy of at least one of the first and second tabulations. include.

[0128] A twenty-second example provides a method according to any one of the first to twenty-first examples, comprising: a first tally of surgical instruments, a second tally of surgical instruments, or at least one of notifications on the display It further includes displaying.

[0129] A twenty-third example provides a method according to any one of the first to twenty-second examples, comprising: The method further includes providing a summary report of the equipment used.

[0130] A twenty-fourth example provides a method according to any one of the first to twenty-third examples, comprising: The provision of notice based on a comparison between two counts is based on the fact that the first count does not match the second count. generating an alert in response to the

[0131] A twenty-fifth example provides a method according to the twenty-fourth example, where the alert follows a predetermined protocol.

[0132] A twenty-sixth example provides a method according to any one of the first to twenty-fifth examples, comprising: At least a portion of the is received prior to the medical procedure.

[0133] A twenty-seventh example provides a method according to any one of the first to twenty-sixth examples, wherein the second aggregation comprises: This is determined during or after a medical procedure.

[0134] A twenty-eighth example provides a method according to any one of the first to twenty-seventh examples, further comprising: The determination uses one or more machine learning models to analyze one or more incoming images. This includes:

[0135] A 29th example provides a method as in the 28th example, wherein the one or more machine learning models are , incorporating deep learning techniques.

[0136] A thirtieth example provides a system for tracking a surgical instrument, the system comprising: The one or more processors are configured to perform operations including: The operation is Receiving a first tally of surgical instruments; receiving one or more images, each image showing one or more surgical instruments; receiving the one or more images of a field of view including: determining a second tally of the surgical instruments based at least in part on the one or more received images; To decide; Providing a notification based on a comparison of a first tally of surgical instruments to a second tally of surgical instruments. Includes and.

[0137] A thirty-first example provides the system of the thirtieth example, further comprising: determining a second tally of the surgical instruments. includes identifying one or more surgical instruments in the one or more received images.

[0138] A thirty-second example provides the system according to the thirty-first example, wherein at least one of the received images is a depth image. This is a 3D image.

[0139] A thirty-third example provides the system of the thirty-second example, wherein the depth image is an infrared image.

[0140] A thirty-fourth example provides a system according to any one of the thirty-first to thirty-third examples, At least one of the images is a color image.

[0141] A thirty-fifth example provides a system according to any one of the thirty-first to thirty-fourth examples, Identifying a plurality of surgical instruments in one or more images of a container including one or more compartments This includes identifying and characterizing the

[0142] A thirty-sixth example provides the system of the thirty-fifth example, wherein the characterization of the container comprises: Or determining a plurality of parameters.

[0143] A thirty-seventh example provides the system of the thirty-sixth example, further comprising: Determining the parameter of the number includes quantifying one or more compartments of the container.

[0144] A thirty-eighth example provides the system according to the thirty-sixth example or the thirty-seventh example, The determination of one or more parameters of the vessel may include determining the geometric arrangement of one or more compartments of the vessel. This includes the determination of:

[0145] A thirty-ninth example provides a system according to any one of the thirty-sixth to thirty-eighth examples, The determination of one or more parameters of the vessel may include determining one or more parameters of each of one or more compartments of the vessel. This includes determining whether or not there are surgical instruments in the area.

[0146] A fortieth example provides the system of any of the thirty-fifth to thirty-ninth examples, wherein the container comprises: A flexible container including a liner and a number of pockets arranged in a rectangular grid.

[0147] A forty-first example provides the system of the forty-first example, wherein the plurality of pockets are made of a transparent material. Includes.

[0148] A forty-second example provides the system of the fortieth example, wherein the backing comprises an opaque material. .

[0149] A forty-third example provides a system according to any one of the thirty to forty-second examples, comprising a processor. and an optical sensor configured to generate one or more images received by the optical sensor. Prepare for the future.

[0150] A forty-fourth example provides the system according to any one of the thirtieth to forty-third examples, further comprising a display screen. It further comprises:

[0151] A forty-fifth example provides the system of the forty-fourth example, wherein the display screen includes a tracked surgical It is configured to display a summary report of the device.

[0152] A 46th example provides the system of the 44th example or the 45th example, wherein the display screen comprises: A notification is configured to be displayed based on a comparison between the first tally and the second tally.

[0153] A 47th example provides the system of the 46th example, wherein the notification is: If the data does not match the actual data, a warning will be issued according to the established protocol.

[0154] A 48th example provides a system according to any one of the 30th to 47th examples, comprising a surgical instrument. The tool is a surgical fabric.

[0155] A forty-ninth example provides a system according to any one of the thirtieth to forty-eight examples, At least one of the processors is a mobile device removably mounted on a stand. The device is located within a computing device.

[0156] A fiftieth example provides a method for tracking a surgical instrument, the method comprising: On one or more processors, receiving an image of a field of view, the field of view including one or more surgical instruments; and, Determining a presence of one or more surgical instruments within the image; and providing an indication of a determined presence of one or more surgical instruments in the image. nothing.

[0157] A 51st example provides a method according to the 50th example, comprising: The aim of the study was to quantify one or more surgical instruments in an image based on a machine learning model. Further includes:

[0158] A 52nd example provides a method according to the 51st example, wherein the machine learning model comprises deep learning techniques. It is incorporated.

[0159] A fifty-third example provides a method according to any one of the fifty to fifty-second examples, comprising one or more Determining the presence of the plurality of surgical instruments includes detecting the presence of a container within the image, the container being a plurality of surgical instruments. each compartment configured to receive at least one respective surgical instrument. will be done.

[0160] A fifty-fourth example provides the method of the fifty-third example, further comprising: determining whether each compartment is a surgical instrument based on the presence or absence of the surgical instrument; The method further includes classifying the

[0161] A 55th example provides a method according to any of the 50th to 54th examples, comprising: Providing an indication of the presence provides an indication of a determined presence of at least one surgical instrument in the image. This includes:

[0162] A fifty-sixth example provides a method according to any one of the fifty to fifty-fifth examples, comprising one or more A number of surgical instruments include surgical fabrics.

[0163] A fifty-seventh example provides a method according to any one of the fifty to fifty-sixth examples, comprising one or more The number of surgical instruments includes surgical instruments.

[0164] The 58th example performs the operation (e.g., the method operation) performed in any one of the previous examples. A carrier medium carrying machine-readable instructions for controlling a machine to perform the steps of the method of the present invention is provided.

[0165] A fifty-ninth example, when executed by one or more processors of a machine, causes the machine to A machine that includes instructions for performing the operations (e.g., method operations) specified in any one of the examples. Provide a readable medium (eg, a non-transitory machine-readable storage medium).

Claims

1. determining, by the one or more processors, a first tally of surgical instruments; and the first count is to quantify the set of surgical instruments. Determining a first tally; The one or more processors then select at least one sub-set of the surgical instruments. accessing an image depicting the set; A second determination of the surgical instrument based on the image by the one or more processors. determining a compilation of The one or more processors perform the first collection of surgical instruments and the hand collection. and providing a notification based on a comparison of the surgical instrument to the second summary.

2. The determination of the second count of the surgical instruments includes determining at least one of the surgical instruments depicted in the image. The method of claim 1 , further comprising identifying a surgical instrument.

3. The identification of the at least one surgical instrument depicted in the image is based on a set of compartments. The method of claim 2, further comprising characterizing a container depicted in the image. Law.

4. The characterization of the vessel includes quantifying the set of compartments of the vessel based on the images. The method of claim 3, comprising:

5. The characterization of the container comprises determining a geometry of the set of compartments of the container based on the image.

4. The method of claim 3, further comprising determining a target location.

6. The characterization of the container comprises determining, based on the image, which of the set of compartments of the container 4. The method of claim 3, further comprising determining the presence or absence of a corresponding surgical instrument in each compartment. Law.

7. The determination of the second count of the surgical instruments includes determining whether at least one of the set of surgical instruments is in a single count. based on a detected change in weight of a container configured to receive at least one surgical instrument. The method according to claim 1 .

8. The determination of the first count of the surgical instruments includes determining the location of the surgical instruments from a further image. The method of claim 1 , further comprising identifying a group of neighboring neighbors.

9. The determination of the first count of the surgical instruments includes determining whether at least one of the set of surgical instruments is in a first group.

10. The method of claim 1, wherein the weight of at least one surgical instrument is based on the weight of the surgical instrument.

10. The providing of the notification based on a comparison of the first tally to the second tally includes:

2. The method of claim 1, further comprising: providing a warning in response to a tally not matching the second tally. The method according to

11. A machine-readable medium containing instructions, The instructions, when executed by one or more processors of a machine, cause the machine to: determining a first tally of the surgical instruments, the first tally being a determining a first tally of the surgical instruments, the first tally being a quantification set; accessing an image depicting at least a subset of the set of surgical instruments; 、 determining a second count of the surgical instruments based on the images; and based on a comparison of the first tally of the surgical instruments and the second tally of the surgical instruments. A machine-readable medium for causing operations to be performed, including providing notification.

12. The determination of the second count of the surgical instruments includes determining a number of surgical instruments depicted in the set of images. The machine-readable medium of claim 11 , further comprising identifying at least one surgical instrument.

13. The identification of the at least one surgical instrument depicted in the image may be based on a set of compartments.

13. The method of claim 12, further comprising characterizing a container depicted in the image. Machine-readable medium.

14. The characterization of the vessel includes quantifying the set of compartments of the vessel based on the images. The machine-readable medium of claim 13 , further comprising:

15. 1. A system comprising: one or more processors; a memory for storing instructions; The instructions are transmitted to at least one of the one or more processors. When executed by the determining a first tally of the surgical instruments, the first tally being a determining a first tally of the surgical instruments, the first tally being a quantification set; accessing an image depicting at least a subset of the set of surgical instruments; 、 determining a second count of the surgical instruments based on the images; and based on a comparison of the first tally of the surgical instruments and the second tally of the surgical instruments. The system causes an operation to be performed, including providing notification.

16. The determination of the second count of the surgical instruments includes determining a number of surgical instruments depicted in the set of images. The system of claim 15 , further comprising identifying at least one surgical instrument.

17. The identification of the at least one surgical instrument depicted in the image is based on a set of compartments.

17. The method of claim 16, further comprising characterizing a container depicted in the image. system.

18. The characterization of the vessel includes quantifying the set of compartments of the vessel based on the images. The system of claim 17, further comprising:

19. The characterization of the container comprises determining a geometry of the set of compartments of the container based on the image.

20. The system of claim 17, further comprising: determining a target location for the first and second sensors.

20. The characterization of the container comprises determining, based on the image, which of the set of compartments of the container 18. The method of claim 17, further comprising determining the presence or absence of a corresponding surgical instrument in each of the compartments. system.

Images