A bleeding score determination and surgical guidance method and system used in surgical operations

Abstract

This invention relates to a bleeding score determination and surgical guidance method and system which allows obtaining a numerical bleeding score by using at least one parameter such as color essence, saturation, brightness, angularity, lightness and / or hue to determine the red color that can be considered as bleeding in instant frames taken from at least one imaging unit that enables frame retrieval in arthroscopic interventions and, guiding of the surgeon based on this score.

Description

[0001] DESCRIPTION

[0002] A BLEEDING SCORE DETERMINATION AND SURGICAL GUIDANCE

[0003] METHOD AND SYSTEM USED IN SURGICAL OPERATIONS

[0004] The present invention relates to a bleeding score determination and surgical performance evaluation method and system using image processing and color metrics, especially suitable for use in arthroscopic surgery.

[0005] Arthroscopic surgery is a minimally invasive surgical method used in the diagnosis and treatment of arthritic diseases. The arthroscope used in this method is connected to a fiber optic light source and images are taken with a video camera system. Arthroscopy is an optical system application and the majority of the structures inside of the joint can be visualized through a monitor connected to a camera attached to the end of the optical system. Since the frames inside the joint are enlarged multiple times, a detailed examination of all intra-articular structure is possible. Frames can be recorded on video and photographs can be taken. By this means, the inside of the joint can be observed, and necessary surgical treatments can be administered with minimal and small incisions without the large incisions required in open surgical treatment methods.

[0006] Favorable outcomes of the arthroscopic procedures are strongly correlated with the quality of the vision during the operation. Current devices provide a highly improved visualization of the joint using 4K visual technologies and improved light sources. However, the adequate visualization of the intra-articular structures is still heavily dependent on the amount of blood in the arthroscopic fluids which reduces the recognition of the tissues and creates a thick red filter in front of camera. Currently, in order to maintain an adequate visualization in arthroscopic surgery, surgeons use a flow controlling pump that maintains a certain pressure within the joint, and cauterize the bleeding vessels with a device called a radiofrequency probe. Many chemicals such as epinephrine or tranexamic acid are known to be added to arthroscopic fluid to experimentally improve visualization during the arthroscopic surgery. However, the main limitation of these studies is that the evaluation of the quality of the vision during the surgical operation is performed by subjective methods such as direct observation and visual analogue scales. In order to objectively evaluate the surgical operation, it is necessary to measure the amount of bleeding in a time -dependent manner. In laparoscopic surgery where the surgical space is inflated with gas, methods are known which use color spaces such as RGB to guide the surgeon where the bleeding is occurring on the surgical site, accumulating on the tissues. The said methods only focus on the bleeding site by detecting the red color on the tissue. In addition, in operations other than arthroscopic surgery, a turbidity of the fluid (arthroscopic fluid) where bleeding affects the operation much more negatively is not experienced. As a result, a method and system for determining the bleeding score to assist the surgeon evaluate their performance in controlling intraarticular bleeding is not suggested in the known methods.

[0007] Visual clarity in arthroscopic procedures has long been evaluated through different methods in the orthopedic literature. These methods include direct subjective visualization, collection and analysis of the arthroscopic fluid by cell counting systems, and the assessment of patient’s total blood loss by comparing preoperative and postoperative total blood counts. However, none of these methods yielded adequate objective solutions.

[0008] In US patent document no. US2018125333, known in the state of the art, a method for interactive bleeding detection in surgical systems is disclosed. By using the pixel values of the red color, the bleeding site is detected. The invention of the said patent aims to detect blood accumulated on the surface of the surgical site during robotic abdominal surgery. In this invention, it is stated that an estimation about the amount of bleeding can be obtained by time-dependent change in the bleeding site which is pooled on the surface but the main focus is on the detection of an inadvertent bleeding that could risk the patient’s life, without any concern on the quality of the visual images. In International patent application no. WO2022066797, known in the state of the art, a method for detecting bleeding in intra-abdominal surgery with a complex tissue pattern in order to prevent overlooking bleeding and thus prevent uncontrolled blood loss of the patient is described. Likewise, a system based on visual clarity to guide surgical operations, especially in arthroscopic surgery, is not described.

[0009] In general, the main difference between quantifying the amount of blood on a tissue in surgical operations and detection of a minimal amount of bleeding in arthroscopic surgery is the use of an arthroscopic fluid during the operation in arthroscopic surgery. Therefore, blood contaminating the fluid has a greater negative impact on the arthroscopic surgery. In arthroscopic surgery, the ability of surgeons to evaluate and improve their perioperative performance is directly correlates with the quantitative measurement or numerical expression of bleeding on a scale. In normal surgical operations, it is aimed to stop bleeding during the operation by detecting the discoloration on the tissue, which is caused only by the red color of the blood. Thus, a quantitative bleeding measurement method which provides the surgeon with the opportunity to evaluate Perioperative performance has not been suggested in implementations in the known state of the art.

[0010] Some scientific publications in the known state of the art describe methods for measuring bleeding in the surgical operation based on the surgeon grading the postoperative image quality with Visual Analog Score (VAS). However, this method is also based on the subjective evaluation of the surgeon.

[0011] The objective of the invention is to provide a bleeding score determination method and system, particularly for use in arthroscopic surgery, which provides an objective measurement of bleeding and allows the surgeon to evaluate and improve their performance based on quantitative bleeding data.

[0012] Thanks to the method and system of the invention, quantitative data using color metrics and an image processing system are provided in arthroscopic surgery by excluding human eye intervention. Direct objective measurement of bleeding into the joint will provide the surgeon with the occasion of comparing the effects of different fluid compositions or methods that are used to reduce bleeding during the arthroscopy. Moreover, the surgeons will be able to assess and track their own technical improvements by achieving better visualization through less bleeding. Surgeons will be able to compare their surgical techniques with other colleagues all around the world using standardized visual adjustments.

[0013] The method of the invention is based on the idea that the main colors all along a video of an arthroscopic procedure is grayish white and a pale pink. Despite all precautions, a varying amount of bleeding occurs during the procedure and the blood acts as a red dye blended in the arthroscopic fluid. This phenomenon is observed as an increase in the amount of red pixels in the color metric analysis of the related frame, which is eventually correlated to the amount of blood in the arthroscopic fluid.

[0014] In the method and system of the invention, HSV (Hue, Saturation, Value) and LAB (Lightness, A, B) color spaces are used together to detect bleeding scores. These color spaces provide a more suitable structure for image processing algorithms since they can represent properties of colors such as angularity, saturation, brightness and color difference.

[0015] In the method of the invention, firstly, threshold values were determined for each of the color essence, saturation and brightness components of the HSV (Hue- Saturation-Value) color space and the lightness, red / green and yellow / blue coordinates of the LAB (Lightness-A-B) color space. The said threshold values are based on selecting the redness in multiple frames that the average human eye would consider as bleeding.

[0016] According to the method of the invention, one frame per second is taken from at least one imaging unit, such as a camera, which allows frames to be taken during the surgical operation. For each frame, a percentage value, which is the bleeding rate, is obtained by using the ratio of the number of pixels above these thresholds to the total number of pixels in that frame. In a preferred embodiment of the invention, the weighted average of the last three frames’ bleeding rates are used to determine the bleeding score of the obtained frame. The bleeding score in each second is calculated over the average of the previous three seconds including that second. For example, to determine the bleeding score at the 5thsecond, the bleeding scores of the frames at the 3rd, 4thand 5thsecond are averaged. In this way, the method of the invention performs scoring in periods of seconds rather than minutes. The obtained value is a pixel-percentage-based metric which places the amount of bleeding on a gradient between 1 and 10 using color features in the frame.

[0017] With the method of the invention, the percentage bleeding data in each frame is placed on a scale between 10 and 100. For example, to calculate the average bleeding score for the frame taken at the 3rdsecond, three frames at the 3rd, 2ndand 1stseconds are taken. In a case where the 3rdsecond bleeding percentage is 80%, the 2ndsecond bleeding percentage is 60% and the 1stsecond bleeding percentage is 40%, the average bleeding percentage is 66%. When calculating this score, the bleeding data in the instant frame is weighted. For this example, the bleeding weight of the 3rdsecond would be 3 units, the 2ndsecond would be 2 units and the 1stsecond would be 1 unit. Thus, this gives an average of (80*3 + 60*2 + 40* 1) / 6= 66%. The equivalent of this percentage average data on a gradient of 1 to 10 would be a bleeding score of 7.

[0018] In the known state of the art, the RGB color space, which is known to be used for bleeding detection, provides the opportunity to analyze only red, blue and green color values. However, when HSV and LAB color spaces are used together, it is possible to perform a versatile and detailed analysis of each frame. With the use of the said color spaces, since changes will be observed not only in the amount of redness when bleeding is observed in the frame, but also in other mentioned components due to the blurring of the fluid, the use of HSV and LAB color spaces improves the accuracy compared to using only RGB space. In addition, although all the structures visualized inside the joint in arthroscopic surgery are in pastel shades of white and beige color, in cases of fracture or severe inflammation, intra- articular tissues can also be observed in pink-red tones. The RGB system alone is insufficient to distinguish these sites of inflammation from an actual bleeding.

[0019] A bleeding score determination and surgical guidance method used in surgical operations according to the invention comprises the steps of

[0020] - retrieving at least one frame from an imaging unit suitable for use during a surgical operation, determining at least one threshold value based on at least one parameter such as color essence, saturation, brightness, angularity, lightness and / or hue to determine the red color which can be considered as bleeding during the surgical operation, comparing the color essence, saturation, brightness, angularity, lightness and / or hue values of each pixel in each frame with predetermined threshold values, converting the bleeding into a numerical data by calculating the ratio of the number of pixels exceeding said threshold values to the total number of pixels in the frame.

[0021] The method of the invention is suitable for use during a surgical operation, especially during arthroscopic surgery. During the surgical operation, frames are taken in periods of seconds from at least one imaging device, such as a camera. For the detection of bleeding in each taken frame, firstly at least one threshold value is determined based on at least one parameter such as color essence, saturation, brightness, angularity, lightness and / or hue. These threshold values may vary for each case. For example, for a threshold value based on the balance of green, red and blue, which are hues, a value of 200 in one frame may be considered as bleeding, while in another frame this value may change. For each frame, the number of pixels exceeding the threshold values of the specified parameters is determined. The said number of pixels is divided by the total number of pixels in that frame and a percentage bleeding value is obtained. By this means, bleeding is scored second by second and provides an opportunity for guidance to the surgeon. In one embodiment, the method of the invention comprises the steps of retrieving multiple frames from the imaging unit during the surgical operation for a predetermined period, calculating the ratio of the number of pixels exceeding the threshold values in all frames to the total number of pixels of each frame individually, and averaging based on the number of frames taken in the determined time period. For example, a frame is taken every one second for 10 seconds of imaging. The bleeding score of the 10thsecond is not calculated only on the frame taken at the 10thsecond. The average value can be calculated by averaging the 1stto 10thseconds together or by averaging the 8th, 9thand 10thseconds. By this means, more accurate scoring is obtained.

[0022] In one embodiment, the method of the invention comprises the steps of retrieving one frame from each one of the last three seconds (including the instant second) together with the second the instant frame was taken to calculate the average in the current second, and averaging the percentage of pixels exceeding the threshold value of the three frames taken, by assigning a multiplier to the frame in the last second so that the weight coefficient is higher. For example, to calculate the average bleeding score for the frame taken at the 3rdsecond, three frames at the 3rd, 2ndand 1stseconds are taken. In a case where the 3rdsecond bleeding percentage is 80%, the 2ndsecond bleeding percentage is 60% and the 1st second bleeding percentage is 40%, the average bleeding percentage is 66%. For this example, the bleeding weight of the 3rdsecond would be 3 units, the 2ndsecond would be 2 units and the 1stsecond would be 1 unit. By this means, a more accurate bleeding score which is weighted to the value in the instant frame can be obtained.

[0023] In one embodiment, the method of the invention comprises the step of matching the time-dependent bleeding data obtained in percentage value type to a bleeding score between 1 and 10 corresponding to the percentage value. By this means, a more practical scoring for surgical guidance is obtained.

[0024] In one embodiment, the method of the invention comprises the step of determining the number of red pixels using HSV and LAB color spaces together. HSV and LAB color spaces are based on multiple parameters which enable much more accurate detection compared to RGB. By this means, the bleeding score is calculated much more precisely and accurately.

[0025] In one embodiment, the method of the invention comprises the step of providing an audible or visual warning to the surgeon to prevent the surgical operation from being performed when a predetermined upper limit bleeding score is reached

[0026] A bleeding score determination and surgical guidance system of the invention comprises a controller which performs the method steps, at least one imaging unit which is used during the surgical operation, and at least one optical system.

[0027] The system of the invention comprises a monitor which provides the surgeon with a time-dependent bleeding score graph during the Perioperative period.

[0028] The system of the invention comprises a monitor which provides the surgeon with an audible or visual warning at the time of operation when the bleeding score exceeds a predetermined upper limit.

[0029] The system of the invention comprises an audible or visual warning unit which, based on the bleeding score, further alerts the surgeon to perform bleeding stopping treatments during the surgical operation to have an improved visual clarity.

[0030] The system of the invention comprises a memory unit which stores frames of the surgical operation and time-based bleeding score information.

Claims

CLAIMS1. A bleeding score determination and surgical guidance method for use in surgical operations, characterized by the steps of- retrieving at least one frame from an imaging unit suitable for use during a surgical operation, determining at least one threshold value based on at least one parameter such as color essence, saturation, brightness, angularity, lightness and / or hue to determine the red color which can be considered as bleeding during the surgical operation, comparing the color essence, saturation, brightness, angularity, lightness and / or hue values of each pixel in each frame with predetermined threshold values, converting the bleeding into a numerical data by calculating the ratio of the number of pixels exceeding said threshold values to the total number of pixels in the frame.

2. A bleeding score determination and surgical guidance method according to Claim 1, characterized by the steps of retrieving multiple frames from the imaging unit during the surgical operation for a predetermined period, calculating the ratio of the number of pixels exceeding the threshold values in all frames to the total number of pixels of each frame individually, and averaging based on the number of frames taken in the determined time period.

3. A bleeding score determination and surgical guidance method according to Claim 2, characterized by the steps of retrieving one frame from each one of the last three seconds together with the second the instant frame was taken to calculate the average in the current second, and averagingthe percentage of pixels exceeding the threshold values of the three frames taken, by assigning a multiplier to the frame in the last second so that the weight coefficient is higher.

4. A bleeding score determination and surgical guidance method according to any one of Claims 1 to 3, characterized by the step of matching the time-dependent bleeding data obtained in percentage value to a bleeding score between 1 and 10 corresponding to the percentage value.

5. A bleeding score determination and surgical guidance method according to any one of Claims 1 to 4, characterized by the step of determining the number of red pixels by using HSV and LAB color spaces together.

6. A bleeding score determination and surgical guidance method according to any one of Claims 1 to 5, characterized by the step of providing an audible or visual warning to the surgeon when a predetermined upper limit bleeding score is reached that would prevent the surgical operation from being performed.

7. A bleeding score determination and surgical guidance method, comprising a controller which performs the steps of the method according to any one of Claims 1 to 6, at least one imaging unit which is used during the surgical operation, and at least one optical system.

8. A bleeding score determination and surgical guidance method according to Claim 7, characterized by a monitor which provides the surgeon with a time-dependent bleeding score graph during the postoperative period.

9. A bleeding score determination and surgical guidance method according to Claim 8, characterized by a monitor which provides the surgeon with an audible or visual warning at the time of operation when the bleeding score exceeds a predetermined upper limit.

10. A bleeding score determination and surgical guidance method according to any one of Claims 7 to 9, characterized by an audible or visualwarning unit which, based on the bleeding score, further alerts the surgeon to perform bleeding stopping treatments during the surgical operation to have a successful operation.

11. A bleeding score determination and surgical guidance method according to any one of Claims 7 to 10, characterized by a memory unit which stores frames of the surgical operation and time-based bleeding score information.