A method and system for intraoperative bleeding recognition and measurement based on computer vision

Abstract

The present invention provides a method and system for intraoperative bleeding recognition and measurement based on computer vision, belonging to the technical field of computer vision. By using a bloodstain recognition model that combines a dynamic threshold and a static threshold for bloodstain recognition, and obtaining a relationship model diagram between each variable based on the proportion of bloodstain pixels and each prognostic index through correlation analysis, and then forming a clinical prognosis plan according to the relationship model diagram between each variable based on the proportion of bloodstain pixels and each prognostic index, the technical effects of high calculation efficiency and applicability to thoracoscopic surgery scenarios with long duration and high picture pixels are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a method, system, electronic device, and storage medium for intraoperative bleeding recognition and quantification based on computer vision. Background Art

[0002] Surgery has gradually shifted towards the minimally invasive surgery (MIS) paradigm, such as laparoscopic surgery and video-assisted thoracoscopic surgery. Compared with open-chest surgery, video-assisted thoracoscopic surgery (VATS) has become the standard method for treating patients with early and locally advanced non-small cell lung cancer due to its fewer postoperative acute phases, less pulmonary function impairment, lower postoperative morbidity, and shorter hospital stay. During VATS surgery, a digital camera records the surgical process as a surgical video. These surgical videos contain rich information about the anatomical structure of the patient's surgical site, the surgical process, the movement of instruments during the surgical process, etc. Intuitively, the bloodstain information at the surgical site may have a certain correlation with the prognosis. For example: VATS surgeries with less bloodstains at the surgical site may result in less postoperative drainage and a shorter hospital stay. For an experienced surgeon, it is an easy task to determine whether the surgical site is "clean", and the intraoperative blood loss can also be obtained by postoperative weighing. However, the surgical video contains more information about the dynamic event of bleeding than just the total amount of bleeding, such as the bleeding period, the bleeding condition after irrigation, etc. How to accurately and effectively quantify the bloodstains in VATS videos through the analysis of surgical videos and extract variables that can reflect the surgical state and are significantly correlated with the prognosis is an unsolved challenge.

[0003] By processing surgical videos through computer vision (CV) algorithms, the surgical stage and surgical instruments can be identified, but laparoscopic surgery analysis cannot be performed. In the prior art, Hassan et al. developed an algorithm based on texture feature descriptors that operates on the normalized gray-level co-occurrence matrix of the image amplitude spectrum to detect bloodstains. With the emergence of computer vision-based intraoperative bleeding recognition and quantification methods for pixel-based bloodstains, Pan et al. extracted bloodstain features and proposed an intelligent bloodstain detection based on a probabilistic neural network. In Usman's research, a pixel-based method using a support vector classifier to detect bloodstains in wireless capsule endoscope videos was proposed. Novozamsky et al. defined a new color space based on a pixel-by-pixel method to maximize the separability of blood pixels and the intestinal wall in capsule endoscope videos; although the bloodstain detection task can basically be completed, the disadvantages are as follows: the algorithm is complex and the computational efficiency is low; it cannot be applied to thoracoscopic surgery scenarios with long durations and high-resolution pixels.

[0004] Therefore, there is an urgent need for an intraoperative bleeding recognition and quantification method that can accurately identify bloodstain pixels from images. Summary of the Invention

[0005] The present invention provides a method, system, electronic device, and storage medium for intraoperative bleeding recognition and measurement based on computer vision to overcome at least one technical problem existing in the prior art.

[0006] To achieve the above object, the present invention provides a method for intraoperative bleeding recognition and measurement based on computer vision, the method comprising:

[0007] Cutting the acquired surgical video into a picture stream at a set frequency;

[0008] Determining the blood stain pixel occupancy ratio of each picture in the picture stream through a pre-acquired blood stain recognition model, and determining a plurality of variables based on the blood stain pixel occupancy ratio through the blood stain pixel occupancy ratio of each picture in the picture stream;

[0009] Based on a preset correlation analysis rule, respectively performing correlation analysis on each variable based on the blood stain pixel occupancy ratio with the significant indicators of the patient's surgical single factors and the prognostic indicators; wherein, the significant indicators of the patient's surgical single factors include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, surgical duration, and estimated blood loss; the prognostic indicators include postoperative drainage volume and whether the postoperative intubation time is prolonged;

[0010] Based on the correlation analysis results, establishing a relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognostic indicator;

[0011] Forming a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognostic indicator.

[0012] Further, preferably,

[0013] The method for obtaining the blood stain pixel occupancy ratio through the blood stain recognition model includes:

[0014] Obtaining the color channel information of each pixel point of the picture to be detected; wherein, the color channel information includes the gray value, red channel value, ratio of the gray value to the red channel value of each pixel point, and the ratio of the sum of the gray values to the sum of the red channel values of the picture to which the pixel point belongs;

[0015] Selecting the pixel points whose color channel values meet the set threshold as blood stain pixel points according to a preset standard; the preset standard is that the red channel value is greater than the red channel set value, the gray value is less than the gray set value, and the ratio of the gray value to the red channel value is less than the ratio set value; wherein, the ratio set value is obtained through the following formula:

[0016] c + d×Frame Gray / R , c, d are both threshold parameters; (Frame Gray / R) is the ratio of the sum of the grayscale values of the pixels belonging to the picture to the sum of the red channel values;

[0017] The proportion of bloodstain pixels among all the pixels in the picture to be detected is used as the bloodstain pixel proportion value.

[0018] Furthermore, preferably,

[0019] The surgical video is marked with the start time of the flushing stage;

[0020] The variables based on the bloodstain pixel proportion include the total sum of the bloodstain pixel proportion and the total sum of the bloodstain pixel proportion after the flushing stage; the total sum of the bloodstain pixel proportion of the surgical video is obtained through the bloodstain pixel proportion values of each picture in all the picture streams, and the total sum of the bloodstain pixel proportion after the flushing stage is obtained through the bloodstain pixel proportion values of each picture in the picture stream after the start time of the flushing stage.

[0021] Furthermore, preferably,

[0022] A training method for a bloodstain recognition model, including:

[0023] Dividing the bloodstain picture data set into a training set and a validation set, where the bloodstain picture data set includes pictures containing bloodstain pixels and pictures containing non-bloodstain pixels;

[0024] Using the training set to establish a pre-trained bloodstain recognition model, and the pre-trained bloodstain recognition model screens out the pixels whose color channel values meet the set threshold as bloodstain pixels according to a preset standard; the preset standard is that the red channel value is greater than the red channel set value a, the grayscale value is less than the grayscale set value b, and the ratio of the grayscale value to the red channel value is less than the ratio set value; among them, the ratio set value is obtained through the following formula:

[0025] c + d×Frame Gray / R , a, b, c, d are all threshold parameters; (Frame Gray / R ) is the ratio of the sum of the grayscale values of the pixels belonging to the picture to the sum of the red channel values;

[0026] Using the validation set to perform threshold parameter tuning and verification on the pre-trained bloodstain recognition model to obtain the best threshold parameter combination.

[0027] Furthermore, preferably,

[0028] Performing correlation analysis on each variable based on the bloodstain pixel proportion with the patient's surgical single-factor significant indicators and prognosis indicators respectively, including,

[0029] Adding each variable based on the bloodstain pixel proportion, the patient's surgical single-factor significant indicators and the postoperative drainage volume into a linear regression model, and performing backward regression to screen out the variables based on the bloodstain pixel proportion that are significantly correlated with the postoperative drainage volume;

[0030] Determine whether the postoperative intubation time is prolonged by whether the postoperative intubation time is greater than or equal to the intubation time threshold; record the prolonged intubation time as 1 and the non-prolonged intubation time as 0;

[0031] Perform logistic regression on each variable based on the blood stain pixel ratio respectively with the significant indicators of the patient's surgery single factor and whether the postoperative intubation time is prolonged, and perform backward regression to screen the variables based on the blood stain pixel ratio that are significantly correlated with whether the postoperative intubation time is prolonged.

[0032] To solve the above problems, the present invention also provides an intraoperative bleeding recognition and measurement system based on computer vision, including:

[0033] An acquisition unit for cutting the acquired surgical video into a picture stream at a set frequency;

[0034] A blood stain recognition unit for determining the blood stain pixel ratio of each picture in the picture stream through a pre-acquired blood stain recognition model, and determining a plurality of variables based on the blood stain pixel ratio through the blood stain pixel ratio of each picture in the picture stream;

[0035] An association analysis unit for performing association analysis on each variable based on the blood stain pixel ratio respectively with the significant indicators of the patient's surgery single factor and the prognosis indicators according to the preset association analysis rules; wherein, the significant indicators of the patient's surgery single factor include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, operation duration and estimated blood loss; the prognosis indicators include postoperative drainage volume and whether the postoperative intubation time is prolonged;

[0036] A clinical prognosis plan forming unit for establishing a relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis indicator based on the association analysis result; forming a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis indicator.

[0037] To solve the above problems, the present invention also provides an electronic device, which includes:

[0038] A memory storing at least one instruction; and

[0039] A processor for executing the instructions stored in the memory to implement the steps in the above-mentioned method for intraoperative bleeding recognition and measurement based on computer vision.

[0040] To solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned method for intraoperative bleeding recognition and measurement based on computer vision.

[0041] A method, system, electronic device, and storage medium for intraoperative bleeding recognition and measurement based on computer vision according to the present invention perform bloodstain recognition by adopting a bloodstain recognition model combining a dynamic threshold and a static threshold, and obtain a relationship model diagram between each variable based on the proportion of bloodstain pixels and each prognostic index through correlation analysis. Furthermore, a clinical prognosis plan is formed according to the relationship model diagram between each variable based on the proportion of bloodstain pixels and each prognostic index, achieving the technical effects of high calculation efficiency and being applicable to the thoracoscopic surgery scenario with a long duration and high picture pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a schematic flowchart of a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention;

[0044] Figure 2 It is a schematic diagram of the principle of a bloodstain recognition model provided by an embodiment of the present invention;

[0045] Figure 3 It is a scatter plot and a linear model diagram of the logarithmic transformation of the drainage volume with respect to the total PBP in a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention;

[0046] Figure 4 It is a scatter plot and a linear model diagram of the logarithmic transformation of the drainage volume with respect to the total PBP in the flushing stage in a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention;

[0047] Figure 5 It is a schematic diagram of the modules of a system for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention;

[0048] Figure 6 It is a schematic diagram of the internal structure of an electronic device for implementing a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention;

[0049] The realization, functional characteristics, and advantages of the objectives of the present invention will be further described in conjunction with the embodiments with reference to the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0051] Figure 1 FIG. is a schematic flow chart of a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention. This method can be executed by a system, and the system can be implemented by software and / or hardware.

[0052] A method for intraoperative bleeding recognition and measurement based on computer vision of the present invention is mainly applicable to the thoracoscopic surgery scenario. By using computer vision technology to analyze the thoracoscopic surgery video, variables based on the proportion of bloodstain pixels are extracted therefrom, and the variables based on the proportion of bloodstain pixels are correlated with clinical prognosis data. The variables based on the proportion of bloodstain pixels are significantly correlated with prognosis data such as drainage volume and whether the intubation time is prolonged, and can be used to predict the postoperative conditions of patients, providing a guiding role for formulating clinical prognosis plans.

[0053] Glossary: Pixel refers to a point in an image. An image is composed of a finite number of pixel points. RGB refers to a way of storing and describing an image in computer vision technology. A pixel point in an image can be represented by a three-dimensional vector, representing the red, green, and blue channel values respectively, and the value range is from 0 to 255. 0 represents black, and 255 represents pure red (or green or blue).

[0054] As Figure 1 shown, in this embodiment, a method for intraoperative bleeding recognition and measurement based on computer vision includes steps S110 to S140.

[0055] S110. Cut the acquired surgical video into a picture stream at a set frequency. Among them, the start time of the flushing stage is marked on the surgical video.

[0056] The surgical video can be but is not limited to an open surgical video, an endoscopic surgical video, and a robotic surgical video. A method for intraoperative bleeding recognition and measurement based on computer vision of the present invention is particularly applicable to the surgical video taken by a digital camera during VATS surgery. These surgical videos contain rich information about the anatomical structure of the patient's surgical site, the surgical process, the movement of instruments during the surgical process, etc. In this embodiment, a thoracoscopic surgery video with a frame rate of 50 FPS is used.

[0057] It should be noted that for the complete surgical video of each patient, a professional thoracic surgeon marks the start time of the flushing stage and cuts the surgical video into a picture stream at a frequency of once per second. During the actual surgical process, whether there is bleeding after the flushing stage has very important clinical significance for the surgical prognosis. Therefore, the start time of the flushing stage is used as an important node.

[0058] After cutting into a picture stream, for each picture in the picture stream, in step S120, a bloodstain recognition model is used to recognize bloodstain pixels, and the proportion of bloodstain pixels in the whole picture (proportion of blood pixels, PBP) is calculated.

[0059] S120. Determine the proportion value of bloodstain pixels in each picture in the picture stream through a pre-obtained bloodstain recognition model, and determine multiple variables based on the proportion of bloodstain pixels through the proportion value of bloodstain pixels in each picture in the picture stream.

[0060] Specifically, the variables based on the proportion of bloodstain pixels include the total sum of the proportion of bloodstain pixels and the total sum of the proportion of bloodstain pixels after the flushing stage; the total sum of the proportion of bloodstain pixels of the surgical video is obtained through the proportion value of bloodstain pixels in each picture in all picture streams, and the total sum of the proportion of bloodstain pixels after the flushing stage is obtained through the proportion value of bloodstain pixels in each picture in the picture stream after the start time of the flushing stage. Among them, the sum of all PBPs in the complete surgical video is SumPBP, which is used to depict the bleeding condition during the whole surgical process; the sum of PBPs after calculating the flushing stage is SumPBP - Flushing, which is used to depict the bleeding condition of the patient after the flushing stage.

[0061] The training method of the bloodstain recognition model includes: S1211. Divide the bloodstain picture data set into a training set and a validation set. The bloodstain picture data set includes pictures containing bloodstain pixel points and pictures containing non-bloodstain pixel points; S1212. Use the training set to establish a bloodstain recognition pre-training model. The bloodstain recognition pre-training model screens pixel points whose color channel values meet the set threshold as bloodstain pixel points according to a preset standard; the preset standard is that the red channel value is greater than the red channel set value a, the gray value is less than the gray set value b, and the ratio of the gray value to the red channel value is less than the ratio set value; among them, the ratio set value is obtained through the following formula: c + d×Frame Gray / R , a, b, c, d are all threshold parameters; (Frame Gray / R ) is the ratio of the total gray value to the total red channel value of the picture to which the pixel point belongs; that is, the standard for determining whether a pixel point is a bloodstain according to the preset standard, and the best criterion is selected by the recognition effect on the training data set. S1213. Use the validation set to perform threshold parameter tuning and verification on the bloodstain recognition pre-training model to obtain the best threshold parameter combination.

[0062] Taking the example of selecting 10,000 bloodstain pixel points and 10,000 non-bloodstain pixel points from the surgical videos of 10 patients as the training data set. For each pixel point, calculate its gray value (Pixel Gray ), red channel value (Pixel Red) The ratio of the grayscale value to the red channel value (Pixel Gray / R ), and the ratio of the total grayscale value to the total red channel value of the original image to which the pixel belongs (Frame Gray / R ). If a pixel satisfies the following preset criteria at the same time, it is determined as a bloodstain pixel: (1) The red channel value is greater than the threshold a; (2) The grayscale value is less than the threshold b; (3) The ratio of the grayscale value to the red channel value is less than the linear function of the ratio of the total grayscale value to the total red channel value of the original image to which the pixel belongs: c + d×Frame Gray / R ; Substituting 20,000 training pixels into the above criteria, the optimal threshold parameter combination, that is, the threshold combination with the highest training accuracy is: a = 14, b = 80, c = 0.24, d = 0.38.

[0063] To verify the recognition effect of the algorithm, 1,000 bloodstain pixels and 1,000 non-bloodstain pixels are selected as the verification set. Finally, the verification results are obtained as follows: accuracy 99.05%, specificity 98.9%, and sensitivity 99.2%. Compared with the bloodstain recognition method of Garcia-Martinez et al., it has higher accuracy and sensitivity, indicating that the bloodstain recognition model with dynamic and static combined thresholds in the method for intraoperative bleeding recognition and measurement based on computer vision of the present invention can identify bloodstain pixels more effectively than the method of Garcia-Martinez et al.

[0064] The method for obtaining the ratio of bloodstain pixels occupied by the bloodstain recognition model includes: S1221. Obtain the color channel information of each pixel of the image to be detected; wherein, the color channel information includes the grayscale value, red channel value, ratio of the grayscale value to the red channel value, and the ratio of the total grayscale value to the total red channel value of the image to which the pixel belongs. S1222. Screen the pixels whose color channel values meet the set thresholds as bloodstain pixels according to the preset criteria; the preset criteria are that the red channel value is greater than the red channel set value, the grayscale value is less than the grayscale set value, and the ratio of the grayscale value to the red channel value is less than the ratio set value; wherein, the ratio set value is obtained through the following formula: c + d×Frame Gray / R , where both c and d are threshold parameters; (Frame Gray / R ) is the ratio of the total grayscale value to the total red channel value of the image to which the pixel belongs. S1223. Take the ratio of the bloodstain pixels in all the pixels of the image to be detected as the ratio of bloodstain pixels occupied.

[0065] Figure 2 is a schematic diagram of the principle of the bloodstain recognition model provided by an embodiment of the present invention; as Figure 2 shown

[0066] Taking the complete surgical video i of the patient as an example, along with the starting time point of water flushing, and denoting the period after the start of water flushing as the water flushing stage. Since the start of the water flushing stage, the video is cut into a picture stream at a frequency of once per second, and the key frames F after the non-water flushing stage are obtained i1 ……F iti and the key frames F after the water flushing stage iti ……F iTi For each pixel point on each picture, through the blood stain pixel point recognizer (i.e., the blood stain recognition model trained with 10,000 blood stain pixel points and 10,000 non-blood stain pixel points), it is determined whether it is a blood stain pixel according to the above three criteria, and then the ratio of the blood stain pixels to the total number of pixel points in the picture is calculated; for the key frame F i1 Obtain the recognition image M i1 and use the blood stain pixel point recognizer to perform blood stain pixel recognition on the image M i1 and obtain the blood stain pixel occupancy ratio PBP i1 = 0.019; for the key frame F iti Obtain the recognition image M iti and use the blood stain pixel point recognizer to perform blood stain pixel recognition on the image M iti and obtain the blood stain pixel occupancy ratio PBP iti = 0.0560; for the key frame F iTi Obtain the recognition image M iTi and use the blood stain pixel point recognizer to perform blood stain pixel recognition on the image M iTi and obtain the blood stain pixel occupancy ratio PBP iTi = 0.164; for the images of the whole surgical process, the sum of the blood stain pixel occupancy ratios is obtained to get the total PBP, that is, SPBP i = 1433; for the images of the surgical video after the water flushing stage, the sum of the blood stain pixel occupancy ratios is obtained to get the PBP total in the water flushing stage, that is, SPBPF i = 50.

[0067] In summary, a method for intraoperative bleeding recognition and measurement based on computer vision adopts a method combining dynamic threshold and static threshold. Among them, static thresholds are adopted for the red value and the gray value, and a dynamic threshold is adopted for the gray value / red value, so as to adjust the judgment criteria for different environments. It has the characteristics of high recognition accuracy, small operation cost, and high calculation efficiency, and realizes the technical effect of accurately identifying blood stain pixels from images by analyzing and processing large-scale thoracoscopic surgical videos through computer vision technology.

[0068] S130. Based on the preset correlation analysis rules, perform correlation analysis on each variable based on the blood stain pixel ratio with the significant single factors of the patient's surgery and the prognosis indicators respectively; among them, the significant single factors of the patient's surgery include the patient's gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, the operation duration, and the estimated blood loss; the prognosis indicators include the postoperative drainage volume and whether the postoperative intubation time is prolonged.

[0069] Among them,

[0070] DV: Drainage volume (ml); EBV: Estimated blood loss by doctor (ml); PTID: Intubation duration;

[0071] Age: Age; SD: Total operation duration (min);

[0072] PBP: Blood stain pixel ratio; IPAL: Whether there is air leakage immediately after surgery;

[0073] SPBP: Total blood stain pixel ratio; SPBPF: Total blood stain pixel ratio in the flushing stage.

[0074] Perform correlation analysis on each variable based on the blood stain pixel ratio with the significant single factors of the patient's surgery and the prognosis indicators respectively, including: 1) Add each variable based on the blood stain pixel ratio, the significant single factors of the patient's surgery, and the postoperative drainage volume into the linear regression model, and perform backward regression to screen the variables based on the blood stain pixel ratio that are significantly correlated with the postoperative drainage volume. 2) Determine whether the postoperative intubation time is prolonged by whether the postoperative intubation time is greater than or equal to the intubation time threshold; record the prolonged intubation time as 1 and the non-prolonged intubation time as 0; perform logistic regression on each variable based on the blood stain pixel ratio with the significant single factors of the patient's surgery and whether the postoperative intubation time is prolonged, and perform backward regression to screen the variables based on the blood stain pixel ratio that are significantly correlated with whether the postoperative intubation time is prolonged.

[0075] In the specific implementation process, the results of performing correlation analysis on each variable based on the blood stain pixel ratio with the significant single factors of the patient's surgery and the prognosis indicators are shown in Table 1 and Table 2.

[0076] Table 1 Univariate analysis of PBP and postoperative drainage volume and multiple linear regression model of PBP and drainage volume (R 2 = 0.228)

[0077]

[0078] Table 2 Univariate analysis of PBP and whether the postoperative intubation time is prolonged and binary logistic regression model of PBP and drainage volume

[0079]

[0080] As can be seen from Table 1 and Table 2, univariate analysis was performed on the sum of PBP (SumPBP) during the operation, the sum of PBP after flushing (SumPBP-Flushing), and the postoperative drainage volume. The p-values of both SumPBP and SumPBP-Flushing were < 0.001. Multivariate analysis was performed on the significantly univariate indicators such as SumPBP, SumPBP-Flushing, patient gender, age, and whether there was immediate air leakage, and the p-value of SumPBP-Flushing was 0.022. Therefore, SumPBP-Flushing was significantly correlated with the postoperative drainage volume. Logistic regression was performed on the above variables and whether the intubation time was prolonged, and the p-value of SumPBP-Flushing was 0.017; therefore, SumPBP-Flushing was significantly correlated with whether the intubation time was prolonged.

[0081] S140. Establish a relationship model diagram between each variable based on the blood stain pixel ratio and each prognostic index according to the results of the association analysis; form a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel ratio and each prognostic index.

[0082] Figure 3 and Figure 4 is the relationship model diagram between each variable based on the blood stain pixel ratio and each prognostic index provided by an embodiment of the present invention; wherein, Figure 3 is a scatter plot and a linear model diagram of the logarithmic transformation of the drainage volume with respect to the sum of PBP of a method for intraoperative bleeding recognition and measurement based on computer vision;

[0083] Figure 4 is a scatter plot and a linear model diagram of the logarithmic transformation of the drainage volume with respect to the sum of PBP during the flushing stage of a method for intraoperative bleeding recognition and measurement based on computer vision provided by an embodiment of the present invention.

[0084] As Figure 3 shown, the horizontal axis is the sum of PBP, and the vertical axis is the logarithmic transformation of the drainage volume. A scatter plot and a linear model of the logarithmic transformation of the drainage volume with respect to the sum of PBP were obtained. That is, the logarithmic transformation was performed on the drainage volume to make it meet the normality assumption of linear regression. Then, a linear regression was performed on the transformed drainage volume using the variables based on the blood stain pixel ratio and the baseline data of the patient (such as age, gender, etc.) to obtain a relationship model diagram. Its coefficient of determination R 2 = 0.1039, p-value < 0.05.

[0085] As Figure 4As shown, the horizontal axis is the total PBP during the flushing stage, and the vertical axis is the logarithmically transformed drainage volume. A scatter plot and a linear model plot of the logarithmically transformed drainage volume against the total PBP during the flushing stage were obtained. The coefficient of determination R 2 = 0.0746, p-value < 0.05.

[0086] In the specific implementation process, the scatter plot and the linear model plot of the logarithmically transformed drainage volume against the total PBP provide guidance for prognosis.

[0087] In a specific embodiment, first, the surgical video of patient A was obtained, and the algorithm was used to analyze the patient's video. The total PBP value of the operation was 3534.13, and the PBP value during the flushing stage was 367.46. Then, combined with other significant single surgical factors such as the patient being male, 41 years old, having immediate postoperative air leakage, and the operation time being 203.88 minutes, based on the above data, the postoperative situation of the patient can be predicted by this algorithm: the patient will have a relatively large drainage volume, the postoperative catheterization time will be > 5 days, and the intraoperative blood loss will also be relatively large, indicating that the short-term postoperative recovery of the patient is worse than that of general patients. The actual drainage volume of the patient is 2275 ml, which is much higher than that of patients undergoing the same operation. The postoperative catheterization time of this patient is 8 days, which is consistent with the clinical prognosis plan generated by the computer vision-based intraoperative blood loss identification and measurement method of the present invention. The intraoperative blood loss of the patient is 100 ml, which is also at a relatively high level among patients undergoing the same operation. This embodiment proves the reliability of the computer vision-based intraoperative blood loss identification and measurement method of the present invention and the clinical prognosis plan generated thereby.

[0088] In another embodiment, after analyzing the surgical video of patient B, the total PBP value of the operation was 2553.84, and the PBP value during the flushing stage was 338.82. Combined with the patient being male, 64 years old, having immediate postoperative air leakage, and the operation duration being 219.02 minutes, the prognosis plan given by the computer vision-based intraoperative blood loss identification and measurement method of the present invention is similar to that of patient A. However, due to the prompting effect of the computer vision-based intraoperative blood loss identification and measurement method of the present invention, the attending physician of patient B gave the patient corresponding treatment measures; for example, keeping the drainage tube unobstructed, delaying and reducing the dosage of postoperative anticoagulant drugs, and fine-tuning the postoperative expectorant drugs. The short-term postoperative recovery process of the patient was significantly improved compared with that of patient A. Compared with patient A, the drainage volume of patient B decreased to 1300 ml, and the postoperative catheterization time was reduced to 5 days. Thus, it can be seen that by using the clinical prognosis plan generated by the computer vision-based intraoperative blood loss identification and measurement method of the present invention and giving the patient targeted and refined treatment, the technical effect of accelerating the patient's postoperative recovery and obtaining better treatment can be achieved.

[0089] A method for intraoperative bleeding recognition and measurement based on computer vision of the present invention performs intraoperative bleeding recognition and measurement through relationship model diagrams between various variables based on the proportion of blood stain pixels and various prognostic indicators, and can thus be used to predict the postoperative recovery time of patients, thereby helping to improve the utilization rate of hospital beds; it can also adjust the postoperative anticoagulation strategy according to the PBP value. If the PBP value of a patient is too high, anticoagulant drugs should not be given or the dosage of anticoagulant drugs should be reduced after surgery; it can also guide the selection of the suction force, diameter, and quantity of drainage tubes, etc.

[0090] In summary, a method for intraoperative bleeding recognition and measurement based on computer vision of the present invention analyzes a thoracoscopic surgery video by using computer vision technology, extracts variables based on the proportion of blood stain pixels therefrom, and performs correlation analysis between the variables based on the proportion of blood stain pixels and clinical prognostic data. The variables based on the proportion of blood stain pixels are significantly correlated with prognostic data such as the drainage volume and whether the intubation time is prolonged, and can be used to predict the postoperative conditions of patients, providing a guiding role for formulating clinical prognostic plans.

[0091] As Figure 5 shown, the present invention provides a system 500 for intraoperative bleeding recognition and measurement based on computer vision, and the present invention can be installed in an electronic device. According to the functions achieved, the system 500 for intraoperative bleeding recognition and measurement based on computer vision can include an acquisition unit 510, a blood stain recognition unit 520, a correlation analysis unit 530, and a clinical prognostic plan formation unit 540. The units of the present invention can also be referred to as modules, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0092] In this embodiment, the functions of each module / unit are as follows:

[0093] The acquisition unit 510 is used to cut the acquired surgical video into a picture stream at a set frequency;

[0094] The blood stain recognition unit 520 is used to determine the proportion value of blood stain pixels of each picture in the picture stream through a pre-acquired blood stain recognition model, and determine multiple variables based on the proportion of blood stain pixels through the proportion value of blood stain pixels of each picture in the picture stream;

[0095] The correlation analysis unit 530 is used to perform correlation analysis between each variable based on the proportion of blood stain pixels and the significant indicators of single factors of the patient's surgery and prognostic indicators respectively based on a preset correlation analysis rule; wherein, the significant indicators of single factors of the patient's surgery include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, the operation duration, and the estimated blood loss; the prognostic indicators include the postoperative drainage volume and whether the postoperative intubation time is prolonged;

[0096] A clinical prognosis plan formation unit 540 is configured to establish a relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis index based on the association analysis result; and form a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis index.

[0097] The intraoperative bleeding recognition and measurement system 500 based on computer vision of the present invention performs blood stain recognition by adopting a blood stain recognition model combining a dynamic threshold and a static threshold, obtains a relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis index through association analysis, and then forms a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis index, achieving the technical effects of high calculation efficiency and being applicable to the thoracoscopic surgery scenario with a long duration and high picture pixels.

[0098] As Figure 6 shown, the present invention provides an electronic device 6 for an intraoperative bleeding recognition and measurement method based on computer vision.

[0099] The electronic device 6 may include a processor 60, a memory 61, and a bus, and may further include a computer program stored in the memory 61 and executable on the processor 60, such as an intraoperative bleeding recognition and measurement program 62 based on computer vision. The memory 61 may further include both an internal storage unit of the intraoperative bleeding recognition and measurement system based on computer vision and an external storage device. The memory 61 may be used not only for storing installed application software and various types of data, such as the code of the intraoperative bleeding recognition and measurement program based on computer vision, but also for temporarily storing data that has been output or will be output.

[0100] Among them, the memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, optical disc, etc. The memory 61 can be an internal storage unit of the electronic device 6 in some embodiments, such as the mobile hard disk of the electronic device 6. The memory 61 can also be an external storage device of the electronic device 6 in some other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 6. Further, the memory 61 can also include both the internal storage unit and the external storage device of the electronic device 6. The memory 61 can be used not only to store application software installed in the electronic device 6 and various types of data, such as the code of the intraoperative bleeding recognition and measurement program based on computer vision, etc., but also to temporarily store the data that has been output or will be output.

[0101] The processor 60 can be composed of integrated circuits in some embodiments. For example, it can be composed of a single packaged integrated circuit, or can also be composed of multiple integrated circuits with the same or different functions packaged, including a combination of one or more Central Processing Units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc. The processor 60 is the control core (Control Unit) of the electronic device, connecting various components of the entire electronic device through various interfaces and lines, and by running or executing programs or modules stored in the memory 61 (such as the intraoperative bleeding recognition and measurement program based on computer vision, etc.), and calling the data stored in the memory 61, to execute various functions of the electronic device 6 and process data.

[0102] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is set to realize the connection and communication between the memory 61 and at least one processor 60, etc.

[0103] Figure 6 Only the electronic device with components is shown. Those skilled in the art can understand that Figure 6The structure shown does not constitute a limitation on the electronic device 6, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0104] For example, although not shown, the electronic device 6 may further include a power source (such as a battery) for powering each component. Preferably, the power source may be logically connected to the at least one processor 60 through a power management system, so as to implement functions such as charging management, discharging management, and power consumption management through the power management system. The power source may also include any components such as one or more DC or AC power sources, a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 6 may also include a variety of sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated here.

[0105] Furthermore, the electronic device 6 may further include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 6 and other electronic devices.

[0106] Optionally, the electronic device 6 may further include a user interface. The user interface may be a display, an input unit (such as a keyboard), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display the information processed in the electronic device 6 and to display a visual user interface.

[0107] It should be understood that the embodiments are only for illustration purposes and are not limited by this structure in the scope of the patent application.

[0108] The intraoperative bleeding recognition and measurement program 62 based on computer vision stored in the memory 61 of the electronic device 6 is a combination of multiple instructions. When running in the processor 60, it can achieve: cutting the acquired surgical video into a picture stream at a set frequency; determining the blood stain pixel ratio of each picture in the picture stream through a pre-acquired blood stain recognition model, and determining multiple variables based on the blood stain pixel ratio through the blood stain pixel ratio of each picture in the picture stream; based on a preset correlation analysis rule, respectively performing correlation analysis on each variable based on the blood stain pixel ratio with the significant indicators of the patient's surgical single factors and the prognosis indicators; wherein, the significant indicators of the patient's surgical single factors include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, surgical duration, and estimated blood loss; the prognosis indicators include postoperative drainage volume and whether the postoperative intubation time is prolonged; establishing a relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis indicator based on the correlation analysis result; forming a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel ratio and each prognosis indicator.

[0109] Specifically, the specific implementation method of the processor 60 for the above instructions can refer to Figure 1 the description of the relevant steps in the corresponding embodiment, which will not be elaborated here. It should be emphasized that to further ensure the privacy and security of the above intraoperative bleeding recognition and measurement program based on computer vision, the intraoperative bleeding recognition and measurement data based on computer vision are stored in the nodes of the blockchain where this server cluster is located.

[0110] Furthermore, if the modules / units integrated in the electronic device 6 are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory) that can carry the computer program code.

[0111] An embodiment of the present invention also provides a computer-readable storage medium, which can be non-volatile or volatile. The storage medium stores a computer program, and when the computer program is executed by a processor, it realizes: cutting the acquired surgical video into a picture stream at a set frequency; determining the blood stain pixel occupancy ratio of each picture in the picture stream through a pre-acquired blood stain recognition model, and determining a plurality of variables based on the blood stain pixel occupancy ratio through the blood stain pixel occupancy ratio of each picture in the picture stream; based on a preset association analysis rule, respectively performing association analysis on each variable based on the blood stain pixel occupancy ratio with the significant single-factor indicators of the patient's surgery and the prognosis indicators; wherein, the significant single-factor indicators of the patient's surgery include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, surgery duration, and estimated blood loss; the prognosis indicators include postoperative drainage volume and whether the postoperative intubation time is prolonged; establishing a relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognosis indicator based on the association analysis result; forming a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognosis indicator.

[0112] Specifically, the specific implementation method when the computer program is executed by the processor can refer to the description of the relevant steps in the method for intraoperative bleeding recognition and measurement based on computer vision in the embodiment, which will not be elaborated here.

[0113] In several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation.

[0114] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional module in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0116] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0117] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be encompassed within the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0118] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithms. Blockchain, in essence, is a decentralized database, a series of data blocks generated by using cryptographic methods. Each data block contains information on a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer, etc.

[0119] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems stated in the system claims can also be implemented by one unit or system through software or hardware. Words such as "second" are used to denote names and do not denote any particular order.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for intraoperative bleeding recognition and measurement based on computer vision, characterized in that, Including: Cutting the acquired surgical video into a picture stream at a set frequency; Determining the blood stain pixel occupancy ratio of each picture in the picture stream through a pre-acquired blood stain recognition model, and determining multiple variables based on the blood stain pixel occupancy ratio through the blood stain pixel occupancy ratio of each picture in the picture stream; Based on a preset correlation analysis rule, respectively performing correlation analysis on each variable based on the blood stain pixel occupancy ratio with the significant single-factor indicators of the patient's surgery and the prognosis indicators; wherein, the significant single-factor indicators of the patient's surgery include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, operation duration, and estimated blood loss; the prognosis indicators include the volume of postoperative drainage and whether the postoperative intubation time is prolonged; Based on the correlation analysis results, establishing a relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognosis indicator; Forming a clinical prognosis plan according to the relationship model diagram between each variable based on the blood stain pixel occupancy ratio and each prognosis indicator.

2. The method for intraoperative bleeding recognition and measurement based on computer vision according to claim 1, characterized in that The method for obtaining the blood stain pixel occupancy ratio through the blood stain recognition model includes: Obtaining the color channel information of each pixel point of the picture to be detected; wherein, the color channel information includes the gray value of each pixel point, the red channel value, the ratio of the gray value to the red channel value, and the ratio of the total gray value of the picture to which the pixel point belongs to the total red channel value; Screening the pixel points whose color channel values meet the set threshold according to a preset standard as blood stain pixel points; the preset standard is that the red channel value is greater than the set red channel value, the gray value is less than the set gray value, and the ratio of the gray value to the red channel value is less than the set ratio value; wherein, the set ratio value is obtained through the following formula: c + d×Frame Gray / R , where c and d are both threshold parameters; (Frame Gray / R ) is the ratio of the sum of the grayscale values to the sum of the red channel values of the picture to which the pixel belongs; Taking the ratio of the blood stain pixel points in all pixel points of the picture to be detected as the blood stain pixel occupancy ratio.

3. The method for intraoperative bleeding recognition and measurement based on computer vision according to claim 1, characterized in that The start time of the flushing stage is marked on the surgical video; The variables based on the blood stain pixel occupancy ratio include the total blood stain pixel occupancy ratio and the total blood stain pixel occupancy ratio after the flushing stage; the total blood stain pixel occupancy ratio of the surgical video is obtained through the blood stain pixel occupancy ratio of each picture in all picture streams, and the total blood stain pixel occupancy ratio after the flushing stage is obtained through the blood stain pixel occupancy ratio of each picture in the picture stream after the start time of the flushing stage.

4. The method for intraoperative bleeding recognition and measurement based on computer vision according to claim 1, characterized in that The training method of the blood stain recognition model includes: Dividing the blood stain picture data set into a training set and a validation set, and the blood stain picture data set includes pictures containing blood stain pixel points and pictures containing non-blood stain pixel points; Establish a pre-training model for bloodstain recognition using the training set. The pre-training model for bloodstain recognition screens out pixel points with color channel values meeting the set threshold as bloodstain pixel points according to a preset standard; the preset standard is that the red channel value is greater than the set value a of the red channel, the grayscale value is less than the set value b of the grayscale, and the ratio of the grayscale value to the red channel value is less than the set ratio value; among them, the set ratio value is obtained through the following formula: c + d × Frame Gray / R , where a, b, c, and d are all threshold parameters; (Frame Gray / R ) is the ratio of the sum of the grayscale values to the sum of the red channel values of the picture to which the pixel belongs; Use the validation set to perform threshold parameter tuning and verification on the pre-training model for bloodstain recognition to obtain the optimal threshold parameter combination.

5. The method for intraoperative bleeding recognition and measurement based on computer vision according to claim 1, wherein Perform correlation analysis on each variable based on the bloodstain pixel ratio with the significant indicators of the patient's surgery single factor and the prognosis indicators respectively, including Add each variable based on the bloodstain pixel ratio, the significant indicators of the patient's surgery single factor, and the postoperative drainage volume into a linear regression model, and perform backward regression to screen out the variables based on the bloodstain pixel ratio that are significantly correlated with the postoperative drainage volume; Determine whether the postoperative intubation time is prolonged by whether the postoperative intubation time is greater than or equal to the intubation time threshold; record the prolonged intubation time as 1 and the non-prolonged intubation time as 0; Perform logistic regression on each variable based on the bloodstain pixel ratio, the significant indicators of the patient's surgery single factor, and whether the postoperative intubation time is prolonged, and perform backward regression to screen out the variables based on the bloodstain pixel ratio that are significantly correlated with whether the postoperative intubation time is prolonged.

6. An intraoperative bleeding recognition and measurement system based on computer vision, characterized in that, Including: An acquisition unit for cutting the acquired surgical video into a picture stream at a set frequency; A bloodstain recognition unit for determining the bloodstain pixel ratio of each picture in the picture stream through a pre-obtained bloodstain recognition model, and determining multiple variables based on the bloodstain pixel ratio through the bloodstain pixel ratio of each picture in the picture stream; A correlation analysis unit for performing correlation analysis on each variable based on the bloodstain pixel ratio with the significant indicators of the patient's surgery single factor and the prognosis indicators respectively based on a preset correlation analysis rule; wherein, the significant indicators of the patient's surgery single factor include patient gender, age, BMI, whether smoking, whether there is air leakage immediately after surgery, operation duration, and estimated blood loss; the prognosis indicators include postoperative drainage volume and whether the postoperative intubation time is prolonged; A clinical prognosis plan forming unit for establishing a relationship model diagram between each variable based on the bloodstain pixel ratio and each prognosis indicator based on the correlation analysis result, and forming a clinical prognosis plan according to the relationship model diagram between each variable based on the bloodstain pixel ratio and each prognosis indicator.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps in the method for intraoperative bleeding recognition and measurement based on computer vision according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements a method for intraoperative bleeding recognition and measurement based on computer vision as described in any one of claims 1 to 5.

Images