Intraoperative blood dilution control method and system based on image recognition

Abstract

The present invention discloses a method and system for controlling blood dilution during surgery based on image recognition, which relates to the field of intelligent control technology for blood transfusion. The present invention can accurately obtain blood bag information and blood color classification results by real-time acquisition of surgical images (blood bag position images, mixed bag position images) and preoperative information of patients, combined with regional detection classification models and blood color classification models, and accurately control the dilution speed through intelligent valves according to the specific conditions of the patients and the actual state of the blood, thereby reducing manual intervention and operation time and improving the efficiency of blood management during surgery. It avoids the risk of blood being too thick or too diluted, effectively solves the problem of slow infusion speed or / and blockage in traditional blood transfusion processes, helps to efficiently maintain the stability of patients' vital signs during surgery, reduces surgical risks and related complications, ensures that patients can receive the required amount of blood in a timely manner, and ensures the smooth progress of the operation.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent blood transfusion control, and in particular to an intraoperative blood dilution control method and system based on image recognition. Background Art

[0002] During surgery, bleeding is inevitable, and intraoperative blood transfusion or even rescue blood transfusion is often required due to excessive blood loss. Accurate blood transfusion operation is crucial to the safety of patients. However, due to the differences in blood components in different blood bags, when the blood is too thick, it is easy to block the infusion tube, resulting in the inability to deliver blood to the patient in time, thus delaying treatment. In addition, increased blood viscosity may also cause blood vessel blockage, increase the risk of thrombosis, affect blood circulation, cause tissue hypoxia and malnutrition, and may even cause organ failure. If the blood is too diluted, the patient will not be able to get the required amount of blood in time, delaying treatment. Hemodilution will lead to a decrease in the hemoglobin concentration in the blood (different degrees of anemia), thereby weakening the body's oxygen-carrying capacity, causing tissue hypoxia, and affecting the patient's physical function.

[0003] At present, the commonly used methods for controlling blood dilution in clinical practice have some shortcomings. For example, although acute isovolumetric hemodilution (ANH) is simple to operate, low in cost, and can effectively reduce the risk of allogeneic blood transfusion, there is a risk of blood contamination and blood coagulation during the operation. At the same time, ANH is prone to cause volume overload. During bloodletting or surgical bleeding, a large amount of crystalloid or colloid fluid has been transfused to maintain volume balance. If a large amount of stored autologous blood is transfused, it may cause volume overload and even cause serious complications such as pulmonary edema and heart failure. In addition, ANH may cause coagulation disorders and increase the risk of intraoperative and postoperative bleeding. The current blood dilution control method lacks real-time monitoring means, and it is difficult to accurately grasp the degree of blood dilution and the specific situation of the patient, which may lead to excessive or insufficient dilution, affecting the treatment effect and safety of the patient. Summary of the invention

[0004] The object of the present invention is to provide an intraoperative blood dilution control method and system based on image recognition to improve the above technical problems.

[0005] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0006] A method for controlling blood dilution during surgery based on image recognition, comprising:

[0007] S1. Real-time acquisition of blood bag position images and patient preoperative information; the patient preoperative information is the preoperative blood test report;

[0008] S2, constructing a region detection classification model and a blood color classification model respectively;

[0009] S3, inputting the blood bag position image into the regional detection classification model to obtain corresponding blood bag information data; the blood bag information data includes first blood text information and second blood text information;

[0010] S4, selecting a blood bag based on the blood bag information data and the patient's preoperative information;

[0011] S5, start the intelligent valve to dilute the blood;

[0012] S6. Real-time acquisition of the patient's vital sign parameters and the mixing bag position image at the current moment; the patient's vital sign parameters include blood pressure, temperature, heart rate and intraoperative blood gas analysis results;

[0013] S7, inputting the mixed bag position image into the region detection classification model and the blood color classification model in sequence to obtain corresponding blood color classification results; the blood color classification results include moderate blood color, lighter blood color and darker blood color;

[0014] S8. Based on the patient's vital signs parameters and blood color classification results, the dilution rate is controlled by an intelligent valve.

[0015] Furthermore, the region detection classification model includes a region detection module, a classification module, a text recognition module, and a QR code scanning module; the output of the region detection module is input to the classification module; the output of the classification module is respectively input to the text recognition module, the QR code scanning module, and the blood color classification model;

[0016] The region detection module adopts the improved YOLOv3 model; the improved YOLOv3 model includes a Backbone module, a Neck module and a Head module connected in series; the Backbone module includes a first CBL layer, a first MobileNetV2 module, a second MobileNetV2 module, a C-Attention module and a third MobileNetV2 module connected in series;

[0017] The Neck module includes a first neck submodule, a second neck submodule and a second CBL layer in parallel; the first neck submodule and the second neck submodule have the same structure, both including a third CBL layer, an Upsample layer and a Concat layer connected in series;

[0018] The Head module includes a first head submodule, a second head submodule and a third head submodule in parallel; the structures of the first head submodule to the third head submodule are the same, and all include a fourth CBL layer, a Conv layer and a Head layer connected in series in sequence; the structures of the first CBL to the fourth CBL layers are the same; the structures of the first MobileNetV2 module to the third MobileNetV2 module are the same, and all adopt the MobileNetV2 model;

[0019] The C-Attention module includes a parallel HSV module and a Grayscale Contrast layer; the HSV module includes a serially connected HSV layer and an Attention layer;

[0020] The classification module uses KNN network or SVM classifier;

[0021] The text recognition module includes a first preprocessing submodule and an OCR recognition submodule which are connected in series.

[0022] Furthermore, the training process of the region detection classification model includes the following steps:

[0023] S3-1, obtain blood bag position training image data, mixed bag position training image data and their labels, and pre-process the blood bag position training image data and mixed bag position training image data; the label is the QR code, text and mixed bag corresponding to the blood bag;

[0024] S3-2, inputting the preprocessed blood bag position training image data into the first CBL layer, and outputting a blood bag shallow layer feature map;

[0025] S3-3, inputting the shallow feature map of the blood bag into the first MobileNetV2 module, and outputting the first deep feature map of the blood bag;

[0026] S3-4, inputting the first blood bag deep feature map into the second MobileNetV2 module, and outputting the second blood bag deep feature map;

[0027] S3-5, input the second blood bag deep feature map into the C-Attention module, and output the blood bag key color deep feature map;

[0028] S3-6, inputting the blood bag key color deep feature map into the third MobileNetV2 module, and outputting the third blood bag deep feature map;

[0029] S3-7, inputting the third blood bag deep layer feature map into the second CBL layer, and outputting the first blood bag fusion feature map;

[0030] S3-8, inputting the first blood bag fusion feature map and the blood bag key color deep feature map into the first neck submodule, and outputting a second blood bag fusion feature map;

[0031] S3-9, inputting the second blood bag fusion feature map and the first blood bag deep feature map into the second neck submodule, and outputting a third blood bag fusion feature map;

[0032] S3-10, input the first blood bag fusion feature map, the second blood bag fusion feature map and the third blood bag fusion feature map into the corresponding first head submodule, the second head submodule and the third head submodule respectively, and output the first detection result, the second detection result and the third detection result; the first detection result to the third detection result respectively include the corresponding detection box and confidence score;

[0033] S3-11, judging whether the confidence corresponding to each detection frame is greater than the confidence threshold; if so, retaining the corresponding detection frame to obtain the filtered detection frame; otherwise, removing the corresponding detection frame;

[0034] S3-12, using the same method from S3-2 to S3-11 to process the pre-processed mixed bag position training image data to obtain a screened detection frame;

[0035] S3-13, inputting the screened detection frame and the pre-processed blood bag position training image data / pre-processed mixed bag position training image data into the classification module to obtain a classification result; the classification result includes text image data, two-dimensional code image data, and mixed bag image data;

[0036] S3-14, inputting the text image data into a text recognition module, and outputting the first blood training text information;

[0037] S3-15, inputting the two-dimensional code image data into a two-dimensional code scanning module, and outputting the second blood training text information;

[0038] S3-16. Calculate the final loss function of the region detection classification model;

[0039] S3-17. Based on the final loss function, the weight parameters of the region detection classification model are adjusted to complete the training of the region detection classification model.

[0040] Furthermore, in S3-12, only the Grayscale Contrast layer in the C-Attention module is used to process the output data of the second MobileNetV2 module corresponding to the preprocessed blood training data to obtain the blood initial key color deep feature map; the blood initial key color deep feature map and the output data of the second MobileNetV2 module corresponding to the preprocessed blood training data are added element by element to obtain the blood key color deep feature map.

[0041] Furthermore, S7 includes:

[0042] S7-1, inputting the mixed bag position image into the region detection classification model, and outputting the classification result, that is, obtaining the mixed bag image;

[0043] S7-2. Input the mixed bag image into the blood color classification model to obtain the blood color classification result.

[0044] Furthermore, S7-2 includes:

[0045] S7-2-1, performing color space conversion on the mixed bag image to obtain a mixed bag HSV image;

[0046] S7-2-2, randomly sampling the pixel points of the mixed bag HSV image to obtain sampled pixel point data;

[0047] S7-2-3, obtaining the red channel pixel value of the sampled pixel point data;

[0048] S7-2-4, based on the red channel pixel value, calculating the mean of each sampled pixel point and the difference between each sampled pixel point and the surrounding sampled pixel points;

[0049] S7-2-5, based on each difference, obtaining a corresponding degree of mixing; the degree of mixing is fully mixed, relatively fully mixed, and insufficiently mixed;

[0050] S7-2-6. Determine whether the mean value is within the color threshold range ; If yes, the blood color classification result is determined to be moderate; otherwise, the process goes to S7-2-7; , All are expressed as constants;

[0051] S7-2-7, determine whether the mean is less than ; If yes, the blood color classification result is determined to be lighter in color; otherwise, the blood color classification result is determined to be darker in color.

[0052] Further, the smart valve includes a first smart valve, a second smart valve and a third smart valve;

[0053] S8 includes:

[0054] S8-1, obtaining the hemoglobin concentration of the mixing bag at the current moment; based on the hemoglobin concentration, determining the concentration level of the mixing bag at the current moment; the concentration level includes low concentration, moderate concentration and high concentration;

[0055] S8-2, judging whether the mixing degree is insufficient; if so, proceeding to S8-6; otherwise, proceeding to S8-3;

[0056] S8-3, judging that the blood color classification result is moderate blood color and moderate concentration; if so, maintaining the current dilution rate and proceeding to S8-7; otherwise, proceeding to S8-4;

[0057] S8-4, determine whether the blood color classification result and the concentration level meet the first judgment condition; if so, proceed to S8-6; otherwise, proceed to S8-5; the first judgment condition is: the blood color classification result and the concentration level meet any one of [lighter blood color, higher concentration] and [darker blood color, lighter concentration];

[0058] S8-5. Determine whether the blood color classification result and the concentration level meet the second judgment condition. If so, the dilution speed is accelerated by the first intelligent valve. Otherwise, the dilution speed is reduced by the first intelligent valve. Enter S8-7. The second judgment condition is: the blood color classification result and the concentration level meet any one of [darker blood color, higher concentration], [darker blood color, moderate concentration], and [moderate blood color, lower concentration].

[0059] S8-6, vibrate the blood transfusion bag, and return to S8-1;

[0060] S8-7, determine whether the patient's vital sign parameters are within the normal range; if so, maintain the current infusion rate; otherwise, enter S8-8;

[0061] S8-8. Determine whether the patient's vital sign parameters meet the third judgment condition and whether the intraoperative blood gas analysis results meet the fourth judgment condition; then speed up the current infusion speed through the third intelligent valve; otherwise, reduce the current infusion speed; the third judgment condition is: the patient's vital sign parameters meet any two of the following: the blood pressure at the current moment is less than the blood pressure at the previous moment, the temperature at the current moment is less than the temperature at the previous moment, and the heart rate at the current moment is less than the heart rate at the previous moment.

[0062] Provided is an intraoperative blood dilution control system based on image recognition, which includes an acquisition module, a region detection and classification module, a blood color classification module, a blood bag selection module, and an intelligent control module; wherein:

[0063] An acquisition module is used to acquire the blood bag position image in real time and obtain the patient's preoperative information, the patient's vital sign parameters at the current moment, and the mixing bag position image;

[0064] The regional detection and classification module is used to input the blood bag position image and the mixed bag position image into the regional detection and classification model respectively to obtain the corresponding blood bag information data and the detection frame corresponding to the mixed bag position image;

[0065] A blood color classification module is used to input the detection frame corresponding to the mixed bag position image into the blood color classification model to obtain the corresponding blood color classification result;

[0066] A blood bag selection module is used to select a blood bag based on the blood bag information data and the patient's preoperative information;

[0067] Intraoperative blood dilution device, used to dilute the blood;

[0068] The intelligent control module is used to control the intraoperative blood dilution device and adjust the dilution speed based on the patient's vital sign parameters and blood color classification results.

[0069] Furthermore, the intraoperative blood dilution device includes a dilution liquid bag, a blood transfusion bag, a first intelligent valve, a second intelligent valve, a mixing bag and a third intelligent valve; the mixing bag is an inverted cone-shaped liquid bag, and vacuum cavities are arranged on both sides of the bottom; a spiral sealing cover is arranged on the outside of the mixing bag corresponding to the vacuum cavity; a photoelectric sensor and a micro vibration motor are arranged in the vacuum cavity; wherein:

[0070] A dilution liquid bag is used to store dilution liquid; a first intelligent valve is used to control the dilution speed of the dilution liquid in the dilution liquid bag; a blood transfusion bag is used to store blood; a second intelligent valve is used to open the corresponding blood transfusion channel and control the corresponding speed; a mixing bag is used to receive the dilution liquid in the dilution liquid bag and the blood in the blood transfusion bag; a third intelligent valve is used to control the corresponding infusion speed; a micro vibration motor is used to provide vibration to achieve the purpose of mixing the dilution liquid and blood; a photoelectric sensor is used to measure the liquid concentration of the mixing bag, that is, the hemoglobin concentration.

[0071] The beneficial effects of the present invention are:

[0072] 1. This method can accurately obtain blood bag information and blood color classification results by real-time acquisition of surgical images (blood bag position images, mixed bag position images) and preoperative information of patients, combined with regional detection classification models and blood color classification models, and accurately control the dilution speed through intelligent valves according to the specific conditions of the patients and the actual state of the blood, thereby reducing manual intervention and operation time, and improving the efficiency of blood management during surgery; avoiding the risk of blood being too thick or too diluted, effectively solving the problems of slow infusion speed and / or blockage in traditional blood transfusion processes, and timely delivering blood to the patient's body, which helps to efficiently maintain the stability of the patient's vital signs during surgery, reduce surgical risks and related complications, and ensure that the patient can receive the required amount of blood in a timely manner, thereby ensuring the smooth progress of the operation;

[0073] 2. The device is equipped with three intelligent valves, which can effectively control the output speed of the dilution liquid bag, blood bag and mixing bag; the vacuum chamber has built-in photoelectric sensors and micro-vibration motors, which can effectively achieve the mixing of dilution liquid and blood, and monitor the liquid concentration in real time, providing hardware support for the precise control of the dilution process, and effectively solving the problem of insufficient mixing in the traditional blood transfusion process, ensuring that the blood can be delivered to the patient in a timely and smooth manner, and improving the safety and reliability of the operation;

[0074] 3. This system can collect the patient's vital signs and parameters and the mixing bag position image in real time, and promptly obtain the patient's blood pressure, temperature, heart rate and other information as well as the color and mixing degree of the blood in the mixing bag, and then timely control the dilution rate, thereby realizing dynamic optimization control of the blood dilution process and improving the safety and effectiveness of the operation; based on the blood bag information data and the patient's preoperative information, the system automatically selects the appropriate blood bag for blood transfusion, effectively reducing the errors of manual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 is a flow chart of a method in an embodiment of the present invention;

[0077] Figure 2 A specific structural diagram of a region detection classification model in an embodiment of the present invention;

[0078] Figure 3 A specific structural diagram of a region detection module in an embodiment of the present invention;

[0079] Figure 4 is a specific structural diagram of the C-Attention module in an embodiment of the present invention;

[0080] Figure 5 This is a specific structural diagram of the intraoperative blood dilution control system in an embodiment of the present invention;

[0081] Figure 6 1 is a specific structural diagram of the intraoperative blood dilution device in an embodiment of the present invention; wherein: 1. dilution liquid bag; 2. blood transfusion bag; 3. first intelligent valve; 4. second intelligent valve; 5. mixing bag; 6. vacuum chamber; 7. third intelligent valve; 61. micro vibration motor; 62. spiral sealing cover; 63. photoelectric sensor; 64. temperature sensor. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0083] See also Figure 1 This embodiment provides a method for controlling blood dilution during surgery based on image recognition, which includes:

[0084] S1. Real-time collection of blood bag position images and acquisition of preoperative information of the patient; the preoperative information of the patient is a preoperative blood test report, which includes the number of red blood cells, hemoglobin concentration, etc.;

[0085] S2, constructing a region detection classification model and a blood color classification model respectively;

[0086] like Figure 2 As shown, the regional detection and classification model includes a regional detection module, a classification module, a text recognition module, and a QR code scanning module; the output of the regional detection module is input to the classification module; the output of the classification module is respectively input to the text recognition module, the QR code scanning module, and the blood color classification model;

[0087] like Figure 3As shown, the region detection module adopts the improved YOLOv3 model; the improved YOLOv3 model includes a Backbone module, a Neck module and a Head module connected in series in sequence; the Backbone module includes a first CBL layer, a first MobileNetV2 module, a second MobileNetV2 module, a C-Attention module and a third MobileNetV2 module connected in series in sequence;

[0088] The Neck module includes a first neck submodule, a second neck submodule and a second CBL layer in parallel; the first neck submodule and the second neck submodule have the same structure, both including a third CBL layer, an Upsample layer and a Concat layer connected in series;

[0089] The Head module includes a first head submodule, a second head submodule and a third head submodule in parallel; the structures of the first head submodule to the third head submodule are the same, and all include a fourth CBL layer, a Conv layer and a Head layer connected in series in sequence; the structures of the first CBL to the fourth CBL layers are the same; the structures of the first MobileNetV2 module to the third MobileNetV2 module are the same, and all adopt the MobileNetV2 model;

[0090] like Figure 4 As shown, the C-Attention module includes a parallel HSV module and a Grayscale Contrast layer; the HSV module includes a serially connected HSV layer and an Attention layer;

[0091] The classification module uses KNN network or SVM classifier;

[0092] The text recognition module includes a first preprocessing submodule and an OCR recognition submodule which are connected in series.

[0093] S3, inputting the blood bag position image into the regional detection classification model to obtain corresponding blood bag information data; the blood bag information data includes first blood text information and second blood text information;

[0094] The training process of the region detection classification model includes the following steps:

[0095] S3-1, obtain blood bag position training image data, mixed bag position training image data and their labels, and pre-process the blood bag position training image data and mixed bag position training image data; the label is the QR code, text and mixed bag corresponding to the blood bag;

[0096] S3-2, input the pre-processed blood bag position training image data into the first CBL layer, and output the shallow feature map of the blood bag , the corresponding formula is:

[0097] ;

[0098] in, represents the preprocessed blood bag position training image data, , represents the convolution kernel parameters, represents the LeakyReLU activation function, Represents the batch normalization function.

[0099] S3-3. Input the shallow feature map of the blood bag into the first MobileNetV2 module, and output the deep feature map of the first blood bag. , the corresponding formula is:

[0100] ;

[0101] in, Represents the MobileNetV2 lightweight network.

[0102] S3-4, the first blood bag deep characteristic map Input to the second MobileNetV2 module, and output the second blood bag deep feature map , the corresponding formula is the same as that of S3-3;

[0103] S3-5, the second blood bag deep characteristic map Input to the C-Attention module, and output the deep feature map of the key colors of the blood bag ;

[0104] According to the color of blood, QR code and text, the focus of target recognition for the blood bag position image is on black, white and red. Therefore, a C-Attention module is designed to extract the image features corresponding to the three colors of black, white and red. The specific process corresponding to S3-5 is as follows:

[0105] The second blood bag deep feature map is input into the HSV layer, the corresponding red area is extracted through the HSV mask, and then input into the Attention layer to generate the corresponding attention weight map and process the second blood bag deep feature map to obtain the first initial key color deep feature map of the blood bag; the second blood bag deep feature map is input into the Grayscale Contrast layer, and the Grayscale Contrast Module is used to grayscale the feature map, enhance the contrast, and extract the edge features to obtain the edge response map data; the edge response map data is normalized by the attention weight map to generate the corresponding attention weight; the attention weight and the second blood bag deep feature map are element-by-element multiplied to obtain the second initial key color deep feature map of the blood bag; the first initial key color deep feature map of the blood bag and the second initial key color deep feature map of the blood bag are element-by-element added to obtain the key color deep feature map of the blood bag.

[0106] Therefore, the formula corresponding to S3-5 is:

[0107] ;

[0108] ;

[0109] ;

[0110] in, represents the attention mechanism, represents the convolution function, Represents the HSV mask operation, Represents the output data of the HSV layer, , They represent element-by-element addition and element-by-element multiplication, respectively. Represents the output data of the Grayscale Contrast layer. Represents grayscale contrast operation.

[0111] The designed C-Attention module effectively improves the speed of feature extraction, reduces model calculation time, and improves the running speed of the method of the present invention.

[0112] S3-6, the key color deep feature map of the blood bag Input to the third MobileNetV2 module, and output the third blood bag deep feature map , the formula is the same as that of S3-3;

[0113] S3-7, input the third blood bag deep characteristic map To the second CBL layer, the output is the first blood bag fusion feature map , the corresponding formula is the same as that of S3-2;

[0114] S3-8, merge the first blood bag with the characteristic graph , Blood bag key color deep feature map Input to the first neck submodule, and output the second blood bag fusion feature map , the corresponding formula is:

[0115] ;

[0116] in, represents the concatenation function, Represents an upsampling operation.

[0117] S3-9, merge the second blood bag with the characteristic graph 、Deep characteristic diagram of the first blood bag Input to the second neck submodule, and output the third blood bag fusion feature map , the corresponding formula is the same as that of S3-8;

[0118] S3-10, merge the first blood bag with the characteristic graph , Second blood bag fusion feature map And the third blood bag fusion feature map The first detection result, the second detection result and the third detection result are respectively input into the corresponding first head submodule, the second head submodule and the third head submodule, and the first detection result to the third detection result respectively include the corresponding detection box and the confidence score;

[0119] S3-11, judging whether the confidence corresponding to each detection frame is greater than the confidence threshold; if so, retaining the corresponding detection frame to obtain the filtered detection frame; otherwise, removing the corresponding detection frame;

[0120] S3-12, using the same method from S3-2 to S3-11 to process the pre-processed mixed bag position training image data to obtain a screened detection frame;

[0121] In S3-12, since blood color is used as a key color feature when performing target recognition on the mixed bag position image, only the Grayscale Contrast layer in the C-Attention module is used to process the output data of the second MobileNetV2 module corresponding to the preprocessed blood training data to obtain a blood initial key color deep feature map; the blood initial key color deep feature map and the output data of the second MobileNetV2 module corresponding to the preprocessed blood training data are added element by element to obtain a blood key color deep feature map.

[0122] S3-13, inputting the screened detection frame and the pre-processed blood bag position training image data / pre-processed mixed bag position training image data into the classification module to obtain a classification result; the classification result includes text image data, two-dimensional code image data, and mixed bag image data;

[0123] S3-14, inputting the text image data into a text recognition module, and outputting first blood training text information, which includes the blood bag validity period and blood bag capacity;

[0124] S3-15, inputting the two-dimensional code image data into the two-dimensional code scanning module, and outputting the second blood training text information, which includes blood components, namely, the number of red blood cells and the hemoglobin concentration;

[0125] S3-16. Calculate the final loss function of the region detection classification model; the final loss function includes a first loss function corresponding to the region detection module and a second loss function corresponding to the classification module; the first loss function adopts the CIOU loss function, and the second loss function adopts the multi-classification cross entropy loss function.

[0126] S3-17. Based on the final loss function, the weight parameters of the region detection classification model are adjusted to complete the training of the region detection classification model.

[0127] After S3-13, there is also a step of coding each blood bag separately.

[0128] S4. Based on the blood bag information data and the patient's preoperative information, select the blood bag that is closest to the blood components and has the longest validity period from multiple blood bags; the blood bag that is closest to the blood components (hemoglobin concentration or red blood cell count) is the first choice target, and the blood bag with the longest validity period is the second choice target.

[0129] The code corresponding to the blood bag selected by S4 is uploaded to the cloud, and transmitted to the first intelligent valve 3 corresponding to the dilution liquid bag 1 and the second intelligent valve 4 corresponding to the blood bag to open the valves corresponding to the blood bag to dilute the blood, thereby solving the problem that the blood in the blood transfusion bag 2 is too thick and easily blocks the infusion tube during the operation, and preventing the problem that the medical staff cannot take into account the impact of the blood concentration on the patient during the operation;

[0130] S6. Real-time collection of the patient's vital signs parameters and mixing bag position images at the current moment; the patient's vital signs parameters include blood pressure, temperature, heart rate and intraoperative blood gas analysis results; intraoperative blood gas analysis results are important examinations to evaluate the patient's oxygenation status, acid-base balance and electrolyte status, which include common indicators such as pH value, arterial oxygen partial pressure, arterial carbon dioxide partial pressure and bicarbonate.

[0131] S7, inputting the mixed bag position image into the region detection classification model and the blood color classification model in sequence to obtain corresponding blood color classification results; the blood color classification results include moderate blood color, lighter blood color and darker blood color;

[0132] S7 includes:

[0133] S7-1, inputting the mixed bag position image into the region detection classification model, and outputting the classification result, that is, obtaining the mixed bag image;

[0134] S7-2. Input the mixed bag image into the blood color classification model to obtain the blood color classification result.

[0135] S7-2 includes:

[0136] S7-2-1, performing color space conversion on the mixed bag image to obtain a mixed bag HSV image;

[0137] S7-2-2, randomly sampling the pixel points of the mixed bag HSV image to obtain sampled pixel point data;

[0138] S7-2-3, obtaining the red channel pixel value of the sampled pixel point data;

[0139] S7-2-4, based on the red channel pixel value, calculating the mean of each sampled pixel point and the difference between each sampled pixel point and the surrounding sampled pixel points;

[0140] S7-2-5. Based on each difference, the corresponding mixing degree is obtained. The corresponding process is:

[0141] If 50% or more of the differences between each sampled pixel and the surrounding sampled pixels exceed the preset red threshold, the sampled pixel is considered to be a color-uneven pixel. If the number of color-uneven pixels exceeds 75% of the number of sampled pixels, the mixing degree is judged to be insufficient; if the number of color-uneven pixels is 50% of the number of sampled pixels, the mixing degree is judged to be relatively sufficient; in other cases, the mixing degree is judged to be sufficient.

[0142] The degree of mixing is fully mixed, relatively fully mixed, and incompletely mixed;

[0143] S7-2-6. Determine whether the mean value is within the color threshold range ; If yes, the blood color classification result is determined to be moderate; otherwise, the process goes to S7-2-7; , All are expressed as constants;

[0144] S7-2-7, determine whether the mean is less than ; If yes, the blood color classification result is determined to be lighter in color; otherwise, the blood color classification result is determined to be darker in color.

[0145] S8. Based on the patient's vital signs parameters and blood color classification results, the dilution rate is controlled by an intelligent valve.

[0146] The smart valve includes a first smart valve 3, a second smart valve 4 and a third smart valve 7;

[0147] S8 includes:

[0148] S8-1. Obtain the hemoglobin concentration of the mixing bag 5 at the current moment; based on the hemoglobin concentration, determine the concentration level of the mixing bag 5 at the current moment according to the set concentration threshold range; the concentration levels include low concentration, moderate concentration and high concentration; when the hemoglobin concentration is within the concentration threshold range, the concentration level is determined to be moderate concentration; when the hemoglobin concentration is lower than the lower value in the concentration threshold range, the concentration level is determined to be low concentration; in other cases, the concentration level is determined to be high concentration.

[0149] S8-2, judging whether the mixing degree is insufficient; if so, proceeding to S8-6; otherwise, proceeding to S8-3;

[0150] S8-3, judging that the blood color classification result is moderate blood color and moderate concentration; if so, maintaining the current dilution rate and proceeding to S8-7; otherwise, proceeding to S8-4;

[0151] S8-4, determine whether the blood color classification result and the concentration level meet the first judgment condition; if so, proceed to S8-6; otherwise, proceed to S8-5; the first judgment condition is: the blood color classification result and the concentration level meet any one of [lighter blood color, higher concentration] and [darker blood color, lighter concentration];

[0152] S8-5. Determine whether the blood color classification result and the concentration level meet the second judgment condition. If so, the dilution speed is accelerated by the first intelligent valve 3. Otherwise, the dilution speed is reduced by the first intelligent valve 3. Enter S8-7. The second judgment condition is: the blood color classification result and the concentration level meet any one of [darker blood color, higher concentration], [darker blood color, moderate concentration], and [moderate blood color, lower concentration]. When the blood color classification result and the concentration level meet [darker blood color, higher concentration], the current dilution speed is increased. ; When the blood color classification result and concentration level meet [moderate blood color, low concentration], the current dilution rate will be increased ; When the blood color classification result and concentration level meet [blood color is darker, concentration is moderate], the current dilution rate will be increased ; When the blood color classification result and concentration level meet [lighter blood color, lower concentration], the current dilution rate will be reduced ; When the blood color classification result and concentration level meet [moderate blood color, high concentration], the current dilution rate will be reduced ; When the blood color classification result and concentration level meet [light blood color, moderate concentration], the current dilution rate will be reduced ;in, , , are constants, and .

[0153] S8-6, sending a start command to the micro vibration motor 61 and starting it, vibrating the blood transfusion bag 2 to mix the blood in the mixing bag 5 evenly, solving the problem of uneven mixing of the blood in the mixing bag 5, and returning to S8-1;

[0154] S8-7, determine whether the patient's vital sign parameters are within the normal range; if so, maintain the current infusion rate; otherwise, enter S8-8;

[0155] S8-8. Determine whether the patient's vital sign parameters meet the third judgment condition and whether the intraoperative blood gas analysis results meet the fourth judgment condition; then speed up the current infusion speed through the third intelligent valve 7; otherwise, reduce the current infusion speed; the third judgment condition is: the patient's vital sign parameters meet any two of the following: the blood pressure at the current moment is less than the blood pressure at the previous moment, the temperature at the current moment is less than the temperature at the previous moment, and the heart rate at the current moment is less than the heart rate at the previous moment; the fourth judgment condition is: 85% or more of the commonly used indicators in the intraoperative blood gas analysis results meet the normal range corresponding to the medical definition.

[0156] Intelligently regulate the release rate of the dilution liquid, and adjust the dilution rate in time according to the real-time situation (blood color, patient's vital signs) to make the concentration of the mixed blood moderate, so as not to block the infusion tube or dilute excessively, to fit the patient's blood concentration, reduce the risk of thrombosis in the patient, and prevent the patient's organ failure.

[0157] The method further comprises S9, wherein the process is as follows:

[0158] The corresponding blood volume is calculated according to the transfusion time and the blood flow rate of the blood bag; the first blood text information (blood bag capacity) obtained in S3 is compared with the blood volume, and when the blood volume and the blood bag capacity are equal, return to S4 and reselect the blood bag.

[0159] In summary, this method can accurately obtain blood bag information and blood color classification results by real-time acquisition of surgical images (blood bag position images, mixed bag position images) and preoperative information of patients, combined with regional detection classification models and blood color classification models, and accurately control the dilution rate through intelligent valves according to the specific conditions of the patients and the actual state of the blood, reducing manual intervention and operation time, and improving the efficiency of blood management during surgery. It avoids the risk of blood being too thick or too diluted, delivers blood to the patient in a timely manner, and effectively solves the problems of slow infusion speed and / or blockage in the traditional blood transfusion process, which helps to efficiently maintain the stability of the patient's vital signs during surgery, reduce surgical risks and related complications, and ensure that the patient can receive the required amount of blood in a timely manner to ensure the smooth progress of the operation.

[0160] like Figure 5 As shown, an intraoperative blood dilution control system based on image recognition includes an acquisition module, a region detection and classification module, a blood color classification module, a blood bag selection module, and an intelligent control module; wherein:

[0161] An acquisition module is used to acquire the blood bag position image in real time and obtain the patient's preoperative information, the patient's vital sign parameters at the current moment, and the mixing bag position image;

[0162] The regional detection and classification module is used to input the blood bag position image and the mixed bag position image into the regional detection and classification model respectively to obtain the corresponding blood bag information data and the detection frame corresponding to the mixed bag position image;

[0163] A blood color classification module is used to input the detection frame corresponding to the mixed bag position image into the blood color classification model to obtain the corresponding blood color classification result;

[0164] A blood bag selection module is used to select a blood bag based on the blood bag information data and the patient's preoperative information;

[0165] Intraoperative blood dilution device, used to dilute the blood;

[0166] The intelligent control module is used to control the intraoperative blood dilution device and adjust the dilution speed based on the patient's vital sign parameters and blood color classification results.

[0167] In summary, the system can collect the patient's vital signs and parameters and the mixing bag position image in real time, and promptly obtain the patient's blood pressure, temperature, heart rate and other information as well as the color and mixing degree of the blood in the mixing bag 5, and then timely control the dilution rate, thereby realizing dynamic optimization control of the blood dilution process, and improving the safety and effectiveness of the operation; based on the blood bag information data and the patient's preoperative information, the system automatically selects the appropriate blood bag for blood transfusion, effectively reducing the errors of manual operation.

[0168] like Figure 6 As shown, the intraoperative blood dilution device includes a dilution liquid bag 1, a blood transfusion bag 2, a first intelligent valve 3, a second intelligent valve 4, a mixing bag 5 and a third intelligent valve 7; the mixing bag 5 is an inverted cone-shaped liquid bag, and vacuum chambers 6 are arranged on both sides of the bottom; a spiral sealing cover 62 is arranged on the outside of the mixing bag 5 corresponding to the vacuum chamber 6; a photoelectric sensor 63 and a micro vibration motor 61 are arranged in the vacuum chamber 6; wherein:

[0169] A dilution liquid bag 1 is used to store dilution liquid; a first intelligent valve 3 is used to control the dilution speed of the dilution liquid in the dilution liquid bag 1; a blood transfusion bag 2 is used to store blood; a second intelligent valve 4 is used to open the corresponding blood transfusion channel and control the corresponding speed; a mixing bag 5 is used to receive the dilution liquid in the dilution liquid bag 1 and the blood in the blood transfusion bag 2; a third intelligent valve 7 is used to control the corresponding infusion speed; a micro vibration motor 61 is used to provide vibration to achieve the purpose of mixing the dilution liquid and blood; a photoelectric sensor 63 is used to measure the liquid concentration of the mixing bag 5, that is, the hemoglobin concentration.

[0170] The second intelligent valve 4 has multiple input valves corresponding to multiple blood bags. When a blood bag is selected, the second intelligent valve 4 receives a corresponding instruction and opens the blood transfusion channel corresponding to the blood bag.

[0171] Any side of the vacuum chamber 6 can be selected, and the corresponding photoelectric sensor 63 can be replaced with a temperature sensor 64 to detect the corresponding blood temperature. When the corresponding blood temperature is too low, the temperature of the operating room can be adjusted or a heating device can be used to heat the blood in the mixing bag 5 before it is transported to the patient.

[0172] The spiral sealing cover 62 is provided to facilitate the installation / removal of the photoelectric sensor 63 and the micro vibration motor 61, thereby reducing consumables of medical equipment and unnecessary expenses.

[0173] In addition, the intraoperative blood dilution device also includes an allergic reaction judgment feedback module. The allergic reaction judgment feedback module is used to judge whether the patient has an acute allergic reaction caused by the blood transfusion according to the patient's vital signs (blood pressure, heart rate). If so, the blood transfusion is stopped and instructions are sent to the first smart valve 3, the second smart valve 4 and the third smart valve 7 to control the closing of the corresponding devices, and switch the dilution rate and blood delivery rate to 0.

[0174] The device is provided with three intelligent valves, which can effectively control the output speed of the dilution liquid bag 1, the blood bag and the mixing bag 5; the vacuum chamber 6 has a built-in photoelectric sensor 63 and a micro vibration motor 61, which can effectively achieve the mixing of the dilution liquid and the blood, and monitor the liquid concentration in real time, providing hardware support for the precise control of the dilution process, and effectively solving the problem of insufficient mixing in the traditional blood transfusion process, ensuring that the blood can be delivered to the patient's body in a timely and smooth manner, and improving the safety and reliability of the operation.

[0175] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for controlling intraoperative blood dilution based on image recognition, characterized in that: include: S1, real-time acquisition of blood bag position images and patient preoperative information; The patient's preoperative information was the preoperative blood test report; S2, constructing a region detection classification model and a blood color classification model respectively; S3, inputting the blood bag position image into the regional detection classification model to obtain corresponding blood bag information data; the blood bag information data includes first blood text information and second blood text information; S4, selecting a blood bag based on the blood bag information data and the patient's preoperative information; S5, start the intelligent valve to dilute the blood; S6. Real-time acquisition of the patient's vital sign parameters and the mixing bag position image at the current moment; the patient's vital sign parameters include blood pressure, temperature, heart rate and intraoperative blood gas analysis results; S7, inputting the mixed bag position image into the region detection classification model and the blood color classification model in sequence to obtain corresponding blood color classification results; the blood color classification results include moderate blood color, lighter blood color and darker blood color; S8. Based on the patient's vital signs parameters and blood color classification results, the dilution rate is controlled by an intelligent valve.

2. The method for controlling blood dilution during surgery based on image recognition according to claim 1, characterized in that: The regional detection and classification model includes a regional detection module, a classification module, a text recognition module, and a two-dimensional code scanning module; the output of the regional detection module is input into the classification module; the output of the classification module is respectively input into the text recognition module, the two-dimensional code scanning module, and the blood color classification model; The region detection module adopts the improved YOLOv3 model; the improved YOLOv3 model includes a Backbone module, a Neck module and a Head module connected in series; the Backbone module includes a first CBL layer, a first MobileNetV2 module, a second MobileNetV2 module, a C-Attention module and a third MobileNetV2 module connected in series; The Neck module includes a first neck submodule, a second neck submodule and a second CBL layer in parallel; the first neck submodule and the second neck submodule have the same structure, both including a third CBL layer, an Upsample layer and a Concat layer connected in series; The Head module includes a first head submodule, a second head submodule and a third head submodule in parallel; the structures of the first head submodule to the third head submodule are the same, and all include a fourth CBL layer, a Conv layer and a Head layer connected in series in sequence; the structures of the first CBL to the fourth CBL layers are the same; the structures of the first MobileNetV2 module to the third MobileNetV2 module are the same, and all adopt the MobileNetV2 model; The C-Attention module includes a parallel HSV module and a Grayscale Contrast layer; the HSV module includes a serially connected HSV layer and an Attention layer; The classification module uses KNN network or SVM classifier; The text recognition module includes a first preprocessing submodule and an OCR recognition submodule which are connected in series.

3. The method for controlling blood dilution during surgery based on image recognition according to claim 2, characterized in that: The training process of the region detection classification model includes the following steps: S3-1, obtain blood bag position training image data, mixed bag position training image data and their labels, and pre-process the blood bag position training image data and mixed bag position training image data; the label is the QR code, text and mixed bag corresponding to the blood bag; S3-2, inputting the preprocessed blood bag position training image data into the first CBL layer, and outputting a blood bag shallow layer feature map; S3-3, inputting the shallow feature map of the blood bag into the first MobileNetV2 module, and outputting the first deep feature map of the blood bag; S3-4, inputting the first blood bag deep feature map into the second MobileNetV2 module, and outputting the second blood bag deep feature map; S3-5, input the second blood bag deep feature map into the C-Attention module, and output the blood bag key color deep feature map; S3-6, inputting the blood bag key color deep feature map into the third MobileNetV2 module, and outputting the third blood bag deep feature map; S3-7, inputting the third blood bag deep layer feature map into the second CBL layer, and outputting the first blood bag fusion feature map; S3-8, inputting the first blood bag fusion feature map and the blood bag key color deep feature map into the first neck submodule, and outputting a second blood bag fusion feature map; S3-9, inputting the second blood bag fusion feature map and the first blood bag deep feature map into the second neck submodule, and outputting a third blood bag fusion feature map; S3-10, input the first blood bag fusion feature map, the second blood bag fusion feature map and the third blood bag fusion feature map into the corresponding first head submodule, the second head submodule and the third head submodule respectively, and output the first detection result, the second detection result and the third detection result; the first detection result to the third detection result respectively include the corresponding detection box and confidence score; S3-11, judging whether the confidence corresponding to each detection frame is greater than the confidence threshold; if so, retaining the corresponding detection frame to obtain the filtered detection frame; otherwise, removing the corresponding detection frame; S3-12, using the same method from S3-2 to S3-11 to process the pre-processed mixed bag position training image data to obtain a screened detection frame; S3-13, inputting the screened detection frame and the pre-processed blood bag position training image data / pre-processed mixed bag position training image data into the classification module to obtain a classification result; the classification result includes text image data, two-dimensional code image data, and mixed bag image data; S3-14, inputting the text image data into a text recognition module, and outputting the first blood training text information; S3-15, inputting the two-dimensional code image data into a two-dimensional code scanning module, and outputting the second blood training text information; S3-16. Calculate the final loss function of the region detection classification model; S3-17. Based on the final loss function, the weight parameters of the region detection classification model are adjusted to complete the training of the region detection classification model.

4. The method for controlling blood dilution during surgery based on image recognition according to claim 3, characterized in that: In S3-12, only the Grayscale Contrast layer in the C-Attention module is used to process the output data of the second MobileNetV2 module corresponding to the preprocessed blood training data to obtain the blood initial key color deep feature map; The output data of the second MobileNetV2 module corresponding to the initial key color deep feature map of blood and the preprocessed blood training data are added element by element to obtain the key color deep feature map of blood.

5. The method for controlling blood dilution during surgery based on image recognition according to claim 1, characterized in that: The S7 includes: S7-1, inputting the mixed bag position image into the region detection classification model, and outputting the classification result, that is, obtaining the mixed bag image; S7-2. Input the mixed bag image into the blood color classification model to obtain the blood color classification result.

6. The method for controlling blood dilution during surgery based on image recognition according to claim 1, characterized in that: The S7-2 includes: S7-2-1, performing color space conversion on the mixed bag image to obtain a mixed bag HSV image; S7-2-2, randomly sampling the pixel points of the mixed bag HSV image to obtain sampled pixel point data; S7-2-3, obtaining the red channel pixel value of the sampled pixel point data; S7-2-4, based on the red channel pixel value, calculating the mean of each sampled pixel point and the difference between each sampled pixel point and the surrounding sampled pixel points; S7-2-5, based on each difference, obtaining a corresponding degree of mixing; the degree of mixing is fully mixed, relatively fully mixed, and insufficiently mixed; S7-2-6. Determine whether the mean value is within the color threshold range ; If yes, the blood color classification result is determined to be moderate; otherwise, the process proceeds to S7-2-7; wherein, , All are expressed as constants; S7-2-7, determine whether the mean is less than ; If yes, the blood color classification result is determined to be lighter in color; otherwise, the blood color classification result is determined to be darker in color.

7. The method for controlling blood dilution during surgery based on image recognition according to claim 6, characterized in that: The smart valve includes a first smart valve, a second smart valve and a third smart valve; The S8 includes: S8-1, obtaining the hemoglobin concentration of the mixing bag at the current moment; based on the hemoglobin concentration, determining the concentration level of the mixing bag at the current moment; the concentration level includes low concentration, moderate concentration and high concentration; S8-2, judging whether the mixing degree is insufficient; if so, proceeding to S8-6; otherwise, proceeding to S8-3; S8-3, judging that the blood color classification result is moderate blood color and moderate concentration; if so, maintaining the current dilution rate and proceeding to S8-7; otherwise, proceeding to S8-4; S8-4, determine whether the blood color classification result and the concentration level meet the first judgment condition; if so, proceed to S8-6; otherwise, proceed to S8-5; the first judgment condition is: the blood color classification result and the concentration level meet any one of [lighter blood color, higher concentration] and [darker blood color, lighter concentration]; S8-5. Determine whether the blood color classification result and the concentration level meet the second judgment condition. If so, the dilution speed is accelerated by the first intelligent valve. Otherwise, the dilution speed is reduced by the first intelligent valve. Enter S8-7. The second judgment condition is: the blood color classification result and the concentration level meet any one of [darker blood color, higher concentration], [darker blood color, moderate concentration], and [moderate blood color, lower concentration]. S8-6, vibrate the blood transfusion bag, and return to S8-1; S8-7, determine whether the patient's vital sign parameters are within the normal range; if so, maintain the current infusion rate; otherwise, enter S8-8; S8-8. Determine whether the patient's vital sign parameters meet the third judgment condition and whether the intraoperative blood gas analysis results meet the fourth judgment condition; then speed up the current infusion speed through the third intelligent valve; otherwise, reduce the current infusion speed; the third judgment condition is: the patient's vital sign parameters meet any two of the following: the blood pressure at the current moment is less than the blood pressure at the previous moment, the temperature at the current moment is less than the temperature at the previous moment, and the heart rate at the current moment is less than the heart rate at the previous moment.

8. An intraoperative blood dilution control system based on image recognition, used to implement an intraoperative blood dilution control method based on image recognition according to any one of claims 1 to 7, characterized in that: It includes a collection module, a regional detection and classification module, a blood color classification module, a blood bag selection module, an intelligent control module, and an intraoperative blood dilution device; among which: An acquisition module is used to acquire the blood bag position image in real time and obtain the patient's preoperative information, the patient's vital sign parameters at the current moment, and the mixing bag position image; The regional detection and classification module is used to input the blood bag position image and the mixed bag position image into the regional detection and classification model respectively to obtain the corresponding blood bag information data and the detection frame corresponding to the mixed bag position image; A blood color classification module is used to input the detection frame corresponding to the mixed bag position image into the blood color classification model to obtain the corresponding blood color classification result; A blood bag selection module is used to select a blood bag based on the blood bag information data and the patient's preoperative information; Intraoperative blood dilution device, used to dilute the blood; The intelligent control module is used to control the intraoperative blood dilution device and adjust the dilution speed based on the patient's vital sign parameters and blood color classification results.

9. The intraoperative blood dilution control system based on image recognition according to claim 8, characterized in that: The intraoperative blood dilution device comprises a dilution liquid bag (1), a blood transfusion bag (2), a first intelligent valve (3), a second intelligent valve (4), a mixing bag (5) and a third intelligent valve (7); the mixing bag (5) is an inverted cone-shaped liquid bag, and vacuum chambers (6) are provided on both sides of the bottom; a spiral-patterned sealing cover (62) is provided on the outside of the mixing bag (5) corresponding to the vacuum chamber (6); a photoelectric sensor (63) and a micro vibration motor (61) are provided in the vacuum chamber (6); wherein: A diluting liquid bag (1) for storing diluting liquid; a first intelligent valve (3) for controlling the dilution rate of the diluting liquid in the diluting liquid bag (1); a blood transfusion bag (2) for storing blood; a second intelligent valve (4) for opening a corresponding blood transfusion channel and controlling the corresponding rate; a mixing bag (5) for receiving the diluting liquid in the diluting liquid bag (1) and the blood in the blood transfusion bag (2); a third intelligent valve (7) for controlling the corresponding transfusion rate; a micro vibration motor (61) for providing vibration to achieve the purpose of mixing the diluting liquid and the blood; and a photoelectric sensor (63) for measuring the liquid concentration of the mixing bag (5), that is, the hemoglobin concentration.

Images