Bleeding volume determination system, method, device, equipment and storage medium

Abstract

The present invention relates to the field of intelligent image recognition technology, and discloses a bleeding volume determination system, method, device, equipment and storage medium, the system comprising: an interactive module and a post-processing module; the interactive module is a lightweight application program installed on an intelligent mobile terminal device, the interactive module is used to obtain a bloody towel image from the intelligent mobile terminal device, and transmit the bloody towel image to the post-processing module; the interactive module is also used to receive the bleeding volume result output by the post-processing module, and display the bleeding volume result using the intelligent mobile terminal device; the post-processing module is used to detect the bloody area from the bloody towel image using a first neural network model; and the bloody area is identified and predicted using a second neural network model to obtain the bleeding volume result. The present invention can quickly, accurately and conveniently estimate the bleeding volume.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent image recognition, and in particular to a bleeding volume determination system, method, device, equipment and storage medium. Background Art

[0002] Antepartum hemorrhage refers to vaginal bleeding after 28 weeks of pregnancy. Postpartum hemorrhage (PPH) refers to bleeding ≥500ml for vaginal delivery and ≥1000ml for cesarean section within 24 hours after the fetus is delivered. Repeated or heavy vaginal bleeding before and after delivery can cause severe anemia in pregnant women and parturients, or a one-time heavy bleeding can cause pregnant women to go into shock. If not treated in time, it can lead to the death of mother and child. Therefore, it is necessary to accurately assess the amount of bleeding to achieve early identification and early diagnosis so as to provide timely treatment.

[0003] At present, the clinical evaluation methods of postpartum hemorrhage include visual measurement, area method, weighing method, colorimetry, hematocrit method and hemoglobin count. However, these methods have problems such as cumbersome operation steps, poor accuracy and long time consumption. Summary of the invention

[0004] In view of this, the present invention provides a bleeding volume determination system, method, apparatus, device and storage medium to solve the problems of cumbersome user operation steps, poor accuracy, and long time consumption in the bleeding volume assessment method.

[0005] In a first aspect, the present invention provides a bleeding volume determination system, the system comprising: an interaction module and a post-processing module;

[0006] The interaction module is a lightweight application program installed on a smart mobile terminal device, and is used to obtain a bloody towel image from the smart mobile terminal device and transmit the bloody towel image to the post-processing module;

[0007] The interaction module is also used to receive the bleeding volume result output by the post-processing module, and display the bleeding volume result using the smart mobile terminal device;

[0008] The post-processing module is used to detect the bloody area from the bloody towel image using the first neural network model; and to identify and predict the bloody area using the second neural network model to obtain the bleeding volume result.

[0009] In an optional embodiment, the first neural network model outputs a target parameter indicating the position of the bounding box of the bloody area;

[0010] The post-processing module is further used to crop the bloody towel image according to the target parameter to obtain the image of the bloody area.

[0011] In an optional embodiment, the post-processing module is also used to mark the bloody area of ​​the bloody towel image based on the bloody area detected by the first neural network model, and send the marked bloody towel image to the interactive module, and the interactive module receives and displays the marked bloody towel image.

[0012] In a second aspect, the present invention provides a method for determining bleeding volume, the method comprising:

[0013] Acquire the bloody towel image;

[0014] Using a first neural network model, detecting a bloody area from the bloody towel image;

[0015] The second neural network model is used to identify and predict the bloody area to obtain the bleeding volume result.

[0016] In an optional embodiment, the first neural network model is a YOLOv8 model; and / or the second neural network model is built based on a ResNet34 model.

[0017] In an optional embodiment, the first neural network model includes an input layer, a convolution layer, a pooling layer and an output layer;

[0018] The convolution layer is used to extract image features of the bloody towel image;

[0019] The pooling layer is used to reduce the data dimension of the image features;

[0020] The output layer is used to output the position and size of the bloody area.

[0021] In an optional embodiment, the second neural network model includes an input layer, a residual block and an output layer, the output layer includes two intermediate layers and an output layer, and the intermediate layers and the output layer are fully connected layers.

[0022] In an optional implementation, the model training loss function of the first neural network model is:

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] in, is the total loss, , , is the weight coefficient of each loss item;

[0029] is the classification loss, is the bounding box regression loss, is the distribution focus loss;

[0030] N is the number of samples, and M is the sum of the classification labels of all bloody areas;

[0031] For the The classification labels of samples;

[0032] For the The classification prediction probability of samples;

[0033] IoU is the intersection over union ratio; CIoU is the complete intersection over union ratio;

[0034] The upper left and lower right coordinates of the predicted bounding box;

[0035] are the coordinates of the upper left and lower right corners of the label bounding box;

[0036] To predict the width and height of the bounding box;

[0037] is the width and height of the label bounding box;

[0038] is a constant value;

[0039] is the target value, is the target value rounded down. Round up the target value;

[0040] pred_dist is the predicted probability distribution, and CE is the cross entropy loss function.

[0041] In an optional implementation, the model training loss function of the second neural network model is:

[0042]

[0043]

[0044] in, , is the bleeding volume label value and the model’s predicted bleeding volume value, is the number of samples, , is a hyperparameter.

[0045] In a third aspect, the present invention provides a device for determining the amount of bleeding, the device comprising:

[0046] An image acquisition module, used for acquiring an image of a bloody towel;

[0047] An object detection module, used to detect the bloody area from the bloody towel image using a first neural network model;

[0048] The bleeding volume detection module is used to use the second neural network model to identify and predict the bloody area to obtain the bleeding volume result.

[0049] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for determining the amount of bleeding of the second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method for determining the amount of bleeding according to the second aspect or any corresponding embodiment thereof.

[0051] In a sixth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for determining the amount of bleeding according to the second aspect or any corresponding embodiment thereof.

[0052] The embodiments of the present invention provide a bleeding volume determination system, method, device, equipment and storage medium, which automatically identifies the bloody area in the bloody towel image based on the deep learning model and performs regression analysis of the bleeding volume, eliminating the subjective error caused by manual visual inspection and significantly improving the accuracy of bleeding volume assessment. It takes a very short time from acquiring the bloody towel image to outputting the bleeding volume prediction result, and can quickly provide the bleeding volume assessment result in an emergency situation, which helps medical staff to make decisions quickly and intervene in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 is a structural block diagram of a system for determining bleeding volume according to an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of a user interaction interface according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of the architecture of a bleeding volume determination system according to an embodiment of the present invention;

[0057] Figure 4 is a flow chart of a method for determining bleeding volume according to an embodiment of the present invention;

[0058] Figure 5 is a structural block diagram of a device for determining bleeding volume according to an embodiment of the present invention;

[0059] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0061] With the widespread application of artificial intelligence technology in the medical field, the method of image processing based on deep learning technology has good feasibility in the assessment of bleeding volume. In addition, the application of image recognition and regression analysis based on deep learning technology in the medical field can provide a more convenient calculation method, which can improve the accuracy and efficiency of bleeding volume calculation. In view of this, an embodiment of the present invention provides a method for determining bleeding volume, which is an automatic prediction method for postpartum bleeding volume based on deep learning.

[0062] The embodiment of the present invention provides a system for determining the amount of bleeding. Figure 1 , the system comprises: an interaction module 101 and a post-processing module 102;

[0063] The interaction module 101 is a lightweight application program installed on a smart mobile terminal device. The interaction module 101 is used to obtain a bloody towel image from the smart mobile terminal device and transmit the bloody towel image to the post-processing module 102. Specifically, the bloody towel image can be obtained by taking a photo of the smart mobile terminal device or obtained from other devices.

[0064] The interaction module 101 is also used to receive the bleeding volume result output by the post-processing module 102, and display the bleeding volume result using the smart mobile terminal device;

[0065] The post-processing module 102 is used to detect the bloody area from the bloody towel image using the first neural network model; and to identify and predict the bloody area using the second neural network model to obtain the bleeding amount result.

[0066] Specifically, the lightweight application can be a small program or other lightweight application. The intelligent mobile terminal device can be various intelligent mobile terminal devices such as smart phones and tablet computers, which have good portability and ease of use. The lightweight application installed on the intelligent mobile terminal device can provide a concise and intuitive user interaction interface and a simple operation process, and support the shooting, uploading and display of bloody pad images and blood volume results. The post-processing module 102 can be implemented based on the Flask framework and can be installed on devices such as background servers.

[0067] In some optional specific embodiments, the post-processing module 102 is also used to mark the bloody area of ​​the bloody towel image based on the bloody area detected by the first neural network model, and send the marked bloody towel image to the interactive module 101, and the interactive module 101 receives and displays the marked bloody towel image.

[0068] See also Figure 2As shown, the user interaction interface includes a title bar, a picture bar, an information bar and a button bar. The title bar is set in the upper area to display the time and the amount of bleeding predicted by the post-processing module 102. The picture bar is used to display the bloody towel image returned by the post-processing module 102. The bloody area in the bloody towel image returned by the post-processing module 102 is marked, so that the user can clearly see the bloody area identified by the post-processing module 102 to determine whether the bloody area identified by the post-processing module 102 is correct. The information bar is used to display the image address, as well as the IP address and port number input box. The backend server connection pairing is performed by entering the IP address and port number of the backend server where the post-processing module 102 is located. The button bar can be set at the bottom, the left button is used to refresh the title bar, and the right button is used to establish a connection with the backend server and send the selected bloody towel image to the backend service end.

[0069] The embodiment of the present invention provides an automatic bleeding volume identification system based on deep learning. By analyzing the image of the pad towel used by the patient through artificial intelligence image recognition technology, the bleeding volume can be estimated quickly and accurately. From uploading the bloody pad towel image to the post-processing module 102 to receiving and displaying the bleeding volume result output by the post-processing module 102, the time consumed in between may be only a few seconds, so that timely processing can be carried out according to the bleeding volume result to avoid serious consequences.

[0070] In addition, the interactive module 101, which can be installed on various smart mobile terminal devices, can realize user-friendly interface display and operation. Medical staff and patients can easily use the system to evaluate bleeding volume without complicated training, and it is suitable for various medical environments.

[0071] In summary, the embodiments of the present invention provide an automatic bleeding volume identification system based on deep learning, which solves the problems of poor accuracy, complex operation, long time consumption, large workload, high cost, etc. in the bleeding volume assessment method in the related art, and realizes a fast, accurate and simple bleeding volume assessment function.

[0072] Specifically, the first neural network model outputs target parameters indicating the position of the bounding box of the bloody area. For example, the first neural network model outputs the coordinates of the upper left corner and the lower right corner of the bounding box of the bloody area identified by it, or the coordinates of the center point of the bounding box and the height and width of the bounding box.

[0073] The post-processing module is further used to crop the bloody towel image according to the target parameter to obtain the image of the bloody area. When the target parameter is the coordinates of the two points at the upper left corner and the lower right corner of the boundary box of the bloody area, the post-processing module crops the bloody towel image according to the coordinates of the two points, and the cropped image area is the bloody area.

[0074] In addition, before the post-processing module 102 inputs the bloody towel image into the first neural network model, it is also necessary to pre-process the bloody towel image, and the pre-processing includes scaling, padding and normalization. For details, please refer to the method embodiment below. Scaling and padding are to make the bloody towel image adapt to the size requirements of the first neural network model for the input image, and normalization is to make the bloody towel image adapt to the pixel value range requirements of the first neural network model for the input image. Pre-processing can enable the system to accurately and stably estimate the amount of bleeding for bloody towel images of different qualities acquired under different lighting and different image acquisition devices, thereby improving the robustness of the system.

[0075] Before the post-processing module 102 inputs the bloody area into the second neural network model for bleeding volume prediction, it is also necessary to pre-process the bloody area cropped from the bloody towel image. The pre-processing here also mainly includes filling, scaling and normalization. The purpose of filling is to fill the bloody area into a square (with the long side of the bloody area as the side length of the square), and scaling is to scale the filled image to the size required by the second neural network model for the input image (for example, 224×224), and finally the filled and scaled bloody area is normalized.

[0076] Figure 3 The diagram shows an architecture of a bleeding volume determination system. The architecture of the bleeding volume determination system includes the following layers: user interface layer, application service layer, model processing layer and basic support layer. The user interface layer is the above-mentioned interaction module 101, which will not be described in detail here. The application service layer includes an API (Application Programming Interface) gateway and a business logic processing module. The API gateway is used to process the image recognition request issued by the interaction module 101, that is, to receive the bloody towel image and route the bloody towel image to the post-processing module 102. In addition, the API gateway is also used to return the bleeding volume result output by the post-processing module 102 to the interaction module 101. The business logic processing module is constructed by the Flask framework, receives the bloody towel image forwarded by the API gateway, and calls the model processing layer. The model processing layer is the above-mentioned post-processing module 102, which will not be described here. The basic support layer includes server hardware and network connection modules, wherein the server hardware is deployed with a post-processing module 102 (i.e., the Flask backend application and deep learning model), and should have sufficient CPU / GPU resources to handle concurrent requests and perform model analysis; the network connection module is used to ensure a stable and high-speed network connection for timely communication between the user interface layer and the application service layer.

[0077] In other words, the mini program communicates with the backend Flask server through the API gateway to upload images and obtain results. After receiving the image, the Flask server calls the deep learning model for processing. The target detection model and the regression model are deployed on the same server to jointly process the image data. The data flow process is: the image uploaded by the user -> the Flask server receives and preprocesses -> the YOLOv8 target detection model detects the bleeding area -> the ResNet34-based regression model estimates the amount of bleeding -> the results are fed back to the user.

[0078] The process of estimating the amount of bleeding using the bleeding volume determination system provided by the embodiment of the present invention is as follows:

[0079] 1. Image upload: Users use the mini program to take images of bloody pads used by parturients and upload them to the backend server. The user can be a patient, a patient's family member, a medical worker, etc. Specifically, the user can open the mini program on a smart mobile terminal device and take a photo of the bloody area of ​​the flattened bloody pad.

[0080] 2. Image preprocessing: Scale and fill the bloody towel image and perform normalization.

[0081] 3. Target detection: The preprocessed image is input into the YOLOv8 model to detect the bloody area and output the bounding box parameters of the bloody area.

[0082] 4. Image processing: using the bounding box parameters, the image of the bloody area is cut out from the original bloody towel image, and scaling, filling, normalization and other processing are performed.

[0083] 5. Estimation of bleeding volume: Input the bloody area image obtained in the previous step into the regression model to estimate the bleeding volume.

[0084] 6. Result feedback: The estimation results are fed back to the user through the mini program and displayed on the mini program interface.

[0085] According to an embodiment of the present invention, an embodiment of a method for determining bleeding volume is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of executable computer instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0086] In this embodiment, a method for determining the amount of bleeding is provided, which can be used in various computer devices, including terminals, the servers described in the above embodiments, etc. Figure 4 is a flow chart of a method for determining bleeding volume according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0087] Step S401, obtaining a bloody towel image.

[0088] Specifically, the bloody towel image may be sent by a smart mobile terminal device.

[0089] After the bloody towel image sent by the smart mobile terminal device is acquired, the bloody towel image may be preprocessed.

[0090] Specifically, the preprocessing of the bloody towel image includes scaling, padding and normalization, etc. These preprocessing can be performed using tools such as the PIL library (Python Imaging Library, a Python library for processing images), the OpenCV library (an open source computer vision and machine learning software library that contains a large number of image processing and computer vision algorithms), and the torchvision library (a part of the PyTorch ecosystem, a Python library for computer vision tasks). The preprocessing is to make the bloody towel image adapt to the input requirements of the deep learning model.

[0091] Scaling is used to adjust the size of the image to meet the input requirements of the deep learning model. If the image size after scaling still does not meet the input requirements of the deep learning model, the image can be padded. For example, the input image size required by the deep learning model is 640×640 pixels, but the image is rectangular and the image size after scaling is 640×224, then the image can be padded with black pixels to obtain a 640×640 pixel image. Normalization is used to scale the image pixel values ​​to a fixed range (for example, the mean of the RGB three channels is [0.485, 0.456, 0.406], and the standard deviation is [0.229, 0.224, 0.225]) to meet the requirements of the deep learning model. These preprocessing of the image can improve the stability of the deep learning model detection prediction.

[0092] Step S402: Detect the bloody area from the bloody towel image using the first neural network model. The first neural network model is used to detect the bloody area.

[0093] In some optional specific implementations, the first neural network model may be a YOLOv8 model.

[0094] Specifically, the first neural network model includes an input layer, a convolution layer, a pooling layer and an output layer;

[0095] The convolution layer is used to extract image features of the bloody towel image;

[0096] The pooling layer is used to reduce the data dimension of the image features;

[0097] The output layer is used to output the position and size of the bloody area.

[0098] In an embodiment of the present invention, the first neural network model can gradually extract high-level features of the image by using multiple convolutional layers and pooling layers, and finally generate the bounding box coordinates and category probability of the bloody area in the output layer.

[0099] Step S403: using the second neural network model, the bloody area is identified and predicted to obtain the bleeding volume result. The second neural network model is used for bleeding volume regression analysis.

[0100] In some optional specific implementations, the second neural network model is constructed based on the ResNet34 model.

[0101] Specifically, the second neural network model includes an input layer, a residual block and an output layer, the output layer includes two intermediate layers and an output layer, and the intermediate layers and the output layer are fully connected layers.

[0102] In the embodiment of the present invention, the second neural network model is constructed based on the ResNet34 model. Specifically, the network structure of the ResNet34 model is modified, the input layer and the residual block are retained, but the output layer is modified to adapt to the regression task. Specifically, the original output layer is a fully connected layer for outputting the category probability. In order to estimate the continuous bleeding volume value, the last fully connected layer is replaced with a layer structure suitable for the regression task, including two intermediate layers and an output layer (all fully connected layers, mapping the high-dimensional feature vector to the dimension of the target value), to output a continuous value, representing the estimated bleeding volume. In addition, in order to enable the model to learn and represent more complex functional relationships, nonlinear transformations can also be introduced, and nonlinear transformations are implemented by activation functions. The residual block is used to solve the gradient vanishing problem in deep network training and improve the training effect of the model. The model gradually extracts the high-level features of the image by using multiple residual blocks, and finally generates an estimated value of the bleeding volume in the output layer.

[0103] The method for determining the amount of bleeding provided in this embodiment automatically identifies the bloody area in the bloody towel image based on a deep learning model, and performs regression analysis of the amount of bleeding, eliminating the subjective error caused by manual visual inspection and significantly improving the accuracy of the bleeding amount assessment. It takes a very short time from acquiring the bloody towel image to outputting the bleeding amount prediction result, and can quickly provide the bleeding amount assessment result in an emergency situation, which helps medical staff to make decisions quickly and intervene in time. The method for determining the amount of bleeding provided in this embodiment can be applied to various medical environments, including emergency scenes, hospital wards, and other occasions where a rapid assessment of the amount of bleeding is required.

[0104] Specifically, the first neural network model is used to quickly and accurately detect the bloody area in the bloody towel image. The training process of the first neural network model includes the following steps:

[0105] 1. Data preparation: Collect sample images (also bloody towel images), annotate the bloody areas of the sample images, and generate a label file for bloody area detection. The label file contains the bounding box coordinates and category information of each bloody area in the sample image. The category information is used to indicate whether the image area within the bounding box is a bloody area.

[0106] 2. Model training: Use sample images to train the model, and adjust the model parameters through multiple iterations so that the model can accurately detect bloody areas. During the training process, cross entropy loss, bounding box regression loss, and distribution focus loss (also called focus distribution loss) can be used as optimization targets.

[0107] 3. Model evaluation: Evaluate the detection accuracy of the model on the validation set to ensure that the model can accurately identify bloody areas. Evaluation indicators include mAP50 (indicating the average precision over multiple categories, mAP50 represents the average precision under a 50% IoU (Intersection over Union) threshold) and recall rate (the proportion of all true positive samples that the evaluation model can find).

[0108] The model training loss function of the first neural network model is:

[0109] ;

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] in, is the total loss, , , is the weight coefficient of each loss item;

[0115] is the classification loss, is the bounding box regression loss, is the distribution focus loss;

[0116] N is the number of samples, and M is the sum of the classification labels of all bloody areas. In other words, The sum of the classification labels excluding non-bloody areas; divided by The weights of the number of targets in the bloody area and the non-bloody area can be balanced, because in general, the number of targets in the non-bloody area is greater than that in the bloody area;

[0117] For the The classification labels of samples;

[0118] For the The classification prediction probability of a sample, using the Sigmoid activation function;

[0119] IoU is the intersection over union ratio; CIoU is the complete intersection over union ratio, which is used to evaluate the overlap between the bounding box predicted by the model and the actual bounding box;

[0120] The upper left and lower right coordinates of the predicted bounding box;

[0121] are the coordinates of the upper left and lower right corners of the label bounding box;

[0122] To predict the width and height of the bounding box;

[0123] is the width and height of the label bounding box;

[0124] is a constant value, which is a very small value used to avoid division by zero errors;

[0125] is the target value, is the target value rounded down. is the target value rounded up; in YOLOv8, the target value It is the offset of the real coordinates of the target bounding box compared to the center of the predicted bounding box in different axes (calculated from the real coordinates). When calculating DFL, is converted into a discrete distribution. That is, is a specific floating point value, and and It is used to calculate the loss and is a specific integer value.

[0126] pred_dist is the predicted probability distribution, which indicates the possibility of the bounding box position at a discrete point; CE is the cross entropy loss function, which is used to measure the difference between pred_dist and the target distribution.

[0127] In this embodiment, the classification loss is used to measure the difference between the category predicted by the model (whether it is a bloody area) and the actual target category (classification label). The bounding box regression loss and distribution focus loss are used to measure the position difference between the bounding box of the bloody area predicted by the model and the real bounding box (bounding box label). When the bounding box predicted by the model does not match the actual bounding box, the bounding box regression loss will become larger. Through back propagation, the model will learn how to adjust the coordinates of the bounding box to make them closer to the real position. The DFL loss assists the model in learning how to predict the bounding box coordinates more finely.

[0128] Specifically, the bounding box regression loss of YOLOv8 is defined as 1 - CIoU. CIoU (Complete Intersection over Union) not only considers the overlap between the predicted bounding box and the true bounding box, but also introduces the difference in center point distance and aspect ratio. By combining these factors, the bounding box regression loss can comprehensively measure the difference between the bounding box predicted by the model and the true bounding box, thereby optimizing the position and shape of the predicted box and improving the accuracy of detection.

[0129] The core goal of the bounding box regression loss is to ensure that the predicted box is as close to the actual situation as possible in terms of overall position, size, and shape. The distribution focus loss (DFL) further refines this process by optimizing the probability distribution of each coordinate to ensure that the model can accurately predict the boundaries of the target in complex scenes (such as blurred boundaries or irregular shapes). In short, the bounding box regression loss is responsible for the accuracy of the "big picture", while DFL ensures the "details" are in place. The combination of the two enables YOLOv8 to handle more complex and diverse scenes in the bounding box regression task and provide more accurate detection results.

[0130] The second neural network model is used to analyze the bloody area detected by the first neural network model and estimate the amount of bleeding. The model needs to learn the relationship between the bloody area image and the amount of bleeding. The embodiment of the present invention uses a ResNet34 model pre-trained based on the ImageNet dataset (a large-scale image dataset that was originally created to promote computer vision research. The ImageNet dataset contains a large number of natural images, and these images are manually labeled with category labels, which makes it an important benchmark in the field of computer vision). The pre-trained ResNet34 model has preliminary image feature extraction capabilities, which helps to improve training efficiency. The process of performing transfer learning training on the pre-trained ResNet34 model includes the following steps:

[0131] 1. Data preparation: Process the sample images to generate the input data and labels required by the regression model. The label is the actual amount of bleeding corresponding to the image, which is used to guide model learning. During the data preparation process, the bloody area of ​​the pad will be cropped from the entire image and resized and padded to a uniform size (for example, 224x224 pixels), and then normalized to scale the pixel values ​​to a fixed range. Specifically, you can use OpenCV to read and convert the image, use PIL to fill the image into a square, and then use the transforms module in torchvision to resize and normalize the image. The purpose is to preprocess the original image data into a standardized tensor for input into the deep learning model for training or inference. These preprocessing steps can improve the performance of the model and the stability of training. At the same time, during training, data enhancement operations including random horizontal flipping, random rotation, random adjustment of image brightness, contrast, saturation and hue, and random affine transformation of the image (translation, scaling, etc.) are performed to increase the diversity of sample images and improve the generalization ability of the model.

[0132] 2. Model training: The main libraries used include PyTorch and Scikit-learn. During the training process, a custom loss function is used to optimize the model parameters. By combining the smooth L1 loss and the Focal mechanism, the prediction error of the model is adjusted more finely to improve the robustness and accuracy of the model.

[0133] 3. Model evaluation: Evaluate the regression accuracy of the model on the validation set to ensure that the model can accurately estimate the amount of bleeding. The evaluation indicator is accuracy: if the model prediction value and the true value are within the error range of 5ml, the prediction is correct.

[0134] In some specific implementations, the model training loss function of the second neural network model is:

[0135]

[0136]

[0137] in, , is the bleeding volume label value and the model’s predicted bleeding volume value, is the number of samples, , is a hyperparameter. Can be set to 4, Can be set to 2.

[0138] In the present embodiment, a kind of bleeding volume determining device is also provided, and this device is used to realize above-mentioned bleeding volume determining method embodiment and preferred implementation mode, and the description that has been made will not be repeated.As used below, term "module" can realize the software and / or hardware combination of predetermined function.Although the device described in the following embodiments is preferably realized with software, hardware, or the realization of the combination of software and hardware is also possible and conceived.

[0139] This embodiment provides a bleeding volume determination device, such as Figure 5 As shown, including:

[0140] An image acquisition module 501 is used to acquire a bloody towel image;

[0141] The target detection module 502 is used to detect the bloody area from the bloody towel image by using the first neural network model;

[0142] The bleeding volume detection module 503 is used to use the second neural network model to identify and predict the bloody area to obtain a bleeding volume result.

[0143] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0144] The bleeding volume determination device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0145] The embodiment of the present invention also provides a computer device having the above Figure 5 The bleeding volume determination device shown.

[0146] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6As shown, the computer device can be a server, specifically including: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Each component utilizes different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0147] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0148] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0149] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0151] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0152] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0153] The computer device also includes a communication interface, which is used for the computer device to communicate with other devices or a communication network.

[0154] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0155] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0156] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A bleeding volume determination system, characterized in that: The system comprises: an interaction module and a post-processing module; The interactive module is a lightweight application program installed on a smart mobile terminal device, and is used to obtain a bloody towel image from the smart mobile terminal device and transmit the bloody towel image to the post-processing module; the bloody towel image is obtained by photographing the smart mobile terminal device; The interaction module is also used to receive the bleeding volume result output by the post-processing module, and display the bleeding volume result using the smart mobile terminal device; The post-processing module is used to detect the bloody area from the bloody towel image using the first neural network model; the first neural network model outputs a target parameter indicating the position of the bounding box of the bloody area; the post-processing module is also used to crop the bloody towel image according to the target parameter to obtain the image of the bloody area; the bloody area is identified and predicted using the second neural network model to obtain the bleeding volume result; the post-processing module is implemented based on the Flask framework and installed on the background server; The user interaction interface in the interactive module includes a title bar, a picture bar, an information bar and a button bar; the title bar is set in the upper area, and is used to display the time and the amount of bleeding predicted by the post-processing module; the picture bar is used to display the bloody towel image returned by the post-processing module, and the bloody area in the bloody towel image returned by the post-processing module is marked, so that the user can see the bloody area identified by the post-processing module to determine whether the bloody area identified by the post-processing module is correct; the information bar is used to display the picture address, as well as the IP address and port number input box; the backend server connection pairing is performed by inputting the IP address and port number of the backend server where the post-processing module is located; The bleeding volume determination system also includes: an application service layer, a model processing layer and a basic support layer; The application service layer includes an API gateway and a business logic processing module. The API gateway is used to process the image recognition request issued by the interaction module, that is, to receive the bloody towel image and route the bloody towel image to the post-processing module; the API gateway is also used to return the bleeding volume result output by the post-processing module to the interaction module; the business logic processing module is built by the Flask framework, receives the bloody towel image forwarded by the API gateway, and calls the model processing layer, which is the post-processing module; the basic support layer includes server hardware and network connection modules, among which the server hardware is deployed with the post-processing module; the network connection module is used for timely communication between the interaction module and the application service layer; Among them, the model training loss function of the first neural network model is: ； ； ； ； ； in, is the total loss, , , is the weight coefficient of each loss item; is the classification loss, is the bounding box regression loss, is the distribution focus loss; N is the number of samples, and M is the sum of the classification labels of all bloody areas; For the The classification labels of samples; For the The classification prediction probability of samples; IoU is the intersection over union ratio; CIoU is the complete intersection over union ratio; The upper left and lower right coordinates of the predicted bounding box; are the coordinates of the upper left and lower right corners of the label bounding box; To predict the width and height of the bounding box; is the width and height of the label bounding box; is a constant value; is the target value, is the target value rounded down. Round up the target value; pred_dist is the predicted probability distribution, CE is the cross entropy loss function; The second neural network model is built based on the ResNet34 model, retaining the input layer and residual block of the ResNet34 model. The original output layer is replaced by two intermediate layers and one output layer. The intermediate layer and the output layer are both fully connected layers, which are used to map high-dimensional feature vectors to the dimensions of the target value to adapt to the regression task and output a numerical value representing the estimated bleeding volume.

2. The system according to claim 1, characterized in that The post-processing module is also used to mark the bloody area of ​​the bloody towel image based on the bloody area detected by the first neural network model, and send the marked bloody towel image to the interactive module, and the interactive module receives and displays the marked bloody towel image.

3. A method for determining the amount of bleeding, characterized in that: Applied to the backend server according to claim 1 or 2, the method comprises: Acquire the bloody towel image; Detecting a bloody area from the bloody towel image using a first neural network model; the first neural network model outputs a target parameter indicating a position of a bounding box of the bloody area; Cropping the bloody towel image according to the target parameter to obtain an image of the bloody area; Using the second neural network model, the bloody area is identified and predicted to obtain a bleeding volume result; Among them, the model training loss function of the first neural network model is: ； ； ； ； ； in, is the total loss, , , is the weight coefficient of each loss item; is the classification loss, is the bounding box regression loss, is the distribution focus loss; N is the number of samples, and M is the sum of the classification labels of all bloody areas; For the The classification labels of samples; For the The classification prediction probability of samples; IoU is the intersection over union ratio; CIoU is the complete intersection over union ratio; The upper left and lower right coordinates of the predicted bounding box; are the coordinates of the upper left and lower right corners of the label bounding box; To predict the width and height of the bounding box; is the width and height of the label bounding box; is a constant value; is the target value, is the target value rounded down. Round up the target value; pred_dist is the predicted probability distribution, CE is the cross entropy loss function; The second neural network model is built based on the ResNet34 model, retaining the input layer and residual block of the ResNet34 model. The original output layer is replaced by two intermediate layers and one output layer. The intermediate layer and the output layer are both fully connected layers, which are used to map high-dimensional feature vectors to the dimensions of the target value to adapt to the regression task and output a numerical value representing the estimated bleeding volume.

4. The method for determining the amount of bleeding according to claim 3, characterized in that: The first neural network model is a YOLOv8 model.

5. The method for determining the amount of bleeding according to claim 3 or 4, characterized in that: The first neural network model includes an input layer, a convolution layer, a pooling layer and an output layer; The convolutional layer is used to extract image features of the bloody towel image; The pooling layer is used to reduce the data dimension of the image features; The output layer is used to output the position and size of the bloody area.

6. The method for determining the amount of bleeding according to claim 3 or 4, characterized in that: The model training loss function of the second neural network model is: in, , is the bleeding volume label value and the model’s predicted bleeding volume value, is the number of samples, , is a hyperparameter.

7. A bleeding volume determination device, characterized in that: Applied to the backend server according to claim 1 or 2, the device comprises: An image acquisition module, used for acquiring an image of a bloody towel; A target detection module, configured to detect a bloody area from the bloody towel image using a first neural network model; the first neural network model outputs a target parameter indicating a position of a bounding box of the bloody area; A module for cropping the bloody towel image according to the target parameter to obtain an image of the bloody area; A bleeding volume detection module is used to identify and predict the bloody area using a second neural network model to obtain a bleeding volume result; Among them, the model training loss function of the first neural network model is: ； ； ； ； ； in, is the total loss, , , is the weight coefficient of each loss item; is the classification loss, is the bounding box regression loss, is the distribution focus loss; N is the number of samples, and M is the sum of the classification labels of all bloody areas; For the The classification labels of samples; For the The classification prediction probability of samples; IoU is the intersection over union ratio; CIoU is the complete intersection over union ratio; The upper left and lower right coordinates of the predicted bounding box; are the coordinates of the upper left and lower right corners of the label bounding box; To predict the width and height of the bounding box; is the width and height of the label bounding box; is a constant value; is the target value, is the target value rounded down. Round up the target value; pred_dist is the predicted probability distribution, CE is the cross entropy loss function; The second neural network model is built based on the ResNet34 model, retaining the input layer and residual block of the ResNet34 model. The original output layer is replaced by two intermediate layers and one output layer. The intermediate layer and the output layer are both fully connected layers, which are used to map high-dimensional feature vectors to the dimensions of the target value to adapt to the regression task and output a numerical value representing the estimated bleeding volume.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for determining the amount of bleeding according to any one of claims 3 to 6 by executing the computer instructions.

Images