Deep learning serum quality interpretation method with feedback characteristic

Abstract

The invention discloses a deep learning serum quality interpretation method with a feedback characteristic. The deep learning serum quality interpretation method comprises the following steps: carrying out image data acquisition on a serum sample; identifying the collected serum sample image data, and judging the quality of the serum sample; if the quality of the serum sample is qualified, performing data processing on the serum sample; and outputting a result. According to the serum quality interpretation method provided by the invention, by combining a target detection method and a numerical regression method, an acquired image can be judged, a label, a blood clot, a serum region and a serum index can be distinguished at one time, and a re-acquisition signal and an adjustment amount can be output to a machine vision system for feedback. According to the prior information that the serum index must be a positive number, the exponential function is introduced into the network architecture to perform positive number constraint on the output result, so that the exponential prediction accuracy is improved while the differentiability of the network model is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of serum detection, and particularly relates to a deep learning serum quality interpretation method with feedback characteristics. Background Art

[0002] The timeliness, accuracy, and reliability of medical laboratory test results are of great guiding significance for clinicians to make clinical decisions. However, due to insufficient sample volume, incorrect label pasting, the presence of fibrin filaments in serum, and changes in the appearance of serum / plasma caused by lipemia, hemolysis, and jaundice in patient samples, it usually leads to delayed test reports, incorrect test results, and affects clinical decisions. Therefore, it is of extremely important significance to quickly and accurately detect the sample quality including serum index (SI) before analysis. SI is a description of the appearance of pre-analytical specimens in medical laboratories and an important reference index for clinicians to make medical decisions based on test results.

[0003] Currently, there is no accurate measurement standard or system for serum quality in clinical laboratory tests. The traditional serum quality detection methods in medical laboratories mainly include manual interpretation method and absorbance detection SI method. Sufficient sample volume is crucial for experimental analysis. If the sample volume is insufficient, it may lead to the inability to complete all necessary tests or incorrect test results; incorrect labels can affect the judgment of SI, clot inspection, or incorrect result reporting; the presence of clots in serum samples usually means improper sample handling or incorrect collection techniques, which may affect the accuracy of test results. Clots may interfere with instrument readings or sample contamination. Therefore, the inspection of serum quality is necessary before medical test analysis. However, at present, on the one hand, the currently widely used manual interpretation method can be intuitively understood as relying on the visual discrimination of laboratory workers. Long-term manual interpretation will lead to a decrease in work efficiency and judgment accuracy, and manual judgment often has problems such as strong subjectivity, poor repeatability, and high missed detection rate. Among them, even experienced laboratory workers have a consistency of only 0.573 for hemolysis judgment, 0.522 for lipemia, and only 0.457 for jaundice. On the other hand, the absorbance detection method using a biochemical analyzer to measure the absorbance value can semi-quantitatively give SI. Its advantages are good precision, high accuracy, simple operation, and more objective and accurate judgment of serum appearance. However, limited by the instrument, the detection results are not repeatable due to inconsistent standards among different manufacturers, and it cannot provide completely reliable results for clinicians. Moreover, due to steps such as dilution, it increases the cost consumption, severely limiting its wide application in clinics. Summary of the Invention

[0004] The purpose of the present invention is to provide a deep learning serum quality interpretation method with feedback characteristics to solve the problems in the prior art that long-term manual interpretation leads to reduced work efficiency and judgment accuracy, and manual judgment often has strong subjectivity, poor repeatability, and a high missed inspection rate.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: A deep learning serum quality interpretation method with feedback characteristics, comprising the following steps: Step S1, collecting image data of serum samples; Step S2, identifying the collected serum sample image data to judge the quality of the serum sample; Step S3, if the quality of the serum sample is qualified, then estimating the serum index of the serum sample. If it is unqualified due to incomplete collection or improper angle of the serum area, adjust the angle of the serum sample through a rotating mechanism, re-collect the serum sample image data, and then conduct quality judgment again; Step S4, outputting the result of serum quality interpretation.

[0006] According to the above technical solution, in step S2, judging the quality of the serum sample includes: judging whether there is a label in the serum sample image data, judging whether the serum area of the serum sample is qualified, and judging whether the serum sample contains blood clots; If there is a label in the serum sample image data, and the serum area of the serum sample is qualified and the serum sample does not contain blood clots, then the quality of the serum sample is considered qualified; If there is no label in the serum sample image data, or the serum area of the serum sample is unqualified, or the serum sample contains blood clots, then the quality of the serum sample is unqualified.

[0007] According to the above technical solution, the steps of judging whether the serum area of the serum sample is qualified include: Judging whether the serum area of the serum sample is less than 16×16 pixels. If it is not less than, then the serum area of the serum sample is qualified; if it is less than, then judge that the serum area of the serum sample is unqualified; If it is judged that the serum area of the serum sample is qualified, then through the target detection algorithm, remove the interference of the redundant area, obtain the accurate serum area, and based on the obtained accurate serum area, predict and identify the hemolysis index, jaundice index, and chylomicron index.

[0008] According to the above technical solution, the method of prediction and identification includes: extracting the characteristics of the hemolysis index, jaundice index, and chylomicron index from the obtained accurate serum area; Based on the extracted features, a serum index regression network model is established, and the prediction and identification of hemolysis index, jaundice index, and chylomicron index are completed according to the serum index regression network model.

[0009] According to the above technical solution, determining whether there is a label on the serum sample specifically is: Convert the collected serum sample image into a grayscale image, and then perform Gaussian filtering on the converted image to remove the noise in the image, smooth the image, and avoid the influence of noise on the subsequent detection results; Use the Canny edge detection algorithm to find the edge information of the objects in the image; perform morphological operations on the edge-detected image; Match the processed image with the template to find the region in the image that is most similar to the template, which is the position of the label.

[0010] According to the above technical solution, determining whether the serum sample contains a blood clot specifically is: Convert the collected serum sample image into a grayscale image, and perform denoising processing on the grayscale image using median filtering to remove the salt-and-pepper noise in the image and protect the edge information of the image; According to the difference in grayscale between the serum and the blood clot, select a threshold T to segment the grayscale image; Perform an erosion operation on the segmented grayscale image to remove the isolated noise points and the burrs on the edge in the image, making the boundary of the blood clot clearer and smoother; Perform a dilation operation on the eroded image to restore the approximate shape of the blood clot and connect some parts that are disconnected due to erosion; Judge whether there is a blood clot in the serum according to the processed image.

[0011] According to the above technical solution, in step S3, adjusting the angle of the serum sample through the rotation mechanism and re-collecting the serum sample image data specifically is: Step S301, output a re-collection signal and a rotation angle to the machine vision system according to the relative position relationship between the test tube and the serum area for re-collection to improve the recognition accuracy; Step S302, assume that the detected test tube bounding box is , where, is the center x coordinate, is the center y coordinate, is the width, is the height, and the bounding box of the serum area is , when and are both less than 16, it is necessary to output the rotation angle of the serum sample to the machine vision system, and the rotation angle is: Among them, is the equivalent focal length of the camera in the machine vision system; In step S303, after obtaining the serum region information, cut out the serum region, adjust the image size to 64×64, and send it into the serum index regression network model.

[0012] According to the above technical solution, the method for prediction by the serum index regression network model specifically includes: Initial feature extraction: Input the obtained accurate serum region image into the layer; 3 input channels are processed by 15 3×3 convolutional kernels with a stride of 1 and padding of 1 to extract preliminary feature map information; Downsampling: The feature map after ReLU processing enters the MaxPool2d(2,2) layer, and performs max pooling operation with a 2×2 pooling window, taking the maximum pixel value within the window as the output, reducing the size of the feature map, reducing the amount of data and computational complexity, while retaining the main features; Deep feature extraction: Multiple Block modules process the feature map in sequence; different Block modules extract features from different scales and levels through different parameter settings; Feature fusion: After the feature map undergoes a column expansion operation, the features of different branches are concatenated in the channel dimension through the Concat operation to fuse multi-scale and multi-level features; Fully connected layer mapping: The concatenated high-dimensional features first enter the Linear(2304,64) layer to map the 2304-dimensional feature vector to 64 dimensions, and then pass through the Linear(64,3) layer to further map the 64-dimensional features to a 3-dimensional vector. These 3 values respectively correspond to the predicted values of the hemolysis index, jaundice index, and chylomicron index; Output transformation: Perform exponential operation transformation on the values in the 3-dimensional vector through the Exp function to make the predicted values conform to the range or distribution characteristics of the actual index, and obtain the final prediction result.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method for judging serum quality. This method combines object detection and numerical regression methods, can judge the collected images, can discriminate labels, blood clots, serum regions and serum indices at one time, and can output a re-acquisition signal and adjustment amount to the machine vision system for feedback.

[0014] The present invention proposes a lightweight deep learning network model that can simultaneously regress three serum indices. This model can achieve an inference speed of more than 10fps on a general-purpose CPU and has strong practicability.

[0015] Based on the prior information that the serum index must be a positive number, an exponential function is introduced into the network architecture to impose a positive constraint on the output result, ensuring the differentiability of the network model while improving the accuracy of exponential prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the system structure diagram of the present invention; Figure 2 It is the detailed algorithm flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Embodiment 1 As Figure 2 shown, a deep learning serum quality interpretation method with feedback characteristics includes the following steps: Step S1, collect image data of the serum sample; Step S2, identify the collected image data of the serum sample to judge the quality of the serum sample; Step S3, if the quality of the serum sample is qualified, estimate the serum index of the serum sample. If the quality of the serum sample is unqualified, adjust the angle of the serum sample through the rotation mechanism, re-collect the image data of the serum sample, and then perform quality judgment; Step S4, output the result of serum quality interpretation.

[0019] The present invention proposes a serum quality interpretation method, which combines object detection and numerical regression methods, can judge the collected images, can discriminate labels, blood clots, serum regions and serum indexes at one time, and can output a re-collection signal and an adjustment amount to the machine vision system for feedback.

[0020] The present invention proposes a lightweight deep learning network model that can simultaneously regress three serum indexes. This model can achieve an inference speed of more than 10fps on a general-purpose CPU, and has strong practicability.

[0021] Based on the prior information that the serum index must be a positive number, an exponential function is introduced into the network architecture to impose a positive constraint on the output result, ensuring the differentiability of the network model while improving the accuracy of exponential prediction.

[0022] Embodiment 2 The specific implementation method of the present invention is: As Figure 1 shown, the core scheduling center of the entire system is the serum quality interpretation software deployed on the computer, including label integrity detection, clot detection, and serum index recognition. It drives the camera to collect images, and at the same time calls the detection algorithm to detect the serum area. If there are situations such as the serum area not being captured, the serum area being too small, or no label, it outputs instructions to control the imaging system to readjust and collect again. After ensuring that each frame of the image has serum sample information, serum index regression is performed, and finally the results are visually output on the software interface.

[0023] Among them, the image acquisition system is responsible for collecting the color images of serum samples to obtain the color information of serum samples, and mainly consists of three major parts: a uniform white light illumination system, an optical imaging lens, and an image sensor. To ensure that the serum area is effectively collected and avoid interference from the labels on the serum samples to image acquisition, a transparent area should be reserved during the production of serum samples.

[0024] When the serum indices are different, the colors of the areas framed by the images will show differences. The present invention proposes a two-stage recognition algorithm: in the first stage, the accurate serum area is obtained through the object detection algorithm to remove the interference of redundant areas; in the second stage, the serum area is cut out to predict and recognize the serum indices (haemolysis index (H), icterus index (I), and lipaemia index (L)). At the same time, based on the collected images, the labels and blood clots can also be detected to achieve multi-dimensional discrimination of serum quality. The detailed algorithm steps are as Figure 2 shown.

[0025] After the main software obtains the image data transmitted back by the image acquisition system, it calls the object detection algorithm to detect the serum area. The object detection algorithm used in the present invention is yolov5s. In terms of task definition, the detection targets of the present invention are four categories: test tube samples, serum areas, labels, and blood clots. Detecting the test tube and the serum area at the same time is to more accurately determine the quality of the serum area: one is to minimize the probability of false detection by judging the connection relationship between the test tube and the serum area; the other is to obtain the relative position of the serum area relative to the test tube, which is used to feedback to the machine vision system to adjust the imaging posture to obtain an image more conducive to recognition. The label and blood clot detections are incorporated into one network to complete the detection of label quality without increasing the detection time and resource consumption, and output whether there is a blood clot in the sample.

[0026] For identifying the serum index, it is necessary to ensure that the machine vision system captures sufficient serum areas. If no target exists, the output result is empty. If a target exists, the output result is a vector of N×7, where N is the number of detected targets. The seven elements of the vector include information such as the target coordinates, target category, and target confidence. If the output result is empty, it means that there is no serum area in the input image, and the serum index cannot be identified.

[0027] The label detection is specifically as follows: Convert the collected serum sample image into a grayscale image, and then perform Gaussian filtering on the converted image to remove the noise in the image, smooth the image, and avoid the influence of noise on the subsequent detection results; Assume the input color image is , convert it to a grayscale image, .

[0028] Perform Gaussian filtering on the grayscale image , specifically as follows: Obtain the filtered image ; the Gaussian filter kernel is , where is the standard deviation of the Gaussian distribution.

[0029] Use the Canny edge detection algorithm to find the edge information of the objects in the image; perform morphological operations on the edge-detected image.

[0030] Match the processed image with the template to find the region in the image that is most similar to the template, which is the position of the label.

[0031] The blood clot detection is specifically as follows: Convert the collected serum sample image into a grayscale image, and use median filtering to denoise the grayscale image, removing the salt-and-pepper noise in the image and protecting the edge information of the image; According to the difference in grayscale between the serum and the blood clot, select a threshold T to segment the grayscale image; generally, the grayscale value of the blood clot is relatively low. By setting the pixel points with grayscale values less than T as the foreground (blood clot) and those greater than T as the background (serum), a binary image is obtained.

[0032] Perform an erosion operation on the segmented grayscale image to remove the isolated noise points and the burrs on the edges in the image, making the boundary of the blood clot clearer and smoother; assume the structuring element is S, and the erosion operation is specifically as follows: Perform a dilation operation on the corroded image to restore the approximate shape of the blood clot and connect some parts that were disconnected due to corrosion. Specifically: Judge whether there is a blood clot in the serum based on the processed image.

[0033] Generally, when the input image is qualified, information about the serum area will be obtained. If the serum sample is placed incorrectly or there are extra false detections, 0 or more than 2 detection target information will be output. At this time, the software outputs a re-acquisition signal to the machine vision system; if the serum area is less than 16×16 pixels, it will seriously affect the recognition of the serum index. At this time, a re-acquisition signal and the rotation angle should be output to the machine vision system according to the relative position relationship between the test tube and the serum area for re-acquisition to improve the recognition accuracy. Let the detected bounding box of the test tube be (the center x coordinate, center y coordinate, width, and height respectively), and the bounding box of the serum area be When and are both less than 16, the rotation angle that the serum sample should be rotated needs to be output to the machine vision system. The rotation angle is: Among them, is the equivalent focal length of the camera of the machine vision system.

[0034] Through the above feedback acquisition, there must be only one serum area with a suitable area in the captured image. After obtaining the serum area information, cut out the serum area, adjust the image size to 64×64, and send it into the serum index recognition network.

[0035] In this embodiment, a specific implementation method of a serum index regression network model is provided.

[0036] The following are the step descriptions of the model for predicting and recognizing hemolysis, jaundice, and chylomicron indexes in the serum area: Step 1, initial feature extraction: Input the obtained accurate serum area image into the Conv2d(3, 15, 3, 1, 1) layer. Among them, 3 input channels (corresponding to the RGB channels of the image) are processed by 15 3×3 convolutional kernels, with a stride of 1 and a padding of 1, to extract preliminary feature information. Then, through the ReLU activation function, the part of the feature values that is less than 0 is changed to 0, and the part that is greater than 0 remains unchanged, introducing non-linearity into the model and enhancing the expression ability of the feature map.

[0037] Step 2, Downsampling: The feature map processed by ReLU enters the MaxPool2d(2,2) layer, where max pooling operation is performed with a pooling window of 2×2. The maximum pixel value within the window is taken as the output, reducing the size of the feature map, the amount of data, and the computational load, while retaining the main features.

[0038] Step 3, Deep Feature Extraction: Multiple Block modules process the feature map sequentially. Different Block modules extract features from different scales and levels through different parameter settings (such as the number of channels, the size of the convolutional kernel, the stride, etc.). For example, the change in the number of channels can capture features with different richness, and the size of the convolutional kernel and the stride determine the receptive field and granularity of feature extraction.

[0039] Step 4, Feature Fusion: After the feature maps of some paths undergo column expansion operations, the features of different branches are concatenated in the channel dimension through the Concat operation, fusing multi-scale and multi-level features, enabling the model to comprehensively utilize various feature information.

[0040] Step 5, Fully Connected Layer Mapping: The concatenated high-dimensional features first enter the Linear(2304,64) layer to map the 2304-dimensional feature vector to 64 dimensions, and then pass through the Linear(64,3) layer to further map the 64-dimensional features to a 3-dimensional vector. These 3 values respectively correspond to the predicted values of the hemolysis index, the jaundice index, and the chylomicron index.

[0041] Step 6, Output Transformation: The values in the 3-dimensional vector are transformed through the Exp function to make the predicted values conform to the range or distribution characteristics of the actual index, obtaining the final prediction result.

[0042] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0043] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A deep learning serum quality interpretation method with feedback characteristics, characterized in that: The following steps are involved: Step S1, collecting image data of serum samples; Step S2, identifying the collected serum sample image data to determine the quality of the serum sample; Step S3, if the quality of the serum sample is qualified, the serum index of the serum sample is estimated; if it is unqualified due to incomplete collection of the serum area or the angle, the angle of the serum sample is adjusted by the rotating mechanism, the serum sample image data is recollected, and the quality is judged again; Step S4, outputting the result of serum quality determination.

2. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: In step S2, judging the quality of the serum sample includes: judging whether there is a label in the serum sample image data, judging whether the serum area of ​​the serum sample is qualified, and judging whether the serum sample contains a blood clot; If there is a label in the serum sample image data, and the serum area of ​​the serum sample is qualified and the serum sample does not contain blood clots, the quality of the serum sample is considered to be qualified; If there is no label in the serum sample image data, or the serum region of the serum sample is unqualified, or the serum sample contains blood clots, the quality of the serum sample is unqualified.

3. A deep learning serum quality interpretation method with feedback characteristics according to claim 2, characterized in that: The steps of determining whether the serum area of ​​the serum sample is qualified include: Determine whether the serum area of ​​the serum sample is less than 16×16 pixels. If not, the serum area of ​​the serum sample is qualified; if less, the serum area of ​​the serum sample is judged to be unqualified; If the serum area of ​​the serum sample is judged to be qualified, the interference of redundant areas is removed through the target detection algorithm to obtain the accurate serum area, and the hemolysis index, icterus index and chylosing index are predicted and identified based on the obtained accurate serum area.

4. The deep learning serum quality interpretation method with feedback characteristics according to claim 3, characterized in that: The predictive identification method includes: extracting the features of hemolytic index, icteric index and chylosing index from the obtained accurate serum region; A serum index regression network model was established based on the extracted features, and the prediction and identification of hemolytic index, icterus index and chylosing index were completed based on the serum index regression network model.

5. The deep learning serum quality interpretation method with feedback characteristics according to claim 2, characterized in that: The specific method for determining whether a serum sample has a label is as follows: The collected serum sample image is converted into a grayscale image, and then the converted image is subjected to Gaussian filtering to remove noise in the image, smooth the image, and avoid the influence of noise on subsequent test results; Use the Canny edge detection algorithm to find the edge information of objects in the image; perform morphological operations on the image after edge detection; Match the processed image with the template and find the area in the image that is most similar to the template, which is the location of the label.

6. A deep learning serum quality interpretation method with feedback characteristics according to claim 2, characterized in that: The specific method for determining whether a serum sample contains a blood clot is as follows: The collected serum sample images are converted into grayscale images, and the grayscale images are denoised using median filtering to remove salt and pepper noise in the images and protect the edge information of the images; According to the difference in grayscale between serum and blood clot, a threshold T is selected to segment the grayscale image; The segmented grayscale image is corroded to remove isolated noise points and edge burrs in the image, making the boundary of the blood clot clearer and smoother. Perform a dilation operation on the eroded image to restore the approximate shape of the blood clot and connect some parts that were disconnected due to corrosion; The processed images are used to determine whether there is a blood clot in the serum.

7. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: In step S3, the serum sample angle is adjusted by the rotating mechanism, and the serum sample image data is recollected specifically as follows: Step S301, outputting a re-collection signal and a rotation angle to a machine vision system according to the relative position relationship between the test tube and the serum area for re-collection to improve recognition accuracy; Step S302, set the detected test tube bounding box to ,in, is the center x coordinate, is the center y coordinate, For width, is the height, and the bounding box of the serum area is ,when and When both are less than 16, the rotation angle of the serum sample needs to be output to the machine vision system. The rotation angle is: in, is the equivalent focal length of the camera in the machine vision system; Step S303, after obtaining the serum region information, the serum region is cut out, the image size is adjusted to 64×64, and sent to the serum index regression network model.

8. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: The method for predicting by serum index regression network model specifically includes: Initial feature extraction: The obtained accurate serum region image is input into Layer; the three input channels are processed by 15 3×3 convolution kernels with a step size of 1 and a padding of 1 to extract preliminary feature map information; Downsampling: The feature map processed by ReLU enters the MaxPool2d(2,2) layer, and the maximum pooling operation is performed with a 2×2 pooling window. The maximum pixel value in the window is taken as the output, and the size of the feature map is reduced, reducing the amount of data and calculation while retaining the main features; Deep feature extraction: Multiple Block modules process feature maps in sequence; different Block modules extract features from different scales and levels through different parameter settings; Feature fusion: After the feature map is expanded, the features of different branches are concatenated in the channel dimension through the Concat operation to fuse multi-scale and multi-level features; Fully connected layer mapping: The concatenated high-dimensional features first enter the Linear (2304, 64) layer to map the 2304-dimensional feature vector to 64-dimensional, and then pass through the Linear (64, 3) layer to further map the 64-dimensional features to a 3-dimensional vector. These three values ​​correspond to the predicted values ​​of the hemolytic index, icteric index, and chylomicron index, respectively. Output transformation: The Exp function is used to perform exponential transformation on the values ​​in the 3D vector so that the predicted value conforms to the range or distribution characteristics of the actual exponent to obtain the final prediction result.

Images