A deep learning serum quality interpretation method with feedback characteristics

Abstract

The present invention discloses a deep learning serum quality interpretation method with feedback characteristics, comprising the following steps: collecting image data of serum samples; identifying the collected serum sample image data to determine the quality of the serum samples; if the quality of the serum samples is qualified, processing the serum samples; and outputting the results. The present invention proposes a serum quality interpretation method, which can judge the collected images by combining target detection and numerical regression methods, and can identify labels, blood clots, serum areas and serum indexes at one time, and can output re-collection signals and adjustment amounts to the machine vision system for feedback. Based on the prior information that the serum index must be a positive number, the present invention introduces an exponential function in the network architecture to impose a positive constraint on the output result, thereby ensuring that the network model is differentiable while improving the accuracy of the index prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of serum detection, and specifically provides a deep learning serum quality interpretation method with feedback characteristics. Background Art

[0002] The timeliness, accuracy, and reliability of medical laboratory test results are crucial for clinicians' clinical decision-making. However, insufficient sample size, incorrect labeling, the presence of fibrin in serum, and altered serum / plasma appearance due to lipemia, hemolysis, and icterus often lead to delayed test reporting and erroneous results, impacting clinical decision-making. Therefore, rapid and accurate preanalysis testing of sample quality, including serum indices (SIs), is crucial. The SI, a description of the specimen's appearance before analysis, serves as a crucial reference for clinicians to compare test results and make informed medical decisions.

[0003] Currently, there is no accurate standard or system for measuring serum quality in clinical laboratory testing. Traditional methods for testing serum quality in medical laboratories primarily include manual interpretation and absorbance-based (SI) analysis. Adequate sample size is crucial for laboratory analysis. Insufficient sample size can result in inability to complete all required tests or inaccurate test results. Labeling errors can affect SI interpretation, clot detection, or misreporting of results. The presence of clots in serum samples often indicates improper sample handling or incorrect collection technique, which can affect the accuracy of test results. Clots can interfere with instrument readings, leading to erroneous readings or sample contamination. Therefore, serum quality testing is essential before medical laboratory analysis. However, the currently widely used manual interpretation method, which intuitively relies on visual judgment by laboratory workers, reduces efficiency and accuracy. Manual interpretation is often subject to high subjectivity, poor reproducibility, and a high rate of missed detections. Even experienced laboratory workers have a consistency of only 0.573 for hemolysis, 0.522 for lipemia, and 0.457 for icterus. On the other hand, absorbance detection, which uses a biochemical analyzer to measure absorbance, can semi-quantitatively determine the SI. Its advantages include high precision, accuracy, simplicity, and objective and accurate assessment of serum appearance. However, due to instrument limitations and inconsistent standards across manufacturers, test results are not reproducible and cannot provide clinicians with completely reliable results. Furthermore, the need for dilution and other steps increases costs, severely limiting its widespread clinical application. Summary of the Invention

[0004] The purpose of the present invention is to provide a deep learning serum quality interpretation method with feedback characteristics to solve the problems in the prior art proposed in the background technology, in which long-term manual interpretation leads to reduced work efficiency and judgment accuracy, and manual judgment often has strong subjectivity, poor repeatability, and high missed detection rate.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A deep learning serum quality interpretation method with feedback characteristics comprises the following steps:

[0007] Step S1, collecting image data of the serum sample;

[0008] Step S2, identifying the collected serum sample image data to determine the quality of the serum sample;

[0009] Step S3: If the quality of the serum sample is qualified, the serum index of the serum sample is estimated. If it is unqualified due to incomplete serum area acquisition or angle, the serum sample angle is adjusted by the rotation mechanism, the serum sample image data is re-acquired, and the quality judgment is performed again;

[0010] Step S4: output the result of serum quality determination.

[0011] According to the above technical solution, in step S2, determining the quality of the serum sample includes: determining whether there is a label in the serum sample image data, determining whether the serum area of the serum sample is qualified, and determining whether the serum sample contains blood clots;

[0012] If the label exists in the serum sample image data, the serum area of the serum sample is qualified, and the serum sample does not contain blood clots, then the quality of the serum sample is considered qualified;

[0013] If the label does not exist in the serum sample image data, or the serum region of the serum sample is unqualified, or the serum sample contains blood clots, the quality of the serum sample is unqualified.

[0014] According to the above technical solution, the steps of determining whether the serum region of the serum sample is qualified include:

[0015] Determine whether the serum area of the serum sample is less than 16×16 pixels. If not, the serum area of the serum sample is qualified; if less, the serum area of the serum sample is judged to be unqualified.

[0016] If the serum area of the serum sample is judged to be qualified, the interference of redundant areas is removed through the target detection algorithm to obtain the accurate serum area, and the hemolysis index, icterus index and chylosing index are predicted and identified based on the obtained accurate serum area.

[0017] According to the above technical solution, the prediction and identification method includes: extracting the features of hemolysis index, icterus index and chylosing index from the obtained accurate serum region;

[0018] A serum index regression network model was established based on the extracted features, and the prediction and identification of hemolysis index, icterus index and chylosing index were completed based on the serum index regression network model.

[0019] According to the above technical solution, the determination of whether a serum sample has a label is specifically as follows:

[0020] The collected serum sample image is converted into a grayscale image, and then the converted image is subjected to Gaussian filtering to remove noise in the image and smooth the image to prevent the noise from affecting subsequent test results;

[0021] Use the Canny edge detection algorithm to find the edge information of objects in the image; perform morphological operations on the image after edge detection;

[0022] Match the processed image with the template and find the area in the image that is most similar to the template, which is the location of the label.

[0023] According to the above technical solution, determining whether a serum sample contains a blood clot is specifically as follows:

[0024] The collected serum sample images were converted into grayscale images, and the grayscale images were denoised using median filtering to remove salt and pepper noise and protect the edge information of the images.

[0025] According to the grayscale difference between serum and blood clot, a threshold T is selected to segment the grayscale image;

[0026] Performing an erosion operation on the segmented grayscale image to remove isolated noise points and edge burrs in the image, making the boundary of the blood clot clearer and smoother;

[0027] Perform a dilation operation on the eroded image to restore the approximate shape of the blood clot and connect some parts that were disconnected due to corrosion;

[0028] The processed images are used to determine whether there is a blood clot in the serum.

[0029] According to the above technical solution, in step S3, the serum sample angle is adjusted by the rotation mechanism, and the serum sample image data is recollected specifically as follows:

[0030] Step S301: Outputting a re-collection signal and a rotation angle to a machine vision system based on the relative positional relationship between the test tube and the serum area for re-collection to improve recognition accuracy;

[0031] Step S302: Set the detected test tube bounding box as ,in, is the center x coordinate, is the center y coordinate, For width, is the height, and the bounding box of the serum area is ,when and When both are less than 16, the rotation angle of the serum sample needs to be output to the machine vision system. The rotation angle is:

[0032]

[0033] in, is the equivalent focal length of the machine vision system camera;

[0034] Step S303: After obtaining the serum region information, the serum region is cut out, the image size is adjusted to 64×64, and then fed into the serum index regression network model.

[0035] According to the above technical solution, the method for prediction using the serum index regression network model specifically includes:

[0036] Initial feature extraction: The obtained accurate serum region image is input to the layer; the three input channels are processed by 15 3×3 convolution kernels with a step size of 1 and a padding of 1 to extract preliminary feature map information;

[0037] Downsampling: The feature map processed by ReLU enters the MaxPool2d(2,2) layer, which performs a maximum pooling operation with a 2×2 pooling window, taking the maximum pixel value in the window as the output, reducing the size of the feature map, reducing the amount of data and calculation, while retaining the main features;

[0038] Deep feature extraction: Multiple Block modules process feature maps in sequence; different Block modules extract features from different scales and levels through different parameter settings;

[0039] Feature fusion: After the feature map undergoes column expansion, the features of different branches are concatenated in the channel dimension through the Concat operation to fuse multi-scale and multi-level features.

[0040] Fully connected layer mapping: The concatenated high-dimensional features first enter the Linear(2304,64) layer, mapping the 2304-dimensional feature vector to a 64-dimensional one. Then, the Linear(64,3) layer further maps the 64-dimensional feature vector to a 3-dimensional vector. These three values correspond to the predicted values of the hemolytic index, icteric index, and chylomicron index, respectively.

[0041] Output transformation: The Exp function is used to perform exponential transformation on the values in the 3D vector so that the predicted value conforms to the range or distribution characteristics of the actual index and obtains the final prediction result.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention proposes a method for serum quality interpretation. This method combines target detection and numerical regression methods to judge the collected images. It can identify labels, blood clots, serum areas and serum indexes at one time, and can output re-collection signals and adjustment amounts to the machine vision system for feedback.

[0044] The present invention proposes a lightweight deep learning network model that can simultaneously regress three serum indices. The model can achieve an inference speed of more than 10fps on a general-purpose CPU and is highly practical.

[0045] Based on the prior information that the serum index must be a positive number, the present invention introduces an exponential function into the network architecture to constrain the output result to a positive number, thereby ensuring the differentiability of the network model and improving the accuracy of index prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a system structure diagram of the present invention;

[0047] Figure 2 Detailed algorithm flow chart of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 2 As shown, a deep learning serum quality interpretation method with feedback characteristics includes the following steps:

[0051] Step S1, collecting image data of the serum sample;

[0052] Step S2, identifying the collected serum sample image data to determine the quality of the serum sample;

[0053] Step S3: If the quality of the serum sample is qualified, the serum index of the serum sample is estimated; if the quality of the serum sample is unqualified, the angle of the serum sample is adjusted by the rotation mechanism, the image data of the serum sample is recollected, and the quality judgment is performed again;

[0054] Step S4: output the result of serum quality determination.

[0055] The present invention proposes a method for serum quality interpretation. This method combines target detection and numerical regression methods to judge the collected images. It can identify labels, blood clots, serum areas and serum indexes at one time, and can output re-collection signals and adjustment amounts to the machine vision system for feedback.

[0056] The present invention proposes a lightweight deep learning network model that can simultaneously regress three serum indices. The model can achieve an inference speed of more than 10fps on a general-purpose CPU and is highly practical.

[0057] Based on the prior information that the serum index must be a positive number, the present invention introduces an exponential function into the network architecture to constrain the output result to a positive number, thereby ensuring the differentiability of the network model and improving the accuracy of index prediction.

[0058] Example 2

[0059] The specific implementation method of the present invention is:

[0060] like Figure 1 As shown, the core scheduling hub of the entire system is the serum quality interpretation software deployed on a computer, which includes label integrity detection, clot detection, and serum index identification. It drives the camera to capture images and simultaneously calls the detection algorithm to detect serum regions. If the serum region is not captured, the serum region is too small, or there is no label, the output command controls the imaging system to readjust and recapture, ensuring that each frame contains serum sample information. After that, serum index regression is performed, and the results are finally visualized on the software interface.

[0061] The image acquisition system is responsible for capturing color images of serum samples to obtain color information. It consists of three main components: a uniform white light illumination system, an optical imaging lens, and an image sensor. To ensure that the serum area is effectively captured and to prevent interference from labels on the serum sample, a transparent area should be retained during the serum sample preparation process.

[0062] When the serum index is different, the color of the area within the image frame will show differences. The present invention proposes a two-stage recognition algorithm: the first stage uses the target detection algorithm to obtain the accurate serum area and remove the interference of redundant areas; the second stage cuts out the serum area and predicts and identifies the serum index (hemolysis index (Haemolysis, H), icterus index (Icterus, I) and chylosing index (Lipaemia, L)). At the same time, based on the collected images, labels and blood clots can also be detected to achieve multi-dimensional judgment of serum quality. The detailed algorithm steps are as follows: Figure 2 shown.

[0063] After receiving image data from the image acquisition system, the main software uses a target detection algorithm to detect the serum region. The target detection algorithm used in this invention is yolov5s. In terms of task definition, the present invention detects four types of targets: test tube samples, serum regions, labels, and blood clots. Simultaneous detection of test tubes and serum regions is intended to more accurately determine the quality of the serum region. First, by determining the connectivity between the test tubes and serum regions, the probability of false detection can be minimized. Second, the relative position of the serum region relative to the test tube can be determined, which is used to provide feedback to the machine vision system to adjust the imaging posture to obtain an image that is more conducive to recognition. Label and blood clot detection are integrated into a single network, completing label quality testing without increasing detection time and resource consumption, and outputting whether there is a blood clot in the sample.

[0064] To identify the serum index, it's crucial to ensure that the machine vision system captures sufficient serum regions. If no target is present, the output is empty. If a target is present, the output is an N x 7 vector, where N is the number of detected targets. The seven elements of this vector contain information such as target coordinates, target category, and target confidence. If the output is empty, it indicates that no serum region exists in the input image, making serum index identification impossible.

[0065] The specific tag detection is:

[0066] The collected serum sample image is converted into a grayscale image, and then the converted image is subjected to Gaussian filtering to remove noise in the image and smooth the image to prevent the noise from affecting subsequent test results;

[0067] Assume that the input color image is , converted to grayscale image, .

[0068] For grayscale images Perform Gaussian filtering, specifically:

[0069]

[0070] Get the filtered image ; Gaussian filter kernel is ,in is the standard deviation of the Gaussian distribution.

[0071] Use the Canny edge detection algorithm to find the edge information of objects in the image; perform morphological operations on the image after edge detection.

[0072] Match the processed image with the template and find the area in the image that is most similar to the template, which is the location of the label.

[0073] Blood clot detection is as follows:

[0074] The collected serum sample images were converted into grayscale images, and the grayscale images were denoised using median filtering to remove salt and pepper noise and protect the edge information of the images.

[0075] According to the difference in grayscale between serum and blood clots, a threshold T is selected to segment the grayscale image. Generally speaking, the grayscale value of blood clots is relatively low. By setting the pixels with grayscale values less than T as the foreground (blood clots) and the pixels with grayscale values greater than T as the background (serum), a binary image is obtained. .

[0076] The segmented grayscale image is eroded to remove isolated noise points and edge burrs in the image, making the boundary of the blood clot clearer and smoother. Let the structural element be S, and the specific erosion operation is:

[0077]

[0078] Perform a dilation operation on the eroded image to restore the approximate shape of the blood clot and connect some parts that were disconnected due to corrosion; specifically:

[0079]

[0080] The processed images are used to determine whether there is a blood clot in the serum.

[0081] In general, if the input image is qualified, the information of a serum area will be obtained. If the serum sample is not placed correctly or there are redundant false detections, 0 or more detection target information will be output. At this time, the software outputs a re-acquisition signal to the machine vision system; if the serum area is less than 16×16 pixels, it will seriously affect the serum index recognition. At this time, the re-acquisition signal and rotation angle should be output to the machine vision system according to the relative position relationship between the test tube and the serum area for re-acquisition to improve the recognition accuracy. Let the detected test tube bounding box be (center x coordinate, center y coordinate, width, height respectively), the bounding box of the serum area is ,when and When both are less than 16, the rotation angle of the serum sample needs to be output to the machine vision system. The rotation angle is:

[0082]

[0083] in, is the equivalent focal length of the machine vision system camera.

[0084] Through the above feedback collection, the captured image is guaranteed to contain only one serum region of appropriate size. After obtaining the serum region information, the serum region is cut out, the image size is adjusted to 64×64, and then fed into the serum index recognition network.

[0085] In this embodiment, a specific implementation method of a serum index regression network model is provided.

[0086] The following is a description of the steps used by the model to predict and identify hemolysis, icterus, and chylosing indices in the serum region:

[0087] Step 1: Initial Feature Extraction: The obtained accurate serum region image is input into the Conv2d(3,15,3,1,1) layer. The three input channels (corresponding to the image's RGB channels) are processed through 15 3×3 convolution kernels with a stride of 1 and padding of 1 to extract preliminary feature information. Next, a ReLU activation function is used to convert eigenvalues less than 0 to 0, while those greater than 0 remain unchanged. This introduces nonlinearity into the model and enhances the expressiveness of the feature map.

[0088] Step 2, downsampling: The feature map processed by ReLU enters the MaxPool2d(2,2) layer, and the maximum pooling operation is performed with a 2×2 pooling window. The maximum pixel value in the window is taken as the output, which reduces the size of the feature map, reduces the amount of data and calculation, and retains the main features.

[0089] Step 3: Deep Feature Extraction: Multiple block modules sequentially process the feature maps. Different block modules extract features at different scales and levels by varying parameter settings (such as the number of channels, convolution kernel size, and stride). For example, varying the number of channels can capture features of varying richness, while the convolution kernel size and stride determine the receptive field and granularity of feature extraction.

[0090] Step 4, feature fusion: After the feature maps of some paths are expanded by columns, the features of different branches are concatenated in the channel dimension through the Concat operation, fusing multi-scale and multi-level features so that the model can comprehensively utilize various feature information.

[0091] Step 5, fully connected layer mapping: The concatenated high-dimensional features first enter the Linear (2304, 64) layer to map the 2304-dimensional feature vector to 64 dimensions. Then, they pass through the Linear (64, 3) layer to further map the 64-dimensional features to a 3-dimensional vector. These three values correspond to the predicted values of the hemolytic index, icteric index, and chylomicron index, respectively.

[0092] Step 6, output transformation: perform exponential transformation on the values in the 3D vector through the Exp function, so that the predicted value conforms to the range or distribution characteristics of the actual index, and obtains the final prediction result.

[0093] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0094] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A deep learning serum quality interpretation method with feedback characteristics, characterized by: The following steps are involved: Step S1, collecting image data of the serum sample; Step S2, identifying the collected serum sample image data to determine the quality of the serum sample; Judging the quality of the serum sample includes: judging whether there is a label in the serum sample image data, judging whether the serum area of the serum sample is qualified, and judging whether the serum sample contains blood clots; If the label exists in the serum sample image data, the serum area of the serum sample is qualified, and the serum sample does not contain blood clots, then the quality of the serum sample is considered qualified; If there is no label in the serum sample image data, or the serum area of the serum sample is unqualified, or the serum sample contains blood clots, the quality of the serum sample is unqualified; The steps for determining whether the serum area of the serum sample is qualified include: Determine whether the serum area of the serum sample is less than 16×16 pixels. If not, the serum area of the serum sample is qualified; if less, the serum area of the serum sample is judged to be unqualified. If the serum region of the serum sample is determined to be qualified, the target detection algorithm is used to remove the interference of redundant regions to obtain the accurate serum region. Based on the obtained accurate serum region, the hemolysis index, icterus index and chylosing index are predicted and identified. The predictive identification method includes: extracting the features of hemolysis index, icterus index and chylosing index from the obtained accurate serum region; A serum index regression network model was established based on the extracted features, and the hemolysis index, icterus index, and chylosing index were predicted and identified based on the serum index regression network model. The method for predicting by serum index regression network model specifically includes: Initial feature extraction: The obtained accurate serum region image is input into Layer; the three input channels are processed by 15 3×3 convolution kernels with a step size of 1 and a padding of 1 to extract preliminary feature map information; Downsampling: The feature map processed by ReLU enters the MaxPool2d(2,2) layer, which performs a maximum pooling operation with a 2×2 pooling window, taking the maximum pixel value in the window as the output, reducing the size of the feature map, reducing the amount of data and calculation, while retaining the main features; Deep feature extraction: Multiple Block modules process feature maps in sequence; different Block modules extract features from different scales and levels through different parameter settings; Feature fusion: After the feature map undergoes column expansion, the features of different branches are concatenated in the channel dimension through the Concat operation to fuse multi-scale and multi-level features. Fully connected layer mapping: The concatenated high-dimensional features first enter the Linear(2304,64) layer, mapping the 2304-dimensional feature vector to a 64-dimensional one. Then, the Linear(64,3) layer further maps the 64-dimensional feature vector to a 3-dimensional vector. These three values correspond to the predicted values of the hemolytic index, icteric index, and chylomicron index, respectively. Output transformation: The Exp function is used to perform exponential transformation on the values in the 3D vector so that the predicted value conforms to the range or distribution characteristics of the actual index, and the final prediction result is obtained; Step S3: If the quality of the serum sample is qualified, the serum index of the serum sample is estimated. If it is unqualified due to incomplete serum area acquisition or angle, the serum sample angle is adjusted by the rotation mechanism, the serum sample image data is re-acquired, and the quality judgment is performed again; Step S4: output the result of serum quality determination.

2. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: The specific method for determining whether a serum sample has a label is as follows: The collected serum sample image is converted into a grayscale image, and then the converted image is subjected to Gaussian filtering to remove noise in the image and smooth the image to prevent the noise from affecting subsequent test results; Use the Canny edge detection algorithm to find the edge information of objects in the image; perform morphological operations on the image after edge detection; Match the processed image with the template and find the area in the image that is most similar to the template, which is the location of the label.

3. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: The specific methods for determining whether a serum sample contains a blood clot are: The collected serum sample images were converted into grayscale images, and the grayscale images were denoised using median filtering to remove salt and pepper noise and protect the edge information of the images. According to the grayscale difference between serum and blood clot, a threshold T is selected to segment the grayscale image; Performing an erosion operation on the segmented grayscale image to remove isolated noise points and edge burrs in the image, making the boundary of the blood clot clearer and smoother; Perform a dilation operation on the eroded image to restore the approximate shape of the blood clot and connect some parts that were disconnected due to corrosion; The processed images are used to determine whether there is a blood clot in the serum.

4. The deep learning serum quality interpretation method with feedback characteristics according to claim 1, characterized in that: In step S3, the serum sample angle is adjusted by the rotation mechanism, and the serum sample image data is recollected as follows: Step S301: Outputting a re-collection signal and a rotation angle to a machine vision system based on the relative positional relationship between the test tube and the serum area for re-collection to improve recognition accuracy; Step S302: Set the detected test tube bounding box as ,in, is the center x coordinate, is the center y coordinate, For width, is the height, and the bounding box of the serum area is ,when and When both are less than 16, the rotation angle of the serum sample needs to be output to the machine vision system. The rotation angle is: in, is the equivalent focal length of the machine vision system camera; Step S303: After obtaining the serum region information, the serum region is cut out, the image size is adjusted to 64×64, and then fed into the serum index regression network model.

Images