Deep Learning-Based Immunoblot Image Recognition Method and System

Abstract

The present invention discloses a method and system for immunoblot image recognition based on deep learning, which relates to the fields of bioinformatics and computer vision, and solves the technical problems that it is difficult to extract various relevant features of proteins from images through image recognition technology, and it is also difficult to correct the predicted values obtained by predicting the protein expression level with multiple protein feature data for a deep learning model. The method includes the following steps: capturing a band image on a membrane using a high-resolution scanner; recognizing the preprocessed image through image recognition technology to extract features related to immunoblotting; predicting the feature data through a corresponding trained deep learning model to obtain predicted values of area, gray-scale difference degree, and relative brightness; adding three predicted correction values with corresponding weights to obtain a comprehensive predicted correction value; and judging the protein expression level according to the comparison between the comprehensive expression evaluation value and a preset threshold.

Description

Technical Field

[0001] The present invention belongs to the fields of bioinformatics and computer vision, and particularly relates to a method and system for immunoblot image recognition based on deep learning. Background Art

[0002] With the continuous development of artificial intelligence technology, especially the breakthrough of deep learning in the field of image processing, it provides the possibility for automated and high-precision immunoblot image analysis. In biomedical research, the immunoblot technique is a commonly used protein analysis method for detecting the presence of specific antigens. The method and system for immunoblot image recognition based on deep learning extract various relevant features of proteins from images through image recognition technology, respectively obtain the measured values of relevant feature numerical values, correct the predicted values predicted in the corresponding deep learning model, and obtain a comprehensive expression evaluation value by combining the corrected value and the standard optical density value. The expression level of the protein is judged according to the result of comparing the evaluation value with the preset threshold.

[0003] Although the existing methods and systems for immunoblot image recognition based on deep learning have achieved protein detection to a certain extent, traditional methods are difficult to extract various relevant features of proteins from images through image recognition technology, and it is also difficult to correct the predicted values obtained by predicting the protein expression level in the corresponding deep learning model with the measured values of various protein feature data. There is a lack of judging the expression level of proteins based on the comparison results, resulting in limited accuracy and repeatability of the results. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; for this purpose, the present invention proposes a method and system for immunoblot image recognition based on deep learning to solve the following technical problems:

[0005] Although the existing methods and systems for immunoblot image recognition based on deep learning have achieved protein detection to a certain extent, traditional methods are difficult to extract various relevant features of proteins from images through image recognition technology, and it is also difficult to correct the predicted values obtained by predicting the protein expression level in the corresponding deep learning model with the measured values of various protein feature data. There is a lack of judging the expression level of proteins based on the comparison results.

[0006] To solve the above problems, the first aspect of the present invention provides a method for immunoblot image recognition based on deep learning, including the following steps:

[0007] S1: Use a high-resolution scanner to capture the band image on the membrane, convert the signal where the immune reaction occurs into a digital image, and process the collected image using image analysis technology, including background subtraction, band recognition, and optical density measurement;

[0008] S2: Identify the preprocessed image through image recognition technology, extract the features related to immunoblotting, including the area, color, and gray value of the bands, and obtain the area value, gray difference degree, and average relative brightness respectively;

[0009] S3: Construct corresponding deep learning models according to the corresponding features and train them respectively. Use the trained models to predict new immunoblot images, obtain the corresponding predicted values of area, gray difference degree, and relative brightness, and standardize them;

[0010] S4: Denote the difference between the measured value obtained through image recognition technology and the predicted value of the model as the prediction offset value. Obtain the average prediction offset according to historical data. Combine the predicted value and the average prediction offset to obtain the prediction correction value. Assign corresponding weights to the three prediction correction values and add them up to obtain the comprehensive prediction correction value;

[0011] S5: Obtain the comprehensive expression evaluation value by combining the standard optical density value of the band and the comprehensive prediction correction value. Compare the comprehensive expression evaluation value with the preset threshold, and judge whether the expression level of the protein is normal according to the comparison result;

[0012] Among them, step S5 includes the following steps:

[0013] Calculation formula for the comprehensive expression evaluation value:

[0014]

[0015] Among them, M is the comprehensive feature value, L standard is the standard optical density value of the band, Z prediction is the comprehensive prediction correction value, and w1 and w2 are the corresponding weight coefficients respectively;

[0016]

[0017] Among them, P is the comprehensive expression evaluation value, M is the comprehensive feature value, and e is a constant in this calculation formula.

[0018] As a further solution of the present invention: The said step S1 includes the following steps:

[0019] Capture the band image on the membrane using a high-resolution scanner, convert the signal where the immune reaction occurs into a digital image, save the captured digital image, set it to the JPEG image file format, process the obtained image using image analysis software, reduce the background noise of the image through the background subtraction function, the band recognition function automatically recognizes the edges of each band, and the optical density measurement function measures the optical density of each recognized band, record the optical density value of each band, and quantitatively analyze the expression level of the target protein according to the optical density value.

[0020] As a further solution of the present invention: Step S2 includes the following steps:

[0021] Calculate the gradients in the horizontal and vertical directions of each pixel in the image using the Sobel operator. After obtaining the gradient image, use the non-maximum suppression technique to check each pixel and the adjacent gradient magnitudes, and retain the local maximum as the candidate edge points. Obtain the edge points by using two thresholds in the Canny algorithm. Analyze the connectivity of the edge points, and use the findContours function in the OpenCV open-source computer vision library to identify the closed contours in the image. The closed contours are used as the strip regions, and the area of the strip region enclosed by the contours is obtained using the polygon formula;

[0022] Formula for calculating the area of the strip region:

[0023]

[0024] Where is the area of the strip region in the image, ( , ), ( , )... ( , ) are the coordinate points on the contour edge;

[0025] Convert the image to a grayscale image through the OpenCV library. For each contour extracted from the image, initialize a variable for storing the sum of RGB values and a variable for counting the number of pixels. Traverse each pixel within each contour, and add the RGB of the pixel to the corresponding sum variable, and update the pixel number counter. Obtain the average value of the grayscale image of the region through the weighted average of the three RGB components, and compare the grayscale value of a pixel point in the image with the average value of the grayscale image to obtain the grayscale difference degree;

[0026] Formula for calculating the grayscale difference degree:

[0027]

[0028] Where is the grayscale difference degree, is the grayscale value of the th pixel point in the image, is the total number of pixels in the image, is the sum of the red channel values of all pixels in the image, is the sum of the green channel values of all pixels in the image, is the sum of the blue channel values of all pixels in the image, is the number of pixels in the image;

[0029] Smooth the image. In the grayscale image of the cell image, use the rectangular tool in the image analysis software to move the tool pointer to the position of the target band, draw a rectangular area covering the target band, use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this area of the image, and record the obtained grayscale values. Set the numerical value of the grayscale value to be between 0 - 255, and classify the grayscale values according to the color brightness. The grayscale value between 0 - 85 represents the pixel area of the low brightness tone level in the image, the grayscale value between 86 - 170 represents the pixel area of the brightness tone level in the image, and the grayscale value between 171 - 255 represents the pixel area of the high brightness tone level in the image. Obtain the relative brightness average value according to the grayscale value and the background grayscale value;

[0030] Formula for calculating the relative brightness average value:

[0031]

[0032] Among them, is the relative brightness average value, is the grayscale value of pixel point is the grayscale value of pixel point is the background grayscale value of pixel point is the total number of pixel points. As a further solution of the present invention: Measure the optical density of each identified band and record the optical density value of each band, including the following steps:

[0033] For each band, find the center point of the band as the band center point, draw a circle with three - quarters of the total length of the band as the radius of the circle, divide it into equal sectors, find four background points in the four directions of up, down, left, and right of the center point of the band in the background area without the band, record the background points as the background center points, use one - tenth of the total length of the band as the radius to obtain the background area of the area without the band, measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the band, measure the optical density of each identified band, record the optical density value of each band, and perform background subtraction by calculating the optical density ratio of the target protein to the internal reference protein using the internal reference protein as a reference to obtain the formula for the standard optical density value of the band;

[0034] Formula for the standard optical density value of the band:

[0035] Among them,

[0036]

[0037] is the standard optical density value of the band, ​is the light intensity value of the i-th pixel within the strip region, is the total number of pixels within the strip region, is the area of the strip region in the image, is the optical density value of the background region to the left of the strip dot, is the optical density value of the background region above the strip dot, is the optical density value of the background region to the right of the strip dot, is the optical density value of the background region below the strip dot, is the area value of the background region, is the optical density value of the internal reference protein.

[0038] As a further solution of the present invention: the step S3 includes the following steps:

[0039] From the acquired image, using the polygon tool in the image annotation software, draw the boundary of each protein strip along the edge of each protein strip, calculate the number of pixels inside each drawn boundary strip to obtain the area of the strip, record the areas of each obtained strip and summarize them into a data set, including the identification of the image, the identification of the strip, and the strip area, extract the area feature as the input for training the deep learning model, train the model, and use the trained model to predict the area value of a new strip, and the model outputs the predicted strip area value;

[0040] From the acquired image, use the strip recognition function in the image analysis tool to automatically identify the positions of the protein strips, segment each strip, extract it separately from the image, use the image analysis tool to extract the color value of each segmented protein strip, convert the color value to a grayscale value by the weighted average method, record the grayscale value of each strip and mark the corresponding protein strip label and set the label, use the grayscale value of each protein strip as a sample feature vector, and the corresponding label as the target value of the sample to summarize into a data set, use the data set as the input of the model to train the model, and use the trained model to predict the grayscale difference degree of a new strip, and the model outputs the predicted strip grayscale difference degree;

[0041] For the protein strips obtained from each image, calculate the relative brightness value of the strip in the corresponding image through the relative brightness average calculation formula and perform annotation, summarize the annotated relative brightness value and the corresponding image into a data set, use it as the input for training the deep learning model, and use the trained model to predict the relative brightness value of a new strip, and the model outputs the predicted strip relative brightness value;

[0042] Normalize the area value, grayscale difference degree, and relative brightness value predicted in the model respectively.

[0043] As a further solution of the present invention: Step S4 includes the following steps:

[0044]

[0045] Wherein, is the comprehensive prediction correction value, is the strip area value enclosed by the contour in the th image, is the strip area value predicted by the model, is the th strip area value predicted by the model, is the th average relative brightness value, is the strip relative brightness value predicted by the model, is the th strip relative brightness value predicted by the model, is the th gray scale difference degree, is the gray scale difference degree predicted by the model, is the th gray scale difference degree predicted by the model, 、 and are the corresponding weight coefficients respectively.

[0046] As a further solution of the present invention: Comparing the comprehensive expression evaluation value with a preset threshold, and judging whether the expression level of the protein is normal according to the comparison result includes the following steps:

[0047] If the preset threshold > the comprehensive expression evaluation value, then judge that the expression level of the protein is abnormal;

[0048] If the preset threshold ≤ the comprehensive expression evaluation value, then judge that the expression level of the protein is normal.

[0049] The present invention also provides an immunoblot image recognition system based on deep learning, including the following modules:

[0050] Protein image analysis module: Use a high-resolution scanner to capture the strip image on the membrane, convert the signal where the immune reaction occurs into a digital image, and use image analysis technology to process the collected image, including background subtraction, strip recognition, and optical density measurement;

[0051] Image recognition feature extraction module: Recognize the preprocessed image through image recognition technology, extract the features related to immunoblotting, including the area, color, and gray scale value of the strip, and obtain the area value, gray scale difference degree, and average relative brightness value respectively;

[0052] Deep learning prediction module: Construct corresponding deep learning models according to corresponding features respectively and train them. Use the trained models to predict new immunoblot images, obtain corresponding predicted values of area, gray scale difference degree and relative brightness, and standardize them.

[0053] Prediction correction and comprehensive evaluation module: Denote the difference between the measured value obtained by image recognition technology and the predicted value of the model as the prediction offset value. Obtain the average prediction offset according to historical data. Combine the predicted value and the average prediction offset to obtain the prediction correction value. Assign corresponding weights to the three prediction correction values and add them up to obtain the comprehensive prediction correction value.

[0054] Protein expression evaluation and analysis module: Combine the standard optical density value of the strip with the comprehensive prediction correction value to obtain the comprehensive expression evaluation value. Compare the comprehensive expression evaluation value with the preset threshold, and judge whether the protein expression level is normal according to the comparison result.

[0055] Beneficial effects of this aspect:

[0056] The present invention analyzes the acquired image data through image analysis software, uses image recognition technology to analyze a large amount of data in the preprocessed image, extracts features related to immunoblot, including area, color and gray scale value, calculates three measured values of area value, gray scale difference degree and average relative brightness, realizes the improvement of the accuracy and reliability of the results. This method is very useful in biomedical research, especially when a large amount of image data needs to be automatically analyzed. Establish a deep learning model based on a large number of measured value data and train it. Use a new image as the input of the trained corresponding model to predict it, obtain the predicted area value, gray scale difference degree and average relative brightness respectively. Perform corresponding corrections on multiple predicted values through multiple measured values to obtain the comprehensive prediction correction value. Combine the standard optical density value of the strip and the comprehensive prediction correction value to obtain the comprehensive expression evaluation value, realizing multi-modal data fusion analysis. Compare the preset threshold with the comprehensive expression evaluation value to judge the protein expression level. Brief description of the drawings

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0058] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0059] Figure 2 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1 As shown, the present invention is a method for immunoblotting image recognition based on deep learning, comprising the following steps:

[0062] S1: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement;

[0063] S2: The preprocessed image is identified by image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the band, and the area value, grayscale difference and relative brightness average are obtained respectively;

[0064] S3: construct corresponding deep learning models according to corresponding features and train them respectively, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them;

[0065] S4: Record the difference between the measured value obtained by the image recognition technology and the predicted value predicted by the model as the predicted offset value, obtain the predicted offset average value according to the historical data, combine the predicted value and the predicted offset average value to obtain the predicted correction value, assign corresponding weights to the three predicted correction values ​​and add them to obtain the comprehensive predicted correction value;

[0066] S5: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value, and whether the expression level of the protein is normal is determined according to the comparison result.

[0067] Specifically, the membrane is scanned using a high-resolution scanner, and the band images in the immunoblotting experiment are converted into digital format. Appropriate resolution and contrast settings are maintained during the scanning process to obtain a clear image. The acquired digital image is preprocessed, including background subtraction, band identification, and optical density measurement, to improve the accuracy of subsequent analysis and reduce random noise and uneven background in the image. The edge points are obtained by using two thresholds in the Canny algorithm, and the connectivity of the edge points is analyzed. The findContours function in the OpenCV open source computer vision library is used to identify the closed contours in the image. The closed contours are used as the band area, and the polygon formula is used to obtain the area of ​​the band area surrounded by the contour; the weighted average of the three RGB components is used to obtain the average grayscale image of the area. Value, compare the grayscale value of a pixel in the image with the grayscale image average value to obtain the grayscale difference; use the rectangle tool in the image analysis software to move the tool pointer to the position of the target stripe, draw a rectangular area covering the target stripe, use the software to obtain the grayscale value of each pixel in the image and the background grayscale value of each pixel in the area, and record the obtained grayscale value to obtain the relative brightness average; construct corresponding deep learning models of area, grayscale difference and relative brightness respectively, use the trained model to make predictions, and correct the predicted values ​​through the measured values ​​of area, grayscale difference and relative brightness to obtain a comprehensive predicted corrected value; obtain a comprehensive expression evaluation value by combining the standard optical density value of the stripe with the comprehensive predicted corrected value, and analyze the expression level of the protein according to the comparison results.

[0068] In one embodiment of the present invention, the step S1 comprises the following steps:

[0069] Use a high-resolution scanner to capture the band image on the membrane, convert the signal of the immune reaction into a digital image, save the captured digital image and set it to JPEG image file format, use image analysis software to process the acquired image, reduce the background noise of the image through the background subtraction function, automatically identify the edge of each band through the band recognition function and the optical density measurement function, measure the optical density of each identified band, record the optical density value of each band, and quantify the expression level of the target protein according to the optical density value.

[0070] Specifically, a high-resolution scanner can clearly capture the details of the stripes on the membrane, convert the scanned analog signal into a digital image format, and use the background subtraction function in the image analysis software to reduce the background noise of the image, making the stripes clearer. The stripe recognition function of the software is then used to identify the edge of each stripe. After successful recognition, the optical density of each identified stripe is measured and the optical density value of each stripe is recorded.

[0071] In one embodiment of the present invention, step S2 includes the following steps:

[0072] Calculate the gradients in the horizontal and vertical directions of each pixel in the image using the Sobel operator. After obtaining the gradient image, use non-maximum suppression technology to examine each pixel and its adjacent gradient magnitudes, and retain the local maximum as a candidate edge point. Obtain the edge points by using two thresholds in the Canny algorithm. Analyze the connectivity of the edge points, and use the findContours function in the OpenCV open-source computer vision library to identify the closed contours in the image. The closed contours are used as the strip regions, and the area of the strip region enclosed by the contours is obtained using the polygon formula;

[0073] Formula for calculating the area of the strip region:

[0074]

[0075] Where, is the area of the strip region in the image, ( , ), ( , )... ( , ) are the coordinate points on the contour edge;

[0076] Convert the image to a grayscale image through the OpenCV library. For each contour extracted from the image, initialize a variable for storing the sum of RGB values and a variable for counting the number of pixels. Traverse each pixel within each contour, and add the RGB of the pixel to the corresponding sum variable, and update the pixel number counter. Obtain the average value of the grayscale image of the region through the weighted average of the three RGB components, and compare the grayscale value of a pixel point in the image with the average value of the grayscale image to obtain the grayscale difference degree;

[0077] Formula for calculating the grayscale difference degree:

[0078]

[0079] Where, is the grayscale difference degree, is the grayscale value of the th pixel point in the image, is the total number of pixels in the image, is the sum of the red channel values of all pixels in the image, is the sum of the green channel values of all pixels in the image, is the sum of the blue channel values of all pixels in the image, is the number of pixels in the image;

[0080] Smooth the image. In the grayscale image of the cell image, use the rectangle tool in the image analysis software to move the tool pointer to the position of the target band and draw a rectangular area covering the target band. Use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this area of the image, record the obtained grayscale values, set the numerical value of the grayscale value to be between 0 - 255, classify the grayscale values according to the color brightness. The pixel area with a grayscale value between 0 - 85 represents the low-brightness tone level in the image, the pixel area with a grayscale value between 86 - 170 represents the medium-brightness tone level in the image, and the pixel area with a grayscale value between 171 - 255 represents the high-brightness tone level in the image. Obtain the relative brightness average value based on the grayscale value and the background grayscale value;

[0081] Formula for calculating the relative brightness average value:

[0082]

[0083] Among them, is the relative brightness average value, is the grayscale value of pixel point ; is the background grayscale value of pixel point ; is the total number of pixel points.

[0084] Specifically, the Sobel operator is used to calculate the gradients of the image in the horizontal and vertical directions respectively. Based on the horizontal and vertical gradients obtained by the operator, the gradient magnitude and direction of each pixel point are calculated. The non-maximum suppression technique is used to refine the edges, and only the pixel points with the local maximum gradient magnitude are retained as candidate edge points. The double-threshold processing in the Canny algorithm is applied. The pixel points higher than the high threshold are regarded as strong edge points, the pixel points lower than the low threshold are regarded as non-edge points, and the pixel points between the two are determined whether they are edge points according to their connectivity with the strong edge points. The edge tracking algorithm is used to start from the strong edge points and connect the weak edge points to form a complete edge. By using the findContours function of OpenCV, all closed contours in the image are found, and for each closed contour, the area enclosed by it is calculated using the polygon formula; the image is converted into a grayscale image through the OpenCV library, each closed contour is processed, variables for storing the sum of RGB values and a pixel number counter are initialized, each pixel inside the contour is traversed, the RGB values are accumulated and the pixel number is updated, the average RGB value is converted into a grayscale value using the standard grayscale conversion formula, and the grayscale value of each pixel point in the image is compared with the calculated average grayscale value of the grayscale image to obtain the grayscale difference degree; rectangular regions are drawn on the grayscale image, the software is used to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this region of the image, and the calculated grayscale values are recorded, the color brightness is classified, and the relative brightness average value is obtained according to the grayscale value and the background grayscale value.

[0085] In one embodiment of the present invention, the step of measuring the optical density of each recognized strip and recording the optical density value of each strip includes the following steps:

[0086] For each strip, find the center point of the strip as the strip dot, draw a circle with three-quarters of the total length of the strip as the radius of the circle, divide it into equal sectors, find the four background points of the strip-free regions in the four directions of up, down, left, and right located at the center point of the strip, record the background points as background dots, use one-tenth of the total length of the strip as the radius to obtain the background area of the strip-free region, measure the optical density values of the four regions respectively, calculate the average optical density value of the region around the strip, measure the optical density of each recognized strip, record the optical density value of each strip, and perform background subtraction by calculating the optical density ratio of the target protein to the internal reference protein using the internal reference protein as a reference to obtain the calculation formula for the standard optical density value of the strip;

[0087] Calculation formula for the standard optical density value of the strip:

[0088]

[0089] Wherein, is the standard optical density value of the strip, is the light intensity value of the i-th pixel in the strip area, is the total number of pixels in the strip area, is the area of the strip area in the image, is the optical density value of the background area to the left of the strip dot, is the optical density value of the background area above the strip dot, is the optical density value of the background area to the right of the strip dot, is the optical density value of the background area below the strip dot, is the background area value, is the optical density value of the internal reference protein.

[0090] Specifically, for each strip using image analysis software, find the center point of the strip as the strip dot, draw a circle with three-fourths of the total length of the strip as the radius of the circle, divide it into equal sectors, find four background points in the four directions of up, down, left, and right of the center point of the strip where there are no strips, record the background points as background dots, use one-tenth of the total length of the strip as the radius to obtain the background area of the area without strips, measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the strip, obtain the optical density value of the strip through the optical density value calculation formula, select a strip of the internal reference protein as a reference. The expression of the internal reference protein is relatively stable and can be used as a standard for comparing the protein expression levels between different samples. Subtract the average optical density value of the surrounding area from the optical density value of the processed target protein for background deduction. The optical density ratio obtained after background deduction is the normalized optical density value of the strip, which can more accurately reflect the relative expression level of the target protein.

[0091] In one embodiment of the present invention, the step S3 includes the following steps:

[0092] Using the polygon tool in the image annotation software, draw the boundary of each protein strip along the edge of each strip from the obtained image, calculate the number of pixels inside each drawn boundary strip to obtain the area of the strip, record the area of each obtained strip and summarize it into a data set, including the identification of the image, the identification of the strip, and the strip area, use the extracted area feature as the input for training the deep learning model, train the model, and use the trained model to predict the new strip area value, and the model outputs the predicted strip area value;

[0093] From the acquired images, use the band recognition function in the image analysis tool to automatically identify the positions of protein bands, segment each band, extract it separately from the image, extract color values for each segmented protein band using the image analysis tool, convert the color values to grayscale values by the weighted average method, record the grayscale values of each band, mark and set labels for the corresponding protein band labels, use the grayscale value of each protein band as a sample feature vector, and the corresponding label as the target value of the sample to summarize into a data set. Use the data set as the input of the model to train the model, and use the trained model to predict the grayscale difference degree of the new band. The model outputs the predicted grayscale difference degree of the band;

[0094] For the protein bands obtained from each image, calculate the relative brightness value of the bands in the corresponding image through the relative brightness average value calculation formula and make annotations. Summarize the annotated relative brightness values and the corresponding images into a data set, use it as the input for training the deep learning model, and use the trained model to predict the relative brightness value of the new band. The model outputs the predicted relative brightness value of the band;

[0095] Standardize the area value, grayscale difference degree, and relative brightness value predicted in the model respectively.

[0096] Specifically, from the acquired images, use the polygon tool in the image annotation software to draw the boundaries of each protein band along the edges of each band. Calculate the number of pixels inside each drawn boundary band to obtain the area of the band. Record the areas of each obtained band and summarize them into a data set and mark the corresponding labels, use it as the input data for training the area deep learning model, and use the trained model to predict the area of the new image to obtain the predicted area value; extract color values for each segmented protein band, convert the color values to grayscale values by the weighted average method, record the grayscale values of each band, mark and set labels for the corresponding protein band labels, use the grayscale value of each protein band as a sample feature vector, and the corresponding label as the target value of the sample to summarize into a data set. Use the data set as the input for training the model, and use the trained model to predict the grayscale difference degree of the new image to obtain the predicted grayscale difference degree; annotate the relative brightness value of the bands in the image, and summarize the annotated relative brightness values and the corresponding images into a data set as the input for training the deep learning model to obtain the predicted relative brightness.

[0097] In one embodiment of the present invention, step S4 includes the following steps:

[0098]

[0099] Among them, is the comprehensive forecast correction value, For the The area of ​​the strip enclosed by the outline in the secondary image, is the strip area value predicted by the model, For the The band area value predicted by the submodel, For the The average relative brightness, is the relative brightness value of the strip predicted by the model, For the The relative brightness value of the strip predicted by the submodel, For the Sub-grayscale difference, is the grayscale difference predicted by the model, For the The grayscale difference predicted by the sub-model, is the number of detections, , and are the corresponding weight coefficients respectively.

[0100] Specifically, the difference between the measured value obtained by image recognition technology and the predicted value predicted by the model is recorded as the predicted offset value, the predicted offset average value is obtained according to historical data, the predicted value and the predicted offset average value are combined to obtain the predicted correction value, and the three predicted correction values ​​are assigned corresponding weights and added to obtain the comprehensive predicted correction value. , and are the corresponding weight coefficients, is 0.5, is 0.3, is 0.2.

[0101] In one embodiment of the present invention, the step S5 comprises the following steps:

[0102] The calculation formula of comprehensive expression evaluation value is:

[0103]

[0104] in, is the comprehensive eigenvalue, is the standard optical density value of the band, is the comprehensive forecast correction value, and are the corresponding weight coefficients respectively;

[0105]

[0106] in, is the comprehensive expression evaluation value, It is a comprehensive eigenvalue, and e is a constant in this calculation formula.

[0107] Specifically, the standard optical density value of the strip and the comprehensive prediction correction value are jointly analyzed. By assigning different weights to the optical density value and the correction value and adding them together, a comprehensive expression evaluation value is obtained, which is used to represent the relative expression level of the target protein. According to the comprehensive expression evaluation value, the prediction result of the model can be explained.

[0108] In one embodiment of the present invention, comparing the comprehensive expression evaluation value with a preset threshold, and judging whether the expression level of the protein is normal according to the comparison result includes the following steps:

[0109] If the preset threshold > the comprehensive expression evaluation value, then judge the expression level of the protein as abnormal;

[0110] If the preset threshold ≤ the comprehensive expression evaluation value, then judge the expression level of the protein as normal.

[0111] Specifically, the comprehensive expression evaluation value is obtained by jointly analyzing the standard optical density value of the strip and the comprehensive prediction correction value obtained after correcting the predicted value, and is compared with the preset threshold. The preset threshold is 0.7. If the preset threshold is greater than the comprehensive expression evaluation value, the expression level of the protein is judged as abnormal. If the preset threshold is less than or equal to the comprehensive expression evaluation value, then the expression level of the protein is judged as normal.

[0112] Please refer to Figure 2 As shown, the present invention is an immunoblot image recognition system based on deep learning, including the following modules:

[0113] Protein image analysis module: Use a high-resolution scanner to capture the strip image on the membrane, convert the signal where the immune reaction occurs into a digital image, and use image analysis technology to process the collected image, including background subtraction, strip recognition, and optical density measurement;

[0114] Image recognition feature extraction module: Recognize the preprocessed image through image recognition technology, and extract the features related to immunoblotting, including the area, color, and gray value of the strip, and obtain the area value, gray difference degree, and relative brightness average value respectively;

[0115] Deep learning prediction module: Construct corresponding deep learning models according to the corresponding features and train them respectively. Use the trained models to predict new immunoblot images, obtain the corresponding predicted values of area, gray difference degree, and relative brightness, and standardize them;

[0116] Prediction correction and comprehensive evaluation module: the difference between the measured value obtained by image recognition technology and the predicted value predicted by the model is recorded as the prediction offset value, the average prediction offset value is obtained according to historical data, the prediction value and the average prediction offset value are combined to obtain the prediction correction value, and the three prediction correction values ​​are assigned corresponding weights and added to obtain the comprehensive prediction correction value;

[0117] Protein expression evaluation and analysis module: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value. Based on the comparison result, it is determined whether the expression level of the protein is normal.

[0118] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A deep learning-based immunoblot image recognition method, characterized in that, The following steps are involved: S1: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement; S2: The preprocessed image is identified by image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the band, and the area value, grayscale difference and relative brightness average are obtained respectively; S3: construct corresponding deep learning models according to corresponding features and train them respectively, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them; S4: Record the difference between the measured value obtained by the image recognition technology and the predicted value predicted by the model as the predicted offset value, obtain the predicted offset average value according to the historical data, combine the predicted value and the predicted offset average value to obtain the predicted correction value, assign corresponding weights to the three predicted correction values ​​and add them to obtain the comprehensive predicted correction value; S5: obtaining a comprehensive expression evaluation value by combining the standard optical density value of the band with the comprehensive prediction correction value, comparing the comprehensive expression evaluation value with the preset threshold, and judging whether the expression level of the protein is normal according to the comparison result; Wherein, step S5 comprises the following steps: The calculation formula of comprehensive expression evaluation value is: Among them, M is the comprehensive eigenvalue, L standard is the standard optical density value of the strip, Z prediction is the comprehensive prediction correction value, and w1 and w2 are the corresponding weight coefficients respectively; Among them, P is the comprehensive expression evaluation value, M is the comprehensive characteristic value, and e is a constant in the calculation formula.

2. The immunoblot image recognition method based on deep learning according to claim 1, wherein The step S1 comprises the following steps: Use a high-resolution scanner to capture the band image on the membrane, convert the signal of the immune reaction into a digital image, save the captured digital image and set it to JPEG image file format, use image analysis software to process the acquired image, reduce the background noise of the image through the background subtraction function, automatically identify the edge of each band through the band recognition function and the optical density measurement function, measure the optical density of each identified band, record the optical density value of each band, and quantify the expression level of the target protein according to the optical density value.

3. The immunoblot image recognition method based on deep learning according to claim 1, characterized in that The step S2 comprises the following steps: The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the image. After obtaining the gradient image, the non-maximum suppression technique is used to check each pixel and the adjacent gradient amplitude, and the local maximum value is retained as the candidate edge point. The edge point is obtained by using two thresholds in the Canny algorithm. According to the connectivity analysis of the edge points, the findContours function in the OpenCV open source computer vision library is used to identify the closed contours in the image. The closed contours are used as strip areas, and the area of ​​the strip area surrounded by the contours is obtained using the polygon formula. Strip area calculation formula: Among them, is the area of the strip region in the image, ( , ), ([[]] , )... ([[]] , ) are the coordinate points on the contour edge;​​ Convert the image to grayscale through the OpenCV library. For each contour extracted from the image, initialize a variable to store the sum of RGB values and a variable to count the number of pixels. Traverse each pixel within each contour, add the RGB of the pixel to the corresponding sum variable, and update the pixel count counter. Obtain the average grayscale value of the grayscale image of the region through the weighted average of the three RGB components. Compare the grayscale value of a pixel in the image with the average grayscale value of the grayscale image to obtain the grayscale difference degree; Grayscale difference degree calculation formula: Among them, is the grayscale difference degree, is the grayscale value of the th pixel in the image, is the total number of pixels in the image, is the sum of the red channel values of all pixels in the image, is the sum of the green channel values of all pixels in the image, is the sum of the blue channel values of all pixels in the image, is the number of pixels in the image; Smooth the image. In the grayscale image of the cell image, use the rectangle tool in the image analysis software to move the tool pointer to the position of the target band and draw a rectangular area covering the target band. Use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point within this area of the image, and record the obtained grayscale values. Set the numerical value of the grayscale value to be between 0 - 255. Classify the grayscale values according to the color brightness. The grayscale value between 0 - 85 represents the pixel area of the low lightness tone level in the image, the grayscale value between 86 - 170 represents the pixel area of the lightness tone level in the image, and the grayscale value between 171 - 255 represents the pixel area of the high lightness tone level in the image. Obtain the relative brightness average value based on the grayscale value and the background grayscale value; Relative brightness average value calculation formula: Wherein, is the average relative luminance, is the gray value of the pixel , is the background gray value of the pixel , is the total number of pixels.

4. The immunoblot image recognition method based on deep learning according to claim 2, wherein, The optical density measurement of each recognized band, recording the optical density value of each band, includes the following steps: For each band, find the center point of the band as the band dot. Draw a circle with three - quarters of the total length of the band as the radius of the circle, and divide it into equal sectors. Find the four background points of the non - band area in the four directions of up, down, left, and right of the center point of the band, and record the background points as background dots. Take one - tenth of the total length of the band as the radius to obtain the background area of the non - band area. Measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the band. Conduct optical density measurement on each recognized band, record the optical density value of each band. By using the internal reference protein as a reference, calculate the optical density ratio of the target protein to the internal reference protein for background subtraction to obtain the calculation formula for the standard optical density value of the band; Standard optical density value calculation formula of the band: Among them, is the standard optical density value of the band, is the light intensity value of the i-th pixel within the band area, is the total number of pixels within the band area, is the area of the band area in the image, is the optical density value of the background area to the left of the band dot, is the optical density value of the background area above the band dot, is the optical density value of the background area to the right of the band dot, is the optical density value of the background area below the band dot, is the background area value, is the optical density value of the internal reference protein.

5. The immunoblot image recognition method based on deep learning according to claim 1, wherein The step S3 includes the following steps: From the obtained image, use the polygon tool in the image annotation software to draw the boundary of each protein band along the edge of each protein band. Calculate the number of pixels inside each drawn boundary band to obtain the area of the band. Record the area of each obtained band and summarize it into a data set, including the identification of the image, the identification of the band, and the band area. Use the extracted area features as the input for training the deep - learning model, train the model, and use the trained model to predict the new band area value. The model outputs the predicted band area value; From the acquired images, use the band recognition function in the image analysis tool to automatically identify the positions of protein bands, segment each band, extract it separately from the image, use the image analysis tool to extract the color values of each segmented protein band, convert the color values to grayscale values through the weighted average method, record the grayscale values of each band, mark the corresponding protein band labels and set the labels, use the grayscale values of each protein band as a sample feature vector, and the corresponding labels as the target values of the samples to summarize into a data set. Use the data set as the input of the model to train the model, and use the trained model to predict the grayscale difference degree of the new band. The model outputs the predicted grayscale difference degree of the band; For the protein bands obtained from each image, calculate the relative brightness value of the bands in the corresponding image through the relative brightness average calculation formula and make annotations. Combine the annotated relative brightness values with the corresponding images into a data set and use it as the input of the deep learning model for training. Use the trained model to predict the relative brightness value of the new band. The model outputs the predicted relative brightness value of the band; Normalize the area value, grayscale difference degree, and relative brightness value predicted in the model respectively.

6. The immunoblot image recognition method based on deep learning according to claim 1, characterized in that, Step S4 includes the following steps: Among them, is the comprehensive prediction correction value, is the area value of the strip enclosed by the contour in the th image, is the strip area value predicted by the model, is the strip area value predicted by the model in the th time, is the average relative brightness in the th time, is the strip relative brightness value predicted by the model, is the strip relative brightness value predicted by the model in the th time, is the gray level difference degree in the th time, is the gray level difference degree predicted by the model, is the gray level difference degree predicted by the model in the th time, , and are the corresponding weight coefficients respectively.

7. The immunoblot image recognition method based on deep learning according to claim 1, wherein The comparison of the comprehensive expression evaluation value with the preset threshold and the judgment of whether the expression level of the protein is normal according to the comparison result include the following steps: If the preset threshold > comprehensive expression evaluation value, then judge the expression level of the protein as abnormal; If the preset threshold ≤ comprehensive expression evaluation value, then judge the expression level of the protein as normal.

8. An immunoblot image recognition system based on deep learning, using the immunoblot image recognition method based on deep learning according to any one of claims 1-7, characterized in that, It includes the following modules: Protein Image Analysis Module: Use a high-resolution scanner to capture the band images on the membrane, convert the signals where immune reactions occur into digital images, and use image analysis techniques to process the acquired images, including background subtraction, band recognition, and optical density measurement; Image Recognition Feature Extraction Module: Identify the preprocessed images through image recognition techniques, extract the features related to immunoblotting, including the area, color, and grayscale values of the bands, and obtain the area value, grayscale difference degree, and relative brightness average value respectively; Deep Learning Prediction Module: Construct corresponding deep learning models according to the corresponding features respectively and train them. Use the trained models to predict new immunoblot images, obtain the corresponding predicted area, grayscale difference degree, and relative brightness values, and normalize them; Prediction Correction and Comprehensive Evaluation Module: Record the difference between the measured value obtained through image recognition techniques and the predicted value predicted by the model as the prediction offset value, obtain the average prediction offset according to historical data, combine the predicted value and the average prediction offset to obtain the prediction correction value, and add the three prediction correction values with corresponding weights to obtain the comprehensive prediction correction value; Protein Expression Evaluation and Analysis Module: Obtain the comprehensive expression evaluation value by combining the standard optical density value of the band with the comprehensive prediction correction value, compare the comprehensive expression evaluation value with the preset threshold, and judge whether the expression level of the protein is normal according to the comparison result.

Images