Immunoblotting image recognition method and system based on deep learning

Abstract

The invention discloses an immunoblotting image recognition method and system based on deep learning, relates to the field of bioinformatics and computer vision, and solves the problem that it is difficult to extract various related features of protein from an image through an image recognition technology. The method solves the technical problem that a prediction value obtained by predicting a protein expression level obtained by a deep learning model through various protein feature data is difficult to correct. The method comprises the following steps: capturing a strip image on a film by using a high-resolution scanner; recognizing the preprocessed image through an image recognition technology, and extracting features related to the immunoblotting; predicting the feature data through a correspondingly trained deep learning model to obtain an area prediction value, a gray scale difference degree prediction value and a relative brightness prediction value; corresponding weights are given to the three prediction correction values for addition, and a comprehensive prediction correction value is obtained; and comparing the comprehensive expression evaluation value with a preset threshold value to judge the protein expression level.

Description

Technical Field

[0001] The present invention belongs to the fields of bioinformatics and computer vision, and particularly relates to a method and system for immunoblot image recognition based on deep learning. Background Art

[0002] With the continuous development of artificial intelligence technology, especially the breakthrough of deep learning in the field of image processing, it provides the possibility for automated and high-precision immunoblot image analysis. In biomedical research, the immunoblot technique is a commonly used protein analysis method for detecting the presence of specific antigens. The method and system for immunoblot image recognition based on deep learning extract various relevant features of proteins from images through image recognition technology, respectively obtain the measured values of relevant feature numerical values, correct the predicted values predicted in the constructed corresponding deep learning model, and obtain a comprehensive expression evaluation value by combining the corrected value and the standard optical density value. The expression level of the protein is judged according to the result of comparing the evaluation value with a preset threshold.

[0003] Although the existing methods and systems for immunoblot image recognition based on deep learning have achieved protein detection to a certain extent, traditional methods are difficult to extract various relevant features of proteins from images through image recognition technology, and it is also difficult to correct the predicted values obtained by predicting the protein expression level of the corresponding deep learning model with the measured values of various protein feature data, lacking the judgment of the protein expression level through the comparison result, resulting in limited accuracy and repeatability of the results. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present invention provides a method and system for immunoblot image recognition based on deep learning to solve the following technical problems: Although the existing methods and systems for immunoblot image recognition based on deep learning have achieved protein detection to a certain extent, traditional methods are difficult to extract various relevant features of proteins from images through image recognition technology, and it is also difficult to correct the predicted values obtained by predicting the protein expression level of the corresponding deep learning model with the measured values of various protein feature data, lacking the judgment of the protein expression level through the comparison result.

[0005] To solve the above problems, the first aspect of the present invention provides a method for immunoblot image recognition based on deep learning, including the following steps: S1: Use a high-resolution scanner to capture the band image on the membrane, convert the signal where the immune reaction occurs into a digital image, and process the collected image using image analysis technology, including background subtraction, band recognition, and optical density measurement; S2: Identify the preprocessed image through image recognition technology, extract the features related to immunoblotting, including the area, color, and grayscale value of the bands, and obtain the area value, grayscale difference degree, and average relative brightness respectively; S3: Construct corresponding deep learning models according to the corresponding features and train them respectively. Use the trained models to predict new immunoblot images, obtain the corresponding predicted values of area, grayscale difference degree, and relative brightness, and standardize them; S4: Denote the difference between the measured value obtained through image recognition technology and the predicted value of the model as the prediction offset value. Obtain the average prediction offset according to historical data. Combine the predicted value and the average prediction offset to obtain the prediction correction value. Assign corresponding weights to the three prediction correction values and add them to obtain the comprehensive prediction correction value; S5: Obtain the comprehensive expression evaluation value by combining the standard optical density value of the band and the comprehensive prediction correction value. Compare the comprehensive expression evaluation value with the preset threshold, and judge whether the expression level of the protein is normal according to the comparison result.

[0006] As a further solution of the present invention: The step S1 includes the following steps: Use a high-resolution scanner to capture the band image on the membrane, convert the signal where the immune reaction occurs into a digital image, save the captured digital image, set it as the JPEG image file format, use image analysis software to process the obtained image, reduce the background noise of the image through the background subtraction function, the band recognition function automatically recognizes the edges of each band and the optical density measurement function, measure the optical density of each recognized band, record the optical density value of each band, and perform quantitative analysis on the expression level of the target protein according to the optical density value.

[0007] As a further solution of the present invention: The step S2 includes the following steps: Use the Sobel operator to calculate the horizontal and vertical gradients of each pixel point in the image. After obtaining the gradient image, use the non-maximum suppression technique to check each pixel point and the adjacent gradient amplitudes, and retain the local maximum as the candidate edge point. Obtain the edge points by using two thresholds in the Canny algorithm. According to the analysis of the connectivity of the edge points, use the findContours function in the OpenCV open-source computer vision library to identify the closed contours in the image. The closed contours are used as the band regions, and the area of the band region enclosed by the contours is obtained by using the polygon formula; Band area calculation formula: Wherein, is the area of the band region in the image, ( , ), ( , ), … ([[]]END]] , are the coordinate points on the contour edge; Convert the image to grayscale through the OpenCV library. For each contour extracted from the image, initialize a variable to store the sum of RGB values and a variable to count the number of pixels. Traverse each pixel within each contour, add the RGB of the pixel to the corresponding sum variable, update the pixel count counter, obtain the average grayscale value of the grayscale image of the region through the weighted average of the three RGB components, and compare the grayscale value of a pixel point in the image with the average grayscale value of the grayscale image to obtain the grayscale difference degree; Grayscale difference degree calculation formula: where, is the grayscale difference degree, is the grayscale value of the th pixel point in the image, is the total number of pixels in the image, is the sum of the red channel values of all pixels in the image, is the sum of the green channel values of all pixels in the image, is the sum of the blue channel values of all pixels in the image, is the number of pixels in the image; Smooth the image. In the grayscale image of the cell image, use the rectangle tool in the image analysis software to move the tool pointer to the position of the target band, draw a rectangular area covering the target band, use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this area of the image, and record the obtained grayscale values. Set the numerical value of the grayscale value to be between 0 - 255, classify the grayscale values according to the color brightness. The grayscale value between 0 - 85 represents the pixel area of the low brightness tone level in the image, the grayscale value between 86 - 170 represents the pixel area of the medium brightness tone level in the image, and the grayscale value between 171 - 255 represents the pixel area of the high brightness tone level in the image. Obtain the relative brightness average value according to the grayscale value and the background grayscale value; Relative brightness average value calculation formula: where, is the relative brightness average value, is the grayscale value of pixel point , is the background grayscale value of pixel point , is the total number of pixel points.

[0008] As a further solution of the present invention: performing an optical density measurement on each identified band and recording the optical density value of each band, including the following steps: For each band, find the center point of the band as the band dot, draw a circle with three-fourths of the total length of the band as the radius of the circle, divide it into equal sectors, find four background points in the four directions of up, down, left, and right of the center point of the band, denote the background points as background dots, use one-tenth of the total length of the band as the radius to obtain the background area of the stripe-free area, measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the band, perform an optical density measurement on each identified band, record the optical density value of each band, perform background subtraction by calculating the optical density ratio of the target protein to the internal reference protein using the internal reference protein as a reference, and obtain the calculation formula for the standard optical density value of the band; Calculation formula for the standard optical density value of the band: Wherein, is the standard optical density value of the band, is the light intensity value of the i-th pixel in the band area, is the total number of pixels in the band area, is the area of the band area in the image, is the optical density value of the background area in the left direction of the band dot, is the optical density value of the background area in the upper direction of the band dot, is the optical density value of the background area in the right direction of the band dot, is the optical density value of the background area in the lower direction of the band dot, is the background area value, is the optical density value of the internal reference protein.

[0009] As a further solution of the present invention: the step S3 includes the following steps: From the obtained image, use the polygon tool in the image annotation software to draw the boundary of each protein band along the edge of each protein band, calculate the number of pixels inside each drawn boundary band to obtain the area of the band, record the area of each obtained band and summarize it into a data set, including the identification of the image, the identification of the band, and the band area, use the extracted area feature as the input for training the deep learning model, train the model, and use the trained model to predict the new band area value, and the model outputs the predicted band area value; From the acquired images, use the band recognition function in the image analysis tool to automatically identify the positions of protein bands, segment each band, extract it separately from the image, use the image analysis tool to extract the color values of each segmented protein band, convert the color values to grayscale values by the weighted average method, record the grayscale values of each band and mark and set the labels for the corresponding protein band labels, use the grayscale value of each protein band as a sample feature vector, and the corresponding label as the target value of the sample to summarize into a data set. Use the data set as the input of the model to train the model, and use the trained model to predict the grayscale difference degree of the new band. The model outputs the predicted grayscale difference degree of the band; For the protein bands obtained from each image, calculate the relative brightness value of the band in the corresponding image through the relative brightness average calculation formula and perform annotation. Summarize the annotated relative brightness value and the corresponding image into a data set, use it as the input of the deep learning model for training, and use the trained model to predict the relative brightness value of the new band. The model outputs the predicted relative brightness value of the band; Normalize the area value, grayscale difference degree, and relative brightness value predicted by the model respectively.

[0010] As a further solution of the present invention: The step S4 includes the following steps: Among them, is the comprehensive prediction correction value, is the area value of the band enclosed by the contour in the th image, is the area value of the band predicted by the model, is the th area value of the band predicted by the model, is the th relative brightness average value, is the relative brightness value of the band predicted by the model, is the th relative brightness value of the band predicted by the model, is the th grayscale difference degree, is the grayscale difference degree of the band predicted by the model, is the th grayscale difference degree of the band predicted by the model, is the number of detections, , and are the corresponding weight coefficients respectively.

[0011] As a further solution of the present invention: The step S5 includes the following steps: Calculation formula for comprehensive expression evaluation: Among them, is the comprehensive eigenvalue, is the standard optical density value of the band, is the comprehensive prediction correction value, and are the corresponding weight coefficients respectively; Among them, the comprehensive expression evaluation value, is the comprehensive eigenvalue, and e is a constant in this calculation formula.

[0012] As a further solution of the present invention: comparing the comprehensive expression evaluation value with a preset threshold, and judging whether the expression level of the protein is normal according to the comparison result, including the following steps: If the preset threshold > the comprehensive expression evaluation value, then judge the expression level of the protein as abnormal; If the preset threshold ≤ the comprehensive expression evaluation value, then judge the expression level of the protein as normal.

[0013] The present invention also provides an immunoblot image recognition system based on deep learning, including the following modules: Protein image analysis module: Use a high-resolution scanner to capture the band image on the membrane, convert the signal where the immune reaction occurs into a digital image, and use image analysis technology to process the collected image, including background subtraction, band recognition, and optical density measurement; Image recognition feature extraction module: Recognize the preprocessed image through image recognition technology, extract the features related to immunoblotting, including the area, color, and gray value of the band, and obtain the area value, gray difference degree, and relative brightness average value respectively; Deep learning prediction module: Construct corresponding deep learning models according to the corresponding features and train them respectively, use the trained models to predict new immunoblot images, obtain the corresponding predicted values of area, gray difference degree, and relative brightness, and standardize them; Prediction correction and comprehensive evaluation module: Denote the difference between the measured value obtained through image recognition technology and the predicted value predicted by the model as the prediction offset value, obtain the average prediction offset according to historical data, combine the predicted value and the average prediction offset to obtain the prediction correction value, and add the three prediction correction values with corresponding weights to obtain the comprehensive prediction correction value; Protein expression evaluation and analysis module: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value. Based on the comparison result, it is determined whether the expression level of the protein is normal.

[0014] Beneficial effects of this aspect: The present invention analyzes the acquired image data through image analysis software, uses image recognition technology to analyze a large amount of data in the preprocessed image, extracts features related to immunoblotting, including area, color and grayscale value, and obtains three measured values ​​of area value, grayscale difference and relative brightness average value by calculation, thereby improving the accuracy and reliability of the result. This method is very useful in biomedical research, especially when a large amount of image data needs to be automatically analyzed. A deep learning model is established and trained through a large amount of measured value data, and a new image is used as the input of the trained corresponding model to predict it, and the predicted area value, grayscale difference and relative brightness average value are respectively obtained. A plurality of predicted values ​​are correspondingly corrected by a plurality of measured values ​​to obtain a comprehensive predicted corrected value, and the standard optical density value of the strip and the comprehensive predicted corrected value are combined to obtain a comprehensive expression evaluation value, thereby realizing multimodal data fusion analysis, and the expression level of the protein is judged by comparing the preset threshold with the comprehensive expression evaluation value. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the system framework of the present invention; Figure 2 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 As shown, the present invention is a method for immunoblotting image recognition based on deep learning, comprising the following steps: S1: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement; S2: The preprocessed image is identified by image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the band, and the area value, grayscale difference and relative brightness average are obtained respectively; S3: construct corresponding deep learning models according to corresponding features and train them respectively, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them; S4: Record the difference between the measured value obtained by the image recognition technology and the predicted value predicted by the model as the predicted offset value, obtain the predicted offset average value according to the historical data, combine the predicted value and the predicted offset average value to obtain the predicted correction value, assign corresponding weights to the three predicted correction values ​​and add them to obtain the comprehensive predicted correction value; S5: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value, and whether the expression level of the protein is normal is determined according to the comparison result.

[0019] Specifically, the membrane is scanned using a high-resolution scanner, and the band images in the immunoblotting experiment are converted into digital format. Appropriate resolution and contrast settings are maintained during the scanning process to obtain a clear image. The acquired digital image is preprocessed, including background subtraction, band identification, and optical density measurement, to improve the accuracy of subsequent analysis and reduce random noise and uneven background in the image. The edge points are obtained by using two thresholds in the Canny algorithm, and the connectivity of the edge points is analyzed. The findContours function in the OpenCV open source computer vision library is used to identify the closed contours in the image. The closed contours are used as the band area, and the polygon formula is used to obtain the area of ​​the band area surrounded by the contour; the weighted average of the three RGB components is used to obtain the average grayscale image of the area. Value, compare the grayscale value of a pixel in the image with the grayscale image average value to obtain the grayscale difference; use the rectangle tool in the image analysis software to move the tool pointer to the position of the target stripe, draw a rectangular area covering the target stripe, use the software to obtain the grayscale value of each pixel in the image and the background grayscale value of each pixel in the area, and record the obtained grayscale value to obtain the relative brightness average; construct corresponding deep learning models of area, grayscale difference and relative brightness respectively, use the trained model to make predictions, and correct the predicted values ​​through the measured values ​​of area, grayscale difference and relative brightness to obtain a comprehensive predicted corrected value; obtain a comprehensive expression evaluation value by combining the standard optical density value of the stripe with the comprehensive predicted corrected value, and analyze the expression level of the protein according to the comparison results.

[0020] In one embodiment of the present invention, the step S1 comprises the following steps: Use a high-resolution scanner to capture the band image on the membrane, convert the signal of the immune reaction into a digital image, save the captured digital image and set it to JPEG image file format, use image analysis software to process the acquired image, reduce the background noise of the image through the background subtraction function, automatically identify the edge of each band through the band recognition function and the optical density measurement function, measure the optical density of each identified band, record the optical density value of each band, and quantify the expression level of the target protein according to the optical density value.

[0021] Specifically, a high-resolution scanner can clearly capture the details of the stripes on the membrane, convert the scanned analog signal into a digital image format, and use the background subtraction function in the image analysis software to reduce the background noise of the image, making the stripes clearer. The stripe recognition function of the software is then used to identify the edge of each stripe. After successful recognition, the optical density of each identified stripe is measured and the optical density value of each stripe is recorded.

[0022] In one embodiment of the present invention, step S2 includes the following steps: Calculate the gradients in the horizontal and vertical directions of each pixel in the image using the Sobel operator. After obtaining the gradient image, use non-maximum suppression technology to examine each pixel and its adjacent gradient magnitudes, and retain the local maximum as a candidate edge point. Obtain the edge points by using two thresholds in the Canny algorithm. Analyze the connectivity of the edge points, and use the findContours function in the OpenCV open-source computer vision library to identify the closed contours in the image. The closed contours are used as strip regions, and the area of the strip region enclosed by the contours is obtained using the polygon formula; Formula for calculating the area of the strip region: Wherein, is the area of the strip region in the image, ( , ), ( , )... ( , ) are the coordinate points on the contour edge; Convert the image to a grayscale image through the OpenCV library. For each contour extracted from the image, initialize a variable for storing the sum of RGB values and a variable for counting the number of pixels. Traverse each pixel within each contour, add the RGB of the pixel to the corresponding sum variable, update the pixel number counter, and obtain the average value of the grayscale image of the region through the weighted average of the three RGB components. Compare the grayscale value of a pixel in the image with the average value of the grayscale image to obtain the grayscale difference degree; Formula for calculating the grayscale difference degree: Wherein, is the grayscale difference degree, is the grayscale value of the th pixel in the image, is the total number of pixels in the image, is the sum of the red channel values of all pixels in the image, is the sum of the green channel values of all pixels in the image, is the sum of the blue channel values of all pixels in the image, is the number of pixels in the image; Smooth the image. In the grayscale image of the cell image, use the rectangle tool in the image analysis software to move the tool pointer to the position of the target band and draw a rectangular area covering the target band. Use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this area of the image, and record the obtained grayscale values. Set the numerical value of the grayscale value to be between 0 and 255. Classify the grayscale values according to the color brightness. The pixel area with a grayscale value between 0 and 85 represents the low brightness tone level in the image. The pixel area with a grayscale value between 86 and 170 represents the brightness tone level in the image. The pixel area with a grayscale value between 171 and 255 represents the high brightness tone level in the image. Obtain the relative brightness average value based on the grayscale value and the background grayscale value; Formula for calculating the relative brightness average value: Wherein, is the relative brightness average value, is the grayscale value of pixel point and is the background grayscale value of pixel point and is the total number of pixel points.

[0023] Specifically, use the Sobel operator to calculate the gradients of the image in the horizontal and vertical directions respectively. According to the horizontal and vertical gradients obtained by the operator, calculate the gradient magnitude and direction of each pixel point. Use the non-maximum suppression technique to refine the edges, and only retain the pixel points with the local maximum gradient magnitude as candidate edge points. Apply the double-threshold processing in the Canny algorithm. Consider the pixel points higher than the high threshold as strong edge points, the pixel points lower than the low threshold as non-edge points, and the pixel points between the two are determined whether they are edge points according to their connectivity with the strong edge points. Use the edge tracking algorithm to start from the strong edge points and connect the weak edge points to form a complete edge. Use the findContours function of OpenCV to find all closed contours in the image. For each closed contour, calculate the area enclosed by it using the polygon formula; Convert the image to a grayscale image through the OpenCV library, process each closed contour, initialize the variables for storing the sum of RGB values and the pixel number counter, traverse each pixel inside the contour, accumulate the RGB values and update the pixel number, convert the average RGB value to a grayscale value using the standard grayscale conversion formula, compare the grayscale value of each pixel point in the image with the calculated average grayscale value of the grayscale image to obtain the grayscale difference degree; Draw a rectangular area on the grayscale image, use the software to obtain the grayscale value of each pixel point and the background grayscale value of each pixel point in this area of the image, and record the calculated grayscale values, classify the color brightness, and obtain the relative brightness average value based on the grayscale value and the background grayscale value.

[0024] In one embodiment of the present invention, the optical density measurement of each identified band and recording the optical density value of each band includes the following steps: For each band, find the center point of the band as the band dot, draw a circle with three-fourths of the total length of the band as the radius of the circle, divide it into equal sectors, find four background points in the four directions of up, down, left, and right of the center point of the band, record the background points as background dots, use one-tenth of the total length of the band as the radius to obtain the background area of the stripe-free area, measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the band, perform optical density measurement on each identified band, record the optical density value of each band, use the internal reference protein as a reference, calculate the optical density ratio of the target protein to the internal reference protein for background subtraction, and obtain the calculation formula for the standard optical density value of the band; Calculation formula for the standard optical density value of the band: Wherein, is the standard optical density value of the band, is the light intensity value of the i-th pixel in the band area, is the total number of pixels in the band area, is the area of the band area in the image, is the optical density value of the background area to the left of the band dot, is the optical density value of the background area above the band dot, is the optical density value of the background area to the right of the band dot, is the optical density value of the background area below the band dot, is the background area value, is the optical density value of the internal reference protein.

[0025] Specifically, using image analysis software, for each band, find the center point of the band as the band dot, draw a circle with three-fourths of the total length of the band as the radius of the circle, divide it into equal sectors, find four background points in the four directions of up, down, left, and right of the center point of the band, record the background points as background dots, use one-tenth of the total length of the band as the radius to obtain the background area of the stripe-free area, measure the optical density values of the four areas respectively, calculate the average optical density value of the area around the band, obtain the optical density value of the band through the optical density value calculation formula, select a band of the internal reference protein as a reference, the expression of the internal reference protein is relatively stable and can be used as a standard for comparing the protein expression levels between different samples, subtract the average optical density value of the surrounding area from the optical density value of the processed target protein for background subtraction, and the optical density ratio obtained after background subtraction is the standardized optical density value of the band, which can more accurately reflect the relative expression level of the target protein.

[0026] In one embodiment of the present invention, step S3 includes the following steps: From the acquired image, use the polygon tool in the image annotation software to draw the boundary of each protein band along the edge of each protein band. Calculate the number of pixels inside each drawn boundary band to obtain the area of the band. Record the area of each obtained band and summarize it into a data set, including the identification of the image, the identification of the band, and the band area. Use the extracted area feature as the input for training the deep learning model, train the model, and use the trained model to predict the new band area value. The model outputs the predicted band area value; From the acquired image, use the band recognition function in the image analysis tool to automatically identify the positions of the protein bands, segment each band, extract it separately from the image. Use the image analysis tool to extract the color value for each segmented protein band, convert the color value to a grayscale value by the weighted average method, record the grayscale value of each band and mark the corresponding protein band label and set the label. Use the grayscale value of each protein band as a sample feature vector, and the corresponding label as the target value of the sample to summarize into a data set. Use the data set as the input of the model, train the model, and use the trained model to predict the grayscale difference degree of the new band. The model outputs the predicted grayscale difference degree of the band; For the protein bands obtained from each image, calculate the relative brightness value of the bands in the corresponding image through the relative brightness average calculation formula and perform annotation. Summarize the annotated relative brightness value and the corresponding image into a data set, use it as the input for training the deep learning model, and use the trained model to predict the relative brightness value of the new band. The model outputs the predicted relative brightness value of the band; Normalize the area value, grayscale difference degree, and relative brightness value predicted by the model respectively.

[0027] Specifically, from the acquired images, using the polygon tool in the image annotation software, draw the boundaries of each protein band along the edges of each protein band, calculate the number of pixels inside each drawn boundary band to obtain the area of the band, record the areas of each obtained band and summarize them into a data set and mark the corresponding labels, and use it as the input data for the area deep learning model for training. Predict the area of the new image through the trained model to obtain the predicted area value; extract the color values for each segmented protein band, convert the color values to grayscale values through the weighted average method, record the grayscale values of each band and mark the corresponding protein band labels and set the labels. Use the grayscale values of each protein band as a sample feature vector, and the corresponding labels as the target values of the samples to summarize into a data set. Use the data set as the input of the model for training, and predict the grayscale difference degree of the new image through the trained model to obtain the predicted grayscale difference degree; mark the relative brightness values of the bands in the image and summarize the marked relative brightness values and the corresponding images into a data set as the input of the deep learning model for training to obtain the predicted relative brightness.

[0028] In one embodiment of the present invention, the step S4 includes the following steps: Wherein, is the comprehensive prediction correction value, is the th area value of the band enclosed by the contour in the image, is the area value of the band predicted by the model, is the th area value of the band predicted by the model, is the th average relative brightness, is the relative brightness value of the band predicted by the model, is the th relative brightness value of the band predicted by the model, is the th grayscale difference degree, is the grayscale difference degree predicted by the model, is the th grayscale difference degree predicted by the model, is the number of detections, , and are the corresponding weight coefficients respectively.

[0029] Specifically, the difference between the measured value obtained by image recognition technology and the predicted value predicted by the model is recorded as the predicted offset value, the predicted offset average value is obtained according to historical data, the predicted value and the predicted offset average value are combined to obtain the predicted correction value, and the three predicted correction values ​​are assigned corresponding weights and added to obtain the comprehensive predicted correction value. , and are the corresponding weight coefficients, is 0.5, is 0.3, is 0.2.

[0030] In one embodiment of the present invention, the step S5 comprises the following steps: The calculation formula of comprehensive expression evaluation value is:

[0031] in, is the comprehensive eigenvalue, is the standard optical density value of the band, is the comprehensive forecast correction value, and are the corresponding weight coefficients respectively;

[0032] in, is the comprehensive expression evaluation value, is the comprehensive eigenvalue, and e is a constant in this calculation formula.

[0033] Specifically, the standard optical density value of the band and the comprehensive prediction correction value are jointly analyzed, and a comprehensive expression evaluation value is obtained by adding the optical density value and the correction value by assigning different weights to represent the relative expression level of the target protein. According to the comprehensive expression evaluation value, the prediction results of the model can be explained.

[0034] In one embodiment of the present invention, comparing the comprehensive expression evaluation value with a preset threshold value and judging whether the expression level of the protein is normal according to the comparison result comprises the following steps: If the preset threshold value is greater than the comprehensive expression evaluation value, the expression level of the protein is judged to be abnormal; If the preset threshold is ≤ the comprehensive expression evaluation value, the expression level of the protein is judged to be normal.

[0035] Specifically, a comprehensive expression evaluation value is obtained by combining the standard optical density value of the band and the comprehensive prediction correction value obtained after correcting the prediction value, and compared with a preset threshold value, the preset threshold value is 0.7. If the preset threshold value is greater than the comprehensive expression evaluation value, the expression level of the protein is judged to be abnormal. If the preset threshold value is less than or equal to the comprehensive expression evaluation value, the expression level of the protein is judged to be normal.

[0036] See also Figure 2 As shown, the present invention is a deep learning-based immunoblot image recognition system, comprising the following modules: Protein image analysis module: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement; Image recognition feature extraction module: The pre-processed image is identified through image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the strip, and the area value, grayscale difference and relative brightness average are obtained respectively; Deep learning prediction module: construct and train corresponding deep learning models according to corresponding features, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them; Prediction correction and comprehensive evaluation module: the difference between the measured value obtained by image recognition technology and the predicted value predicted by the model is recorded as the prediction offset value, the average prediction offset value is obtained according to historical data, the prediction value and the average prediction offset value are combined to obtain the prediction correction value, and the three prediction correction values ​​are assigned corresponding weights and added to obtain the comprehensive prediction correction value; Protein expression evaluation and analysis module: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value. Based on the comparison result, it is determined whether the expression level of the protein is normal.

[0037] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for immunoblotting image recognition based on deep learning, characterized in that: The following steps are involved: S1: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement; S2: The preprocessed image is identified by image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the band, and the area value, grayscale difference and relative brightness average are obtained respectively; S3: construct corresponding deep learning models according to corresponding features and train them respectively, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them; S4: Record the difference between the measured value obtained by the image recognition technology and the predicted value predicted by the model as the predicted offset value, obtain the predicted offset average value according to the historical data, combine the predicted value and the predicted offset average value to obtain the predicted correction value, assign corresponding weights to the three predicted correction values ​​and add them to obtain the comprehensive predicted correction value; S5: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value, and whether the expression level of the protein is normal is determined according to the comparison result.

2. The method for immunoblotting image recognition based on deep learning according to claim 1, characterized in that: The step S1 comprises the following steps: Use a high-resolution scanner to capture the band image on the membrane, convert the signal of the immune reaction into a digital image, save the captured digital image and set it to JPEG image file format, use image analysis software to process the acquired image, reduce the background noise of the image through the background subtraction function, automatically identify the edge of each band through the band recognition function and the optical density measurement function, measure the optical density of each identified band, record the optical density value of each band, and quantify the expression level of the target protein according to the optical density value.

3. The method for immunoblotting image recognition based on deep learning according to claim 1, characterized in that: The step S2 comprises the following steps: The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the image. After obtaining the gradient image, the non-maximum suppression technique is used to check each pixel and the adjacent gradient amplitude, and the local maximum value is retained as the candidate edge point. The edge point is obtained by using two thresholds in the Canny algorithm. According to the connectivity analysis of the edge points, the findContours function in the OpenCV open source computer vision library is used to identify the closed contours in the image. The closed contours are used as strip areas, and the area of ​​the strip area surrounded by the contours is obtained using the polygon formula. Strip area calculation formula: in, is the area of ​​the stripe in the image, ( , )、( , )…( , ) is the coordinate point on the edge of the contour; The image is converted into a grayscale image through the OpenCV library. For each contour extracted from the image, a variable for storing the sum of RGB values ​​and a variable for counting the number of pixels are initialized. Each pixel in each contour is traversed, and the RGB of the pixel is added to the corresponding sum variable. The pixel number counter is updated, and the average grayscale image value of the area is obtained by the weighted average of the three RGB components. The grayscale value of a pixel in the image is compared with the grayscale image average to obtain the grayscale difference. Grayscale difference calculation formula: in, is the grayscale difference, For the image The gray value of a pixel, is the total number of pixels in the image, is the sum of the red channel values ​​of all pixels in the image, is the sum of the green channel values ​​of all pixels in the image, is the sum of the blue channel values ​​of all pixels in the image, is the number of pixels in the image; The image was smoothed. In the grayscale image of the cell image, the tool pointer was moved to the position of the target stripe by the rectangle tool in the image analysis software to draw a rectangular area covering the target stripe. The grayscale value of each pixel in the area and the background grayscale value of each pixel in the image were obtained by the software, and the obtained grayscale value was recorded. The grayscale value was set to 0-255, and the grayscale value was classified according to the color brightness. The grayscale value between 0-85 represents the pixel area with low brightness tone level in the image, the grayscale value between 86-170 represents the pixel area with brightness tone level in the image, and the grayscale value between 171-255 represents the pixel area with high brightness tone level in the image. The relative brightness average value was obtained according to the grayscale value and the background grayscale value. The relative brightness average calculation formula is: in, is the average relative brightness, Pixel The gray value of Pixel The background gray value, is the total number of pixels.

4. The method for immunoblotting image recognition based on deep learning according to claim 2, characterized in that: The optical density measurement of each identified band and recording the optical density value of each band comprises the following steps: For each band, find the center point of the band as the band dot, draw a circle with three quarters of the total length of the band as the radius, and divide it into equal sectors, find four background points of the band-free area in the four directions of the top, bottom, left and right of the center point of the band, record the background points as background dots, use one tenth of the total length of the band as the radius, and obtain the background area of ​​the band-free area, measure the optical density values ​​of the four areas respectively, calculate the average optical density value of the area around the band, measure the optical density of each identified band, and record the optical density value of each band. By using the internal reference protein as a reference, calculate the optical density ratio of the target protein to the internal reference protein for background subtraction, and obtain the calculation formula for the standard optical density value of the band; Calculation formula for standard optical density of bands: in, is the standard optical density value of the band, is the light intensity value of the i-th pixel in the strip area, is the total number of pixels in the strip area, is the area of ​​the stripe in the image, is the optical density value of the background area to the left of the stripe dot, is the optical density value of the background area above the strip dots, is the optical density value of the background area to the right of the stripe dot, is the optical density value of the background area below the stripe dots, is the background area value, is the optical density value of the internal reference protein.

5. The method for immunoblot image recognition based on deep learning according to claim 1, characterized in that: The step S3 comprises the following steps: Using the polygon tool in the image annotation software, draw the boundary of each protein band along the edge of each band from the acquired image. Calculate the number of pixels inside each drawn boundary band to obtain the area of ​​the band. Record and summarize the area of ​​each band into a data set, including the image identification, band identification and band area. Use the extracted area features as input for training the deep learning model. Train the model. Use the trained model to predict the new band area value. The model outputs the predicted band area value. From the acquired image, the band recognition function in the image analysis tool is used to automatically identify the position of the protein band, each band is segmented and extracted separately from the image, the color value of each segmented protein band is extracted using the image analysis tool, the color value is converted into a gray value by a weighted average method, the gray value of each band is recorded and the corresponding protein band label is marked and set, the gray value of each protein band is used as a sample feature vector, and the corresponding label is used as the target value of the sample to summarize into a data set, the data set is used as the input of the model, the model is trained, and the gray difference of the new band is predicted using the trained model, and the model outputs the predicted band gray difference; The protein bands obtained in each image are annotated by calculating the relative brightness average value of the bands in the corresponding image, and the annotated relative brightness values ​​and the corresponding images are aggregated into a data set, which is used as the input of the deep learning model for training. The trained model is used to predict the relative brightness values ​​of new bands, and the model outputs the predicted relative brightness values ​​of the bands; The area value, grayscale difference and relative brightness value predicted in the model are standardized respectively.

6. The method for immunoblot image recognition based on deep learning according to claim 1, characterized in that: The step S4 comprises the following steps: in, is the comprehensive forecast correction value, For the The area of ​​the strip enclosed by the outline in the secondary image, is the strip area value predicted by the model, For the The band area value predicted by the submodel, For the The average relative brightness, is the relative brightness value of the strip predicted by the model, For the The relative brightness value of the strip predicted by the submodel, For the Sub-grayscale difference, is the grayscale difference predicted by the model, For the The grayscale difference predicted by the sub-model, is the number of detections, , and are the corresponding weight coefficients respectively.

7. The method for immunoblot image recognition based on deep learning according to claim 1, characterized in that: The step S5 comprises the following steps: The calculation formula of comprehensive expression evaluation value is: in, is the comprehensive eigenvalue, is the standard optical density value of the band, is the comprehensive forecast correction value, and are the corresponding weight coefficients respectively; in, Comprehensive expression evaluation value, is the comprehensive eigenvalue, and e is a constant in this calculation formula.

8. The method for immunoblotting image recognition based on deep learning according to claim 1, characterized in that: The comprehensive expression evaluation value is compared with a preset threshold value, and judging whether the expression level of the protein is normal according to the comparison result, comprising the following steps: If the preset threshold value is greater than the comprehensive expression evaluation value, the expression level of the protein is judged to be abnormal; If the preset threshold is ≤ the comprehensive expression evaluation value, the expression level of the protein is judged to be normal.

9. A deep learning-based immunoblot image recognition system, using the deep learning-based immunoblot image recognition method according to any one of claims 1 to 8, characterized in that: Includes the following modules: Protein image analysis module: Use a high-resolution scanner to capture the band images on the membrane, convert the signals of the immune reaction into digital images, and use image analysis technology to process the collected images, including background subtraction, band identification and optical density measurement; Image recognition feature extraction module: The pre-processed image is identified through image recognition technology to extract features related to immunoblotting, including the area, color and grayscale value of the strip, and the area value, grayscale difference and relative brightness average are obtained respectively; Deep learning prediction module: construct and train corresponding deep learning models according to corresponding features, use the trained models to predict new immunoblot images, obtain corresponding area, grayscale difference and relative brightness prediction values, and standardize them; Prediction correction and comprehensive evaluation module: the difference between the measured value obtained by image recognition technology and the predicted value predicted by the model is recorded as the prediction offset value, the average prediction offset value is obtained according to historical data, the prediction value and the average prediction offset value are combined to obtain the prediction correction value, and the three prediction correction values ​​are assigned corresponding weights and added to obtain the comprehensive prediction correction value; Protein expression evaluation and analysis module: The comprehensive expression evaluation value is obtained by combining the standard optical density value of the band with the comprehensive prediction correction value, and the comprehensive expression evaluation value is compared with the preset threshold value. Based on the comparison result, it is determined whether the expression level of the protein is normal.

Images