Auxiliary identification method of diabetic nephropathy based on multi-source data

Abstract

The present invention relates to the field of image recognition technology, and in particular to a method for assisting the identification of diabetic nephropathy based on multi-source data. The method obtains a grayscale image of a renal biopsy specimen; obtains a closed interlayer area according to the distribution of grayscale values; selects any closed interlayer area as a target area, and obtains the degree of damage according to the distribution of pixel points in the target area on the straight line where the gradient direction of the edge pixel points on the outer edge line of the target area is located, the distribution of grayscale values ​​on the outer edge line, and the distribution of grayscale values ​​in the area formed by the inner edge line; obtains a reference abnormality degree according to the fluctuation of the degree of damage and the position distribution of the closed interlayer area, and the distribution of grayscale values ​​in the area formed by the outer edge line, and evaluates the condition of diabetic nephropathy. The present invention accurately obtains the reference abnormality degree, and then accurately evaluates the condition of diabetic nephropathy, which is conducive to accurate treatment of diabetic nephropathy and delays the deterioration of diabetic nephropathy.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for assisting in identifying diabetic nephropathy based on multi-source data. Background Art

[0002] Diabetic nephropathy (DN) is a glomerular disease caused by diabetes. Its main feature is the significant proliferation of mesangial matrix in the glomerular mesangial area. It is also called diabetic glomerulosclerosis. It is one of the common and serious chronic complications of diabetes patients. It is mainly manifested by the gradual deterioration of renal function and may eventually lead to end-stage renal disease. Early diagnosis and treatment of diabetic nephropathy are crucial to delay the progression of the disease. In recent years, the development of auxiliary recognition methods using multi-source data has achieved a comprehensive assessment of diabetic nephropathy, especially the auxiliary recognition methods of imaging data have provided great help in the assessment of the status of diabetic nephropathy.

[0003] The core of the diagnosis of diabetic nephropathy is to evaluate the degree of glomerular damage. The existing method directly observes the glomeruli in the renal biopsy specimen images to evaluate the degree of glomerular damage and then evaluate the condition of diabetic nephropathy. However, in actual situations, direct observation of glomeruli in renal biopsy specimen images will have loopholes, resulting in inaccurate analysis of the condition of diabetic nephropathy and affecting the accurate treatment of the disease. Summary of the invention

[0004] In order to solve the technical problem that direct observation of glomeruli in renal biopsy specimen images may result in loopholes, leading to inaccurate analysis of the condition of diabetic nephropathy, the purpose of the present invention is to provide a diabetic nephropathy auxiliary identification method based on multi-source data, and the technical solution adopted is as follows:

[0005] The embodiment of the present invention provides a method for assisting in identifying diabetic nephropathy based on multi-source data, the method comprising the following steps:

[0006] Obtain grayscale images of renal biopsy specimens after fluorescent labeling;

[0007] According to the distribution of gray values ​​in the gray image, a closed interlayer area is obtained; the closed interlayer area is the area corresponding to the immunoglobulin on the capillary wall of the glomerulus;

[0008] A closed sandwich area is selected as the target area, and the degree of damage of the area formed by the outer edge line of the target area is obtained according to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of the grayscale value on the outer edge line of the target area, and the distribution of the grayscale value in the area formed by the inner edge line of the target area;

[0009] The reference abnormality degree of the renal biopsy specimen is obtained according to the fluctuation of the degree of damage and the position distribution of all closed interlayer areas in the grayscale image, as well as the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area;

[0010] The status of diabetic nephropathy was assessed based on the degree of reference abnormalities.

[0011] Furthermore, the method for obtaining the closed interlayer area is:

[0012] According to the gray value of each pixel in the gray image, the pixels in the gray image are divided to obtain pixel categories;

[0013] Binarize the grayscale image after dividing the pixel points into categories to obtain the connected domain in the grayscale image;

[0014] The connected domains in the grayscale image are screened through the semantic segmentation network to obtain the closed interlayer area; wherein the closed interlayer area is a connected domain composed of the closed edge lines not included in the grayscale image and the maximum closed edge line included in it.

[0015] Furthermore, the method for obtaining the degree of damage is:

[0016] Obtain the degree of edge abnormality of the target area according to the distribution of pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located and the distribution of grayscale values ​​on the outer edge line of the target area;

[0017] According to the distribution of gray values ​​in the area formed by the inner edge line of the target area, the internal abnormality degree of the area formed by the inner edge line of the target area is obtained;

[0018] According to the edge abnormality degree and the internal abnormality degree, the damage degree of the area formed by the outer edge line of the target area is obtained; wherein the edge abnormality degree and the internal abnormality degree are both positively correlated with the damage degree.

[0019] Furthermore, the method for obtaining the edge abnormality degree is:

[0020] According to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the reference thickness of the corresponding edge pixel point is obtained;

[0021] Obtain the average value of the reference thickness of all edge pixels on the outer edge line of the target area as the edge thickness;

[0022] Obtain the variance of the reference thickness of all edge pixels on the outer edge line of the target area as the edge fluctuation value;

[0023] Obtain the average grayscale value of all edge pixels on the outer edge line of the target area as the edge grayscale value;

[0024] The edge anomaly degree of the target area is obtained according to the edge thickness, edge fluctuation value and edge gray value; wherein the edge thickness, edge fluctuation value and edge gray value are all positively correlated with the edge anomaly degree.

[0025] Furthermore, the reference thickness is obtained by:

[0026] For any edge pixel point on the outer edge line of the target area, the straight line where the gradient direction of the edge pixel point lies is taken as the target straight line;

[0027] The intersection point of the target straight line and the inner edge line of the target area is taken as the target intersection point;

[0028] The number of pixels in all target areas on the line segment formed by the edge pixel point and the nearest target intersection point is obtained as the reference thickness of the edge pixel point.

[0029] Furthermore, the method for obtaining the internal abnormality degree is:

[0030] Obtain the mean grayscale value of all pixels in the area formed by the inner edge line of the target area as the internal grayscale value;

[0031] According to the internal gray value and the skewness of the gray value in the area constituted by the inner edge line of the target area, the internal abnormality degree of the area constituted by the inner edge line of the target area is obtained; wherein the internal gray value and the skewness are both positively correlated with the internal abnormality degree.

[0032] Furthermore, the method for obtaining the reference abnormality degree is:

[0033] Get the variance of all damage degree values ​​as the first variance;

[0034] The shortest distance between the outer edge lines of any two closed sandwich regions in the grayscale image is taken as the first distance;

[0035] The average of the grayscale values ​​of all pixels in the area formed by the outer edge line of each closed sandwich area is used as the reference grayscale value of the area formed by the outer edge line of the corresponding closed sandwich area;

[0036] The reference abnormality degree of the renal biopsy specimen is obtained according to the first variance, the mean of the first distance and the mean of the reference gray value; wherein the first variance and the mean of the reference gray value are both positively correlated with the reference abnormality degree, and the mean of the first distance is negatively correlated with the reference abnormality degree.

[0037] Further, the method for evaluating the condition of diabetic nephropathy according to the reference abnormality degree is:

[0038] When the reference abnormal degree does not meet the first abnormal range, the condition of diabetic nephropathy is assessed as early stage;

[0039] When the reference abnormality level meets the first abnormal range but does not meet the second abnormal range, the condition of diabetic nephropathy is assessed to be intermediate stage;

[0040] When the reference abnormality level satisfies the second abnormality range, the condition of diabetic nephropathy is assessed to be in the late stage.

[0041] Furthermore, the method of fluorescent labeling of the renal biopsy specimen is:

[0042] The specific protein activity markers associated with diabetic nephropathy in the renal biopsy specimens were stained with FITC to complete the fluorescent labeling of the renal biopsy specimens.

[0043] Furthermore, the method for obtaining the pixel category is:

[0044] The pixels in the grayscale image are clustered using the K-means clustering algorithm to obtain pixel categories; wherein the K value in the K-means clustering algorithm is 2.

[0045] The present invention has the following beneficial effects:

[0046] According to the distribution of gray values ​​in the gray image, the closed interlayer area is obtained, and the area corresponding to each glomerulus in the gray image is accurately determined, which is conducive to the subsequent accurate analysis of the degree of damage of each glomerulus; in order to analyze the degree of damage of each glomerulus, the degree of damage of the area constituted by the outer edge line of the target area is obtained according to the distribution of pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of gray values ​​on the outer edge line of the target area, and the distribution of gray values ​​in the area constituted by the inner edge line of the target area, which accurately reflects the degree of damage of the glomerulus corresponding to the target area, which is conducive to the subsequent accurate analysis of the condition of diabetic nephropathy; and then according to the fluctuation of the degree of damage and the position distribution of all closed interlayer areas in the gray image, as well as the distribution of gray values ​​in the area constituted by the outer edge line of each closed interlayer area, the reference abnormality of the renal biopsy specimen is obtained, which accurately reflects the overall degree of damage of the glomerulus in the renal biopsy specimen, and then the condition of diabetic nephropathy is accurately evaluated according to the reference abnormality, which is conducive to accurate treatment of diabetic nephropathy and can effectively delay the deterioration of diabetic nephropathy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of a method for assisting in identifying diabetic nephropathy based on multi-source data provided by one embodiment of the present invention;

[0049] Figure 2 A grayscale image of a renal biopsy specimen after fluorescence labeling provided by one embodiment of the present invention;

[0050] Figure 3 An image of a closed sandwich region provided by one embodiment of the present invention;

[0051] Figure 4 A flow chart of a method for obtaining a damage degree provided by an embodiment of the present invention;

[0052] Figure 5 A structural diagram of a diabetic nephropathy auxiliary identification system based on multi-source data provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the diabetic nephropathy auxiliary identification method based on multi-source data proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0055] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0056] The specific scheme of the diabetic nephropathy auxiliary identification method based on multi-source data provided by the present invention is described in detail below with reference to the accompanying drawings.

[0057] Embodiment 1:

[0058] This paper proposes a method for assisting the identification of diabetic nephropathy based on multi-source data. Figure 1 , which shows a schematic flow chart of a method for assisting in identifying diabetic nephropathy based on multi-source data provided by an embodiment of the present invention, the method comprising the following steps:

[0059] Step S1: Acquire a grayscale image of a renal biopsy specimen after fluorescence labeling.

[0060] Specifically, diabetic nephropathy (DN) is a glomerular disease caused by diabetes, which is mainly characterized by the significant proliferation of mesangial matrix in the glomerular mesangial area, also known as diabetic glomerulosclerosis. Its main pathological manifestation is nodular sclerotic glomerulopathy. Under immunofluorescence examination, the characteristics of diabetic nephropathy are mainly manifested by the linear deposition of immunoglobulins (IgG) along the walls of glomerular capillaries, and the degree of deposition varies with the progression of the disease. Specifically, in early diabetic nephropathy, the deposition of immunoglobulins (IgG) on the walls of glomerular capillaries is less, the fluorescence signal is weak, and the distribution is relatively uniform; in mid-stage diabetic nephropathy, the deposition of immunoglobulins (IgG) on the walls of glomerular capillaries is enhanced, the fluorescence signal intensity increases, and focal distribution may occur; in late-stage diabetic nephropathy, the deposition of immunoglobulins (IgG) on the walls of glomerular capillaries is significantly enhanced, the fluorescence signal intensity is high, and the distribution range is wide. Therefore, in order to evaluate the specific condition of the patient's diabetic nephropathy, this embodiment first obtains the patient's renal biopsy specimen, appropriately fixes and slices the renal biopsy specimen, and then stains the specific protein activity markers related to diabetic nephropathy in the renal biopsy specimen with fluorescein isothiocyanate (FITC) to complete the fluorescent labeling of the renal biopsy specimen. Among them, the immunoglobulins (IgG) in all glomeruli in the renal biopsy specimen are stained as green highlighted fluorescent areas, and then the condition of the patient's diabetic nephropathy is evaluated by observing the fluorescent areas. It should be noted that for the sake of accurate description, this embodiment takes a renal biopsy specimen of a patient as an example for analysis, and the renal biopsy specimens appearing in this embodiment are all renal biopsy specimens of the same patient.

[0061] In order to accurately analyze the damage of glomeruli in renal biopsy specimens, this embodiment first obtains a fluorescent image of the renal biopsy specimen after fluorescence labeling through a fluorescence microscope. It is known that the fluorescent area in the fluorescent image is the area corresponding to immunoglobulin (IgG). In order to better analyze the distribution of immunoglobulin (IgG), this embodiment grayscales the fluorescent image and then obtains a grayscale image of the renal biopsy specimen after fluorescence labeling. Figure 2 As shown in Figure 2, the fluorescence area corresponding to immunoglobulin (IgG) is Figure 2 The middle is a highlight area with a large gray value. The grayscale processing is a well-known technology and will not be described in detail.

[0062] Step S2: According to the distribution of grayscale values ​​in the grayscale image, a closed interlayer region is obtained; the closed interlayer region is a region corresponding to the immunoglobulin on the capillary wall of the glomerulus.

[0063] Specifically, in actual situations, there are many glomeruli in a renal biopsy specimen, and the distribution of immunoglobulins in each glomerulus is different. In order to better analyze the damage to each glomerulus, it is necessary to determine the area corresponding to each glomerulus in the grayscale image. It is known that there must be immunoglobulins on the capillary wall of each glomerulus corresponding to diabetic nephropathy. Therefore, in a grayscale image, an independent highlighted connected area is more likely to be the area corresponding to the immunoglobulin on the capillary wall of a glomerulus. Therefore, this embodiment first obtains the grayscale value of each pixel in the grayscale image, and then clusters the pixels in the grayscale image through the K-means clustering algorithm to obtain pixel categories; wherein, the K value in the K-means clustering algorithm is 2, and then the pixels in the grayscale image are divided into two major pixel categories. In order to better display the highlighted area in the grayscale image, the grayscale image after dividing the pixel categories is binarized, and the grayscale values ​​of the pixels in the pixel category with large grayscale values ​​are all set to 255, and the grayscale values ​​of the pixels in the pixel category with small grayscale values ​​are all set to 0. At this point, each highlighted connected domain in the grayscale image is obtained. A highlighted connected domain is more likely to be the area corresponding to the immunoglobulin on the capillary wall of a glomerulus. Among them, the K-means clustering algorithm and binarization are both well-known technologies and will not be described in detail.

[0064] As diabetic nephropathy gradually worsens, immunoglobulin (IgG) will be distributed inside the glomerulus, and a connected domain will also correspond to it in the grayscale image. Therefore, it is inaccurate to directly determine the glomerulus in the grayscale image by the connected domain. Considering that the area corresponding to the immunoglobulin on the capillary wall of each glomerulus corresponding to diabetic nephropathy must be a closed interlayer area, such as Figure 3The image shown is a closed interlayer area provided by an embodiment of the present invention, and the connected domain formed by the inner edge line and the outer edge line is the closed interlayer area. Among them, the outer edge line must not be included in other closed edge lines in the grayscale image, and the inner edge line must be the largest closed edge line included in the outer edge line. In order to obtain the closed interlayer area in the grayscale image, the present embodiment further screens the connected domains in the grayscale image through a semantic segmentation network to obtain the closed interlayer area. The semantic segmentation network of this embodiment uses a U-net neural network, and the input is all the connected domains in the grayscale image; the output is the closed interlayer area; the U-net neural network is trained and labeled in the following manner: the connected domain corresponding to the closed interlayer area is marked as 1, and other connected domains are marked as 0; the loss function of the U-net neural network is the cross entropy loss function. Among them, the U-net neural network is a well-known technology and will not be described in detail here. At this point, the closed interlayer region in the grayscale image is obtained, wherein a closed interlayer region is the region corresponding to the immunoglobulin on the capillary wall of a glomerulus, that is, the closed region formed by the outer edge line of a closed interlayer region is the region corresponding to a glomerulus. It should be noted that this embodiment only analyzes the complete glomerulus in the renal biopsy specimen.

[0065] Step S3: Select any closed interlayer area as the target area, and obtain the degree of damage of the area constituted by the outer edge line of the target area according to the distribution of pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of grayscale values ​​on the outer edge line of the target area, and the distribution of grayscale values ​​in the area constituted by the inner edge line of the target area.

[0066] Specifically, as diabetic nephropathy gradually worsens, the thickness and brightness of the fluorescent area of ​​the capillary wall of the glomerulus gradually increase, and the more the fluorescent area is distributed inside the glomerulus, therefore, this embodiment analyzes the thickness distribution and brightness distribution of each closed interlayer area, as well as the distribution of grayscale values ​​in the area formed by the inner edge line of each closed interlayer area. For better description, this embodiment selects any closed interlayer area as the target area. When the thickness of the target area is more uneven and larger and the grayscale value of the target area is larger, it means that the distribution of immunoglobulin (IgG) on the capillary wall of the glomerulus corresponding to the target area is more abnormal, which indirectly indicates that the degree of damage to the glomerulus corresponding to the target area may be greater. Among them, this embodiment analyzes the thickness of the target area by the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, and analyzes the brightness of the target area by the distribution of the grayscale value on the outer edge line of the target area. In order to more accurately analyze the damage of the glomeruli corresponding to the target area, this embodiment further analyzes the distribution of gray values ​​in the area formed by the inner edge line of the target area. When the gray value of the pixel point in the area formed by the inner edge line of the target area is larger, it further indicates that the degree of damage to the glomeruli corresponding to the target area is greater, and the possibility of aggravation of diabetic nephropathy is greater. Therefore, according to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of the gray value on the outer edge line of the target area, and the distribution of the gray value in the area formed by the inner edge line of the target area, the degree of damage to the area formed by the outer edge line of the target area is obtained. The greater the degree of damage, the more abnormal the glomeruli corresponding to the target area are, which indirectly indicates that the possibility of severe diabetic nephropathy is greater.

[0067] Preferably, in some possible implementations of this embodiment, the method for obtaining the degree of damage can refer to Figure 4 , which shows a flow chart of a method for obtaining the degree of damage provided by an embodiment of the present invention, the method comprising the following steps:

[0068] Step S301: Obtain the edge abnormality degree of the target area according to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located and the distribution of the grayscale value on the outer edge line of the target area.

[0069] It is known that the more uneven and larger the thickness of the target area is and the greater the brightness of the target area is, the more abnormal the target area is, which indirectly indicates that the glomerulus corresponding to the target area is more damaged. This embodiment obtains the reference thickness of the corresponding edge pixel point according to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, and then indicates whether the thickness of the target area is uniform according to the fluctuation degree of the reference thickness, and indicates the thickness size of the target area according to the size of the reference thickness. In some possible implementation methods of this embodiment, the method for obtaining the reference thickness is: for any edge pixel point on the outer edge line of the target area, the straight line where the gradient direction of the edge pixel point is located is used as the target straight line; the intersection of the target straight line and the inner edge line of the target area is used as the target intersection; the number of pixel points of all target areas on the line segment formed by the edge pixel point and the nearest target intersection is obtained as the reference thickness of the edge pixel point. Among them, if there is no target intersection, the reference thickness of the edge pixel point is defaulted to 0. As an example, taking the ith edge pixel point on the outer edge line of the target area as an example, the line segment formed by the ith edge pixel point and the nearest target intersection point is used as the target line segment of the ith edge pixel point, such as Figure 3 The target line segment shown is the target line segment of the i-th edge pixel point. The number of pixels on the target line segment is the reference thickness of the i-th edge pixel point. At this point, the reference thickness of each edge pixel point on the outer edge line of the target area is obtained.

[0070] In order to obtain the thickness of the target area, the mean value of the reference thickness of all edge pixels on the outer edge line of the target area is obtained as the edge thickness. The larger the edge thickness, the larger the thickness of the target area and the more abnormal the target area is. In order to analyze whether the thickness of the target area is uniform, the variance of the reference thickness of all edge pixels on the outer edge line of the target area is further obtained as the edge fluctuation value. The larger the edge fluctuation value, the more uneven the thickness distribution of the target area and the more abnormal the target area is. In order to obtain the brightness of the target area, the mean value of the grayscale value of all edge pixels on the outer edge line of the target area is obtained as the edge grayscale value. The larger the edge grayscale value, the more abnormal the target area is. Therefore, according to the edge thickness, edge fluctuation value and edge grayscale value, the degree of edge abnormality of the target area is obtained; wherein, the edge thickness, edge fluctuation value and edge grayscale value are all positively correlated with the degree of edge abnormality. The greater the degree of edge abnormality, the greater the degree of damage to the glomerulus corresponding to the target area.

[0071] In order to accurately indicate the degree of abnormality of the target area, this embodiment specifically quantifies the degree of edge abnormality as an edge abnormality value. The larger the edge abnormality value, the more abnormal the target area is, and the greater the degree of damage to the glomerulus corresponding to the target area. The calculation formula of the edge abnormality value is: ; In the formula, is the edge abnormality value of the ath closed interlayer area; is the total number of edge pixels on the outer edge line of the a-th closed sandwich area; is the reference thickness of the nth edge pixel point on the outer edge line of the ath closed sandwich area; is the edge thickness of the ath closed interlayer region; is the edge fluctuation value of the ath closed interlayer area; is the edge gray value of the ath closed interlayer region. In other embodiments, the edge abnormality value can be obtained by multiplying the edge thickness, the edge fluctuation value and the edge gray value, which is not limited here. The larger the edge abnormality value, the more uneven the thickness distribution of the corresponding closed interlayer region and the larger the brightness, the greater the degree of damage to the glomerulus corresponding to the corresponding closed interlayer region. So far, the edge abnormality value of each closed interlayer region is obtained.

[0072] Step S302: Obtain the internal abnormality degree of the area formed by the inner edge line of the target area according to the distribution of grayscale values ​​in the area formed by the inner edge line of the target area.

[0073] It is known that the more severe the diabetic nephropathy is, the more immunoglobulins (IgG) are distributed inside the glomeruli, and the display in the grayscale image is that there are more pixels with large grayscale values ​​in the area formed by the inner edge line of the target area. Therefore, this embodiment obtains the internal abnormality of the area formed by the inner edge line of the target area according to the distribution of grayscale values ​​in the area formed by the inner edge line of the target area. The greater the internal abnormality, the greater the degree of damage to the glomerulus corresponding to the target area.

[0074] Preferably, in some possible implementations of this embodiment, the method for obtaining the internal abnormality degree is: obtaining the grayscale value mean of all pixels in the area formed by the inner edge line of the target area as the internal grayscale value; the larger the internal grayscale value, the larger the grayscale value of the pixels in the area formed by the inner edge line of the target area, the more abnormalities there are inside the area formed by the inner edge line of the target area, and the more damage to the glomerulus corresponding to the target area is indirectly reflected. In order to more accurately analyze the abnormality in the area formed by the inner edge line of the target area, and then obtain the skewness of the grayscale value in the area formed by the inner edge line of the target area, the larger the skewness, the more pixels with large grayscale values ​​there are in the area formed by the inner edge line of the target area, and the more accurately it is explained that the more abnormalities there are inside the area formed by the inner edge line of the target area, the more fluorescent areas there are inside the glomerulus corresponding to the target area, and the greater the damage to the glomerulus corresponding to the target area. Among them, the method for obtaining the skewness is a well-known technology and will not be repeated. Then, according to the internal gray value and the skewness of the gray value in the area constituted by the inner edge line of the target area, the internal abnormality degree of the area constituted by the inner edge line of the target area is obtained; wherein, the internal gray value and the skewness are both positively correlated with the internal abnormality degree.

[0075] In order to accurately indicate the degree of abnormality of the area formed by the inner edge line of the target area, that is, to determine the degree of abnormality inside the glomerulus corresponding to the target area, this embodiment specifically quantifies the degree of internal abnormality as an internal abnormality degree value. The larger the internal abnormality degree value, the more abnormal the area formed by the inner edge line of the target area is, and the greater the degree of damage to the glomerulus corresponding to the target area. The calculation formula of the internal abnormality degree value is: ; In the formula, is the internal abnormality value of the area formed by the inner edge line of the ath closed interlayer area; is the internal gray value of the area formed by the inner edge line of the ath closed interlayer area; is the number of all pixels in the area formed by the inner edge line of the ath closed sandwich area; is the gray value of the mth pixel in the area formed by the inner edge line of the ath closed interlayer area; is the standard deviation of the grayscale value in the area formed by the inner edge line of the ath closed interlayer area; is the skewness of the grayscale value in the area formed by the inner edge line of the a-th closed interlayer area. In other embodiments, the internal abnormality value can be obtained by adding the internal grayscale value and the skewness, which is not limited here. So far, the internal abnormality value of the area formed by the inner edge line of each closed interlayer area is obtained.

[0076] Step S303: According to the edge abnormality degree and the internal abnormality degree, the damage degree of the area formed by the outer edge line of the target area is obtained; wherein the edge abnormality degree and the internal abnormality degree are both positively correlated with the damage degree.

[0077] The greater the degree of edge abnormality of the target area, the more immunoglobulins (IgG) deposited on the capillary wall of the glomerulus corresponding to the target area, and the greater the degree of damage to the glomerulus corresponding to the target area. When the degree of internal abnormality of the area formed by the inner edge line of the target area is greater, it means that there are more immunoglobulins (IgG) inside the glomerulus corresponding to the target area, and the wider the distribution range of immunoglobulins (IgG), the more accurately it is explained that the degree of damage to the glomerulus corresponding to the target area is greater. Therefore, according to the degree of edge abnormality and the degree of internal abnormality, the degree of damage to the area formed by the outer edge line of the target area is obtained; among them, the degree of edge abnormality and the degree of internal abnormality are both positively correlated with the degree of damage. It should be noted that the area formed by the outer edge line of the target area is the area corresponding to a glomerulus. The greater the degree of damage, the more serious the condition of diabetic nephropathy may be.

[0078] In order to accurately analyze the damage of each glomerulus, this embodiment specifically quantifies the degree of damage as a damage degree value. The larger the damage degree value, the more the corresponding glomerulus is affected by diabetic nephropathy and the greater the degree of damage. The calculation formula of the damage degree value is: ; In the formula, is the damage degree value of the area formed by the outer edge line of the ath closed interlayer area; is the edge abnormality value of the ath closed interlayer area; is the internal abnormality value of the area formed by the inner edge line of the ath closed interlayer area; norm is the normalization function. So far, the damage degree value of the area formed by the outer edge line of each closed interlayer area is obtained, that is, the damage degree value of the glomerulus corresponding to each closed interlayer area is obtained.

[0079] Step S4: Obtain a reference abnormality degree of the renal biopsy specimen based on the fluctuation of the degree of damage and the position distribution of all closed interlayer areas in the grayscale image, as well as the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area.

[0080] Specifically, in the early stage of diabetic nephropathy, the deposition of immunoglobulin (IgG) on the glomerular capillary wall is less, the fluorescence signal is relatively weak, and there is a certain spacing distance between the glomeruli. At the same time, the difference in the degree of damage between different glomeruli is small. However, with the gradual deterioration of diabetic nephropathy, the deposition of immunoglobulin (IgG) on the glomerular capillary wall increases significantly, the intensity of the fluorescence signal increases significantly, and the spacing distance between the glomeruli decreases significantly. At the same time, there are obvious differences in the degree of damage between different glomeruli. It is known that in a grayscale image, the area formed by the outer edge line of a closed interlayer area is a glomerular area. Therefore, this embodiment first analyzes the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area. When the grayscale values ​​in the area formed by the outer edge line of each closed interlayer area are larger, it means that the intensity of the fluorescence signal is stronger, which indirectly indicates that the condition of diabetic nephropathy is more serious and the abnormality of the renal biopsy specimen is greater. Further, the position distribution of all closed interlayer areas in the grayscale image is analyzed. When the position distance between different closed interlayer areas is smaller, it further indicates that the condition of diabetic nephropathy is more serious and the abnormality of the renal biopsy specimen is greater. Finally, the fluctuation of all damage degrees is analyzed. When the fluctuation of the damage degree is greater, it indicates that there are obvious differences in the damage degree between different glomeruli, which also reflects that the condition of diabetic nephropathy is more serious and the abnormality of the renal biopsy specimen is greater. Therefore, this embodiment obtains the reference abnormality of the renal biopsy specimen according to the fluctuation of the damage degree and the position distribution of all closed interlayer areas in the grayscale image, as well as the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area. The greater the reference abnormality, the more severe the patient's diabetic nephropathy is.

[0081] Preferably, in some possible implementations of this embodiment, the reference abnormality degree is obtained by: obtaining the variance of all damage degree values ​​as the first variance. The larger the first variance is, the greater the difference in the damage degree of the area formed by the outer edge lines of different closed interlayer areas in the grayscale image, that is, the greater the difference in the damage degree of different glomeruli, the greater the abnormality of the renal biopsy specimen. In order to further analyze the abnormality of the renal biopsy specimen, the shortest distance between the outer edge lines of any two closed interlayer areas in the grayscale image is obtained as the first distance; the smaller the first distance is, the more serious the condition of diabetic nephropathy may be. In order to better represent the distance between different closed interlayer areas in the grayscale image, indirectly reflecting the position distribution of glomeruli in the renal biopsy specimen, the mean of the first distance is obtained. When the mean of the first distance is smaller, it indirectly reflects that the spacing distance of glomeruli in the renal biopsy specimen is smaller, indicating that the diabetic nephropathy may be more serious and the abnormality of the renal biopsy specimen is greater. Considering that when diabetic nephropathy worsens, the brighter the fluorescent area, the larger the grayscale value of the pixel reflected in the grayscale image. Therefore, the mean of the grayscale values ​​of all pixels in the area formed by the outer edge line of each closed interlayer area is further obtained as the reference grayscale value of the area formed by the outer edge line of the corresponding closed interlayer area. The larger the reference grayscale value, the greater the possibility of severe diabetic nephropathy and the greater the abnormality of the renal biopsy specimen. In order to better reflect the severity of diabetic nephropathy, the mean of the reference grayscale value is obtained. When the mean of the reference grayscale value is larger, it means that the severity of diabetic nephropathy is greater and the abnormality of the renal biopsy specimen is greater. Then, according to the first variance, the mean of the first distance and the mean of the reference grayscale value, the reference abnormality of the renal biopsy specimen is obtained; wherein, the first variance and the mean of the reference grayscale value are both positively correlated with the reference abnormality, and the mean of the first distance is negatively correlated with the reference abnormality. The greater the reference abnormality, the more severe the diabetic nephropathy.

[0082] In order to more accurately analyze the condition of diabetic nephropathy, this embodiment specifically quantifies the reference abnormality level into a reference abnormality level value. The larger the reference abnormality level value, the more serious the condition of diabetic nephropathy. The calculation formula of the reference abnormality level value is: ; In the formula, Z is the reference abnormality value of the renal biopsy specimen; R is the total number of closed intercalation areas; is the damage degree value of the area formed by the outer edge line of the rth closed interlayer area; It is the average value of the damage degree of the area formed by the outer edge lines of all closed interlayer areas; is the first variance; is the reference gray value of the area formed by the outer edge line of the rth closed interlayer area; is the mean of the first distance; norm is a normalization function. In other embodiments, , obtain Z, which is not limited here. It should be noted that different closed interlayer regions must be separated from each other, so the mean value of the first distance must be greater than 0. So far, the reference abnormality degree value of the renal biopsy specimen is obtained.

[0083] Step S5: Evaluate the condition of diabetic nephropathy according to the reference abnormality level.

[0084] It is known that the greater the reference abnormality, the more serious the condition of diabetic nephropathy. Therefore, this embodiment sets the preset first abnormality value threshold to 0.3, and sets the preset second abnormality value threshold to 0.6. The implementer can set the preset first abnormality value threshold and the preset second abnormality value threshold according to the actual situation, and there is no limitation here. When the reference abnormality value is less than or equal to the preset first abnormality value threshold, the condition of diabetic nephropathy is evaluated as early stage; when the reference abnormality value is greater than the preset first abnormality value threshold and less than or equal to the preset second abnormality value threshold, the condition of diabetic nephropathy is evaluated as middle stage; when the reference abnormality value is greater than the preset second abnormality value threshold, the condition of diabetic nephropathy is evaluated as late stage. At this point, accurate evaluation of the patient's diabetic nephropathy condition is conducive to accurate treatment of diabetic nephropathy and effectively delaying the progression of diabetic nephropathy.

[0085] In summary, this embodiment obtains a grayscale image of a renal biopsy specimen; obtains a closed interlayer area according to the distribution of grayscale values; selects any closed interlayer area as a target area, and obtains the degree of damage according to the distribution of pixels in the target area on the straight line where the gradient direction of the upper edge pixel points on the outer edge line of the target area is located, the distribution of grayscale values ​​on the outer edge line, and the distribution of grayscale values ​​in the area formed by the inner edge line; obtains a reference abnormality degree according to the fluctuation of the degree of damage and the position distribution of the closed interlayer area, and the distribution of grayscale values ​​in the area formed by the outer edge line, and evaluates the condition of diabetic nephropathy. The present invention accurately obtains the reference abnormality degree and then accurately evaluates the condition of diabetic nephropathy, which is conducive to accurate treatment of diabetic nephropathy and delays the deterioration of diabetic nephropathy.

[0086] Embodiment 2:

[0087] The present invention also proposes a diabetic nephropathy auxiliary identification system based on multi-source data, please refer to Figure 5 , which shows a structural diagram of a diabetic nephropathy auxiliary identification system based on multi-source data provided by an embodiment of the present invention, the system includes: an acquisition module 10, a closed interlayer area acquisition module 20, a damage degree acquisition module 30, a reference abnormality degree acquisition module 40 and an evaluation module 50.

[0088] The acquisition module 10 is used to acquire a grayscale image of the renal biopsy specimen after fluorescence labeling.

[0089] The closed interlayer region acquisition module 20 is used to acquire the closed interlayer region according to the distribution of grayscale values ​​in the grayscale image; the closed interlayer region is the region corresponding to the immunoglobulin on the capillary wall of the glomerulus.

[0090] The damage degree acquisition module 30 is used to select any closed interlayer area as the target area, and obtain the damage degree of the area constituted by the outer edge line of the target area according to the distribution of pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of grayscale values ​​on the outer edge line of the target area, and the distribution of grayscale values ​​in the area constituted by the inner edge line of the target area.

[0091] The reference abnormality degree acquisition module 40 is used to acquire the reference abnormality degree of the renal biopsy specimen according to the fluctuation of the damage degree and the position distribution of all closed interlayer areas in the grayscale image, as well as the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area.

[0092] The evaluation module 50 is used to evaluate the condition of diabetic nephropathy according to the reference abnormality level.

[0093] It should be noted that: the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the above embodiment provides a system for auxiliary identification of diabetic nephropathy based on multi-source data and an embodiment of an auxiliary identification method for diabetic nephropathy based on multi-source data, which belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0094] Embodiment 3:

[0095] The present invention also proposes a device for assisting the identification of diabetic nephropathy based on multi-source data, the device comprising a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a method for assisting the identification of diabetic nephropathy based on multi-source data provided in an embodiment of the present application. The device can be specifically a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a method for assisting the identification of diabetic nephropathy based on multi-source data provided in the above embodiment.

[0096] In addition, the present application embodiment also protects a computer device, see Figure 6 The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program 403, the computer device can execute any one of the multi-source data-based diabetic nephropathy auxiliary identification methods introduced above.

[0097] Embodiment 4:

[0098] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a diabetic nephropathy auxiliary identification method based on multi-source data provided in the above embodiment.

[0099] Embodiment 5:

[0100] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the diabetic nephropathy auxiliary identification method based on multi-source data provided by the above embodiment.

[0101] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0102] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for assisting the identification of diabetic nephropathy based on multi-source data, characterized in that: The method comprises the following steps: Obtain grayscale images of renal biopsy specimens after fluorescent labeling; According to the distribution of gray values ​​in the gray image, a closed interlayer area is obtained; the closed interlayer area is the area corresponding to the immunoglobulin on the capillary wall of the glomerulus; A closed sandwich area is selected as the target area, and the degree of damage of the area formed by the outer edge line of the target area is obtained according to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the distribution of the grayscale value on the outer edge line of the target area, and the distribution of the grayscale value in the area formed by the inner edge line of the target area; The reference abnormality degree of the renal biopsy specimen is obtained according to the fluctuation of the degree of damage and the position distribution of all closed interlayer areas in the grayscale image, as well as the distribution of grayscale values ​​in the area formed by the outer edge line of each closed interlayer area; The status of diabetic nephropathy was assessed based on the degree of reference abnormality; The method for obtaining the damage extent is as follows: Obtain the degree of edge abnormality of the target area according to the distribution of pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located and the distribution of grayscale values ​​on the outer edge line of the target area; According to the distribution of gray values ​​in the area formed by the inner edge line of the target area, the internal abnormality degree of the area formed by the inner edge line of the target area is obtained; According to the edge abnormality degree and the internal abnormality degree, the damage degree of the area formed by the outer edge line of the target area is obtained; wherein the edge abnormality degree and the internal abnormality degree are both positively correlated with the damage degree; The method for obtaining the edge abnormality degree is: According to the distribution of the pixel points in the target area on the straight line where the gradient direction of each edge pixel point on the outer edge line of the target area is located, the reference thickness of the corresponding edge pixel point is obtained; Obtain the average value of the reference thickness of all edge pixels on the outer edge line of the target area as the edge thickness; Obtain the variance of the reference thickness of all edge pixels on the outer edge line of the target area as the edge fluctuation value; Obtain the average grayscale value of all edge pixels on the outer edge line of the target area as the edge grayscale value; According to the edge thickness, edge fluctuation value and edge gray value, the edge abnormality degree of the target area is obtained; wherein the edge thickness, edge fluctuation value and edge gray value are all positively correlated with the edge abnormality degree; The method for obtaining the internal abnormality degree is: Obtain the mean grayscale value of all pixels in the area formed by the inner edge line of the target area as the internal grayscale value; According to the internal gray value and the skewness of the gray value in the area constituted by the inner edge line of the target area, the internal abnormality degree of the area constituted by the inner edge line of the target area is obtained; wherein the internal gray value and the skewness are both positively correlated with the internal abnormality degree.

2. The method for assisting in identifying diabetic nephropathy based on multi-source data according to claim 1, characterized in that: The method for obtaining the closed interlayer area is: According to the gray value of each pixel in the gray image, the pixels in the gray image are divided to obtain pixel categories; Binarize the grayscale image after dividing the pixel points into categories to obtain the connected domain in the grayscale image; The connected domains in the grayscale image are screened through the semantic segmentation network to obtain the closed interlayer area; wherein the closed interlayer area is a connected domain composed of the closed edge lines not included in the grayscale image and the maximum closed edge line included in it.

3. The method for assisting in identifying diabetic nephropathy based on multi-source data as claimed in claim 1, characterized in that: The method for obtaining the reference thickness is: For any edge pixel point on the outer edge line of the target area, the straight line where the gradient direction of the edge pixel point lies is taken as the target straight line; The intersection point of the target straight line and the inner edge line of the target area is taken as the target intersection point; The number of pixels in all target areas on the line segment formed by the edge pixel point and the nearest target intersection point is obtained as the reference thickness of the edge pixel point.

4. The method for assisting in identifying diabetic nephropathy based on multi-source data according to claim 1, characterized in that: The method for obtaining the reference abnormality degree is: Get the variance of all damage degree values ​​as the first variance; The shortest distance between the outer edge lines of any two closed sandwich regions in the grayscale image is taken as the first distance; The average of the grayscale values ​​of all pixels in the area formed by the outer edge line of each closed sandwich area is used as the reference grayscale value of the area formed by the outer edge line of the corresponding closed sandwich area; The reference abnormality degree of the renal biopsy specimen is obtained according to the first variance, the mean of the first distance and the mean of the reference gray value; wherein the first variance and the mean of the reference gray value are both positively correlated with the reference abnormality degree, and the mean of the first distance is negatively correlated with the reference abnormality degree.

5. The method for assisting in identifying diabetic nephropathy based on multi-source data as claimed in claim 1, characterized in that: The method for evaluating the condition of diabetic nephropathy according to the reference abnormality degree is: When the reference abnormal degree does not meet the first abnormal range, the condition of diabetic nephropathy is assessed as early stage; When the reference abnormality level meets the first abnormal range but does not meet the second abnormal range, the condition of diabetic nephropathy is assessed to be intermediate stage; When the reference abnormality level satisfies the second abnormality range, the condition of diabetic nephropathy is assessed to be in the late stage.

6. The method for assisting in identifying diabetic nephropathy based on multi-source data as claimed in claim 1, characterized in that: The method for fluorescent labeling of the renal biopsy specimen is: The specific protein activity markers associated with diabetic nephropathy in the renal biopsy specimens were stained with FITC to complete the fluorescent labeling of the renal biopsy specimens.

7. The method for assisting in identifying diabetic nephropathy based on multi-source data as claimed in claim 2, characterized in that: The method for obtaining the pixel category is: The pixels in the grayscale image are clustered using the K-means clustering algorithm to obtain pixel categories; wherein the K value in the K-means clustering algorithm is 2.

Images