A cervical lesion grading method based on deep learning

Abstract

The present invention relates to the field of image processing technology, and more specifically to a method for grading and discriminating cervical lesions based on deep learning, comprising: obtaining a plurality of lesion representation areas from a collected cervical image; analyzing the degree of irregularity and edge abruptness to obtain a reflective interference index for each lesion representation area; dividing each lesion representation area to obtain an internal closed area; analyzing the tubular appearance to obtain blood shape diffusion; obtaining blood accumulation appearance based on grayscale uniformity; obtaining a blood interference index for each lesion representation area based on the blood shape diffusion and blood accumulation appearance; constructing a multidimensional feature vector for each lesion representation area; and training a deep learning model to perform cervical lesion grading and discrimination. The present invention aims to address the problem of errors caused by reflection and blood interference in cervical images when training a cervical lesion grading and discrimination model using deep learning, thereby improving the recognition accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for grading and distinguishing cervical lesions based on deep learning. Background Art

[0002] Cervical cancer is a common malignancy in women. Cervical intraepithelial neoplasia (CIN), a precancerous lesion, is crucial for early detection and accurate grading to prevent cancer. CIN is classified into three grades: CIN1, CIN2, and CIN3. Identifying and grading CIN can effectively improve prevention and treatment.

[0003] There are existing methods for grading and identifying cervical lesions using deep learning models, but cervical images are easily interfered with by factors such as the reflected highlights of the cervical os mirror and blood contamination at the cervical os. As a result, when using deep learning to grade cervical lesions in cervical images, the intensity of the lesion features in the cervical lesion area that exhibits lesion characteristics in the cervical image decreases, causing the deep learning model to misclassify due to image interference. Summary of the Invention

[0004] The present invention provides a cervical lesion grading and discrimination method based on deep learning to solve the existing problems.

[0005] The present invention provides a method for grading and distinguishing cervical lesions based on deep learning, which adopts the following technical solutions:

[0006] An embodiment of the present invention provides a method for grading cervical lesions based on deep learning, the method comprising the following steps:

[0007] The collected cervical images are divided and screened to obtain several lesion characterization areas;

[0008] Selecting each highlighted portion in each lesion representation region as each highlighted region to be determined in each lesion representation region;

[0009] In each lesion representation area, the degree of irregularity and edge abruptness of each highlight area to be determined meeting the requirements of reflective interference is analyzed to obtain the reflective interference index of each lesion representation area;

[0010] Each lesion representation area is divided using closed texture to obtain the internal closed area in each lesion representation area;

[0011] In each lesion characterization region, the tubular appearance of each internal closed region is analyzed to obtain the blood shape diffusion degree of each internal closed region; the blood accumulation expression degree of each internal closed region is obtained based on the grayscale uniformity of each internal closed region; and the blood interference index of each lesion characterization region is obtained based on the blood shape diffusion degree and blood accumulation expression degree of all internal closed regions in each lesion characterization region;

[0012] Constructing a multidimensional feature vector of each lesion representation area according to the blood interference index and the reflection interference index of each lesion representation area;

[0013] The multidimensional feature vector is used to train a deep learning model to perform cervical lesion grading judgment.

[0014] Preferably, the specific steps of dividing and screening the collected cervical images to obtain a plurality of lesion characterization areas include:

[0015] collecting a cervical image to obtain a cervical grayscale image of the cervical image;

[0016] After finely dividing the cervical grayscale image, a number of sub-pixel blocks are obtained; and according to the color and texture information of the sub-pixel blocks and their positions in the cervical grayscale image, all the sub-pixel blocks are clustered to obtain a number of initial divided regions;

[0017] The average value of the ratio of the brightness channel V to the saturation channel S of all pixels corresponding to each initial divided area in the cervical image is recorded as the reference weight of each initial divided area;

[0018] Arrange the reference weights of all initially divided areas in descending order to obtain a descending sequence of reference weights;

[0019] The difference between the reference weight of each number and the next number in the descending sequence of reference weight is recorded as the reference difference of each number in the descending sequence of reference weight;

[0020] Each initially divided region corresponding to each number from the first number to the number with the largest reference difference in the descending sequence of reference weights is recorded as a lesion representation region.

[0021] Preferably, the specific steps of obtaining each highlight area to be determined include:

[0022] The binary segmentation of each lesion representation area is performed using the Otsu threshold algorithm to obtain a binary segmentation map of each lesion representation area;

[0023] After performing a morphological closing operation on the binary segmentation image, each closed area formed by a pixel gray value of 1 is recorded as a highlight area to be determined.

[0024] Preferably, the specific steps of obtaining the reflection interference index include:

[0025] Obtaining the shape irregularity and edge abruptness of each highlight area to be determined in each lesion representation area;

[0026] The reflection interference index of each lesion characterization area is obtained, and the shape irregularity and edge sudden change degree of the lesion characterization area are in direct proportion to all the highlight areas to be determined.

[0027] Preferably, obtaining the shape irregularity and edge abruptness of each to-be-determined highlight region of each lesion characterization region comprises:

[0028] In each lesion representation area, the convex hull area of ​​each highlight area to be determined is obtained, and the ratio of the number of pixels in the convex hull area of ​​each highlight area to be determined to the number of pixels in the convex hull area of ​​the highlight area to be determined is recorded as the shape irregularity of each highlight area to be determined;

[0029] In each lesion representation area, obtain the fitting line segment between each edge pixel point of each highlight area to be determined and the next pixel point in the clockwise direction, and record it as the trend line segment of each edge pixel point of each highlight area to be determined;

[0030] The difference in slope between each edge pixel point of each highlight area to be determined and the next edge pixel point in the clockwise direction is recorded as the gradient sudden change amplitude of each edge pixel point of each highlight area to be determined;

[0031] The edge sudden change degree of each highlight area to be determined is obtained by integrating the gradient sudden change amplitude and edge gradient value of all edge pixels of each highlight area to be determined, wherein the edge sudden change degree is directly proportional to the distribution range of the gradient sudden change amplitude, and the edge sudden change degree is inversely proportional to the edge gradient value.

[0032] Preferably, the specific steps of obtaining the blood shape diffusivity include:

[0033] In each lesion representation region, the principal axis direction of each internal closed region was obtained;

[0034] The projection length of the edge of the internal closed area in the main axis direction is recorded as the main axis projection length of each internal closed area;

[0035] The projection length of the edge of the internal closed area in the tangent direction of the main axis direction is recorded as the tangent projection length of each internal closed area;

[0036] Obtaining a non-tubular expression degree of each internal closed area, wherein the non-tubular expression degree is inversely proportional to the main axis projection length and is directly proportional to the tangent projection length;

[0037] For each pixel point that the main axis direction of each internal closed area passes through, obtain the distance of each pixel point from the edge of the internal closed area in the tangent direction of the main axis direction, and record it as the relative diameter of each pixel point that the main axis direction of each internal closed area passes through;

[0038] Obtain the variance of the relative diameters of all pixels passed by the main axis direction of each internal closed area, which is recorded as the diameter inconsistency of each internal closed area;

[0039] The blood shape diffusion of each internal closed area is obtained, and the blood shape diffusion is in direct proportion to the diameter non-uniformity and the non-tubular expression.

[0040] Preferably, the specific steps of obtaining the blood accumulation expression include:

[0041] Obtain the average of all grayscale values ​​of each pixel point passed by the main axis direction of each internal closed area in the tangent direction of the main axis direction, and record it as the grayscale value of the section of each pixel point passed by the main axis direction of each internal closed area;

[0042] The sequence of the grayscale values ​​of the cross-section in the direction pointed by the main axis direction of each internal closed area is recorded as the grayscale value sequence of the cross-section of each internal closed area;

[0043] The cumulative sum of the absolute values ​​of the differences between each sequence value and the next sequence value in the section grayscale value sequence is recorded as the blood pool expression degree of each internal closed area.

[0044] Preferably, the specific steps of obtaining the blood interference index include:

[0045] A blood interference index is obtained for each lesion characterization region, wherein the blood interference index is in positive proportion to the blood shape diffusion and blood accumulation expression of all internal closed areas in each lesion characterization region.

[0046] Preferably, constructing a multidimensional feature vector of each lesion characterization region according to the blood interference index and the reflection interference index of each lesion characterization region includes:

[0047] The blood interference index of the i-th lesion representation area is recorded as ;

[0048] The reflection interference index of the i-th lesion representation area is recorded as ;

[0049] The multidimensional feature vector of the i-th lesion representation region ,in is the transpose symbol.

[0050] Preferably, the multi-dimensional feature vector is used to train a deep learning model to perform cervical lesion grading judgment, including:

[0051] The multidimensional feature vector of each lesion representation area in the cth cervical image is assigned to each pixel point of each lesion representation area in the cth cervical image, and the spatial attention weight matrix of the cth cervical image is obtained. The size of the spatial attention weight matrix is ,in and are the length and width of the cth cervical image respectively, is the number of dimensions of the multidimensional feature vector representing each lesion region in the cth cervical image;

[0052] Input the c-th image into the CNN neural network to obtain the feature map of the c-th cervical image. The product of the feature map of the c-th cervical image and its spatial attention weight matrix is ​​recorded as the modified attention feature map of the c-th cervical image.

[0053] Pre-construct the training set to obtain the modified attention feature maps of all cervical images in the training set;

[0054] The CNN model is trained with the modified attention feature maps of all cervical images in the training set to obtain a cervical lesion grading discrimination model, which is then used to perform cervical lesion grading judgment.

[0055] The beneficial effects of the technical solution of the present invention are as follows: the present application collects cervical lesion images, divides the images into regions to obtain lesion representation regions, analyzes the reflective features in the highlighted regions to be determined in the lesion representation regions, obtains the reflective interference index of each lesion representation region, and is used to reflect the situation in which each lesion representation region is affected by interference. At the same time, the lesion representation region is divided according to the texture to obtain an internal closed region, and the internal closed region is analyzed to meet the blood features of uneven width and the blood features of uneven grayscale, and the blood interference index of each lesion representation region is obtained to reflect the situation in which each lesion representation region is interfered with by blood. The blood interference index and the reflective interference index are then used to construct a multidimensional feature vector, and the spatial attention weight matrix is ​​generated to adjust the reference when training the cervical lesion grading and discrimination model, so as to achieve the purpose of improving the accuracy of cervical lesion analysis and discrimination. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a flowchart of the steps of a method for grading cervical lesions based on deep learning in the present invention. DETAILED DESCRIPTION

[0058] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a deep learning-based cervical lesion grading method proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0059] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0060] The following describes in detail a specific scheme of a cervical lesion grading and discrimination method based on deep learning provided by the present invention with reference to the accompanying drawings.

[0061] See also Figure 1 , which shows a flowchart of a method for grading cervical lesions based on deep learning provided by one embodiment of the present invention, the method comprising the following steps:

[0062] Step S001: The collected cervical image is divided and screened to obtain a number of lesion representation areas.

[0063] It should be noted that the existing deep learning model can be trained on a training set consisting of collected cervical images to obtain a cervical lesion grading and discrimination model, and the cervical image can be input into the cervical lesion grading and discrimination model to automatically obtain the cervical lesion grade of precancerous lesions. However, since the vagina needs to be dilated with a dilator during the cervical image collection process, the colposcope enters the vagina from the space opened by the dilator and is aimed at the cervical opening for collection. During this process, the dilator will cause slight bleeding on the surface of the cervix, and the colposcope has a light source, which will produce a reflective effect after being reflected by vaginal secretions and mucus, thereby interfering with the acetowhite reflection of the cervical opening. This causes interference in the deep learning model when training the cervical lesion grading and discrimination model, reducing the accuracy and precision of the cervical lesion grading and discrimination model.

[0064] Therefore, this embodiment analyzes the collected cervical images to obtain multidimensional feature vectors for each region in the cervical image, and uses the spatial attention mechanism to generate the corresponding spatial attention weight matrix for the cervical image. This allows the cervical lesion classification model to pay different attention to different regions in the training set, thereby achieving the purpose of avoiding the impact of interference on model accuracy. First, it is necessary to collect cervical images and perform corresponding regional division preprocessing.

[0065] Preferably, the specific steps of dividing and screening the collected cervical image to obtain a number of lesion characterization areas include:

[0066] collecting a cervical image to obtain a cervical grayscale image of the cervical image;

[0067] After finely dividing the cervical grayscale image, a number of sub-pixel blocks are obtained; and according to the color and texture information of the sub-pixel blocks and their positions in the cervical grayscale image, all the sub-pixel blocks are clustered to obtain a number of initial divided regions;

[0068] According to the brightness and saturation of the pixels in each initial divided area, a reference weight of each initial divided area is obtained, and the initial divided areas are screened using the reference weight to obtain a number of lesion characterization areas.

[0069] Specifically, the specific method of collecting the cervical image and obtaining the cervical grayscale image is:

[0070] It should be noted that, due to the degeneration of the epithelial tissue in cervical lesions, acetic acid solution can be applied to the epithelial tissue to cause the protein in the abnormal epithelial tissue to coagulate, thus presenting an acetowhite area at the cervical os. Therefore, when performing cervical lesion grading, the cervical image collected in this embodiment is an image in HSV format, which can better show the color information of the cervical os. The specific cervical image collection method is as follows:

[0071] After dilating the vagina with a dilator, a 3% acetic acid solution was applied to the patient's cervix, and a colposcope was used to obtain an RGB source image of the cervix;

[0072] The cervical source image is converted from RGB format to HSV format to obtain a cervical image; and the cervical source image is weighted grayscaled to obtain a cervical grayscale image of the cervical image.

[0073] It should be noted that the weighted grayscale conversion and the conversion of RGB format to HSV format described in this embodiment are both existing well-known technologies and will not be described in detail in this embodiment.

[0074] Furthermore, the cervical grayscale image is finely divided to obtain a number of sub-pixel blocks. All sub-pixel blocks are clustered according to the color, texture information and position of the sub-pixel blocks in the cervical grayscale image to obtain a number of initial divided areas. The specific method is as follows:

[0075] It should be noted that the cervical grayscale image includes cervical tissue areas, lesion areas, vaginal walls or surrounding background, shadows or reflective areas, as well as some secretions, blood, and air bubble areas, all of which will interfere with the cervical grayscale image. Therefore, it is necessary to first divide the cervical grayscale image into several sub-pixel blocks, and then merge the sub-pixel blocks with similar attributes to obtain several initial divided areas, so that each initial divided area represents the same type of tissue or interference. The specific operations are:

[0076] Use superpixel segmentation algorithm to divide the cervical grayscale image into several sub-pixel blocks;

[0077] The HSV three-channel values ​​of each sub-pixel block in the corresponding area of ​​the cervical image, the LBP code mean of the sub-pixel block, and the center of gravity position of the sub-pixel block are used as clustering parameters to construct a multidimensional clustering space. After clustering all sub-pixel blocks in the multidimensional clustering space using the K-means clustering algorithm, several clusters are obtained. The adjacent sub-pixel blocks belonging to the same cluster in the cervical grayscale image are spliced ​​to obtain several initial division areas.

[0078] It should be noted that the superpixel segmentation algorithm and the K-means clustering algorithm are both existing publicly known technologies, wherein the LBP code mean is obtained using the publicly known LBP algorithm. The LBP algorithm can reflect the texture information composed of each pixel point in the image and the surrounding pixels. The specific method of obtaining the LBP code mean is: using the LBP algorithm to obtain the LBP code of each pixel point in each sub-pixel block, and taking the mean of the LBP codes of all pixels in each sub-pixel block as the LBP code mean of the sub-pixel block, which is used to represent the texture information of the sub-pixel block.

[0079] Furthermore, according to the brightness and saturation of the pixels in each initial divided area, a reference weight of each initial divided area is obtained, and the reference weight is used to screen the initial divided areas to obtain several lesion characterization areas. The specific method is as follows:

[0080] It should be noted that the epithelial tissue of cervical lesions will produce an acetowhite reaction after being smeared with acetowhite solution, which appears as high brightness and low saturation in the HSV format image. Therefore, the reference weight of each initial divided area is obtained by the brightness and saturation of the pixels in each initial divided area. The reference weight is used to screen the initial divided areas to obtain several lesion representation areas. The specific steps are:

[0081] The average value of the ratio of the brightness channel V to the saturation channel S of all pixels corresponding to each initial divided area in the cervical image is recorded as the reference weight of each initial divided area;

[0082] Arrange the reference weights of all initially divided areas in descending order to obtain a descending sequence of reference weights;

[0083] The difference between the reference weight of each number and the next number in the descending sequence of reference weight is recorded as the reference difference of each number in the descending sequence of reference weight;

[0084] Each initially divided region corresponding to each number from the first number to the number with the largest reference difference in the descending sequence of reference weights is recorded as a lesion representation region.

[0085] It should be noted that if there is a pixel whose saturation channel S has a value of 0, the value of the saturation channel S of the pixel is assigned to 0.01.

[0086] It should be further explained that since the color of the abnormal epithelial tissue after the acetowhite reaction is closer to white than that of other areas, the value of the reference weight will be much larger than that of the normal area. This embodiment uses the maximum descending difference to achieve a polarized division of the reference weight, thereby obtaining the lesion representation area.

[0087] Step S002: Select each highlighted part in each lesion representation area as each highlighted area to be determined in each lesion representation area; in each lesion representation area, analyze the irregularity and edge sudden change degree of each highlighted area to be determined to meet the reflection interference, and obtain the reflection interference index of each lesion representation area.

[0088] The grading of cervical precancerous lesions is primarily based on the cervical intraepithelial neoplasia (CIN) grading criteria to determine the severity of cervical lesions. The core basis for grading is the longitudinal accumulation of atypical cells within the thickness of the cervical epithelium. In the evaluation of cervical images, the corresponding CIN grade should also be indirectly inferred based on imaging features such as the degree of acetowhite reaction, color, surface structure, and vascular pattern of the lesion area. However, in this process, when the colposcopy light source is reflected on the secreted fluid on the surface of the cervix, it easily forms a reflective highlight area. In the cervical lesion area, the abnormal epithelial tissue degenerates after the action of acetic acid, and the protein coagulates, resulting in increased reflectivity and a milky white appearance, similar to the appearance of the reflective highlight area. As a result, when training the cervical lesion grading and discrimination model, the reflective highlight area is easily selected as the acetowhite area of ​​the acetowhite reaction, affecting the accuracy of the cervical lesion grading and discrimination model.

[0089] Therefore, this embodiment distinguishes between reflection and acetowhite reaction in the lesion representation area, thereby obtaining the reflection interference index of the lesion representation area.

[0090] Preferably, the specific steps of selecting each highlighted portion in each lesion representation region as each highlighted region to be determined in each lesion representation region include:

[0091] The binary segmentation of each lesion representation area is performed using the Otsu threshold algorithm to obtain a binary segmentation map of each lesion representation area;

[0092] After performing a morphological closing operation on the binary segmentation image, each closed area formed by a pixel gray value of 1 is recorded as a highlight area to be determined.

[0093] It should be noted that the Otsu threshold algorithm and morphological closing operation described in this embodiment are both existing well-known technologies and will not be described in detail in this embodiment.

[0094] It should be further explained that the area with a gray value of 1 in the binary segmentation image obtained by Otsu threshold segmentation corresponds to the highlight area in the lesion representation area, so it is analyzed as the highlight area to be determined to determine whether it belongs to the acetowhite area or the reflective highlight area.

[0095] It should be noted that the reflective highlight area appears as a dot or irregular patch with a clear edge; whereas the acetowhite area of ​​a cervical lesion treated with acetic acid exhibits a characteristic of deriving from the center to the edge. This means that the acetowhite area is a relatively regular, patchy structure, and the boundary of the acetowhite area becomes blurred due to the gradual expansion of the affected area. This indicates that the lesion area exhibits a derivative characteristic of natural diffusion. Therefore, based on the shape regularity and edge derivative characteristics of the acetowhite area in the highlight area to be determined, this embodiment analyzes the areas that meet the requirements for irregularity and edge abruptness for reflective interference, and obtains a reflective interference index for each lesion characterization area.

[0096] Furthermore, in each lesion representation area, the irregularity and edge abruptness of each highlight area to be determined meeting the requirements for reflective interference are analyzed, and the specific steps of obtaining the reflective interference index of each lesion representation area include:

[0097] In each lesion representation area, the shape irregularity of each highlight area to be determined is obtained according to the area difference between each highlight area to be determined and its convex hull area;

[0098] Obtaining the edge sudden change degree of each highlight area to be determined according to the edge gradient value of each edge pixel point of each highlight area to be determined and the gradient sudden change amplitude with adjacent edge pixels;

[0099] The shape irregularity and edge abruptness of all highlight areas to be determined are comprehensively considered to obtain the reflection interference index of each lesion characterization area.

[0100] Specifically, according to the area difference between each highlight region to be determined and its convex hull region, the shape irregularity of each highlight region to be determined is obtained in the following manner:

[0101] In each lesion representation area, the convex hull area of ​​each highlight area to be determined is obtained, and the ratio of the number of pixels in the convex hull area of ​​each highlight area to be determined to the number of pixels in itself is recorded as the shape irregularity of each highlight area to be determined.

[0102] It should be noted that since the convex hull area of ​​a region must be smaller than or equal to the region, when the region is regular, the convex hull area is equal to the region, and the greater the difference between the area of ​​the convex hull area and the area of ​​the region, the more irregular the region.

[0103] Furthermore, according to the edge gradient value of each edge pixel point of each highlight area to be determined and the gradient sudden change amplitude with the adjacent edge pixel points, the specific method of obtaining the edge sudden change degree of each highlight area to be determined is:

[0104] In each lesion representation area, obtain the fitting line segment between each edge pixel point of each highlight area to be determined and the next pixel point in the clockwise direction, and record it as the trend line segment of each edge pixel point of each highlight area to be determined;

[0105] The difference in slope between each edge pixel point of each highlight area to be determined and the next edge pixel point in the clockwise direction is recorded as the gradient sudden change amplitude of each edge pixel point of each highlight area to be determined;

[0106] The edge sudden change degree of each highlight area to be determined is obtained by integrating the gradient sudden change amplitude and edge gradient value of all edge pixels of each highlight area to be determined, wherein the edge sudden change degree is directly proportional to the distribution range of the gradient sudden change amplitude, and the edge sudden change degree is inversely proportional to the edge gradient value.

[0107] As an example, the edge sudden change degree of the jth highlight area to be determined in the i-th lesion representation area is The calculation method is:

[0108] ;

[0109] in, is the mean edge gradient value of all edge pixels in the jth highlight area to be determined in the i-th lesion representation area; is the variance of the gradient sudden change amplitude of all edge pixels in the jth highlight area to be determined in the i-th lesion representation area, and is used to represent the distribution range of the gradient sudden change amplitude.

[0110] Furthermore, the shape irregularity and edge abruptness of all highlight areas to be determined are comprehensively considered to obtain the reflection interference index of each lesion characterization area. The specific method is as follows:

[0111] Obtain the reflective interference index of each lesion characterization area, wherein the shape irregularity and edge abruptness of the lesion characterization area are in direct proportion to the shape irregularity and edge abruptness of all the highlight areas to be determined. As an example, the reflective interference index of the i-th lesion characterization area is obtained as follows:

[0112] Obtain the product of the shape irregularity and the edge abruptness of each highlight area to be determined in the i-th lesion representation area, and record it as the reflective performance of each highlight area to be determined in the i-th lesion representation area;

[0113] The result obtained by normalizing the cumulative sum of the reflective performance of all the highlight areas to be determined in the i-th lesion representation area is recorded as the reflective interference index of the i-th lesion representation area.

[0114] It should be noted that this embodiment uses a sigmoid function to normalize the accumulated sum of the reflective performance, wherein the sigmoid function is a well-known technology.

[0115] Step S003: Divide each lesion characterization area using closed textures to obtain internal closed areas in each lesion characterization area; within each lesion characterization area, analyze the tubular expression of each internal closed area to obtain the blood shape diffusion of each internal closed area; obtain the blood accumulation expression of each internal closed area based on the grayscale uniformity of each internal closed area; obtain the blood interference index of each lesion characterization area based on the blood shape diffusion and blood accumulation expression of all internal closed areas in each lesion characterization area.

[0116] Precancerous lesions contain numerous blood vessels. When capturing cervical images, the fragile cervical os can easily damage the vaginal and cervical mucosa, resulting in minor bleeding. This blood interferes with the blood vessels when training the cervical lesion grading model, reducing the recognition accuracy of the model. Therefore, this embodiment analyzes blood vessels and diffuse blood to obtain a blood interference index for each lesion representation area, which is then used as the spatial attention weight in the adaptive attention mechanism.

[0117] It should be noted that the distribution of blood on the surface of cervical tissue appears as irregular spots or patches, lacking clear linear structural features. However, the blood vessels within the lesion area appear as elongated, linear, or cord-like tubular structures, with distinct geometric linear edge features. Therefore, this embodiment divides the lesion representation area into internal closed regions by edge segmentation, and then analyzes the blood or blood vessel characteristics presented in these internal closed regions to obtain the blood interference index for the lesion representation area.

[0118] Preferably, the specific steps of dividing each lesion representation region by using closed textures to obtain the inner closed region in each lesion representation region are:

[0119] Edge detection is performed on each lesion representation area, and the area formed by each closed edge in the edge detection result is recorded as an internal closed area.

[0120] It should be noted that the region formed by the closed edges described in this embodiment includes the region formed by the edges detected by the edge detection, and also includes the region formed by the edges detected by the edge detection and the edges of the lesion representation region.

[0121] Preferably, in each lesion characterization region, the tubular manifestation of each internal closed region is analyzed, and the specific method for obtaining the blood shape diffusion degree of each internal closed region is:

[0122] In each lesion representation region, the principal axis direction of each internal closed region was obtained;

[0123] The non-tubular expression degree of each internal closed area is obtained according to the difference in the projection length of the edge of each internal closed area in the main axis direction;

[0124] The diameter distribution of each internal closed area in the main axis direction and the tangent direction of each pixel point is analyzed, and the blood shape diffusion degree of each internal closed area is obtained in combination with the non-tubular expression.

[0125] Specifically, in each lesion representation region, the specific method for obtaining the principal axis direction of each internal closed region is:

[0126] The grayscale value surface of each internal closed area is constructed using the grayscale value of each pixel in each internal closed area;

[0127] The principal component analysis algorithm is used to obtain the principal component direction of the gray value surface, which is recorded as the principal axis direction of each internal closed area.

[0128] It should be noted that the principal component analysis algorithm described in this embodiment is a well-known technique and will not be further described in this embodiment. The principal component analysis algorithm can be used to determine the direction of the data. If the internal closed region is blood, then the internal closed region has no clear direction. However, due to the distinct tubular structure of the blood vessel, a clear direction is evident. Therefore, the principal component analysis algorithm can be used to determine the principal axis direction of the internal closed region.

[0129] Furthermore, based on the difference in the projected lengths of the edges of each internal closed region in the main axis direction, the specific method for obtaining the non-tubular expression degree of each internal closed region is as follows:

[0130] The projection length of the edge of the internal closed area in the main axis direction is recorded as the main axis projection length of each internal closed area;

[0131] The projection length of the edge of the internal closed area in the tangent direction of the main axis direction is recorded as the tangent projection length of each internal closed area;

[0132] The non-tubular expression degree of each internal closed area is obtained, where the non-tubular expression degree is inversely proportional to the main axis projection length and is directly proportional to the tangent projection length.

[0133] As an example, the calculation method for obtaining the non-tubular representation of each internal closed area is:

[0134] The ratio of the tangent projection length to the principal axis projection length of each internal closed region is recorded as the non-tubular expression degree of each internal closed region.

[0135] It should be noted that if the internal closed area is a blood vessel, then the main axis projection length is the length of the blood vessel, and the tangent projection length is the width of the blood vessel. When the width-to-length ratio of the internal closed area is larger, it means that it does not conform to the long strip shape of the blood vessel, and is more likely to be blood interference.

[0136] It should be noted that since the cervical opening is annular and protrudes from the plane of the vaginal wall, wrinkles will form on the surface. If blood flows into the wrinkles, it will also present a tubular structure. However, the diameter of the wrinkles is different from that of the blood vessels, and the diameter of the blood vessels changes little over a short distance. Therefore, this embodiment obtains the blood shape diffusion of each internal closed area by analyzing the diameter change of the internal closed area in the main axis direction and combining it with the non-tubular expression.

[0137] Specifically, the diameter distribution of each internal closed area in the tangential direction of each pixel point in the main axis direction is analyzed, and combined with the non-tubular expression, the specific method of obtaining the blood shape diffusion degree of each internal closed area is as follows:

[0138] For each pixel point that the main axis direction of each internal closed area passes through, obtain the distance of each pixel point from the edge of the internal closed area in the tangent direction of the main axis direction, and record it as the relative diameter of each pixel point that the main axis direction of each internal closed area passes through;

[0139] Obtain the variance of the relative diameters of all pixels passed by the main axis direction of each internal closed area, which is recorded as the diameter inconsistency of each internal closed area;

[0140] The blood shape diffusion of each internal closed area is obtained, and the blood shape diffusion is in direct proportion to the diameter non-uniformity and the non-tubular expression.

[0141] It should be noted that the variance of the relative diameters of all pixel points indicates the distribution range of the width of the internal closed area. The smaller the value, the smaller the distribution range, that is, the more consistent the diameter, and the internal closed area is more likely to be a blood vessel. When the variance value is larger, it indicates that the internal closed area is more likely to be blood, that is, the diameter inconsistency value is larger.

[0142] As an example, the specific calculation method for obtaining the blood shape diffusion of each internal closed area is:

[0143] The product of the diameter non-uniformity and the non-tubular expression of each internal closed area is recorded as the blood shape diffusion index of each internal closed area.

[0144] It should be noted that, since the blood in the blood vessels is wrapped by the blood vessel walls, the grayscale values ​​of the internal closed areas are approximately the same. Even if the blood vessels are at different depths from the mucosal surface, the changes are uniform. However, the interfering blood is easily accumulated in the folds, resulting in uneven distribution of the internal closed areas formed. Therefore, this embodiment obtains the blood accumulation expression of each internal closed area by analyzing the grayscale uniformity of the back closed area.

[0145] Preferably, the specific method of obtaining the blood pooling expression of each internal closed area according to the grayscale uniformity of each internal closed area is:

[0146] Obtain the average of all grayscale values ​​of each pixel point passed by the main axis direction of each internal closed area in the tangent direction of the main axis direction, and record it as the grayscale value of the section of each pixel point passed by the main axis direction of each internal closed area;

[0147] The sequence of the grayscale values ​​of the cross-section in the direction pointed by the main axis direction of each internal closed area is recorded as the grayscale value sequence of the cross-section of each internal closed area;

[0148] The cumulative sum of the absolute values ​​of the differences between each sequence value and the next sequence value in the section grayscale value sequence is recorded as the blood pool expression degree of each internal closed area.

[0149] It should be noted that when the grayscale value in the internal closed area is uniform, the absolute value of the difference between each sequence value and the next sequence value is small, and the cumulative sum is also small. Therefore, the larger the cumulative sum, that is, the greater the blood accumulation expression, the more likely the internal closed area is to be interfered with by blood.

[0150] Preferably, the specific steps of obtaining the blood interference index of each lesion characterization area according to the blood shape diffusivity and blood accumulation expression of all internal closed areas in each lesion characterization area are:

[0151] A blood interference index is obtained for each lesion characterization region, wherein the blood interference index is in positive proportion to the blood shape diffusion and blood accumulation expression of all internal closed areas in each lesion characterization region.

[0152] As an example, the specific method of obtaining the blood interference index of each lesion characterization area is:

[0153] The product of the blood shape diffusivity and the blood congestion expression of each inner closed area in each lesion characterization area is recorded as the local blood influence degree of each inner closed area in each lesion characterization area;

[0154] The normalized result of the cumulative sum of the local blood influence degrees of all internal closed areas in each lesion characterization area is recorded as the blood interference index of each lesion characterization area.

[0155] Step S004: construct a multidimensional feature vector for each lesion characterization area based on the blood interference index and the reflection interference index of each lesion characterization area; train a deep learning model with the multidimensional feature vector to perform cervical lesion grading judgment.

[0156] It should be noted that after obtaining the blood interference index and reflection interference index of each lesion representation area, they both reflect the interference situation of the lesion representation area. When training the deep learning model, it is necessary to reduce the attention to the interfered area. Therefore, this embodiment analyzes each cervical image and uses the blood interference index of each lesion representation area in the cervical grayscale image corresponding to the cervical image to form a multidimensional feature vector. All multidimensional feature vectors construct the spatial attention weight matrix of the cervical image. After the cervical image generates its feature map through the deep learning model, the spatial attention weight matrix is ​​multiplied by the feature map to obtain the modified attention feature map of the cervical image, thereby adjusting the contribution of different areas in the cervical image to the cervical lesion grading and discrimination model obtained after deep learning training.

[0157] Preferably, the specific steps of constructing a multidimensional feature vector of each lesion characterization region according to the blood interference index and the reflection interference index of each lesion characterization region are:

[0158] The blood interference index of the i-th lesion representation area is recorded as ;

[0159] The reflection interference index of the i-th lesion representation area is recorded as ;

[0160] The multidimensional feature vector of the i-th lesion representation region ,in is the transpose symbol.

[0161] As another example in the embodiment of the present application, the multidimensional feature vector may further include other features added to the blood interference index and the reflection interference index to form a multidimensional feature vector. Specifically, the multidimensional feature vector includes:

[0162] The energy in the gray level co-occurrence matrix of all pixels in the i-th lesion representation area is recorded as ;

[0163] The contrast of the gray level co-occurrence matrix of all pixels in the i-th lesion representation area is recorded as ;

[0164] The entropy of the gray level co-occurrence matrix of all pixels in the i-th lesion representation area is denoted as ;

[0165] The multidimensional feature vector of the i-th lesion representation region ,in is the transpose symbol.

[0166] Furthermore, for the cth cervical image in the training set, the multidimensional feature vector of each lesion representation area in the cth cervical image is assigned to each pixel point of each lesion representation area in the cth cervical image to obtain the spatial attention weight matrix of the cth cervical image. The size of the spatial attention weight matrix is ,in and are the length and width of the cth cervical image respectively, is the number of dimensions of the multidimensional feature vector representing each lesion region in the cth cervical image;

[0167] The c-th image is input into the CNN neural network to obtain the feature map of the c-th cervical image. The product of the feature map of the c-th cervical image and its spatial attention weight matrix is ​​recorded as the modified attention feature map of the c-th cervical image.

[0168] Similarly, a training set is pre-constructed and the modified attention feature maps of all cervical images in the training set are obtained. The CNN model is trained with the modified attention feature maps of all cervical images in the training set to obtain a cervical lesion grading and discrimination model, and then the cervical lesion grading and discrimination model is used to perform cervical lesion grading judgment.

[0169] It should be noted that the training of the CNN model is an existing well-known technology and will not be described in detail in this embodiment. The loss function in the training process is taken as an example in this embodiment by the cross entropy loss function.

[0170] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for grading cervical lesions based on deep learning, characterized in that: The method comprises the following steps: The collected cervical images are divided and screened to obtain several lesion characterization areas; Selecting each highlighted portion in each lesion representation region as each highlighted region to be determined in each lesion representation region; In each lesion representation area, the degree of irregularity and edge abruptness of each highlight area to be determined meeting the requirements of reflective interference is analyzed to obtain the reflective interference index of each lesion representation area; Each lesion representation area is divided using closed texture to obtain the internal closed area in each lesion representation area; In each lesion characterization region, the tubular appearance of each internal closed region is analyzed to obtain the blood shape diffusion degree of each internal closed region; the blood accumulation expression degree of each internal closed region is obtained based on the grayscale uniformity of each internal closed region; and the blood interference index of each lesion characterization region is obtained based on the blood shape diffusion degree and blood accumulation expression degree of all internal closed regions in each lesion characterization region; Constructing a multidimensional feature vector of each lesion representation area according to the blood interference index and the reflection interference index of each lesion representation area; The multidimensional feature vector is used to train a deep learning model to perform cervical lesion grading judgment.

2. According to the method for grading cervical lesions based on deep learning in claim 1, it is characterized in that: The specific steps of dividing and screening the collected cervical images to obtain a number of lesion representation areas include: collecting a cervical image to obtain a cervical grayscale image of the cervical image; After finely dividing the cervical grayscale image, a number of sub-pixel blocks are obtained; and according to the color and texture information of the sub-pixel blocks and their positions in the cervical grayscale image, all the sub-pixel blocks are clustered to obtain a number of initial divided regions; The average value of the ratio of the brightness channel V to the saturation channel S of all pixels corresponding to each initial divided area in the cervical image is recorded as the reference weight of each initial divided area; Arrange the reference weights of all initially divided areas in descending order to obtain a descending sequence of reference weights; The difference between the reference weight of each number and the next number in the descending sequence of reference weight is recorded as the reference difference of each number in the descending sequence of reference weight; Each initially divided region corresponding to each number from the first number to the number with the largest reference difference in the descending sequence of reference weights is recorded as a lesion representation region.

3. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The specific steps of obtaining each highlight area to be determined include: The binary segmentation of each lesion representation area is performed using the Otsu threshold algorithm to obtain a binary segmentation map of each lesion representation area; After performing a morphological closing operation on the binary segmentation image, each closed area formed by a pixel gray value of 1 is recorded as a highlight area to be determined.

4. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The specific steps of obtaining the reflection interference index include: Obtaining the shape irregularity and edge abruptness of each highlight area to be determined in each lesion representation area; The reflection interference index of each lesion characterization area is obtained, and the shape irregularity and edge sudden change degree of the lesion characterization area are in direct proportion to all the highlight areas to be determined.

5. The method for grading cervical lesions based on deep learning according to claim 4, characterized in that: The step of obtaining the shape irregularity and edge abruptness of each to-be-determined highlight region of each lesion characterization region comprises: In each lesion representation area, the convex hull area of ​​each highlight area to be determined is obtained, and the ratio of the number of pixels in the convex hull area of ​​each highlight area to be determined to the number of pixels in the convex hull area of ​​the highlight area to be determined is recorded as the shape irregularity of each highlight area to be determined; In each lesion representation area, obtain the fitting line segment between each edge pixel point of each highlight area to be determined and the next pixel point in the clockwise direction, and record it as the trend line segment of each edge pixel point of each highlight area to be determined; The difference in slope between each edge pixel point of each highlight area to be determined and the next edge pixel point in the clockwise direction is recorded as the gradient sudden change amplitude of each edge pixel point of each highlight area to be determined; The edge sudden change degree of each highlight area to be determined is obtained by integrating the gradient sudden change amplitude and edge gradient value of all edge pixels of each highlight area to be determined, wherein the edge sudden change degree is directly proportional to the distribution range of the gradient sudden change amplitude, and the edge sudden change degree is inversely proportional to the edge gradient value.

6. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The specific steps of obtaining the blood shape diffusivity include: In each lesion representation region, the principal axis direction of each internal closed region was obtained; The projection length of the edge of the internal closed area in the main axis direction is recorded as the main axis projection length of each internal closed area; The projection length of the edge of the internal closed area in the tangent direction of the main axis direction is recorded as the tangent projection length of each internal closed area; Obtaining a non-tubular expression degree of each internal closed area, wherein the non-tubular expression degree is inversely proportional to the main axis projection length and is directly proportional to the tangent projection length; For each pixel point that the main axis direction of each internal closed area passes through, obtain the distance of each pixel point from the edge of the internal closed area in the tangent direction of the main axis direction, and record it as the relative diameter of each pixel point that the main axis direction of each internal closed area passes through; Obtain the variance of the relative diameters of all pixels passed by the main axis direction of each internal closed area, which is recorded as the diameter inconsistency of each internal closed area; The blood shape diffusion of each internal closed area is obtained, and the blood shape diffusion is in direct proportion to the diameter non-uniformity and the non-tubular expression.

7. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The specific steps of obtaining the blood accumulation expression include: Obtain the average of all grayscale values ​​of each pixel point passed by the main axis direction of each internal closed area in the tangent direction of the main axis direction, and record it as the grayscale value of the section of each pixel point passed by the main axis direction of each internal closed area; The sequence of the grayscale values ​​of the cross-section in the direction pointed by the main axis direction of each internal closed area is recorded as the grayscale value sequence of the cross-section of each internal closed area; The cumulative sum of the absolute values ​​of the differences between each sequence value and the next sequence value in the section grayscale value sequence is recorded as the blood pool expression degree of each internal closed area.

8. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The specific steps of obtaining the blood interference index include: A blood interference index is obtained for each lesion characterization region, wherein the blood interference index is in positive proportion to the blood shape diffusion and blood accumulation expression of all internal closed areas in each lesion characterization region.

9. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The step of constructing a multidimensional feature vector of each lesion characterization region according to the blood interference index and the reflection interference index of each lesion characterization region includes: The blood interference index of the i-th lesion representation area is recorded as ; The reflection interference index of the i-th lesion representation area is recorded as ; The multidimensional feature vector of the i-th lesion representation region ,in is the transpose symbol.

10. The method for grading cervical lesions based on deep learning according to claim 1, characterized in that: The multi-dimensional feature vector is used to train the deep learning model to perform cervical lesion grading judgment, which includes: The multidimensional feature vector of each lesion representation area in the cth cervical image is assigned to each pixel point of each lesion representation area in the cth cervical image, and the spatial attention weight matrix of the cth cervical image is obtained. The size of the spatial attention weight matrix is ,in and are the length and width of the cth cervical image respectively, is the number of dimensions of the multidimensional feature vector representing each lesion region in the cth cervical image; Input the c-th image into the CNN neural network to obtain the feature map of the c-th cervical image. The product of the feature map of the c-th cervical image and its spatial attention weight matrix is ​​recorded as the modified attention feature map of the c-th cervical image. Pre-construct the training set to obtain the modified attention feature maps of all cervical images in the training set; The CNN model is trained with the modified attention feature maps of all cervical images in the training set to obtain a cervical lesion grading discrimination model, which is then used to perform cervical lesion grading judgment.

Images