Cervical lesion grading discrimination method based on deep learning

Abstract

The invention relates to the technical field of image processing, in particular to a cervical lesion grading and distinguishing method based on deep learning, which comprises the following steps: acquiring a plurality of lesion characterization areas from an acquired cervical image; the irregularity degree and the edge sudden change degree are analyzed, and the reflection interference index of each lesion characterization area is obtained; dividing to obtain an internal closed region in each lesion characterization region; analyzing the tubular expression condition to obtain the blood shape dispersivity; according to the gray level uniformity, obtaining a blood fovea expression degree; acquiring a blood interference index of each lesion characterization area according to the blood shape dispersivity and the blood fovea expression degree; constructing a multi-dimensional feature vector of each lesion characterization area; and training a deep learning model to carry out cervical lesion grading judgment. The invention aims to solve the problem of training result error caused by reflection and blood interference of a cervical image during deep learning training of a cervical lesion grading discrimination model, and achieves the purpose of improving the model identification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for grading and discriminating cervical lesions based on deep learning. Background Art

[0002] Cervical cancer is one of the common malignant tumors in women. As a precancerous lesion of cervical cancer, early detection and accurate grading are crucial for preventing canceration. Cervical intraepithelial neoplasia is divided into three grades: CIN1, CIN2, and CIN3. Discriminating and grading cervical intraepithelial neoplasia can effectively improve the efficiency of prevention and treatment.

[0003] There are existing methods for grading and discriminating cervical lesions using deep learning models. However, cervical images are easily interfered by factors such as the reflected highlights of the cervical orifice mirror surface and blood contamination at the cervical orifice. When using deep learning to grade cervical lesions in cervical images, the intensity of the lesion characteristics shown in the cervical lesion areas that show lesion characteristics in the cervical images decreases, resulting in misgrading of the deep learning model due to image interference. Summary of the Invention

[0004] The present invention provides a method for grading and discriminating cervical lesions based on deep learning to solve the existing problems.

[0005] A method for grading and discriminating cervical lesions based on deep learning of the present invention adopts the following technical solutions: An embodiment of the present invention provides a method for grading and discriminating cervical lesions based on deep learning, and the method includes the following steps: Divide and screen the collected cervical images to obtain several lesion characterization regions; Select each highlighted part in each lesion characterization region as each to-be-determined highlighted region in each lesion characterization region; In each lesion characterization region, analyze the irregular degree and edge sudden change degree that each to-be-determined highlighted region satisfies the reflection interference to obtain the reflection interference index of each lesion characterization region; Use closed texture to divide each lesion characterization region to obtain the internal closed region in each lesion characterization region; In each lesion characterization region, analyze the tubular performance of each internal closed region to obtain the blood shape dispersion degree of each internal closed region; according to the gray level uniformity of each internal closed region, obtain the blood accumulation performance degree of each internal closed region; according to the blood shape dispersion degree and blood accumulation performance degree of all internal closed regions in each lesion characterization region, obtain the blood interference index of each lesion characterization region; Construct a multi-dimensional feature vector for each lesion characterization region according to the blood interference index and reflection interference index of each lesion characterization region; Train a deep learning model with the multi-dimensional feature vector to judge the grading of cervical lesions.

[0006] Preferably, the specific acquisition steps for obtaining several lesion characterization regions after partitioning and screening the collected cervical images are as follows: Collect a cervical image to obtain a cervical grayscale image of the cervical image; After finely partitioning the cervical grayscale image to obtain several sub-pixel blocks, cluster all sub-pixel blocks according to the color, texture information of the sub-pixel blocks and their positions in the cervical grayscale image to obtain several initial partitioning regions; Denote the mean value of the ratio of the brightness channel V to the saturation channel S of all pixel points corresponding to each initial partitioning region in the cervical image as the reference weight of each initial partitioning region; Arrange the reference weights of all initial partitioning regions in descending order to obtain a descending sequence of reference weights; Denote the difference between the reference weight of each serial number and the reference weight of the next serial number in the descending sequence of reference weights as the reference difference degree of each serial number in the descending sequence of reference weights; In the descending sequence of reference weights, denote each initial partitioning region corresponding to each serial number from the first serial number to the serial number with the largest reference difference degree as a lesion characterization region.

[0007] Preferably, the specific acquisition steps for each region to be determined with high brightness are as follows: Perform binary segmentation on each lesion characterization region using the Otsu threshold algorithm to obtain a binary segmentation map of each lesion characterization region; After performing morphological closing operation on the binary segmentation map, denote each closed region composed of pixel points with a gray value of 1 as a region to be determined with high brightness.

[0008] Preferably, the specific acquisition steps for the specular interference index are as follows: Obtain the irregularity degree of the shape and the degree of abrupt change of the edge of each region to be determined with high brightness in each lesion characterization region; Obtain the specular interference index of each lesion characterization region, and the specular interference index of the lesion characterization region is in a direct proportional relationship with the irregularity degree of the shape and the degree of abrupt change of the edge of all its regions to be determined with high brightness.

[0009] Preferably, obtaining the irregularity degree of the shape and the degree of abrupt change of the edge of each region to be determined with high brightness in each lesion characterization region includes: In each lesion characterization region, obtain the convex hull region of each region to be determined with high brightness, and denote the ratio of the number of pixel points in the convex hull region of each region to be determined with high brightness to the number of its own pixel points as the irregularity degree of the shape of each region to be determined with high brightness; In each lesion characterization region, obtain the fitting line segment of each edge pixel point of each highlighted region to be determined and the next pixel point in the clockwise direction, which is denoted as the trend line segment of each edge pixel point of each highlighted region to be determined; Denote the slope difference between the trend line segments of each edge pixel point of each highlighted region to be determined and the next edge pixel point in the clockwise direction as the gradient mutation amplitude of each edge pixel point of each highlighted region to be determined; Based on the gradient mutation amplitudes and edge gradient values of all edge pixel points of each highlighted region to be determined, obtain the edge mutation degree of each highlighted region to be determined, where the edge mutation degree is in a direct proportional relationship with the distribution range of the gradient mutation amplitude, and the edge mutation degree and the edge gradient value are in an inverse proportional relationship.

[0010] Preferably, the specific steps for obtaining the blood shape dispersion degree include: In each lesion characterization region, obtain the main axis direction of each internal closed region; Denote the projection length of the edge of the internal closed region on the main axis direction as the main axis projection length of each internal closed region; Denote the projection length of the edge of the internal closed region on the tangent direction of the main axis direction as the tangent projection length of each internal closed region; Obtain the non-tubular manifestation degree of each internal closed region, where the non-tubular manifestation degree is in an inverse proportional relationship with the main axis projection length and in a direct proportional relationship with the tangent projection length; For each pixel point passed by the main axis direction of each internal closed region, obtain the distance from each pixel point to the edge of the internal closed region in the tangent direction of the main axis direction, which is denoted as the relative diameter of each pixel point passed by the main axis direction of each internal closed region; Obtain the variance of the relative diameters of all pixel points passed by the main axis direction of each internal closed region, which is denoted as the diameter non-uniformity of each internal closed region; Obtain the blood shape dispersion degree of each internal closed region, where the blood shape dispersion degree is in a direct proportional relationship with the diameter non-uniformity and the non-tubular manifestation degree.

[0011] Preferably, the specific steps for obtaining the blood pooling manifestation degree include: Obtain the mean value of all gray values of each pixel point passed by the main axis direction of each internal closed region in the tangent direction of the main axis direction and within the internal closed region, which is denoted as the sectional gray value of each pixel point passed by the main axis direction of each internal closed region; Denote the sequence formed by the sectional gray values in the direction pointed by the main axis direction of each internal closed region as the sectional gray value sequence of each internal closed region; The sum of the absolute values of the differences between each sequence value and the next sequence value in the sequence of cross-sectional gray-scale values is denoted as the blood accumulation expression degree of each internal closed region.

[0012] Preferably, the specific steps for obtaining the blood interference index include: Obtain the blood interference index of each lesion characterization region, and the blood interference index is in a direct proportional relationship with both the blood shape dispersion degree and the blood accumulation expression degree of all internal closed regions in each lesion characterization region.

[0013] Preferably, constructing the multi-dimensional feature vector of each lesion characterization region according to the blood interference index and the reflection interference index of each lesion characterization region includes: Denote the blood interference index of the i-th lesion characterization region as ; Denote the reflection interference index of the i-th lesion characterization region as ; The multi-dimensional feature vector of the i-th lesion characterization region , where is the transpose symbol.

[0014] Preferably, using the multi-dimensional feature vector to train a deep learning model for cervical lesion grading judgment includes: Assign the multi-dimensional feature vector of each lesion characterization region in the c-th cervical image to each pixel point of each lesion characterization region in the c-th cervical image to obtain the spatial attention weight matrix of the c-th cervical image, and the size of the spatial attention weight matrix is , where and are the length and width of the c-th cervical image respectively, is the dimension number of the multi-dimensional feature vector of each lesion characterization region in the c-th cervical image; Input the c-th image into the CNN neural network to obtain the feature map of the c-th cervical image, and denote the product of the feature map of the c-th cervical image and its spatial attention weight matrix as the modified attention feature map of the c-th cervical image; Pre-construct a training set to obtain the modified attention feature maps of all cervical images in the training set; Train the CNN model with the modified attention feature maps of all cervical images in the training set to obtain a cervical lesion grading discrimination model, and then use the cervical lesion grading discrimination model to perform the grading determination of cervical lesions.

[0015] The beneficial effects of the technical solution of the present invention are as follows: By collecting cervical lesion images, dividing the images into regions, obtaining the lesion characterization regions, analyzing the reflection features in the to-be-determined high-brightness regions that belong to the high-brightness regions in the lesion characterization regions, obtaining the reflection interference index of each lesion characterization region, which is used to reflect the interference situation in each lesion characterization region. At the same time, dividing the lesion characterization regions according to the texture to obtain internal closed regions, and analyzing the blood characteristics that meet the uneven width and the blood characteristics that meet the uneven gray level in the internal closed regions, obtaining the blood interference index of each lesion characterization region, which reflects the blood interference situation in each lesion characterization region. Furthermore, using the blood interference index and the reflection interference index to construct a multi-dimensional feature vector, generating a spatial attention weight matrix to adjust the reference when training the cervical lesion grading discrimination model, so as to achieve the purpose of improving the accuracy of cervical lesion analysis and discrimination. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 It is a flowchart of the steps of a method for grading and discriminating cervical lesions based on deep learning according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, describe in detail the specific implementation manner, structure, features and effects of a method for grading and discriminating cervical lesions based on deep learning proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0020] The following will specifically describe the specific solution of a method for grading and discriminating cervical lesions based on deep learning provided by the present invention in conjunction with the accompanying drawings.

[0021] Please refer to Figure 1 , which shows a flowchart of the steps of a method for grading and discriminating cervical lesions based on deep learning provided by an embodiment of the present invention. The method includes the following steps: Step S001: After dividing and screening the collected cervical images, several lesion characterization regions are obtained.

[0022] It should be noted that existing deep learning models can be trained on a training set composed of collected cervical images to obtain a cervical lesion grading discrimination model, so as to input the cervical images into the cervical lesion grading discrimination model and automatically obtain the cervical lesion grades of precancerous lesions. However, during the collection of cervical images, a dilator is needed to dilate the vagina, and the colposcope enters the vagina through the space opened by the dilator and aligns with the cervical os for collection. During this process, the dilator will cause slight bleeding on the cervical surface, and the colposcope has a light source. After reflecting through vaginal secretions and mucus, a reflection effect will be generated, which will interfere with the manifestation of the acetic acid white reaction at the cervical os, resulting in interference during the training of the cervical lesion grading discrimination model by the deep learning model and reducing the accuracy and precision of the cervical lesion grading discrimination model.

[0023] Therefore, in this embodiment, after analyzing the collected cervical images, multi-dimensional feature vectors of each region in the cervical images are obtained, and a spatial attention weight matrix corresponding to the cervical images is generated using the spatial attention mechanism, so that the cervical lesion grading discrimination model has different attentions to different regions in the training set, achieving the purpose of avoiding the influence of interference on the model's precision. First, cervical images need to be collected and corresponding regional division preprocessing is performed.

[0024] Preferably, the specific steps for dividing and screening the collected cervical images to obtain several lesion characterization regions include: Collect cervical images to obtain the cervical grayscale image of the cervical images; The cervical grayscale image is finely divided to obtain several sub-pixel blocks. According to the color, texture information of the sub-pixel blocks and their positions in the cervical grayscale image, all sub-pixel blocks are clustered to obtain several initial division regions; According to the brightness and saturation of the pixel points in each initial division region, the reference weights of each initial division region are obtained, and the initial division regions are screened using the reference weights to obtain several lesion characterization regions.

[0025] Specifically, the specific method for collecting cervical images to obtain the cervical grayscale image of the cervical images is: It should be noted that since the epithelial tissue degenerates during cervical lesions, acetic acid solution can be applied to the epithelial tissue, and the proteins of abnormal epithelial tissue coagulate, thus presenting an acetic acid white area at the cervical os. Therefore, when judging cervical lesion grades, the cervical images collected in this embodiment are HSV format images, which can better show the color information of the cervical os. The specific cervical image collection method is as follows: After dilating the vagina with a dilator, apply an acetic acid solution with a concentration of 3% to the patient's cervix, and use a colposcope to obtain a cervical source image in RGB format; Convert the cervical source image from RGB format to HSV format to obtain a cervical image; and perform weighted grayscale processing on the cervical source image to obtain a cervical grayscale image of the cervical image.

[0026] It should be noted that the weighted grayscale processing and the conversion from RGB format to HSV format described in this embodiment are both well-known prior arts, and will not be elaborated in this embodiment.

[0027] Further, after finely dividing the cervical grayscale image to obtain a number of sub-pixel blocks, clustering all the sub-pixel blocks according to the color, texture information and the position in the cervical grayscale image of the sub-pixel blocks to obtain a number of initial division regions, the specific method is: It should be noted that the cervical grayscale image includes a cervical tissue region, a lesion region, a vaginal wall or surrounding background, a shadow or reflection region, and some secretion, blood, bubble regions, etc., which will all interfere with the cervical grayscale image. Therefore, it is necessary to first divide the cervical grayscale image to obtain a number of sub-pixel blocks, and then merge the sub-pixel blocks with similar attributes to obtain a number of initial division regions, so that each initial division region represents the same type of tissue or interference. The specific operation is: Use a superpixel segmentation algorithm to divide the cervical grayscale image to obtain a number of sub-pixel blocks; Take the HSV three-channel values of each sub-pixel block in the corresponding region of the cervical image, the mean value of the LBP codes of the sub-pixel block, and the centroid position of the sub-pixel block as clustering parameters to construct a multi-dimensional clustering space. Use the K-means clustering algorithm to cluster all the sub-pixel blocks in the multi-dimensional clustering space to obtain a number of clusters, and splice the sub-pixel blocks that belong to the same cluster and are adjacent in the cervical grayscale image to obtain a number of initial division regions.

[0028] It should be noted that the superpixel segmentation algorithm and the K-means clustering algorithm are both well-known prior arts. Among them, the mean value of the LBP codes is obtained by using the well-known LBP algorithm. The LBP algorithm can reflect the texture information formed by each pixel point in the image and the surrounding pixel points. The specific method for obtaining the mean value of the LBP codes is: use the LBP algorithm to obtain the LBP codes of each pixel point in each sub-pixel block, and take the mean value of the LBP codes of all pixel points in each sub-pixel block as the mean value of the LBP codes of the sub-pixel block to represent the texture information of the sub-pixel block.

[0029] Further, according to the brightness and saturation of the pixel points in each initial division region, obtain the reference weight of each initial division region, and use the reference weight to screen the initial division regions to obtain a number of lesion characterization regions. The specific method is: It should be noted that after the epithelial tissue of cervical lesions is smeared with acetic acid white solution, an acetic acid white reaction will occur, which appears as high brightness and low saturation in HSV-format images. Therefore, the reference weights of each initial divided region are obtained through the brightness and saturation of the pixel points in each initial divided region, and the initial divided regions are screened using the reference weights to obtain several lesion characterization regions. The specific steps are as follows: The mean value of the ratio of the brightness channel V to the saturation channel S of all pixel points corresponding to each initial divided region in the cervical image is denoted as the reference weight of each initial divided region; The reference weights of all initial divided regions are arranged in descending order to obtain a reference weight descending sequence; The difference between the reference weight of each serial number and the reference weight of the next serial number in the reference weight descending sequence is denoted as the reference difference degree of each serial number in the reference weight descending sequence; In the reference weight descending sequence, each initial divided region corresponding to each serial number from the first serial number to the serial number with the largest reference difference degree is denoted as a lesion characterization region.

[0030] It should be noted that if the value of the saturation channel S of a pixel point is 0, the value of the saturation channel S of this pixel point is assigned 0.01.

[0031] Furthermore, it should be noted that since the color of the abnormal epithelial tissue of the lesion is closer to white after the acetic acid white reaction compared to the colors of other regions, the value of the reference weight will be much larger than that of the normal region. In this embodiment, the maximum descending difference is used to achieve the bipolar division of the reference weight, thereby obtaining the lesion characterization regions.

[0032] Step S002: Select each highlighted part in each lesion characterization region as each to-be-determined highlighted region in each lesion characterization region; in each lesion characterization region, analyze the irregularity degree and edge sudden change degree of each to-be-determined highlighted region satisfying the reflection interference to obtain the reflection interference index of each lesion characterization region.

[0033] The grading of cervical precancerous lesions mainly determines the severity of cervical lesions based on the grading criteria of cervical intraepithelial neoplasia. The core basis for grading is the longitudinal cumulative range of atypical cells within the thickness of the cervical epithelium. In the evaluation of cervical images, the corresponding CIN grade should also be indirectly inferred based on imaging features such as the acetowhite reaction degree, color, surface structure, and vascular pattern in the lesion area. However, during this process, when the colposcope light source reflects on the cervical surface secretion fluid, it is easy to form a reflective high-brightness area. In the cervical lesion area, after the action of acetic acid, the abnormal epithelial tissue degenerates, and protein coagulation leads to enhanced reflectivity and appears milky white, which is similar to the performance of the reflective high-brightness area. When training the cervical lesion grading discrimination model, the reflective high-brightness is easily selected as the acetowhite area of the acetowhite reaction, affecting the accuracy of the cervical lesion grading discrimination model.

[0034] Therefore, in this embodiment, the reflection and acetowhite reaction are distinguished in the lesion characterization area, so as to obtain the reflection interference index of the lesion characterization area.

[0035] Preferably, the specific steps of selecting each high-brightness part in each lesion characterization area as each to-be-determined high-brightness area in each lesion characterization area include: Performing binary segmentation on each lesion characterization area using the Otsu threshold algorithm to obtain the binary segmentation map of each lesion characterization area; After performing morphological closing operation on the binary segmentation map, each closed area composed of pixel point gray values of 1 is recorded as a to-be-determined high-brightness area.

[0036] It should be noted that the Otsu threshold algorithm and morphological closing operation described in this embodiment are both existing well-known technologies, and will not be elaborated in this embodiment.

[0037] Furthermore, it should be noted that the area with a gray value of 1 in the binary segmentation map obtained by Otsu threshold segmentation corresponds to the high-brightness area in the lesion characterization area. Therefore, it is used as the to-be-determined high-brightness area to analyze whether it belongs to the acetowhite area or the reflective high-brightness area.

[0038] It should be noted that the reflective high-brightness area appears as a dot-shaped or irregular small patchy area, and the edge is clear; while the acetowhite area after acetic acid treatment of cervical lesions shows the characteristic of deriving from the center to the edge, that is, the acetowhite area is a relatively regular patchy structure, and due to the gradually expanding influence range of the lesion, the boundary of the acetowhite area is blurred, that is, the lesion area has the derivative characteristic of natural diffusion. Therefore, in this embodiment, based on the shape regularity and edge derivative characteristics of the acetowhite area in the to-be-determined high-brightness area, the area that meets the irregular degree and edge abrupt change degree of reflection interference is analyzed to obtain the reflection interference index of each lesion characterization area.

[0039] Further, in each lesion characterization region, the specific steps of analyzing the irregularity degree and edge sudden change degree of each highlighted region to be determined that meet the specular interference and obtaining the specular interference index of each lesion characterization region are as follows: In each lesion characterization region, according to the area difference between each highlighted region to be determined and its convex hull region, obtain the shape irregularity degree of each highlighted region to be determined; According to the edge gradient value of each edge pixel point of each highlighted region to be determined and the gradient sudden change amplitude with the adjacent edge pixel point, obtain the edge sudden change degree of each highlighted region to be determined; Integrate the shape irregularity degree and edge sudden change degree of all highlighted regions to be determined to obtain the specular interference index of each lesion characterization region.

[0040] Specifically, the specific method of obtaining the shape irregularity degree of each highlighted region to be determined according to the area difference between each highlighted region to be determined and its convex hull region is as follows: In each lesion characterization region, obtain the convex hull region of each highlighted region to be determined, and record the ratio of the number of pixel points in the convex hull region of each highlighted region to be determined to the number of its own pixel points as the shape irregularity degree of each highlighted region to be determined.

[0041] It should be noted that since the convex hull region of a region must be less than or equal to the region, when the region is regular, the convex hull region is equal to the region, and the greater the area difference between the convex hull region and the area of the region, the more irregular the region indicates.

[0042] Further, the specific method of obtaining the edge sudden change degree of each highlighted region to be determined according to the edge gradient value of each edge pixel point of each highlighted region to be determined and the gradient sudden change amplitude with the adjacent edge pixel point is as follows: In each lesion characterization region, obtain the fitting line segment of each edge pixel point of each highlighted region to be determined and the next pixel point in the clockwise direction, and record it as the trend line segment of each edge pixel point of each highlighted region to be determined; Record the slope difference between the trend line segment of each edge pixel point of each highlighted region to be determined and the next edge pixel point in the clockwise direction as the gradient sudden change amplitude of each edge pixel point of each highlighted region to be determined; Integrate the gradient sudden change amplitude and edge gradient value of all edge pixel points of each highlighted region to be determined to obtain the edge sudden change degree of each highlighted region to be determined, where the edge sudden change degree is in a proportional relationship with the distribution range of the gradient sudden change amplitude, and the edge sudden change degree and the edge gradient value are in an inverse proportional relationship.

[0043] As an example, the edge sudden change degree of the jth highlighted region to be determined in the ith lesion characterization region The calculation method is as follows: ; Wherein, is the average value of the edge gradient values of all edge pixel points of the j-th to-be-determined highlighted region in the i-th lesion characterization region; is the variance of the gradient abrupt change amplitude of all edge pixel points of the j-th to-be-determined highlighted region in the i-th lesion characterization region, which is used to represent the distribution range of the gradient abrupt change amplitude.

[0044] Furthermore, by comprehensively considering the irregularity of the shape and the degree of edge abrupt change of all to-be-determined highlighted regions, the specific method for obtaining the reflection interference index of each lesion characterization region is as follows: Obtain the reflection interference index of each lesion characterization region, and the lesion characterization region is in a direct proportional relationship with the irregularity of the shape and the degree of edge abrupt change of all its to-be-determined highlighted regions. As an example, the method for obtaining the reflection interference index of the i-th lesion characterization region is as follows: Obtain the product of the irregularity of the shape and the degree of edge abrupt change of each to-be-determined highlighted region in the i-th lesion characterization region, which is denoted as the reflection performance of each to-be-determined highlighted region in the i-th lesion characterization region; The result obtained after normalizing the sum of the reflection performances of all to-be-determined highlighted regions in the i-th lesion characterization region is denoted as the reflection interference index of the i-th lesion characterization region.

[0045] It should be noted that in this embodiment, the sigmoid function is used to normalize the sum of the reflection performances, and the sigmoid function is a well-known prior art.

[0046] Step S003: Use the closed texture to divide each lesion characterization region to obtain the internal closed regions in each lesion characterization region; in each lesion characterization region, analyze the tubular performance of each internal closed region to obtain the blood shape dispersion degree of each internal closed region; according to the gray uniformity of each internal closed region, obtain the blood accumulation performance of each internal closed region; according to the blood shape dispersion degree and the blood accumulation performance of all internal closed regions in each lesion characterization region, obtain the blood interference index of each lesion characterization region.

[0047] There are a large number of blood vessels in the pre-cancerous lesion area. When taking cervical images, due to the fragility of the cervical orifice, the vaginal and cervical orifice mucosa are prone to breakage and slight bleeding, resulting in interference of the bleeding blood on the blood vessels during the training of the cervical lesion grading discrimination model, reducing the recognition accuracy of the cervical lesion grading discrimination model. Therefore, in this embodiment, by analyzing blood vessels and diffused blood, the blood interference index of each lesion characterization area is obtained, so as to be used for the spatial attention weight of the adaptive attention mechanism.

[0048] It should be noted that the distribution of blood on the surface of cervical tissue shows irregular scattered points or patches, lacking clear linear structural features, while the blood vessels in the lesion area are in the shape of slender, linear or cord-like tubular structures, with obvious geometric linear edge features. Therefore, in this embodiment, the internal closed area is obtained by dividing the edge of the lesion characterization area, and then the blood or blood vessel features presented by the internal closed area are analyzed to obtain the blood interference index of the lesion characterization area.

[0049] Preferably, the specific steps of using the closed texture to divide each lesion characterization area and obtaining the internal closed area in each lesion characterization area are as follows: Perform edge detection on each lesion characterization area, and record the area formed by each closed edge in the edge detection result as an internal closed area.

[0050] It should be noted that the area formed by the closed edge described in this embodiment includes not only the area formed by the edge detected by the edge detection, but also the area jointly formed by the edge detected by the edge detection and the edge of the lesion characterization area.

[0051] Preferably, in each lesion characterization area, the specific method of analyzing the tubular performance of each internal closed area and obtaining the blood shape dispersion degree of each internal closed area is as follows: In each lesion characterization area, obtain the principal axis direction of each internal closed area; According to the projection length difference of the edge of each internal closed area in the principal axis direction, obtain the non-tubular performance degree of each internal closed area; Analyze the diameter distribution of each pixel point in the principal axis direction and the tangent direction of each pixel point of each internal closed area, and combine the non-tubular performance degree to obtain the blood shape dispersion degree of each internal closed area.

[0052] Specifically, in each lesion characterization area, the specific method of obtaining the principal axis direction of each internal closed area is as follows: Use the gray value of each pixel point in each internal closed area to form the gray value surface of each internal closed area; Use the principal component analysis algorithm to obtain the principal component direction of the gray value surface, which is recorded as the principal axis direction of each internal closed area.

[0053] It should be noted that the principal component analysis algorithm described in this embodiment is a well-known existing technology and will not be elaborated in this embodiment. Among them, the principal component analysis algorithm can obtain the trend of data. If the internal closed area is blood, then there is no clear trend in the internal closed area, while the blood vessel area has an obvious tubular structure and an obvious trend. Therefore, the principal axis direction of the internal closed area can be obtained through the principal component analysis algorithm.

[0054] Further, the specific method for obtaining the non-tubular expression degree of each internal closed area according to the difference in the projection length of the edge of each internal closed area in the principal axis direction is as follows: The projection length of the edge of the internal closed area projected on the principal axis direction is denoted as the principal axis projection length of each internal closed area; The projection length of the edge of the internal closed area projected on the tangent direction of the principal axis direction is denoted as the tangent projection length of each internal closed area; Obtain the non-tubular expression degree of each internal closed area, and the non-tubular expression degree is inversely proportional to the principal axis projection length and directly proportional to the tangent projection length.

[0055] As an example, the calculation method for obtaining the non-tubular expression degree of each internal closed area is as follows: The ratio of the tangent projection length to the principal axis projection length of each internal closed area is denoted as the non-tubular expression degree of each internal closed area.

[0056] It should be noted that if the internal closed area is a blood vessel, then the principal axis projection length is the length of the blood vessel, and the tangent projection length is the width of the blood vessel. When the ratio of the width to the length of the internal closed area is larger, it means that it does not conform to the long strip shape of the blood vessel more, and it is more likely to belong to blood interference.

[0057] It should be noted that since the cervix is annular and protrudes from the vaginal wall plane, folds will be formed on the surface. If blood flows into the folds, it will also show a tubular structure. However, there is a difference in diameter between the folds and the blood vessels, and the diameter of the blood vessels changes little within a short distance. Therefore, in this embodiment, by analyzing the diameter change of the internal closed area in the principal axis direction and combining the non-tubular expression degree, the blood shape dispersion degree of each internal closed area is obtained.

[0058] Specifically, the specific method for analyzing the diameter distribution of each pixel point in the tangent direction of the principal axis direction of each internal closed area and combining the non-tubular expression degree to obtain the blood shape dispersion degree of each internal closed area is as follows: For each pixel point passed by the principal axis direction of each internal closed region, obtain the distance from each pixel point to the edge of the internal closed region in the tangent direction of the principal axis direction, and denote it as the relative diameter of each pixel point passed by the principal axis direction of each internal closed region; Obtain the variance of the relative diameters of all pixel points passed by the principal axis direction of each internal closed region, and denote it as the diameter non-uniformity of each internal closed region; Obtain the blood shape dispersion degree of each internal closed region, and the blood shape dispersion degree is in a direct proportional relationship with the diameter non-uniformity and the non-tubular manifestation degree.

[0059] It should be noted that the variance of the relative diameters of all pixel points indicates the distribution range of the width of the internal closed region. The smaller the value, the smaller the distribution range, that is, the more consistent the diameter, and the more likely the internal closed region is a blood vessel. When the variance value is larger, it indicates that the internal closed region is more likely to be blood, that is, the diameter non-uniformity value is larger.

[0060] As an example, the specific calculation method for obtaining the blood shape dispersion degree of each internal closed region is as follows: Denote the product of the diameter non-uniformity and the non-tubular manifestation degree of each internal closed region as the blood shape dispersion degree of each internal closed region.

[0061] It should be noted that in blood vessels, since blood is wrapped by the blood vessel wall, the gray values of the internal closed regions are approximately the same. Even if the depth of the blood vessels from the mucosal surface is different, the change is uniform. However, the interfering blood is prone to accumulate in the folds, resulting in uneven distribution of the internal closed regions formed by it. Therefore, in this embodiment, by analyzing the gray uniformity of the dorsal closed regions, the blood accumulation manifestation degree of each internal closed region is obtained.

[0062] Preferably, the specific method for obtaining the blood accumulation manifestation degree of each internal closed region according to the gray uniformity of each internal closed region is as follows: Obtain the mean value of all gray values of each pixel point passed by the principal axis direction of each internal closed region in the internal closed region in the tangent direction of the principal axis direction, and denote it as the section gray value of each pixel point passed by the principal axis direction of each internal closed region; Denote the sequence formed by the section gray values in the direction pointed by the principal axis direction of each internal closed region as the section gray value sequence of each internal closed region; Denote the accumulated sum of the absolute values of the differences between each sequence value and the next sequence value in the section gray value sequence as the blood accumulation manifestation degree of each internal closed region.

[0063] It should be noted that when the gray value in the internal closed area is uniform, the absolute value of the difference between each sequence value and the next sequence value is small, and its cumulative sum is also small. Therefore, when the cumulative sum is larger, that is, when the blood pooling manifestation degree is larger, it indicates that the internal closed area is more likely to be blood interference.

[0064] Preferably, the specific steps for obtaining the blood interference index of each lesion characterization area according to the blood shape dispersion degree and blood pooling manifestation degree of all internal closed areas in each lesion characterization area are as follows: Obtain the blood interference index of each lesion characterization area, and the blood interference index is in a direct proportional relationship with the blood shape dispersion degree and blood pooling manifestation degree of all internal closed areas in each lesion characterization area.

[0065] As an example, the specific method for obtaining the blood interference index of each lesion characterization area is as follows: Multiply the blood shape dispersion degree and blood pooling manifestation degree of each internal closed area in each lesion characterization area, and record it as the local blood influence degree of each internal closed area in each lesion characterization area; Record the normalization result of the cumulative sum of the local blood influence degrees of all internal closed areas in each lesion characterization area as the blood interference index of each lesion characterization area.

[0066] Step S004: Construct a multi-dimensional feature vector for each lesion characterization area according to the blood interference index and specular reflection interference index of each lesion characterization area; train a deep learning model with the multi-dimensional feature vector to perform cervical lesion grading judgment.

[0067] It should be noted that after obtaining the blood interference index and specular reflection interference index of each lesion characterization area, they both reflect the interference situation of the lesion characterization area. For the interfered area during the training of the deep learning model, it is necessary to reduce the attention to the lesion characterization area. Therefore, in this embodiment, after analyzing each cervical image, a multi-dimensional feature vector is formed using the blood interference index of each lesion characterization area in the cervical gray image corresponding to the cervical image. All the multi-dimensional feature vectors construct the spatial attention weight matrix of the cervical image, and after the cervical image generates its feature map through the deep learning model, the spatial attention weight matrix is multiplied by the feature map to obtain the corrected attention feature map of the cervical image, thereby adjusting the contribution degree of different areas in the cervical image to the cervical lesion grading discrimination model obtained after deep learning training.

[0068] Preferably, the specific steps for constructing a multi-dimensional feature vector for each lesion characterization area according to the blood interference index and specular reflection interference index of each lesion characterization area are as follows: Record the blood interference index of the i-th lesion characterization area as ; Denote the specular interference index of the i-th lesion characterization region as ; The multi-dimensional feature vector of the i-th lesion characterization region , where is the transpose symbol.

[0069] As another example in the embodiments of the present application, other features can also be added to the multi-dimensional feature vector based on the blood interference index and the specular interference index to form a multi-dimensional feature vector. Specifically, it includes: Denote the energy in the gray-level co-occurrence matrix of all pixel points in the i-th lesion characterization region as ; Denote the contrast in the gray-level co-occurrence matrix of all pixel points in the i-th lesion characterization region as ; Denote the entropy in the gray-level co-occurrence matrix of all pixel points in the i-th lesion characterization region as ; The multi-dimensional feature vector of the i-th lesion characterization region , where is the transpose symbol.

[0070] Furthermore, for the c-th cervical image in the training set, assign the multi-dimensional feature vector of each lesion characterization region in the c-th cervical image to each pixel point in each lesion characterization region of the c-th cervical image to obtain the spatial attention weight matrix of the c-th cervical image. The size of the spatial attention weight matrix is , where and are the length and width of the c-th cervical image respectively, is the dimension number of the multi-dimensional feature vector of each lesion characterization region in the c-th cervical image; Input the c-th image into the CNN neural network to obtain the feature map of the c-th cervical image. Denote the product of the feature map of the c-th cervical image and its spatial attention weight matrix as the modified attention feature map of the c-th cervical image.

[0071] Similarly, pre-construct a training set and obtain the modified attention feature maps of all cervical images in the training set. Use the modified attention feature maps of all cervical images in the training set to train the CNN model to obtain a cervical lesion grading discrimination model, and then use the cervical lesion grading discrimination model to perform the grading determination of cervical lesions.

[0072] It should be noted that the training of the CNN model is a well-known prior art, which will not be specifically described in this embodiment. The loss function in the training process takes the cross-entropy loss function as an example in this embodiment.

[0073] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A cervical lesion grading and discrimination method based on deep learning, characterized in that, The method includes the following steps: After dividing and screening the collected cervical images, several lesion characterization regions are obtained; Each highlighted part in each lesion characterization region is selected as each to-be-determined highlighted region in each lesion characterization region; In each lesion characterization region, analyze the irregularity degree and edge abrupt change degree that each to-be-determined highlighted region satisfies the reflective interference, and obtain the reflective interference index of each lesion characterization region; Use the closed texture to divide each lesion characterization region to obtain the internal closed region in each lesion characterization region; In each lesion characterization region, analyze the tubular manifestation of each internal closed region to obtain the blood shape dispersion degree of each internal closed region; according to the gray level uniformity of each internal closed region, obtain the blood accumulation manifestation degree of each internal closed region; according to the blood shape dispersion degree and blood accumulation manifestation degree of all internal closed regions in each lesion characterization region, obtain the blood interference index of each lesion characterization region; According to the blood interference index and reflective interference index of each lesion characterization region, construct the multi-dimensional feature vector of each lesion characterization region; Use the multi-dimensional feature vector to train a deep learning model to perform cervical lesion grading judgment.

2. The cervical lesion grading and discrimination method based on deep learning according to claim 1, characterized in that, The specific obtaining steps of obtaining several lesion characterization regions after dividing and screening the collected cervical images include: Collect cervical images to obtain the cervical gray-scale image of the cervical images; After finely dividing the cervical gray-scale image to obtain several sub-pixel blocks, cluster all sub-pixel blocks according to the color, texture information of the sub-pixel blocks and the position in the cervical gray-scale image to obtain several initial division regions; Denote the mean value of the ratio of the luminance channel V to the saturation channel S of all pixel points corresponding to each initial division region in the cervical image as the reference weight of each initial division region; Arrange the reference weights of all initial division regions in descending order to obtain a reference weight descending sequence; Denote the difference between the reference weight of each serial number and the reference weight of the next serial number in the reference weight descending sequence as the reference difference degree of each serial number in the reference weight descending sequence; In the reference weight descending sequence, denote each initial division region corresponding to each serial number from the first serial number to the serial number with the largest reference difference degree as a lesion characterization region.

3. The cervical lesion grading discrimination method based on deep learning according to claim 1, characterized in that, The specific obtaining steps of each to-be-determined highlighted region include: Use the Otsu threshold algorithm to perform binary segmentation on each lesion characterization region to obtain the binary segmentation map of each lesion characterization region; After performing morphological closing operation on the binary segmentation map, denote each closed region composed of pixel points with a gray value of 1 as a to-be-determined highlighted region.

4. The cervical lesion grading discrimination method based on deep learning according to claim 1, wherein The specific obtaining steps of the reflective interference index include: Obtain the shape irregularity degree and edge abrupt change degree of each to-be-determined highlighted region of each lesion characterization region; Obtain the reflective interference index of each lesion characterization region, and the lesion characterization region is in a direct proportional relationship with the shape irregularity degree and edge abrupt change degree of all its to-be-determined highlighted regions.

5. The cervical lesion grading discrimination method based on deep learning according to claim 4, characterized in that The obtaining of the shape irregularity degree and edge abrupt change degree of each to-be-determined highlighted region of each lesion characterization region includes: In each lesion characterization region, obtain the convex hull region of each highlight region to be determined, and denote the ratio of the number of pixel points in the convex hull region of each highlight region to be determined to the number of its own pixel points as the shape irregularity degree of each highlight region to be determined; In each lesion characterization region, obtain the fitting line segment of each edge pixel point of each highlight region to be determined and the next pixel point in the clockwise direction, and denote it as the trend line segment of each edge pixel point of each highlight region to be determined; Denote the slope difference between the trend line segments of each edge pixel point of each highlight region to be determined and the next edge pixel point in the clockwise direction as the gradient abrupt change amplitude of each edge pixel point of each highlight region to be determined; Integrate the gradient abrupt change amplitude and the edge gradient value of all edge pixel points of each highlight region to be determined to obtain the edge abrupt change degree of each highlight region to be determined, wherein the edge abrupt change degree is in a direct proportional relationship with the distribution range of the gradient abrupt change amplitude, and the edge abrupt change degree and the edge gradient value are in an inverse proportional relationship.

6. The cervical lesion grading discrimination method based on deep learning according to claim 1, wherein The specific steps for obtaining the blood shape dispersion degree are as follows: In each lesion characterization region, obtain the main axis direction of each internal closed region; Denote the projection length of the edge of the internal closed region on the main axis direction as the main axis projection length of each internal closed region; Denote the projection length of the edge of the internal closed region on the tangent direction of the main axis direction as the tangent projection length of each internal closed region; Obtain the non-tubular manifestation degree of each internal closed region, where the non-tubular manifestation degree is in an inverse proportional relationship with the main axis projection length and in a direct proportional relationship with the tangent projection length; For each pixel point passed by the main axis direction of each internal closed region, obtain the distance from each pixel point to the edge of the internal closed region in the tangent direction of the main axis direction, and denote it as the relative diameter of each pixel point passed by the main axis direction of each internal closed region; Obtain the variance of the relative diameters of all pixel points passed by the main axis direction of each internal closed region, and denote it as the diameter non-uniformity of each internal closed region; Obtain the blood shape dispersion degree of each internal closed region, where the blood shape dispersion degree is in a direct proportional relationship with the diameter non-uniformity and the non-tubular manifestation degree.

7. The cervical lesion grading discrimination method based on deep learning according to claim 1, characterized in that, The specific steps for obtaining the blood pooling manifestation degree are as follows: Obtain the mean value of all gray values of each pixel point passed by the main axis direction of each internal closed region in the tangent direction of the main axis direction and within the internal closed region, and denote it as the sectional gray value of each pixel point passed by the main axis direction of each internal closed region; Denote the sequence formed by the sectional gray values in the direction pointed by the main axis direction of each internal closed region as the sectional gray value sequence of each internal closed region; Denote the cumulative sum of the absolute values of the differences between each sequence value and the next sequence value in the sectional gray value sequence as the blood pooling manifestation degree of each internal closed region.

8. The cervical lesion grading discrimination method based on deep learning according to claim 1, wherein The specific steps for obtaining the blood interference index are as follows: Obtain the blood interference index of each lesion characterization region, and the blood interference index is in a direct proportional relationship with both the blood shape dispersion degree and the blood pooling manifestation degree of all internal closed regions in each lesion characterization region.

9. The cervical lesion grading and discrimination method based on deep learning according to claim 1, characterized in that Constructing the multi-dimensional feature vector of each lesion characterization region according to the blood interference index and the reflection interference index of each lesion characterization region includes: Denote the blood interference index of the i-th lesion characterization region as ; Denote the specular interference index of the i-th lesion characterization region as ; The multi-dimensional feature vector of the i-th lesion characterization region , where is the transpose symbol.

10. The cervical lesion grading and discrimination method based on deep learning according to claim 1, characterized in that, Using the multi-dimensional feature vector to train a deep learning model for cervical lesion grading judgment includes: Assign the multi-dimensional feature vectors of each lesion characterization region in the c-th cervical image to each pixel point of each lesion characterization region in the c-th cervical image to obtain the spatial attention weight matrix of the c-th cervical image, where the size of the spatial attention weight matrix is , where and are the length and width of the c-th cervical image respectively, is the dimension number of the multi-dimensional feature vectors of each lesion characterization region in the c-th cervical image; Input the c-th image into the CNN neural network to obtain the feature map of the c-th cervical image, and denote the product of the feature map of the c-th cervical image and its spatial attention weight matrix as the modified attention feature map of the c-th cervical image; Pre-construct a training set to obtain the modified attention feature maps of all cervical images in the training set; Train the CNN model with the modified attention feature maps of all cervical images in the training set to obtain a cervical lesion grading discrimination model, and then use the cervical lesion grading discrimination model to perform cervical lesion grading determination.

Images