An image processing method for oral mucosal squamous cell carcinoma

Abstract

The present invention discloses an image processing method for oral mucosal squamous cell carcinoma, which relates to the technical field of image processing. The method comprises the following steps: inputting #imgabs0# into a first semantic segmentation model, inputting #imgabs1# into a second semantic segmentation model, and obtaining a saliva mask and a local positive lesion mask #imgabs2#; obtaining a transition mask #imgabs3#; calculating boundary mutation strength, constructing a boundary-guided diffusion map #imgabs4# according to the boundary mutation strength, using #imgabs5# to segment a local positive lesion mask #imgabs7# from #imgabs6#, and merging #imgabs8# and #imgabs9# to obtain an overall lesion mask. The present invention can achieve more complete and accurate segmentation of oral mucosal squamous cell carcinoma lesion areas under conditions of strong reflective occlusion and complex background.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method for oral mucosal squamous cell carcinoma. Background Art

[0002] Oral mucosal squamous cell carcinoma is a common oral malignant lesion. In its early stages, it is characterized by irregular tissue structure, blurred boundaries, and a small area. With the development of digital medical imaging technology, more and more clinical application scenarios use oral endoscopic images or digital scanning images to analyze the lesion area to achieve auxiliary detection and judgment of positive lesions.

[0003] Existing oral image processing methods mostly perform image segmentation based on color features or shallow semantic information. However, in actual images, due to interference factors such as saliva reflection, imaging angle, and lighting changes, there are a large number of highlighted areas in the image. These areas usually block the visual information of the actual lesion tissue, thereby significantly reducing the segmentation accuracy. In addition, the color differences and texture features of early lesion areas are highly similar to those of normal tissue, making them difficult to accurately identify using traditional methods.

[0004] On the other hand, existing methods usually process based on a single feature dimension and lack the ability to jointly model multi-dimensional structural features such as color information, texture features, and spatial boundaries, resulting in insufficient recognition robustness in complex backgrounds, especially in areas with blurred edges and occluded lesions, resulting in a high missed detection rate.

[0005] Therefore, there is an urgent need for an image processing method that can improve the ability to identify lesion areas with blurred boundaries while dealing with reflection interference, and has strong boundary understanding capabilities to improve the automatic processing quality and segmentation accuracy of oral mucosal squamous cell carcinoma images. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an image processing method for oral mucosal squamous cell carcinoma.

[0007] To achieve the above objectives, the present invention provides an image processing method for oral mucosal squamous cell carcinoma, comprising:

[0008] Acquire the first oral image , and copy Second oral image after ;

[0009] Will Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ;

[0010] Get in and After performing the AND operation, a transition mask of the overlapping parts is generated. ;

[0011] Calculate the boundary mutation intensity and construct the boundary guided diffusion map based on the boundary mutation intensity ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask .

[0012] Preferably, the first semantic segmentation model is trained as follows:

[0013] Obtain a historical oral image sample set, and divide the historical oral image sample set into a first semantic segmentation training set and a first semantic segmentation test set, wherein the historical oral image sample set includes multiple grayscale sample oral images and the corresponding first annotation label;

[0014] The first label includes a first positive label and a first negative label, and the first positive label is represented by a saliva mask. ,The first negative label represents the non-saliva mask;

[0015] Initialize the first convolutional neural network and train the first semantic segmentation training set As input of the first convolutional neural network and the first annotation label as output, the first convolutional neural network is trained to obtain a first semantic segmentation network to be verified;

[0016] The first semantic segmentation network is model verified using the first semantic segmentation test set, and a first semantic segmentation network with a test error less than or equal to a preset test error threshold is output as the trained first semantic segmentation model.

[0017] Preferably, the first annotation label is obtained as follows:

[0018] Acquire standard lesion images and sample oral images and from mid-saliva area Get the first color feature, and get A second color feature of each pixel in ;

[0019] Wherein, the first color feature includes a target saturation range and a target brightness range, and the second color feature includes a saturation value and a brightness value;

[0020] Will The second color feature of each pixel in the image is compared with the color feature of the first color feature to filter out The first target pixel and the non-first target pixel in:

[0021] ;

[0022] Where: Represents a sample oral image exist The color characteristics of the pixel at , including saturation and brightness values; is the first target pixel, represents the non-first target pixel, represents the target saturation range, Indicates the target brightness range;

[0023] All pixel areas connected by the first target pixel are used as saliva masks , and marked as the first positive label, and all pixel areas formed by connecting non-first target pixels are taken as non-saliva masks and marked as the first negative label;

[0024] Repeat the above steps until all the historical oral image samples are Marked, get all The first annotation label.

[0025] Preferably, the target saturation interval is obtained as follows:

[0026] Standard lesion image , convert from RGB color space to HSV color space to get saturation image , where the saturation of each pixel ranges from [0,1];

[0027] from The saturation values corresponding to the pixels in the saliva area and the lesion area are extracted to form the first saturation set and the second saturation set ;

[0028] The saturation range [0,1] is evenly divided into K saturation subintervals, and each saturation subinterval is defined as: , is the i-th saturation subinterval;

[0029] Statistical first saturation set Among them, the number of pixels falling within each saturation sub-interval is denoted as , and the number of pixels falling within each saturation sub-interval in the second saturation set is also counted and denoted as ;

[0030] According to and , the first saturation histogram and the second saturation histogram are constructed respectively;

[0031] Any two consecutive saturation sub-intervals are selected as the saturation sub-interval combination , where 0 ≤ m ≤ n < K, and the first error term and the second error term are defined as follows:

[0032] Among them, the calculation method of the first error term is:

[0033] ;

[0034] Among them, the calculation method of the second error term is:

[0035] ;

[0036] Based on the first error term and the second error term , the discriminant error total loss function is constructed as follows:

[0037] ;

[0038] In the formula: and are the weighting coefficients of the first error term and the second error term respectively, ;

[0039] All saturation sub-interval combinations are traversed, and the corresponding discriminant error total loss function is calculated for each saturation sub-interval combination;

[0040] For all saturation sub-interval combinations corresponding to , they are sorted in ascending order, and the saturation sub-interval combination corresponding to the first is used as the target saturation interval .

[0041] Preferably, the training method of the second semantic segmentation model is as follows:

[0042] Obtain a historical oral image sample set, divide the historical oral image sample set into a second semantic segmentation training set and a second semantic segmentation test set, and the historical oral image sample set also includes a sample oral image corresponding to the sample oral image. The second annotation label;

[0043] The second annotation label includes a second positive label and a second negative label, and the second positive label is represented by the first local positive lesion mask , the second negative label represents the non-first local positive lesion mask;

[0044] Initialize the second convolutional neural network and use the second semantic segmentation training set As input of the second convolutional neural network, and taking the second annotation label as output, the second convolutional neural network is trained to obtain a second semantic segmentation network to be verified;

[0045] The second semantic segmentation network is model verified using the second semantic segmentation test set, and a second semantic segmentation network with a test error less than or equal to a preset threshold is output as the trained second semantic segmentation model.

[0046] Preferably, the second annotation label is obtained as follows:

[0047] The grayscale sample oral image is processed by sliding window Perform window division to obtain a set of local window area blocks , where the sliding step is 1, the window size is W×W, and Z and W are positive integers;

[0048] Acquire standard lesion images The first texture feature of the lesion area, and The second texture feature of each local window area in;

[0049] Wherein, the first texture feature and the second texture feature are both statistical feature sets of gray-level co-occurrence matrices, comprising multiple statistical features, including energy, average contrast, and entropy;

[0050] Calculate the texture similarity between the first texture feature of the lesion area and the second texture feature of the local window area , which is calculated as follows:

[0051] ;

[0052] Where: is the eigenvalue of the rth statistical feature in the first texture feature of the lesion area, is the eigenvalue of the rth statistical feature in the second texture feature of the local window area, is the total number of statistical features, is a positive real number;

[0053] The texture similarity is compared with the preset texture similarity threshold. If the texture similarity is greater than or equal to the texture similarity threshold, the corresponding local window area is used as the first local positive lesion mask. , and marked as the second positive label, otherwise, the corresponding local window area is used as the non-first local positive lesion mask and marked as the second negative label;

[0054] Repeat the above steps until all the historical oral image samples are Marked, get all The second annotation label.

[0055] Preferably, the calculation of the boundary mutation strength includes:

[0056] for Every pixel in , perform the following operations:

[0057] Pixel Calculate the component of the Sobel gradient in the x direction and the y-direction component :

[0058] ;

[0059] ;

[0060] Where: Representing an image The partial derivative in the horizontal x direction is, Representing an image Partial derivative in the vertical y direction;

[0061] The amount and quantity Combined into a gradient vector , and calculate its counterclockwise perpendicular vector based on the gradient vector as the boundary normal direction vector:

[0062] ;

[0063] Normal direction vector to the boundary Normalize to get the unit normal direction ;

[0064] by As the starting point, along the unit normal direction Sample e pixels in sequence to form a sampling sequence:

[0065] ;

[0066] For each sampling point , in the original first oral image Calculate the Sobel gradient magnitude:

[0067] ;

[0068] The gradient amplitudes of e pixels are combined into a gradient sequence , and calculate the maximum value of the first-order difference of the gradient sequence, which is defined as the boundary mutation intensity : .

[0069] Preferably, the boundary-guided diffusion map is constructed according to the boundary mutation intensity. ,include:

[0070] Construct a new oral image that is the same as the original first oral image Two-dimensional floating-point images with exactly the same size , and initialize all its pixel positions to zero values:

[0071] ;

[0072] Traversing the transition mask All pixels in , and judge each pixel Is the boundary mutation intensity greater than the preset structural mutation discrimination threshold? ;

[0073] If not, the corresponding pixel Marked as non-diffusive source point;

[0074] If so, the corresponding pixel Mark as diffusion source point, and for each diffusion source point, re- As the starting point, and along the unit normal direction vector Sampling in sequence pixels to form a new sampling sequence:

[0075] ;

[0076] For each sampling point , calculate its diffusion weight:

[0077] ;

[0078] Where: is the standard deviation of the diffusion kernel, is an exponential function;

[0079] The boundary mutation strength , by weight Weighted accumulation to boundary guided diffusion map The update formula is: .

[0080] Preferably, the Segment the second local positive lesion mask ,include:

[0081] Setting the lesion likelihood threshold ;

[0082] for Each pixel in Perform pixel differentiation as follows:

[0083] ;

[0084] Where: , representing pixels exist is marked as 1 and guides the diffusion map at the boundary The value in is greater than or equal to the lesion possibility threshold ; Represents pixels Dissatisfied or not satisfied , 1 represents the second target pixel, and 0 represents the non-second target pixel;

[0085] The area formed by the second target pixel obtained after differentiation is obtained from Segmented out and marked as the second local positive lesion mask .

[0086] An image processing system for oral mucosal squamous cell carcinoma is implemented based on the above-mentioned image processing method for oral mucosal squamous cell carcinoma, comprising:

[0087] An acquisition module for acquiring a first oral cavity image , and copy Second oral image after ;

[0088] The initial segmentation module is used to Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ;

[0089] Merge module to get and After performing the AND operation, a transition mask of the overlapping parts is generated. ;

[0090] The sub-segmentation module is used to calculate the boundary mutation strength and construct the boundary guided diffusion map based on the boundary mutation strength. ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask .

[0091] Compared with the prior art, the present invention has the following beneficial effects:

[0092] The oral mucosal squamous cell carcinoma image processing method provided by the present invention can effectively distinguish between high-reflective interference areas and real lesion areas in the image by introducing a dual-model collaborative segmentation mechanism of saliva mask and lesion initial segmentation mask, thereby reducing the impact of saliva areas on the accuracy of lesion identification; at the same time, by constructing a boundary-guided diffusion map based on boundary mutation strength, it can explore potential obscured lesion edges in the transition area, effectively improving the ability to recover and detect lesion structures in fuzzy areas.

[0093] Compared with existing methods based on single feature or single model processing, the present invention utilizes the multi-dimensional fusion of color features, texture features and boundary structure features to enhance the robustness of detection of lesions with blurred boundaries and small scale, and achieves a more complete and accurate segmentation of oral mucosal squamous cell carcinoma lesion areas under strong reflective occlusion and complex background conditions; overall, the present invention can improve the boundary coherence and regional integrity of lesion segmentation, reduce the missed detection rate caused by reflective occlusion, and has good application prospects and practical promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0095] Figure 1 Schematic diagram of the process of the present invention;

[0096] Figure 2 Schematic diagram of the structure of the system of the present invention. DETAILED DESCRIPTION

[0097] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0098] See also Figure 1 The first embodiment of the present invention provides an image processing method for oral mucosal squamous cell carcinoma, comprising:

[0099] Step 1: Obtain the first oral image , and copy Second oral image after ;

[0100] It should be understood that: the first oral cavity image The second oral image is obtained by photographing the oral cavity of patients with oral mucosal squamous cell carcinoma using an oral endoscope (such as Olympus ENF-VH) or a digital oral scanner, with a resolution of 1024×1024, an output image format of PNG, and RGB channels. First oral image It is worth noting that the first oral cavity image With the second oral image All aspects such as size, resolution and orientation are exactly the same.

[0101] Step 2: Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ;

[0102] Specifically, the training method of the first semantic segmentation model is as follows:

[0103] Obtain a historical oral image sample set, and divide the historical oral image sample set into a first semantic segmentation training set and a first semantic segmentation test set, wherein the historical oral image sample set includes multiple grayscale sample oral images and the corresponding first annotation label;

[0104] The first label includes a first positive label and a first negative label, and the first positive label is represented by a saliva mask. ,The first negative label represents the non-saliva mask;

[0105] In implementation, the first annotation label is obtained as follows:

[0106] Acquire standard lesion images and sample oral images and from mid-saliva area Get the first color feature, and get A second color feature of each pixel in ;

[0107] Wherein, the standard lesion image Based on pre-annotated saliva regions and lesion area ;

[0108] Wherein, the first color feature includes a target saturation range and a target brightness range, and the second color feature includes a saturation value and a brightness value;

[0109] Specifically, the target saturation range is obtained as follows:

[0110] Standard lesion image , convert from RGB color space to HSV color space to get saturation image , where the saturation of each pixel ranges from [0,1];

[0111] It should be noted that: the standard lesion image Pre-stored in the system database, including saliva area and lesion area, among which the standard lesion image The saliva area and lesion area in the image were manually annotated by oral mucosal squamous cell carcinoma physician experts;

[0112] The conversion from RGB color space to HSV color space includes:

[0113] 1) Calculate hue as follows:

[0114] ;

[0115] 2) Calculate saturation as follows:

[0116] ;

[0117] 3) Calculate the brightness as follows:

[0118] ;

[0119] Where: is color difference, , is the maximum value of the color channel, is the minimum value of the color channel, For hue, is the saturation, is the lightness, is the pixel value of the red channel, G is the pixel value of the green channel, is the pixel value of the blue channel;

[0120] From extract the saturation values corresponding to the pixels belonging to the saliva region and the lesion region respectively, and form the first saturation set and the second saturation set ;

[0121] Among them, , represents the saturation values of all pixels belonging to the saliva region; , represents the saturation values of all pixels belonging to the lesion region;

[0122] Divide the saturation range [0, 1] evenly into K saturation sub - intervals, and define each saturation sub - interval as: , is the i - th saturation sub - interval;

[0123] Count the number of pixels in the first saturation set falling into each saturation sub - interval, denoted as , and count the number of pixels in the second saturation set falling into each saturation sub - interval, denoted as ;

[0124] According to and construct the first saturation histogram and the second saturation histogram ;

[0125] Arbitrarily select two consecutive saturation sub - intervals as the saturation sub - interval combination , where 0 ≤ m ≤ n < K, and define the first error term and the second error term :

[0126] Among them, the first error term , represents the number of pixels in the selected saturation sub - interval combination that are mis - identified as saliva - region pixels in the lesion region, and the calculation method is:

[0127] ;

[0128] Among them, the second error term , represents the number of saliva - region pixels that are missed and identified as non - saliva regions outside the interval , and the calculation method is:

[0129] ;

[0130] According to the first error term and the second error term Construct the total loss function of the discrimination error :

[0131] ;

[0132] Where: and are the weighted coefficients of the first error term and the second error term, respectively, and are used to adjust the influence of misjudgment of lesions and missed reflections in the objective function. , set by technical personnel based on historical data;

[0133] Traverse all saturation sub-interval combinations, and calculate the corresponding total loss function of the discrimination error for each saturation sub-interval combination. ;

[0134] For all saturation subinterval combinations Corresponding Sort in ascending order, sort first Corresponding saturation sub-interval combination As the target saturation range ;

[0135] It is worth noting that the target brightness interval is the same as the target saturation interval acquisition logic. The difference is that when determining the target saturation interval, the brightness range is first obtained as [0,1], and the first brightness set is extracted. (representing the brightness values of all pixels belonging to the saliva area) and the second brightness set (represents the brightness value of all pixels belonging to the lesion area), divided into K brightness sub-intervals, and the number of pixels falling into each brightness sub-interval is counted and the number of pixels falling into each brightness subrange , the first brightness histogram is constructed and the second brightness histogram , which defines the third error term and the fourth error term , the constructed total loss function of the discrimination error , and are the weighted coefficients of the third and fourth error terms, , the last thing to traverse is all the combinations of brightness sub-intervals, and by combining all the combinations of brightness sub-intervals Corresponding Perform ascending sorting and selection to obtain the target brightness range. For details, refer to the relevant part about the target saturation range above, which will not be described in detail here;

[0136] Will The second color feature of each pixel in the image is compared with the color feature of the first color feature to filter out The first target pixel and the non-first target pixel in:

[0137] ;

[0138] Where: Represents a sample oral image exist The color characteristics of the pixel at , including saturation and brightness values; is the first target pixel, represents the non-first target pixel, represents the target saturation range, Indicates the target brightness range;

[0139] All pixel areas connected by the first target pixel are used as saliva masks , and marked as the first positive label, and all pixel areas formed by connecting non-first target pixels are taken as non-saliva masks and marked as the first negative label;

[0140] Repeat the above steps until all the historical oral image samples are Marked, get all The first annotation label of

[0141] Initialize the first convolutional neural network and train the first semantic segmentation training set As input of the first convolutional neural network and the first annotation label as output, the first convolutional neural network is trained to obtain a first semantic segmentation network to be verified;

[0142] Using the first semantic segmentation test set to perform model verification on the first semantic segmentation network, and outputting a first semantic segmentation network with a test error less than or equal to a preset test error threshold as the trained first semantic segmentation model;

[0143] Specifically, the training method of the second semantic segmentation model is as follows:

[0144] Obtain a historical oral image sample set, divide the historical oral image sample set into a second semantic segmentation training set and a second semantic segmentation test set, and the historical oral image sample set also includes a sample oral image corresponding to the sample oral image. The second annotation label;

[0145] It should be noted that: the sample oral images in the historical oral image sample set , obtained by technicians by taking medical images of the oral cavity of several patients or non-patients in advance, and pre-stored in the system database;

[0146] The second annotation label includes a second positive label and a second negative label, and the second positive label is represented by the first local positive lesion mask , the second negative label represents the non-first local positive lesion mask;

[0147] In implementation, the second annotation tag is obtained as follows:

[0148] The grayscale sample oral image is processed by sliding window Perform window division to obtain a set of local window area blocks , where the sliding step is 1, the window size is W×W (e.g. W=32), and Z and W are positive integers;

[0149] It should be noted that if the sliding window exceeds the image boundary, mirror padding or zero padding is used to ensure the integrity of the window block;

[0150] Acquire standard lesion images The first texture feature of the lesion area, and The second texture feature of each local window area in;

[0151] Wherein, the first texture feature and the second texture feature are both statistical feature sets of gray-level co-occurrence matrices, including multiple statistical features, including but not limited to energy, average contrast, and entropy;

[0152] Calculate the texture similarity between the first texture feature of the lesion area and the second texture feature of the local window area , which is calculated as follows:

[0153] ;

[0154] Where: is the eigenvalue of the rth statistical feature in the first texture feature of the lesion area, is the eigenvalue of the rth statistical feature in the second texture feature of the local window area, is the total number of statistical features, is a (very small) positive real number used to avoid division by zero errors;

[0155] The texture similarity is compared with the preset texture similarity threshold. If the texture similarity is greater than or equal to the texture similarity threshold, the corresponding local window area is used as the first local positive lesion mask. , and marked as the second positive label, otherwise, the corresponding local window area is used as the non-first local positive lesion mask and marked as the second negative label;

[0156] Repeat the above steps until all the historical oral image samples are Marked, get all The second annotation label;

[0157] Initialize the second convolutional neural network and use the second semantic segmentation training set As input of the second convolutional neural network, and taking the second annotation label as output, the second convolutional neural network is trained to obtain a second semantic segmentation network to be verified;

[0158] Using the second semantic segmentation test set to perform model verification on the second semantic segmentation network, and outputting a second semantic segmentation network with a test error less than or equal to a preset test error threshold as the trained second semantic segmentation model;

[0159] By driving dual-model training through the three features of saturation, brightness, and texture, the accuracy of initial segmentation is improved and the ability to handle small local lesions and reflective interference is enhanced.

[0160] Step 3: Get the and After performing the AND operation, a transition mask of the overlapping parts is generated. ;

[0161] The expression of the AND operation is as follows: ;in, The overlapping part of the image that is identified as both saliva and lesion is called the transition area. The transition area represents the area where the lesion is partially blurred due to saliva, but there is still the possibility of lesion.

[0162] It should be understood that the image will be binarized before the operation, and a fixed threshold (0.5) or an empirical threshold setting based on the model output probability distribution will be used during binarization; among them, the AND operation is a common logical operation in computer image processing, and its purpose is to find the transition area. It is further explained that for two segmented images (binary images (each pixel value is 0 or 1)), if the pixel values ​​at the same position in the two images are both 1, then 1 is output (indicating that it is an overlapping part), and if any of the pixel values ​​are 0, then 0 is output (indicating that this part is not an overlapping part).

[0163] Step 4: Calculate the boundary mutation intensity and construct the boundary guided diffusion map based on the boundary mutation intensity ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask ;

[0164] In implementation, the calculation of the boundary mutation strength includes:

[0165] for Every pixel in , perform the following operations:

[0166] Pixel Calculate the component of the Sobel gradient in the x direction and the y-direction component :

[0167] ;

[0168] ;

[0169] Where: Representing an image The partial derivative in the horizontal x direction is, Representing an image The partial derivative in the vertical y direction is actually calculated by the Sobel filter;

[0170] The amount and quantity Combined into a gradient vector , and calculate its counterclockwise perpendicular vector based on the gradient vector as the boundary normal direction vector:

[0171] ;

[0172] Normal direction vector to the boundary Normalize to get the unit normal direction ;

[0173] It should be noted that the unitization formula is as follows: ;in, represents the Euclidean modulus of the normal vector;

[0174] by As the starting point, along the unit normal direction Sample e pixels in sequence to form a sampling sequence:

[0175] ;

[0176] For each sampling point , in the original first oral image Calculate the Sobel gradient magnitude:

[0177] ;

[0178] In an optional embodiment, if the sampling point If it is a non-integer coordinate position, then in the original first oral image Use bilinear interpolation to estimate the Sobel gradient component from the grayscale values of the four neighboring pixels;

[0179] The gradient amplitudes of e pixels are combined into a gradient sequence , and calculate the maximum value of the first-order difference of the gradient sequence, which is defined as the boundary mutation intensity :

[0180]

[0181] Where: Indicates that at the border The maximum mutation degree of the image gradient intensity along the normal direction is 0.001, which indicates that the boundary interruption is more obvious and the location of the lesion structure is more likely to be blocked by saliva. In practice, the boundary-guided diffusion map is constructed according to the boundary mutation intensity. ,include:

[0182] Construct a new oral image that is the same as the original first oral image Two-dimensional floating-point images with exactly the same size , and initialize all its pixel positions to zero values:

[0183] ;

[0184] Where: Represents a universal quantifier, indicating that the initialization operation is performed on all pixel coordinates in the image;

[0185] Traversing the transition mask All pixels in , and judge each pixel Is the boundary mutation intensity greater than the preset structural mutation discrimination threshold? ;

[0186] If not, the corresponding pixel Marked as non-diffusive source point;

[0187] If so, the corresponding pixel Mark as diffusion source point, and for each diffusion source point, re- As the starting point, and along the unit normal direction vector Sampling in sequence pixels to form a new sampling sequence:

[0188] ;

[0189] For each sampling point , calculate its diffusion weight:

[0190] ;

[0191] Where: is the standard deviation of the diffusion kernel, is an exponential function;

[0192] The boundary mutation strength , by weight Weighted accumulation to boundary guided diffusion map The update formula is: ; For example, assuming pixel , its normal vector is [0.6, 0.8], and the gradient sequence calculated after sampling e=5 points outward is {20, 25, 40, 30, 15}, then ,like , it is marked as a diffusion source point.

[0193] In implementation, the Segment the second local positive lesion mask ,include:

[0194] Setting the lesion likelihood threshold ;

[0195] for Each pixel in Perform pixel differentiation as follows:

[0196] ;

[0197] Where: , representing pixels exist is marked as 1 and guides the diffusion map at the boundary The value in is greater than or equal to the lesion possibility threshold ; Represents pixels Dissatisfied or not satisfied , 1 represents the second target pixel, 0 represents the non-second target pixel, The typical value range is 5-20, which is set empirically based on the blurriness of the lesion boundary in the image;

[0198] The area formed by the second target pixel obtained after differentiation is obtained from Segmented out and marked as the second local positive lesion mask ;

[0199] By diffusing information along the normal direction of the boundary at the image structure mutation point, the blurred boundary information obscured by saliva can be restored, thereby achieving more precise lesion recovery and compensating for segmentation omissions, significantly improving the detection rate, segmentation coherence and structural integrity of early blurred lesions in oral squamous cell carcinoma images.

[0200] See also Figure 2 Based on the same inventive concept, the second aspect of the present invention provides an image processing system for oral mucosal squamous cell carcinoma. For details not provided in this embodiment, please refer to the description of the relevant parts in Example 1. The system includes:

[0201] An acquisition module for acquiring a first oral cavity image , and copy Second oral image after ;

[0202] The initial segmentation module is used to Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ;

[0203] Merge module to get and After performing the AND operation, a transition mask of the overlapping parts is generated. ;

[0204] The sub-segmentation module is used to calculate the boundary mutation strength and construct the boundary guided diffusion map based on the boundary mutation strength. ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask .

[0205] The above embodiments can be implemented in whole or in part via software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0206] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only one type. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0209] Some of the data in the above formula are calculated by removing the dimensions and taking their numerical values. The formula is a formula that is closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0210] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An image processing method for oral mucosal squamous cell carcinoma, characterized in that: include: Acquire the first oral image , and copy Second oral image after ; Will Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ; Get in and After performing the AND operation, a transition mask of the overlapping parts is generated. ; Calculate the boundary mutation intensity and construct the boundary guided diffusion map based on the boundary mutation intensity ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask .

2. The image processing method for oral mucosal squamous cell carcinoma according to claim 1, characterized in that: The first semantic segmentation model is trained as follows: Obtain a historical oral image sample set, and divide the historical oral image sample set into a first semantic segmentation training set and a first semantic segmentation test set, wherein the historical oral image sample set includes multiple grayscale sample oral images and the corresponding first annotation label; The first label includes a first positive label and a first negative label, and the first positive label is represented by a saliva mask. ,The first negative label represents the non-saliva mask; Initialize the first convolutional neural network and train the first semantic segmentation training set As input of the first convolutional neural network and the first annotation label as output, the first convolutional neural network is trained to obtain a first semantic segmentation network to be verified; The first semantic segmentation network is model verified using the first semantic segmentation test set, and a first semantic segmentation network with a test error less than or equal to a preset test error threshold is output as the trained first semantic segmentation model.

3. The image processing method for oral mucosal squamous cell carcinoma according to claim 2, characterized in that: The first annotation label is obtained as follows: Acquire standard lesion images and sample oral images and from mid-saliva area Get the first color feature, and get A second color feature of each pixel in ; Wherein, the first color feature includes a target saturation range and a target brightness range, and the second color feature includes a saturation value and a brightness value; Will The second color feature of each pixel in the image is compared with the color feature of the first color feature to filter out The first target pixel and the non-first target pixel in: ； Where: Represents a sample oral image exist The color characteristics of the pixel at , including saturation and brightness values; is the first target pixel, represents the non-first target pixel, represents the target saturation range, Indicates the target brightness range; All pixel areas connected by the first target pixel are used as saliva masks , and marked as the first positive label, and all pixel areas formed by connecting non-first target pixels are taken as non-saliva masks and marked as the first negative label; Repeat the above steps until all the historical oral image samples are Marked, get all The first annotation label.

4. The image processing method for oral mucosal squamous cell carcinoma according to claim 3, characterized in that: The target saturation interval is obtained as follows: Standard lesion image , convert from RGB color space to HSV color space to get saturation image , where the saturation of each pixel ranges from [0,1]; from The saturation values corresponding to the pixels in the saliva area and the lesion area are extracted to form the first saturation set and the second saturation set ; The saturation range [0,1] is evenly divided into K saturation subintervals, and each saturation subinterval is defined as: , is the i-th saturation subinterval; Statistical first saturation set The number of pixels falling into each saturation subinterval is recorded as , and the statistical second saturation set The number of pixels falling into each saturation subinterval is recorded as ; according to and Construct the first saturation histogram respectively and the second saturation histogram ; Arbitrarily select two consecutive saturation sub - intervals as the saturation sub - interval combination , where \(0\leq m\leq n < K\), and define the first error term and the second error term : Among them, the first error term The calculation method is: ； Among them, the second error term The calculation method is: ； According to the first error term and the second error term Construct the total loss function of the discrimination error : ； Where: and are the weighting coefficients of the first error term and the second error term, ; Traverse all saturation sub-interval combinations, and calculate the corresponding total loss function of the discrimination error for each saturation sub-interval combination. ; For all saturation subinterval combinations Corresponding Sort in ascending order, sort first Corresponding saturation sub-interval combination As the target saturation range .

5. The image processing method for oral mucosal squamous cell carcinoma according to claim 4, characterized in that: The second semantic segmentation model is trained as follows: Obtain a historical oral image sample set, divide the historical oral image sample set into a second semantic segmentation training set and a second semantic segmentation test set, and the historical oral image sample set also includes a sample oral image corresponding to the sample oral image. The second annotation label; The second annotation label includes a second positive label and a second negative label, and the second positive label is represented by the first local positive lesion mask , the second negative label represents the non-first local positive lesion mask; Initialize the second convolutional neural network and use the second semantic segmentation training set As input of the second convolutional neural network, and taking the second annotation label as output, the second convolutional neural network is trained to obtain a second semantic segmentation network to be verified; The second semantic segmentation network is model verified using the second semantic segmentation test set, and a second semantic segmentation network with a test error less than or equal to a preset threshold is output as the trained second semantic segmentation model.

6. The image processing method for oral mucosal squamous cell carcinoma according to claim 5, characterized in that: The second annotation label is obtained as follows: The grayscale sample oral image is processed by sliding window Perform window division to obtain a set of local window area blocks , where the sliding step is 1, the window size is W×W, and Z and W are positive integers; Acquire standard lesion images The first texture feature of the lesion area, and The second texture feature of each local window area in; Wherein, the first texture feature and the second texture feature are both statistical feature sets of gray-level co-occurrence matrices, comprising multiple statistical features, including energy, average contrast, and entropy; Calculate the texture similarity between the first texture feature of the lesion area and the second texture feature of the local window area , which is calculated as follows: ； Where: is the eigenvalue of the rth statistical feature in the first texture feature of the lesion area, is the eigenvalue of the rth statistical feature in the second texture feature of the local window area, is the total number of statistical features, is a positive real number; The texture similarity is compared with the preset texture similarity threshold. If the texture similarity is greater than or equal to the texture similarity threshold, the corresponding local window area is used as the first local positive lesion mask. , and marked as the second positive label, otherwise, the corresponding local window area is used as the non-first local positive lesion mask and marked as the second negative label; Repeat the above steps until all the historical oral image samples are Marked, get all The second annotation label.

7. The image processing method for oral mucosal squamous cell carcinoma according to claim 6, characterized in that: The calculation of the boundary mutation strength includes: for Every pixel in , perform the following operations: Pixel Calculate the component of the Sobel gradient in the x direction and the y-direction component : ； ； Where: Representing an image The partial derivative in the horizontal x direction is, Representing an image Partial derivative in the vertical y direction; The amount and quantity Combined into a gradient vector , and calculate its counterclockwise perpendicular vector based on the gradient vector as the boundary normal direction vector: ； Normal direction vector to the boundary Normalize to get the unit normal direction ; by As the starting point, along the unit normal direction Sample e pixels in sequence to form a sampling sequence: ； For each sampling point , in the original first oral image Calculate the Sobel gradient magnitude: ； The gradient amplitudes of e pixels are combined into a gradient sequence , and calculate the maximum value of the first-order difference of the gradient sequence, which is defined as the boundary mutation intensity : .

8. The image processing method for oral mucosal squamous cell carcinoma according to claim 7, characterized in that: The boundary-guided diffusion map is constructed according to the boundary mutation intensity ,include: Construct a new oral image that is the same as the original first oral image Two-dimensional floating-point images with exactly the same size , and initialize all its pixel positions to zero values: ； Traversing the transition mask All pixels in , and judge each pixel Is the boundary mutation intensity greater than the preset structural mutation discrimination threshold? ; If not, the corresponding pixel Marked as non-diffusive source point; If so, the corresponding pixel Mark as diffusion source point, and for each diffusion source point, re- As the starting point, and along the unit normal direction vector Sampling in sequence pixels to form a new sampling sequence: ； For each sampling point , calculate its diffusion weight: ； Where: is the standard deviation of the diffusion kernel, is an exponential function; The boundary mutation strength , by weight Weighted accumulation to boundary guided diffusion map The update formula is: .

9. The image processing method for oral mucosal squamous cell carcinoma according to claim 8, characterized in that: The said Segment the second local positive lesion mask ,include: Setting the lesion likelihood threshold ; for Each pixel in Perform pixel differentiation as follows: ； Where: , representing pixels exist is marked as 1 and guides the diffusion map at the boundary The value in is greater than or equal to the lesion possibility threshold ; Represents pixels Dissatisfied or not satisfied , 1 represents the second target pixel, and 0 represents the non-second target pixel; The area formed by the second target pixel obtained after differentiation is obtained from Segmented out and marked as the second local positive lesion mask .

10. An image processing system for oral mucosal squamous cell carcinoma, implemented based on the image processing method for oral mucosal squamous cell carcinoma according to any one of claims 1 to 9, characterized in that: include: An acquisition module for acquiring a first oral cavity image , and copy Second oral image after ; The initial segmentation module is used to Input the pre-trained first semantic segmentation model to obtain the saliva mask ; and will Input the pre-trained second semantic segmentation model for initial segmentation to obtain the first local positive lesion mask ; Merge module to get and After performing the AND operation, a transition mask of the overlapping parts is generated. ; The sub-segmentation module is used to calculate the boundary mutation strength and construct the boundary guided diffusion map based on the boundary mutation strength. ,use from Segment the second local positive lesion mask , and and Merge to obtain the overall lesion mask .

Images