Tongue Image Classification Method for Systemic Lupus Erythematosus Patients with Different Traditional Chinese Medicine Syndromes

Abstract

The present invention relates to the technical field of image analysis, and specifically relates to a tongue image classification method for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes. The method includes: obtaining a target tongue surface image corresponding to a lupus erythematosus patient to be detected, and performing tongue image classification through a tongue image classification network; dividing the S-channel values in each obtained tongue surface image of lupus erythematosus into intervals; extracting connected components for the regions corresponding to each S-channel interval; analyzing and processing the missing conditions of the holes in each multi-connected region; performing adaptive supplementary division analysis based on the determined comprehensive supplementary index; and screening out a reference image set based on the obvious lupus erythematosus characteristic factors. Through image analysis, the present invention quantifies the obviousness of lupus erythematosus characteristics in the tongue surface images of lupus erythematosus patients, and to a certain extent reduces the possibility that the tongue image classification network learns inaccurate lupus erythematosus characteristic information, thereby improving the accuracy of tongue image classification for lupus erythematosus patients.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and particularly to a method for classifying tongue images of patients with systemic lupus erythematosus for different traditional Chinese medicine syndromes. Background Art

[0002] With the development of technology, neural networks are increasingly widely used. For example, they can be applied to the classification of tongue images of patients with systemic lupus erythematosus for different traditional Chinese medicine syndromes. Since the selection of the training set of a neural network often affects the training result of the neural network, the acquisition of the training set is crucial. Currently, when acquiring the training set of a neural network, the commonly used method is to acquire images of objects of the same category as the object to be detected to form the training set.

[0003] However, when acquiring a set of lupus erythematosus tongue surface images and a set of normal tongue surface images to form the training set of a neural network and training the neural network for classifying the tongue images of patients with lupus erythematosus, the following technical problems often exist:

[0004] Since a large number of training samples are often required for training a neural network, and the sources of the training samples are often diverse, and their shooting methods and the resolutions of the shooting devices are often different, the lupus erythematosus features in some lupus erythematosus tongue surface images in the set of lupus erythematosus tongue surface images may not be obvious, which may cause the neural network to learn inaccurate information in the training set, affect the generalization ability of the neural network, and further lead to poor accuracy in classifying the tongue images of patients with lupus erythematosus. Summary of the Invention

[0005] In order to solve the technical problem of poor accuracy in classifying the tongue images of patients with lupus erythematosus, the present invention proposes a method for classifying tongue images of patients with systemic lupus erythematosus for different traditional Chinese medicine syndromes.

[0006] In a first aspect, the present invention provides a method for classifying tongue images of patients with systemic lupus erythematosus for different traditional Chinese medicine syndromes, the method comprising:

[0007] Obtaining a target tongue surface image corresponding to a patient with lupus erythematosus to be detected;

[0008] According to the target tongue surface image, performing tongue image classification on the target tongue surface image through a pre-trained tongue image classification network, wherein the training set of the tongue image classification network is a target training set, and the target training set includes a set of normal tongue surface images and a reference image set;

[0009] The method for obtaining the reference image set includes:

[0010] Obtaining a set of lupus erythematosus tongue surface images and the S-channel value corresponding to each pixel point in each lupus erythematosus tongue surface image therein;

[0011] Divide the S-channel values corresponding to all pixel points in each lupus erythematosus tongue image into intervals to obtain the S-channel interval corresponding to each lupus erythematosus tongue image;

[0012] Extract the connected regions of the areas corresponding to each S-channel interval in its corresponding lupus erythematosus tongue image, and screen out the multi-connected regions from the extracted connected regions;

[0013] Analyze and process the missing situation of all holes in each multi-connected region to obtain the target missing index corresponding to each multi-connected region;

[0014] Determine the comprehensive supplement index corresponding to each S-channel interval according to the target missing index corresponding to all multi-connected regions corresponding to each S-channel interval and the distribution of the connected regions whose corresponding S-channel intervals are adjacent S-channel intervals;

[0015] Perform adaptive supplement division analysis based on the comprehensive supplement indexes corresponding to all S-channel intervals in each lupus erythematosus tongue image to obtain the obvious lupus erythematosus feature factor corresponding to each lupus erythematosus tongue image;

[0016] Screen out the lupus erythematosus tongue images in the lupus erythematosus tongue image set whose corresponding obvious lupus erythematosus feature factors are greater than the preset feature threshold to form a reference image set.

[0017] Combined with the first aspect above, in a possible implementation manner, the dividing the S-channel values corresponding to all pixel points in each lupus erythematosus tongue image into intervals to obtain the S-channel interval corresponding to each lupus erythematosus tongue image includes:

[0018] Determine any lupus erythematosus tongue image as a marked image, and make an S-channel histogram corresponding to the marked image, denoted as the marked S-channel histogram;

[0019] The S-channel values between the S-channel values corresponding to every two adjacent minima in the marked S-channel histogram form the S-channel interval.

[0020] Combined with the first aspect above, in a possible implementation manner, the analyzing and processing the missing situation of all holes in each multi-connected region to obtain the target missing index corresponding to each multi-connected region includes:

[0021] Determine the area ratio of each hole in its corresponding multi-connected region as the initial missing factor corresponding to each hole;

[0022] Determine the erythema missing contribution factor corresponding to each hole in each multi-connected region according to the difference between the initial missing factor corresponding to each hole in each multi-connected region and the initial missing factors corresponding to other holes;

[0023] Determine the target missing index corresponding to each multiply connected region according to the erythema missing contribution factor and the initial missing factor corresponding to all holes in each multiply connected region, where both the erythema missing contribution factor and the initial missing factor are positively correlated with the target missing index.

[0024] Combined with the first aspect above, in a possible implementation manner, the determining the erythema missing contribution factor corresponding to each hole in each multiply connected region according to the difference between the initial missing factor corresponding to each hole in each multiply connected region and the initial missing factors corresponding to other holes includes:

[0025] Determine any multiply connected region as a marked multiply connected region, determine any hole in the marked multiply connected region as a marked hole, and determine each hole in the marked multiply connected region other than the marked hole as a reference hole;

[0026] Determine the cumulative value of the absolute values of the differences between the initial missing factor corresponding to the marked hole and the initial missing factors corresponding to each reference hole as the target missing difference corresponding to the marked hole;

[0027] Determine the erythema missing contribution factor corresponding to the marked hole according to the target missing difference corresponding to the marked hole, where the target missing difference is positively correlated with the erythema missing contribution factor.

[0028] Combined with the first aspect above, in a possible implementation manner, the determining the comprehensive supplement index corresponding to each S-channel interval according to the target missing index corresponding to all multiply connected regions corresponding to each S-channel interval and the distribution of the connected regions whose corresponding S-channel intervals within them are adjacent S-channel intervals includes:

[0029] Screen out supplementary candidate connected regions from each hole according to all the connected regions corresponding to the adjacent S-channel intervals of the S-channel interval corresponding to each hole;

[0030] Determine the initial supplement factor corresponding to each supplementary candidate connected region in each hole in each multiply connected region according to the target missing index corresponding to each multiply connected region;

[0031] Determine the supplementary effectiveness corresponding to each supplementary candidate connected region in each hole in each multiply connected region according to the number of edge pixels of each hole and each supplementary candidate connected region within it;

[0032] Determine the adjacent supplement index corresponding to each hole in each multiply connected region according to the initial supplement factor and the supplementary effectiveness corresponding to all the supplementary candidate connected regions in each hole in each multiply connected region, where both the initial supplement factor and the supplementary effectiveness are positively correlated with the adjacent supplement index;

[0033] The mean value of the adjacent supplementary indicators corresponding to all holes in all multiply connected regions corresponding to each S-channel interval is determined as the comprehensive supplementary indicator corresponding to each S-channel interval.

[0034] Combined with the above first aspect, in a possible implementation manner, the screening of supplementary candidate connected regions from each hole according to all connected regions corresponding to adjacent S-channel intervals of the S-channel interval corresponding to each hole includes:

[0035] Determine any one hole as a candidate hole, and determine the S-channel interval corresponding to the multiply connected region to which the candidate hole belongs as the candidate S-channel interval;

[0036] Determine each S-channel interval adjacent to the candidate S-channel interval as a to-be-supplemented S-channel interval, and determine each connected region corresponding to each to-be-supplemented S-channel interval as a calibrated connected region;

[0037] Determine each calibrated connected region in the candidate hole as a supplementary candidate connected region.

[0038] Combined with the above first aspect, in a possible implementation manner, the formula for the initial supplementary factor corresponding to the supplementary candidate connected region in the hole within the multiply connected region is:

[0039] ; where is the initial supplementary factor corresponding to the th supplementary candidate connected region in the th hole within the th multiply connected region in the th lupus erythematosus tongue surface image; is the serial number of the lupus erythematosus tongue surface image; is the serial number of the multiply connected region in the th lupus erythematosus tongue surface image; is the serial number of the hole within the th multiply connected region; is the serial number of the supplementary candidate connected region within the th hole; is the normalization function; is the target missing index corresponding to the th multiply connected region in the th lupus erythematosus tongue surface image; is the target missing index corresponding to the target connected region; the method for obtaining the target connected region is: filling the th supplementary candidate connected region into the th hole within the th multiply connected region, so that the The reduced area of the holes is equal to the area corresponding to the th supplementary candidate connected component, so that the increased area of the th multi-connected region is equal to the area corresponding to the th supplementary candidate connected component, thereby realizing the update of the th multi-connected region in the th lupus erythematosus tongue image, and the updated multi-connected region is denoted as the target connected component.

[0040] Combined with the above first aspect, in a possible implementation, the formula for the supplementary effectiveness corresponding to the supplementary candidate connected component in the hole within the multi-connected region is:

[0041] ; where is the supplementary effectiveness corresponding to the th supplementary candidate connected component in the th hole within the th multi-connected region in the th lupus erythematosus tongue image; is the serial number of the lupus erythematosus tongue image; is the serial number of the multi-connected region in the th lupus erythematosus tongue image; is the serial number of the hole within the th multi-connected region; is the serial number of the supplementary candidate connected component within the th hole; is the normalization function; is the number of pixel points in the intersection between the edge of the th hole and the edge of the th supplementary candidate connected component within the th multi-connected region in the th lupus erythematosus tongue image; is the number of pixel points on the edge of the th hole within the th multi-connected region in the th lupus erythematosus tongue image; is the number of pixel points on the edge of the th supplementary candidate connected component within the th hole and within the th multi-connected region in the th lupus erythematosus tongue image.

[0042] Combined with the above first aspect, in a possible implementation, the adaptive supplementary division analysis is performed based on the comprehensive supplementary indicators corresponding to all S-channel intervals in each systemic lupus erythematosus tongue image to obtain the systemic lupus erythematosus feature obvious factor corresponding to each systemic lupus erythematosus tongue image, including:

[0043] According to the comprehensive supplementary indicators corresponding to all S-channel intervals in each systemic lupus erythematosus tongue image, the S-channel intervals in each systemic lupus erythematosus tongue image are clustered into two categories, and the continuous S-channel intervals in the category with higher comprehensive supplementary indicators are formed into continuous supplementary intervals, and the continuous S-channel intervals in the other category with lower comprehensive supplementary indicators are formed into continuous separation intervals;

[0044] According to the area filling situation between adjacent S-channel intervals in each continuous supplementary interval, the characteristic separation intervals are screened out from the continuous separation intervals adjacent to each continuous supplementary interval;

[0045] According to all the continuous supplementary intervals and their corresponding characteristic separation intervals in each systemic lupus erythematosus tongue image, the formula for determining the systemic lupus erythematosus feature obvious factor corresponding to each systemic lupus erythematosus tongue image is:

[0046] ; where is the systemic lupus erythematosus feature obvious factor corresponding to the th systemic lupus erythematosus tongue image; is the serial number of the systemic lupus erythematosus tongue image; is the normalization function; is the number of continuous supplementary intervals in the th systemic lupus erythematosus tongue image; is the serial number of the continuous supplementary interval in the th systemic lupus erythematosus tongue image; is the number of S-channel values included in the rd continuous supplementary interval in the th systemic lupus erythematosus tongue image; is the mean value of the target missing indicators corresponding to all multi-connected regions corresponding to all S-channel intervals in the characteristic separation interval corresponding to the th continuous supplementary interval in the th systemic lupus erythematosus tongue image.

[0047] Combined with the above first aspect, in a possible implementation, the screening of the characteristic separation intervals from the continuous separation intervals adjacent to each continuous supplementary interval according to the area filling situation between adjacent S-channel intervals in each continuous supplementary interval includes:

[0048] Determine any one continuous supplementary interval as the marked continuous supplementary interval, and sequentially determine any two adjacent S-channel intervals within the marked continuous supplementary interval as the marked left S-channel interval and the marked right S-channel interval;

[0049] Determine the total area of all connected domains of the corresponding S-channel intervals within all the holes corresponding to the marked right S-channel interval that belong to the marked left S-channel interval as the left filling area between the marked left S-channel interval and the marked right S-channel interval;

[0050] Determine the total area of all connected domains of the corresponding S-channel intervals within all the holes corresponding to the marked left S-channel interval that belong to the marked right S-channel interval as the right filling area between the marked left S-channel interval and the marked right S-channel interval;

[0051] Determine the cumulative value of the left filling areas between all adjacent S-channel intervals within the marked continuous supplementary interval as the overall left filling factor corresponding to the marked continuous supplementary interval;

[0052] Determine the cumulative value of the right filling areas between all adjacent S-channel intervals within the marked continuous supplementary interval as the overall right filling factor corresponding to the marked continuous supplementary interval;

[0053] If the overall left filling factor corresponding to the marked continuous supplementary interval is greater than or equal to its corresponding overall right filling factor, then determine the continuous separation interval adjacent to the marked continuous supplementary interval and on the left side of the marked continuous supplementary interval as the characteristic separation interval corresponding to the marked continuous supplementary interval;

[0054] If the overall left filling factor corresponding to the marked continuous supplementary interval is less than its corresponding overall right filling factor, then determine the continuous separation interval adjacent to the marked continuous supplementary interval and on the right side of the marked continuous supplementary interval as the characteristic separation interval corresponding to the marked continuous supplementary interval.

[0055] In a second aspect, the present invention provides a tongue image classification system for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes, and the system includes:

[0056] A tongue surface image acquisition module, configured to acquire a target tongue surface image corresponding to a lupus erythematosus patient to be detected;

[0057] A tongue image classification module, configured to perform tongue image classification on the target tongue surface image according to the target tongue surface image through a pre-trained tongue image classification network.

[0058] In a third aspect, a server is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the method in the above first aspect or any possible implementation manner of the first aspect.

[0059] In a fourth aspect, a computer program product is provided, which includes: computer program code. When the computer program code runs on a computer, the computer is enabled to execute the method in the above first aspect or any possible implementation manner of the first aspect.

[0060] In a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer program code. When the computer program code runs on a computer, the computer is enabled to execute the method in the above first aspect or any possible implementation manner of the first aspect.

[0061] The present invention has the following beneficial effects:

[0062] The method for classifying tongue images of patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes according to the present invention quantifies the obviousness of lupus erythematosus characteristics in the tongue surface images of lupus erythematosus through image analysis, and to a certain extent reduces the possibility that the tongue image classification network learns inaccurate lupus erythematosus characteristic information, solves the technical problem of poor accuracy in classifying tongue images of patients with lupus erythematosus, and thus improves the accuracy of classifying tongue images of patients with lupus erythematosus. When training the tongue image classification network, compared with directly constructing a training set from the tongue surface image set of lupus erythematosus and the tongue surface image set of normal people, the present invention comprehensively considers multiple indicators related to the obviousness of lupus erythematosus characteristics, such as the target missing index and the comprehensive supplement index, etc. Thereby quantifying the obvious lupus erythematosus characteristic factor corresponding to each tongue surface image of lupus erythematosus, and then screening out the tongue surface images of lupus erythematosus whose corresponding obvious lupus erythematosus characteristic factor is greater than the preset characteristic threshold, so that the tongue surface images of lupus erythematosus with a smaller obvious lupus erythematosus characteristic factor do not form a training set, and to a certain extent reduces the possibility that the tongue image classification network learns inaccurate lupus erythematosus characteristic information, thereby improving the accuracy of classifying tongue images of patients with lupus erythematosus. Description of the Drawings

[0063] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0064] Figure 1Flow chart of the tongue image classification method for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes of the present invention;

[0065] Figure 2 Schematic diagram of the acquisition process of the reference image set of the present invention;

[0066] Figure 3 Schematic diagram of the composition structure of the tongue image classification system for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes of the present invention;

[0067] Figure 4 Schematic diagram of the structure of a computer device of the present invention. Detailed implementation manners

[0068] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and their effects of the technical solutions proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0070] Systemic lupus erythematosus is a complex autoimmune disease in which the patient's immune system attacks its own tissues and organs, resulting in inflammation and damage. The harms of systemic lupus erythematosus mainly include damage to multiple organs, which may cause kidney inflammation, heart and lung problems, nervous system damage and anemia, etc. In addition, long-term inflammation and immune system disorders may increase the risk of infection, and in severe cases, it may affect the patient's quality of life and even endanger life. Improper treatment may lead to complications, and regular monitoring and management are required.

[0071] In traditional Chinese medicine, the symptoms of systemic lupus erythematosus mainly lie in the abnormal manifestations of the tongue and pulse. Combining the possible occurrences of fever, fatigue, joint pain and skin erythema, etc., the condition of systemic lupus erythematosus is comprehensively diagnosed. When diagnosing systemic lupus erythematosus through tongue images, the classification of tongue images of different traditional Chinese medicine syndromes has important clinical significance. Tongue images often reflect the patient's overall health status and visceral functions, and their accurate classification can help traditional Chinese medicine doctors better identify the causes, judge the development of the condition, and formulate personalized treatment plans. In addition, the changes in tongue images can be used as an important basis for disease monitoring and curative effect evaluation, which helps to improve the treatment effect and the patient's quality of life.

[0072] Reference Figure 1, showing the flow of some embodiments of the tongue image classification method for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes according to the present invention. The tongue image classification method for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes includes the following steps:

[0073] Step S1, obtaining a target tongue surface image corresponding to the patient with lupus erythematosus to be detected.

[0074] Among them, the patient with lupus erythematosus to be detected can be a patient to be divided according to the syndrome condition of lupus erythematosus. The division of the syndrome condition of lupus erythematosus can be achieved through tongue image classification. The syndrome condition of lupus erythematosus can be, but is not limited to: normal, rheumatism and heat arthralgia syndrome, yin deficiency and internal heat syndrome, phlegm-heat obstructing the lung syndrome, and liver depression and blood stasis syndrome. The target tongue surface image can be the tongue surface image of the patient with lupus erythematosus to be detected.

[0075] As an example, an RGB (Red Green Blue, color mode) image of the tongue surface of the patient with lupus erythematosus to be detected can be collected by a camera, and through threshold segmentation, the tongue surface area can be segmented from the RGB image, and the image block where the segmented tongue surface area is located at this time is used as the target tongue surface image.

[0076] It should be noted that the tongue surface area can also be recognized by a neural network. For example, it can be recognized by a convolutional neural network (CNN, Convolutional Neural Network) with a U-Net structure. The loss function used in its training process can be a cross-entropy loss function. The input of this neural network can be the captured tongue surface image, and the output can be the main tongue surface image of the tongue surface image.

[0077] Step S2, according to the target tongue surface image, through the pre-trained tongue image classification network, performing tongue image classification on the target tongue surface image.

[0078] Among them, the tongue image classification network can be ResNet (Residual Neural Network, residual neural network). The training set of the tongue image classification network can be the target training set. The target training set can include a normal tongue surface image set and a reference image set. The normal tongue surface images in the normal tongue surface image set can be the tongue surface images of people without lupus erythematosus. The reference images in the reference image set can be the tongue surface images with relatively obvious lupus erythematosus characteristics. The acquisition method of the normal tongue surface image set can be: by a camera, collecting RGB images of the tongue surfaces of multiple people without lupus erythematosus, and through threshold segmentation, segmenting the tongue surface areas from these RGB images, and the image blocks where these segmented tongue surface areas are located at this time constitute the normal tongue surface image set.

[0079] As an example, the target tongue surface image can be input into the tongue image classification network, and through the tongue image classification network, the tongue image classification of the target tongue surface image can be realized.

[0080] Reference Figure 2 , the method for obtaining the reference image set may include the following steps:

[0081] Step 201, obtain the discoid lupus erythematosus tongue surface image set and the S-channel value corresponding to each pixel point in each discoid lupus erythematosus tongue surface image in the set.

[0082] Among them, the discoid lupus erythematosus tongue surface images in the discoid lupus erythematosus tongue surface image set may be the tongue surface images of patients with discoid lupus erythematosus. The method for obtaining the S-channel value corresponding to a pixel point may be: converting the discoid lupus erythematosus tongue surface image to which the pixel point belongs into an HSV (Hue Saturation Value, color model) image to obtain the S (Saturation) channel value of the pixel point.

[0083] As an example, multiple RGB images of the tongue surfaces of patients with discoid lupus erythematosus may be collected by a camera, and the tongue surface regions may be segmented from these RGB images through threshold segmentation. The image blocks where the segmented tongue surface regions are located at this time constitute the discoid lupus erythematosus tongue surface image set.

[0084] Step 202, perform interval division on the S-channel values corresponding to all pixel points in each discoid lupus erythematosus tongue surface image to obtain the S-channel interval corresponding to each discoid lupus erythematosus tongue surface image.

[0085] As an example, this step may include the following steps:

[0086] The first step is to determine any one discoid lupus erythematosus tongue surface image as a marked image, and make an S-channel histogram corresponding to the marked image, denoted as the marked S-channel histogram.

[0087] For example, a histogram constructed with the S-channel value corresponding to the pixel points in the marked image as the abscissa and the number of pixel points with the same S-channel value in the marked image as the ordinate is the S-channel histogram corresponding to the marked image.

[0088] The second step is to form an S-channel interval with the S-channel values between the S-channel values corresponding to every two adjacent minimum values in the above-mentioned marked S-channel histogram, denoted as the S-channel interval in the marked image.

[0089] For example, if the S-channel values in the marked S-channel histogram are 0.1, 0.2, 0.3, 0.5, 0.6, 0.7, 0.8, 0.9, and 1 respectively, and the number of pixel points corresponding to these S-channel values is 10, 20, 100, 50, 60, 70, 40, 50, and 30 in sequence, then 3 S-channel intervals can be obtained, which are [0.1, 0.5), [0.5, 0.8), and [0.8, 1] respectively, and [0.1, 0.5) and [0.5, 0.8) are adjacent, and [0.5, 0.8) and [0.8, 1] are adjacent.

[0090] It should be noted that since the number of pixel points belonging to different S-channel values in the same image is often different, the amount of information contained in each S-channel value in the same image is often different. In the S-channel of the tongue surface image, the S-channel value with more pixel points occupies a larger area proportion in the tongue surface image and contains more information. And because the saturation of the pixel points on the tongue surface often shows transitional changes, there is often an aggregation phenomenon in the S-channel histogram of the tongue surface image, which is mainly manifested as multiple peaks in the histogram, and each peak and the area aggregated around it represent a main area in the tongue surface image. In order to completely segment several main areas in the main tongue surface image, it is often necessary to merge consecutive S-channel values. Therefore, the S-channel values between the S-channel values corresponding to every two adjacent minimum values can often be the interval where a peak in the S-channel histogram and the S-channel values around it are aggregated, and can represent a main area in the tongue surface image.

[0091] Step 203, perform connected component extraction on the region corresponding to each S-channel interval in its corresponding systemic lupus erythematosus tongue surface image, and screen out multi-connected regions from the extracted connected components.

[0092] As an example, any S-channel interval in the marked image is determined as a temporary S-channel interval, and all pixel points whose corresponding S-channel in the marked image belongs to the temporary S-channel interval constitute the target region corresponding to the temporary S-channel interval. Connected component extraction is performed on the target region corresponding to the temporary S-channel interval. At this time, all the extracted connected components are denoted as all the connected components corresponding to the temporary S-channel interval. Generally speaking, connected components are often divided into two categories, one is a single-connected region, and the other is a multi-connected region, and the multi-connected region is often a region with holes. The holes in the multi-connected region are also called cavities.

[0093] It should be noted that the color of the normal tongue surface often shows uniform transitional changes, while the tongue surface of patients with systemic lupus erythematosus often has erythema, and the erythema often causes the color of some positions in the main area to change, thus showing cavities of different colors in the image. Therefore, the cavities in the multi-connected region may be erythema.

[0094] Step 204: Analyze and process the missing conditions of all holes in each multiply connected region to obtain a target missing index corresponding to each multiply connected region.

[0095] As an example, this step may include the following steps:

[0096] In the first step, the area ratio of each hole in the multiply connected region to which it belongs is determined as the initial missing factor corresponding to each hole.

[0097] For example, the formula for determining the initial missing factor corresponding to the hole can be:

[0098] ;in, It is In the image of lupus erythematosus tongue In a multiply connected region The initial missing factor corresponding to the holes. It is the serial number of the lupus erythematosus tongue image. It is The serial numbers of the multiply connected regions in the lupus erythematosus tongue image. It is The sequence number of the holes in a multiply connected region. It is In the image of lupus erythematosus tongue In a multiply connected region, The number of pixels in a hole can represent the The area of ​​the hole. It is In the lupus erythematosus tongue image, The number of pixels in a multiply connected region, which can represent the The area of ​​a multiply connected region.

[0099] It should be noted that, in actual situations, erythema often appears as holes in the normal connected domain, and holes can be understood as missing parts of the normal connected domain. The larger the In the image of lupus erythematosus tongue In a multiply connected region The larger the area of ​​the first hole, the The greater the degree of missing multiply connected regions.

[0100] In the second step, according to the difference between the initial missing factor corresponding to each hole in each multiply connected region and the initial missing factors corresponding to other holes, determining the erythema missing contribution factor corresponding to each hole in each multiply connected region may include the following sub-steps:

[0101] In the first sub-step, any multiply-connected region is determined as the marked multiply-connected region, and any hole within the above-mentioned marked multiply-connected region is determined as the marked hole. Each hole within the above-mentioned marked multiply-connected region except the above-mentioned marked hole is determined as the reference hole.

[0102] In the second sub-step, the cumulative value of the absolute values of the differences between the initial missing factor corresponding to the above-mentioned marked hole and the initial missing factors corresponding to each reference hole is determined as the target missing difference corresponding to the above-mentioned marked hole.

[0103] In the third sub-step, according to the target missing difference corresponding to the above-mentioned marked hole, the erythema missing contribution factor corresponding to the above-mentioned marked hole is determined.

[0104] Among them, the target missing difference can have a positive correlation with the erythema missing contribution factor.

[0105] For example, the formula for determining the erythema missing contribution factor corresponding to a hole can be:

[0106] ; where is the erythema missing contribution factor corresponding to the th hole in the th multiply-connected region in the th lupus erythematosus tongue image. is the serial number of the lupus erythematosus tongue image. is the serial number of the multiply-connected region in the th lupus erythematosus tongue image. and are the serial numbers of different holes in the th multiply-connected region. is the normalization function. is the th lupus erythematosus tongue image. is the number of holes in the th multiply-connected region in the is the th lupus erythematosus tongue image. is the th hole in the th multiply-connected region in the th lupus erythematosus tongue image. is the th hole in the is the th lupus erythematosus tongue image. is the th hole in the

[0107] It should be noted that in actual situations, the number of large holes in a connected component is often smaller than that of small holes. For large holes, there are often fewer holes with similar areas in the multi-connected region to which they belong. Therefore, the difference between a large hole and other holes in the multi-connected region to which it belongs is relatively large. For small holes, there are often more holes with similar areas in the multi-connected region to which they belong. Therefore, the difference between a small hole and other holes in the multi-connected region to which it belongs is relatively small. Thus, when is larger, it often indicates that the th lupus erythematosus tongue surface image has a larger area difference between the th hole and other holes in the th multi-connected region, which often indicates that the th hole is more likely to be a hole with more missing parts.

[0108] Step 3: Determine the target missing index corresponding to each multi-connected region according to the erythema missing contribution factor and the initial missing factor corresponding to all holes in each multi-connected region.

[0109] Among them, both the erythema missing contribution factor and the initial missing factor can have a positive correlation with the target missing index.

[0110] For example, the formula for determining the target missing index corresponding to a multi-connected region can be:

[0111] ; where is the target missing index corresponding to the th multi-connected region in the th lupus erythematosus tongue surface image. is the serial number of the lupus erythematosus tongue surface image. is the serial number of the multi-connected region in the th lupus erythematosus tongue surface image. is the serial number of the hole in the th multi-connected region in the th lupus erythematosus tongue surface image. is the serial number of the hole in the th multi-connected region. is the erythema missing contribution factor corresponding to the th hole in the th multi-connected region in the th lupus erythematosus tongue surface image. is the initial missing factor corresponding to the th hole in the th multi-connected region in the th lupus erythematosus tongue surface image.

[0112] It should be noted that when The larger it is, it often indicates that in the th multiple-connected region of the th erythematosus tongue surface image, the area difference between the th multiple-connected region and other holes is larger, which often indicates that the th hole is more likely to be a hole with more missing parts. When is larger, it often indicates that in the th

[0113] erythematosus tongue surface image, the area of the

[0114] th

[0115] multiple-connected region is relatively larger, which often indicates that the degree of missing parts in the

[0116] th multiple-connected region is greater. Therefore, when

[0117] is larger, it often indicates that the overall degree of missing parts in the

[0118] th multiple-connected region of the

[0119] erythematosus tongue surface image is greater.

[0120] Step 205: Determine the comprehensive supplement index corresponding to each S-channel interval according to the target missing index corresponding to all multiple-connected regions corresponding to each S-channel interval and the distribution of the connected regions whose S-channel intervals are adjacent S-channel intervals within them.

[0114] As an example, this step may include the following steps:

[0115] First step: Screening out the supplementary candidate connected regions from each hole according to all the connected regions corresponding to the adjacent S-channel intervals of the S-channel interval corresponding to each hole may include the following sub-steps:

[0116] First sub-step: Determine any one hole as the candidate hole, and determine the S-channel interval corresponding to the multiple-connected region to which the above candidate hole belongs as the candidate S-channel interval.

[0117] Second sub-step: Determine each S-channel interval adjacent to the above candidate S-channel interval as the to-be-supplemented S-channel interval, and determine each connected region corresponding to each to-be-supplemented S-channel interval as the calibrated connected region.

[0118] Third sub-step: Determine each calibrated connected region within the above candidate hole as the supplementary candidate connected region.

[0119] It should be noted that the holes caused by the normal tongue surface transition process can often be filled by the connected regions corresponding to the adjacent S-channel intervals, while the holes caused by the erythema region are often difficult to be filled by the connected regions corresponding to the adjacent S-channel intervals due to color changes. Therefore, screening out the supplementary candidate connected regions from each hole can facilitate the subsequent analysis of the erythema degree of the tongue surface.

[0120] Step 2: According to the target missing index corresponding to each multiply connected region, determine the initial supplementation factor corresponding to each supplementary candidate connected region in each hole within each multiply connected region.

[0121] For example, the formula for determining the initial supplementation factor corresponding to the supplementary candidate connected region in the hole within the multiply connected region can be:

[0122] ; where is the initial supplementation factor corresponding to the th supplementary candidate connected region in the th hole within the th multiply connected region in the th lupus erythematosus tongue surface image. is the serial number of the lupus erythematosus tongue surface image. is the serial number of the multiply connected region in the th lupus erythematosus tongue surface image. is the serial number of the hole within the th multiply connected region. is the serial number of the supplementary candidate connected region in the th hole. is the normalization function. is the target missing index corresponding to the th multiply connected region in the th lupus erythematosus tongue surface image. is the target missing index corresponding to the target connected region; the method for obtaining the target connected region is: filling the th supplementary candidate connected region into the th hole within the th multiply connected region, so that the reduced area of the th hole is equal to the area corresponding to the th supplementary candidate connected region, and the increased area of the th multiply connected region is equal to the area corresponding to the th supplementary candidate connected region, thereby realizing the update of the th multiply connected region in the th lupus erythematosus tongue surface image, and denoting the updated multiply connected region as the target connected region.

[0123] It should be noted that and can respectively represent the missing situations corresponding to the multiply connected regions before and after filling the th supplementary candidate connected region. When is larger, it often indicates that the difference in the missing situations between the multiply connected regions before and after filling the th supplementary candidate connected region is larger, and it often indicates that the The greater the degree of supplementation of the th supplementary candidate connected component to the

[0124] th multiply-connected region.

[0125] Step 3: Determine the corresponding effective supplementation degree of each supplementary candidate connected component within each hole in each multiply-connected region according to the number of edge pixels of each hole and each supplementary candidate connected component within it.

[0126] ; where is the th effective supplementation degree of the th supplementary candidate connected component within the th hole in the th multiply-connected region in the th lupus erythematosus tongue image. is the serial number of the lupus erythematosus tongue image. is the serial number of the multiply-connected region in the th lupus erythematosus tongue image. is the serial number of the hole in the th multiply-connected region. is the serial number of the supplementary candidate connected component within the th hole. is the th lupus erythematosus tongue image. is the serial number of the multiply-connected region in the th lupus erythematosus tongue image. is the number of pixels in the intersection between the edge of the th hole and the edge of the th supplementary candidate connected component in the th multiply-connected region in the th lupus erythematosus tongue image. is the th lupus erythematosus tongue image. is the serial number of the multiply-connected region in the th lupus erythematosus tongue image. is the number of pixels on the edge of the

[0127] It should be noted that when is larger, it often indicates that the overlapping part between the edge of the th hole and the edge of the th supplementary candidate connected component is more, and it often indicates that the th hole and the The closer the supplementary candidate connected components are likely to fit tightly, it often indicates that the th supplementary candidate connected component is relatively more effective in supplementing the th multi-connected region.

[0128] Step 4: Determine the adjacent supplementation index corresponding to each hole in each multi-connected region according to the initial supplementation factor and the supplementation effectiveness corresponding to all supplementary candidate connected components in each hole within each multi-connected region.

[0129] Among them, both the initial supplementation factor and the supplementation effectiveness can be positively correlated with the adjacent supplementation index.

[0130] For example, the formula for determining the adjacent supplementation index corresponding to the hole in the multi-connected region can be:

[0131] ; where is the adjacent supplementation index corresponding to the th hole in the th multi-connected region in the th lupus erythematosus tongue surface image. is the serial number of the lupus erythematosus tongue surface image. is the serial number of the multi-connected region in the th lupus erythematosus tongue surface image. is the serial number of the hole in the th multi-connected region. is the serial number of the supplementary candidate connected component in the th hole. is the number of supplementary candidate connected components in the th hole. is the supplementation effectiveness corresponding to the th supplementary candidate connected component in the th multi-connected region in the th hole in the th lupus erythematosus tongue surface image. is the initial supplementation factor corresponding to the th supplementary candidate connected component in the th multi-connected region in the th hole in the th lupus erythematosus tongue surface image.

[0132] It should be noted that when is larger, it often indicates that the difference in the missing situation between the multi-connected regions before and after the filling of the th supplementary candidate connected component is larger, and it often indicates that the th supplementary candidate connected component has a relatively greater degree of supplementation for the th multi-connected region. When The larger it is, it often indicates that the more overlapping parts there are between the edge of the th hole and the edge of the th supplementary candidate connected region. It often indicates that the th hole and the th supplementary candidate connected region are more likely to fit closely. It often indicates that the th supplementary candidate connected region is relatively more effective in supplementing the th multi-connected region. Therefore, when the value is larger, it often indicates that the supplementary candidate connected regions within the th hole have a better supplementary effect on the th hole, and it often indicates that the supplementary candidate connected regions within the th hole have a better supplementary effect on the

[0133] Fifth step, determine the mean value of the adjacent supplementary indicators corresponding to all holes within all multi-connected regions corresponding to each S-channel interval as the comprehensive supplementary indicator corresponding to each S-channel interval.

[0134] It should be noted that the larger the comprehensive supplementary indicator corresponding to the S-channel interval is, the better the supplementary effect of the adjacent S-channel intervals on the corresponding multi-connected region is often indicated.

[0135] Step 206, perform adaptive supplementary division analysis based on the comprehensive supplementary indicators corresponding to all S-channel intervals in each lupus erythematosus tongue image, and obtain the lupus erythematosus feature obvious factor corresponding to each lupus erythematosus tongue image.

[0136] As an example, this step may include the following steps:

[0137] First step, according to the comprehensive supplementary indicators corresponding to all S-channel intervals in each lupus erythematosus tongue image, use the Fisher optimal solution method to cluster the S-channel intervals in each lupus erythematosus tongue image into two categories, and form continuous supplementary intervals for the continuous S-channel intervals in the category with higher comprehensive supplementary indicators, and form continuous separation intervals for the continuous S-channel intervals in the other category with lower comprehensive supplementary indicators.

[0138] Among them, there are at least two S-channel intervals in the continuous supplementary interval. There are at least two S-channel intervals in the continuous separation interval.

[0139] For example, if a category with higher comprehensive supplementary indicators includes: [0.1, 0.3), [0.3, 0.5), [0.5, 0.6) and [0.9, 1], then this category can construct a continuous supplementary interval, which can be [0.1, 0.6).

[0140] In the second step, according to the area filling situation between adjacent S-channel intervals within each continuous supplementary interval, screening out characteristic separation intervals from the continuous separation intervals adjacent to each continuous supplementary interval may include the following sub-steps:

[0141] In the first sub-step, determine any one continuous supplementary interval as the marked continuous supplementary interval, and successively determine any two adjacent S-channel intervals within the above-mentioned marked continuous supplementary interval as the marked left S-channel interval and the marked right S-channel interval.

[0142] In the second sub-step, determine the total area of all connected domains where the corresponding S-channel intervals within all the holes corresponding to the above-mentioned marked right S-channel interval belong to the above-mentioned marked left S-channel interval as the left filling area between the marked left S-channel interval and the marked right S-channel interval.

[0143] In the third sub-step, determine the total area of all connected domains where the corresponding S-channel intervals within all the holes corresponding to the above-mentioned marked left S-channel interval belong to the above-mentioned marked right S-channel interval as the right filling area between the marked left S-channel interval and the marked right S-channel interval.

[0144] In the fourth sub-step, determine the cumulative value of the left filling areas between all adjacent S-channel intervals within the above-mentioned marked continuous supplementary interval as the overall left filling factor corresponding to the above-mentioned marked continuous supplementary interval.

[0145] In the fifth sub-step, determine the cumulative value of the right filling areas between all adjacent S-channel intervals within the above-mentioned marked continuous supplementary interval as the overall right filling factor corresponding to the above-mentioned marked continuous supplementary interval.

[0146] In the sixth sub-step, if the overall left filling factor corresponding to the above-mentioned marked continuous supplementary interval is greater than or equal to its corresponding overall right filling factor, then determine the continuous separation interval adjacent to the above-mentioned marked continuous supplementary interval and on the left side of the above-mentioned marked continuous supplementary interval as the characteristic separation interval corresponding to the above-mentioned marked continuous supplementary interval.

[0147] In the seventh sub-step, if the overall left filling factor corresponding to the above-mentioned marked continuous supplementary interval is less than its corresponding overall right filling factor, then determine the continuous separation interval adjacent to the above-mentioned marked continuous supplementary interval and on the right side of the above-mentioned marked continuous supplementary interval as the characteristic separation interval corresponding to the above-mentioned marked continuous supplementary interval.

[0148] In the third step, according to all the continuous supplementary intervals and their corresponding characteristic separation intervals in each systemic lupus erythematosus tongue surface image, the formula for determining the systemic lupus erythematosus characteristic obvious factor corresponding to each systemic lupus erythematosus tongue surface image is:

[0149] ; where is the The lupus erythematosus feature obvious factor corresponding to a lupus erythematosus tongue surface image. Is the serial number of the lupus erythematosus tongue surface image. Is the normalization function. Is the Number of consecutive supplementary intervals in the Is the Serial number of the consecutive supplementary interval in the Is the Number of S-channel values included in the th consecutive supplementary interval in the Is the Mean of the target missing indicators corresponding to all multi-connected regions corresponding to all S-channel intervals in the characteristic separation interval corresponding to the th consecutive supplementary interval in the

[0150] It should be noted that in actual situations, the longer the length of the consecutive supplementary interval, the higher the degree of filling of the missing holes. If the degree of missing of the connected domain corresponding to its adjacent consecutive separation interval is still relatively high at this time, it often indicates the phenomenon of hole filling fault. There may be lupus erythematosus defects in the holes, and the lupus erythematosus features are relatively more obvious. When is larger, it often indicates that the length of the th consecutive supplementary interval is longer. When is larger, it often indicates that the degree of missing of the connected domain corresponding to the adjacent consecutive separation interval of the th consecutive supplementary interval is relatively larger. Therefore, when is larger, it often indicates that the lupus erythematosus features in the th lupus erythematosus tongue surface image are relatively more obvious, and it should be more involved in the training of the subsequent tongue image classification network.

[0151] Step 207, screen out the lupus erythematosus tongue surface images in the lupus erythematosus tongue surface image set whose corresponding lupus erythematosus feature obvious factors are greater than the preset feature threshold to form a reference image set.

[0152] Among them, the preset feature threshold can be a threshold set in advance for judging whether the lupus erythematosus features are obvious. For example, the preset feature threshold can be 0.3.

[0153] As an example, the lupus erythematosus tongue surface images whose corresponding lupus erythematosus feature obvious factors are greater than the preset feature threshold can be used as reference images, and all reference images form a reference image set.

[0154] Optionally, the training process of the tongue image classification network may include the following steps:

[0155] First, construct a ResNet as the tongue image classification network before training.

[0156] Second, manually annotate the lupus erythematosus syndrome condition for each image in the target training set.

[0157] Among them, the lupus erythematosus syndrome condition can be the manifestation of lupus erythematosus in patients. For example, the lupus erythematosus syndrome condition can be, but is not limited to: normal, rheumatic heat arthralgia syndrome, yin deficiency and internal heat syndrome, phlegm-heat stagnating in the lung syndrome, and liver depression and blood stasis syndrome.

[0158] Third, use the target training set as the training set and the lupus erythematosus syndrome condition annotated for each image in the target training set as the training label to train the constructed tongue image classification network, and obtain the trained tongue image classification network.

[0159] Among them, the loss function of the tongue image classification network uses the cross-entropy loss function.

[0160] Reference Figure 3 , based on the same inventive concept as the above method embodiment, the present invention provides a tongue image classification system for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes. The system includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the above computer program is executed by the processor, it implements the steps of the tongue image classification method for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes, which may specifically include:

[0161] A tongue surface image acquisition module 301, configured to acquire a target tongue surface image corresponding to a lupus erythematosus patient to be detected;

[0162] A tongue image classification module 302, configured to perform tongue image classification on the target tongue surface image through a pre-trained tongue image classification network according to the target tongue surface image.

[0163] Figure 4 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Exemplarily, as Figure 4 shown, the computer device 400 includes: a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402. Among them, when the processor 402 executes the computer program 403, the computer device can execute any one of the aforementioned tongue image classification methods for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes.

[0164] Based on the same inventive concept as the above method embodiments, the present invention provides a server, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes any one of the above-mentioned tongue image classification methods for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes.

[0165] Based on the same inventive concept as the above method embodiments, the present invention provides a computer program product, which includes: computer program code, when the computer program code runs on a computer, it causes the computer to execute any one of the above-mentioned tongue image classification methods for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes.

[0166] Based on the same inventive concept as the above method embodiments, the present invention provides a computer-readable storage medium, which stores computer program code, and when the computer program code runs on a computer, it causes the computer to execute any one of the above-mentioned tongue image classification methods for patients with systemic lupus erythematosus facing different traditional Chinese medicine syndromes.

[0167] In summary, when training the tongue image classification network, compared with directly forming a training set with the systemic lupus erythematosus tongue surface image set and the normal tongue surface image set, the present invention comprehensively considers multiple indicators related to the obviousness of systemic lupus erythematosus features, such as the target missing indicator and the comprehensive supplement indicator, etc. Thus, the obvious factor of systemic lupus erythematosus corresponding to each systemic lupus erythematosus tongue surface image is quantified, and then the systemic lupus erythematosus tongue surface images with the corresponding obvious factor of systemic lupus erythematosus greater than the preset feature threshold are screened out, so that the systemic lupus erythematosus tongue surface images with a smaller obvious factor of systemic lupus erythematosus do not form a training set, which reduces the possibility of the tongue image classification network learning inaccurate systemic lupus erythematosus feature information to a certain extent, thereby improving the accuracy of tongue image classification for patients with systemic lupus erythematosus.

[0168] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A tongue image classification method for patients with systemic lupus erythematosus with different TCM syndromes, characterized in that: The following steps are involved: Acquire a target tongue surface image corresponding to a lupus erythematosus patient to be detected; According to the target tongue surface image, the target tongue surface image is classified by a pre-trained tongue image classification network, wherein the training set of the tongue image classification network is the target training set, and the target training set includes a normal tongue surface image set and a reference image set; The method for obtaining the reference image set includes: Obtain the lupus erythematosus tongue surface image set and the S channel value corresponding to each pixel point in each lupus erythematosus tongue surface image in the lupus erythematosus tongue surface image; The S channel values ​​corresponding to all the pixels in each lupus erythematosus tongue surface image are divided into intervals to obtain the S channel interval corresponding to each lupus erythematosus tongue surface image; For each S channel interval, the connected domain is extracted from the region in the lupus erythematosus tongue surface image to which it belongs, and multi-connected regions are screened out from the extracted connected domains; Analyze and process the missing conditions of all holes in each multi-connected region to obtain the target missing index corresponding to each multi-connected region; According to the target missing index corresponding to all multi-connected regions corresponding to each S channel interval and the distribution of the connected domains of the corresponding S channel intervals as adjacent S channel intervals, the comprehensive supplementary index corresponding to each S channel interval is determined; Based on the comprehensive supplementary indexes corresponding to all S channel intervals in each lupus erythematosus tongue surface image, an adaptive supplementary division analysis is performed to obtain lupus erythematosus characteristic obvious factors corresponding to each lupus erythematosus tongue surface image, including: according to the comprehensive supplementary indexes corresponding to all S channel intervals in each lupus erythematosus tongue surface image, the S channel intervals in each lupus erythematosus tongue surface image are clustered into continuous supplementary intervals and continuous separation intervals; characteristic separation intervals are screened out from the continuous separation intervals adjacent to each continuous supplementary interval; according to all the continuous supplementary intervals in each lupus erythematosus tongue surface image and their corresponding characteristic separation intervals, the lupus erythematosus characteristic obvious factors corresponding to each lupus erythematosus tongue surface image are determined; The lupus erythematosus tongue surface images whose corresponding lupus erythematosus characteristic obvious factors are greater than the preset characteristic threshold are screened out from the lupus erythematosus tongue surface image set to form a reference image set.

2. A method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 1, characterized in that: The S channel values ​​corresponding to all the pixels in each lupus erythematosus tongue surface image are divided into intervals to obtain the S channel interval corresponding to each lupus erythematosus tongue surface image, including: Determine any lupus erythematosus tongue image as a marked image, and make an S channel histogram corresponding to the marked image, recorded as a marked S channel histogram; The S channel values ​​between the S channel values ​​corresponding to every two adjacent minimum values ​​in the marked S channel histogram constitute an S channel interval.

3. According to claim 1, a tongue image classification method for systemic lupus erythematosus patients with different TCM syndromes is characterized by: The missing condition analysis of all holes in each multi-connected region is performed to obtain the target missing index corresponding to each multi-connected region, including: The area ratio of each hole in the multiply connected region to which it belongs is determined as the initial missing factor corresponding to each hole; Determine the erythema missing contribution factor corresponding to each hole in each multi-connected region according to the difference between the initial missing factor corresponding to each hole in each multi-connected region and the initial missing factors corresponding to other holes; According to the erythema missing contribution factor and the initial missing factor corresponding to all the holes in each multi-connected region, the target missing index corresponding to each multi-connected region is determined, wherein the erythema missing contribution factor and the initial missing factor are positively correlated with the target missing index.

4. A method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 3, characterized in that: Determining the erythema missing contribution factor corresponding to each hole in each multiply connected region according to the difference between the initial missing factor corresponding to each hole in each multiply connected region and the initial missing factors corresponding to other holes includes: Determine any multiply connected region as a marked multiply connected region, determine any hole in the marked multiply connected region as a marked hole, and determine each hole in the marked multiply connected region except the marked hole as a reference hole; The accumulated value of the absolute value of the difference between the initial missing factor corresponding to the marked hole and the initial missing factor corresponding to each reference hole is determined as the target missing difference corresponding to the marked hole; According to the target loss difference corresponding to the marked hole, the erythema loss contribution factor corresponding to the marked hole is determined, wherein the target loss difference is positively correlated with the erythema loss contribution factor.

5. The method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 1, characterized in that: The comprehensive supplementary index corresponding to each S channel interval is determined according to the target missing index corresponding to all multi-connected regions corresponding to each S channel interval and the distribution of the connected domains of the corresponding S channel intervals as adjacent S channel intervals, including: According to all connected domains corresponding to the adjacent S channel intervals of the S channel interval corresponding to each hole, a supplementary candidate connected domain is screened out from each hole; According to the target missing index corresponding to each multi-connected region, an initial supplement factor corresponding to each supplementary candidate connected domain in each hole in each multi-connected region is determined; Determine the supplementation effectiveness corresponding to each supplementary candidate connected domain in each hole in each multi-connected region according to the number of edge pixels of each hole and each supplementary candidate connected domain in each multi-connected region; According to the initial supplementation factors and supplementation effectiveness corresponding to all supplementary candidate connected domains in each hole in each multi-connected region, the adjacent supplementation index corresponding to each hole in each multi-connected region is determined, wherein the initial supplementation factor and the supplementation effectiveness are both positively correlated with the adjacent supplementation index; The average of the adjacent supplementary indices corresponding to all holes in all multi-connected regions corresponding to each S channel interval is determined as the comprehensive supplementary index corresponding to each S channel interval.

6. A method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 5, characterized in that: The method of selecting a supplementary candidate connected domain from each hole according to all connected domains corresponding to the adjacent S channel intervals of the S channel interval corresponding to each hole includes: Determine any hole as a candidate hole, and determine the S channel interval corresponding to the multiply connected region to which the candidate hole belongs as a candidate S channel interval; Determine each S channel interval adjacent to the candidate S channel interval as an S channel interval to be supplemented, and determine each connected domain corresponding to each S channel interval to be supplemented as a calibrated connected domain; Each calibrated connected domain within the candidate hole is determined as a supplementary candidate connected domain.

7. The method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 5, characterized in that: The formula for the initial supplementary factor corresponding to the supplementary candidate connected domains in the holes in the multi-connected regions is: ;in, It is In the image of lupus erythematosus tongue In a multiply connected region In the hole, The initial supplementary factor corresponding to the supplementary candidate connected domains; It is the serial number of the lupus erythematosus tongue surface image; It is The serial number of the multi-connected regions in the lupus erythematosus tongue image; It is The sequence number of the holes in the multiply connected regions; It is The serial number of the candidate connected domain to be added in each hole; is the normalization function; It is In the image of lupus erythematosus tongue The target missing index corresponding to the multi-connected regions; is the target missing index corresponding to the target connected domain; the method for obtaining the target connected domain is: The supplementary candidate connected regions are filled into the In a multiply connected region holes, so that the The area reduced by the first hole is equal to the The area corresponding to the supplementary candidate connected domains is The area of ​​the multiply connected region is equal to the The area corresponding to the supplementary candidate connected domain is realized. In the image of lupus erythematosus tongue The updated multi-connected region is recorded as the target connected domain.

8. The method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 5, characterized in that: The formula for the supplementary effectiveness of the supplementary candidate connected domains in the holes in the multi-connected regions is: ;in, It is In the image of lupus erythematosus tongue In a multiply connected region In the hole, The supplementary effectiveness corresponding to the supplementary candidate connected domains; It is the serial number of the lupus erythematosus tongue surface image; It is The serial number of the multi-connected regions in the lupus erythematosus tongue image; It is The sequence number of the holes in the multiply connected regions; It is The serial number of the candidate connected domain to be added in each hole; is the normalization function; It is In the image of lupus erythematosus tongue In a multiply connected region The edge of the hole and the The number of pixels in the intersection between the edges of the supplementary candidate connected domains; It is In the image of lupus erythematosus tongue In a multiply connected region The number of pixels on the edge of a hole; It is In the image of lupus erythematosus tongue In a multiply connected region In the hole, The number of pixels on the edge of the supplementary candidate connected domain.

9. The method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 1, characterized in that: The method of clustering the S channel intervals in each lupus erythematosus tongue surface image into continuous supplementary intervals and continuous separation intervals according to the comprehensive supplementary indexes corresponding to all S channel intervals in each lupus erythematosus tongue surface image comprises: According to the comprehensive supplementary index corresponding to all S channel intervals in each lupus erythematosus tongue surface image, the S channel intervals in each lupus erythematosus tongue surface image are clustered into two categories, and the continuous S channel intervals in the category with a higher comprehensive supplementary index constitute a continuous supplementary interval, and the continuous S channel intervals in the other category with a lower comprehensive supplementary index constitute a continuous separation interval; The step of selecting the characteristic separation interval from the continuous separation intervals adjacent to each continuous supplementary interval is specifically as follows: selecting the characteristic separation interval from the continuous separation intervals adjacent to each continuous supplementary interval according to the area filling condition between the adjacent S channel intervals in each continuous supplementary interval; According to all the continuous supplement intervals in each lupus erythematosus tongue surface image and the corresponding characteristic separation intervals, the lupus erythematosus characteristic obvious factor corresponding to each lupus erythematosus tongue surface image is determined, and the corresponding formula is: ;in, It is Lupus erythematosus characteristic obvious factors corresponding to lupus erythematosus tongue images; It is the serial number of the lupus erythematosus tongue surface image; is the normalization function; It is The number of consecutive supplementation intervals in the lupus erythematosus tongue surface images; It is The serial number of the continuous supplement interval in the lupus erythematosus tongue image; It is In the image of lupus erythematosus tongue The number of S channel values ​​included in a continuous supplementary interval; It is In the image of lupus erythematosus tongue The mean value of the target missing index corresponding to all multi-connected regions corresponding to all S channel intervals in the feature separation interval corresponding to the continuous supplementary interval.

10. A method for classifying tongue images of patients with systemic lupus erythematosus with different TCM syndromes according to claim 9, characterized in that: The method of selecting a characteristic separation interval from the continuous separation intervals adjacent to each continuous supplementary interval according to the area filling condition between the adjacent S channel intervals in each continuous supplementary interval includes: Determine any one of the continuous supplement intervals as a marked continuous supplement interval, and sequentially determine any two adjacent S channel intervals in the marked continuous supplement interval as a marked left S channel interval and a marked right S channel interval; The total area of ​​all connected domains of the corresponding S channel intervals contained in all holes corresponding to the marked right S channel interval and belonging to the marked left S channel interval is determined as the left filling area between the marked left S channel interval and the marked right S channel interval; The total area of ​​all connected domains of the corresponding S channel intervals contained in all holes corresponding to the marked left S channel interval and belonging to the marked right S channel interval is determined as the right filling area between the marked left S channel interval and the marked right S channel interval; Determine the accumulated value of the left padding areas between all adjacent S channel intervals in the marked continuous supplement interval as the overall left padding factor corresponding to the marked continuous supplement interval; Determine the accumulated value of the right filling area between all adjacent S channel intervals in the marked continuous supplement interval as the overall right filling factor corresponding to the marked continuous supplement interval; If the overall left padding factor corresponding to the continuous mark supplement interval is greater than or equal to the overall right padding factor corresponding to it, then the continuous separation interval adjacent to the continuous mark supplement interval and on the left side of the continuous mark supplement interval is determined as the characteristic separation interval corresponding to the continuous mark supplement interval; If the overall left padding factor corresponding to the marked continuous supplement interval is smaller than its corresponding overall right padding factor, the continuous separation interval adjacent to the marked continuous supplement interval and on the right side of the marked continuous supplement interval will be determined as the characteristic separation interval corresponding to the marked continuous supplement interval.

Images