A method for constructing a coal maceral group automatic segmentation model

Abstract

The application discloses a method for constructing a coal rock maceral group automatic segmentation model, which comprises the following steps: in a full-supervised learning stage, labeled images are input into a convolutional neural network in batches, and a teacher model is obtained by optimizing the coal rock maceral group automatic segmentation model with the aim of minimizing cross-entropy loss at a segmentation head and contrast loss at a representation head; in a pseudo-label learning stage, the teacher model is used as an initial student model; the teacher model performs pseudo-label inference on unlabeled images to obtain pseudo-labeled images; reliable pixels and unreliable pixels are distinguished according to the probability distribution entropy of pixels; the reliable pixels are added to the student model, and the student model is optimized with the aim of minimizing cross-entropy loss at the segmentation head and contrast loss at the representation head; and the unreliable pixels participate in a contrast learning process to optimize the student model. The model constructed by the method can realize high-quality automatic segmentation of coal rock micro images only by using a small amount of labeled coal rock micro images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coal rock identification, and particularly relates to a method for constructing a coal rock maceral group automatic segmentation model. BACKGROUND

[0002] Coal is an organic sedimentary rock, and the use of methods for studying rocks to study coal has generated a branch of coal science, coal petrology. Coal petrology analysis refers to a method for qualitatively describing and quantitatively measuring the rock composition, structure, properties and coalification degree of coal by mainly using an optical microscope and other means such as naked eye, and is an important means for studying coal petrology. The most common analysis items are the determination of coal maceral groups and mineral matter, the determination of vitrinite reflectance, the determination of microscopic coal petrology types and macroscopic description. The three items have international standards or international standard drafts, and China has also formulated corresponding national standards (GB6948). Since coal rocks formed under different time and geological conditions have great randomness and complexity in terms of gray value, structure, size, anisotropy, prominence and shape, there is still room for improvement for the existing methods for such a complex task as the segmentation of coal rock maceral groups.

[0003] The traditional determination of coal rock maceral groups is performed manually, mainly according to the differences in reflectivity, morphological structure and other indicators of maceral groups. Subsequently, a segmentation method based on traditional image processing is widely used, which not only saves time and effort, but also makes the analysis results more objective. With the development of machine learning, multilayer perception, nearest neighbor and other methods have been applied to the classification of coal rock maceral groups, and good results have been achieved. In recent years, semantic segmentation models based on deep learning have gradually become one of the mainstream methods for the segmentation of coal rock maceral groups. Such models mainly use deep convolutional neural networks to extract features of coal rock microscopic images, and then realize the automatic segmentation of coal rock maceral groups. The method based on deep learning can automatically learn more abstract and high-level feature expressions from training samples, and has good segmentation effect.

[0004] The existing coal maceral group identification method has the following problems: the manual method not only requires the tester to have strong professionalism, but also is time-consuming, laborious, subjective and non-reproducible. The analysis using the traditional image processing method is mostly based on the single feature of the image pixel gray value, and cannot accurately identify all the coal maceral groups. Due to the complex structure and various forms of coal maceral groups, the coal maceral groups show diversity and gradual change, which is reflected in the low contrast and weak edge of the coal maceral image. The method of manually designing features has poor generalization performance and cannot realize end-to-end identification, which cannot meet the application requirements in industrial production. The full-supervised deep learning method needs a large number of labeled samples for training, but the pixel-level labeling of the coal maceral image needs a lot of time and effort of experts, and the training time cost is high. SUMMARY

[0005] In order to solve the above problems, the present application provides a method for constructing an automatic segmentation model of coal maceral groups, which can complete high-quality segmentation of coal maceral images according to a small amount of labeled coal maceral images in the segmentation task of coal maceral groups.

[0006] In order to achieve the above purpose, the present application provides a method for constructing an automatic segmentation model of coal maceral groups, which includes a convolutional neural network for extracting feature maps from input images, a segmentation head and a representation head for processing the feature maps. The construction process includes the following steps:

[0007] S1, full-supervised learning stage: input the labeled image into the convolutional neural network in batches, and optimize the automatic segmentation model of coal maceral groups to minimize the cross-entropy loss at the segmentation head and the contrast loss at the representation head, thereby obtaining a teacher model;

[0008] S2, pseudo-label learning stage: the teacher model is used as the initial student model; during the training process using unlabeled images in batches, the teacher model first performs pseudo-label inference on the unlabeled images to obtain pseudo-labeled images; the probability distribution entropy of each pixel in the pseudo-labeled image is calculated to distinguish reliable pixels from unreliable pixels; the reliable pixels are added to the student model for training, and the student model is optimized to minimize the cross-entropy loss at the segmentation head and the contrast loss at the representation head; the unreliable pixels participate in the contrast learning process, and the student model is optimized;

[0009] The labeled image and the unlabeled image are both coal maceral images; in the labeled image, each pixel is labeled with a corresponding pixel class.

[0010] A further improvement of the present application is that in step S1, the contrast loss of pixel representation i is The expression of H(p ij ) is:

[0011]

[0012] Wherein: P i and N i are the pixel representation set of positive and negative samples of pixel i respectively, i + and i - are the positive and negative sample pixel representations selected in P i and N i respectively, '·' represents inner product, τ>0 is a temperature coefficient; the positive and negative samples include difficult positive and negative samples selected from the pixel representation memory bank and ordinary positive and negative samples randomly selected from the current batch feature map.

[0013] Further improvement of the present application is that: in the pixel representation memory bank, each pixel category c has a corresponding pixel representation memory queue, and the pixel representation memory bank further comprises a regional mean representation memory bank; the regional mean representation memory bank stores the pixel representation mean of each pixel representation corresponding to each pixel category c in the feature map corresponding to the current batch of labeled images;

[0014] After each batch of labeled images is input into the convolutional neural network, the regional mean representation memory bank is updated according to the output feature map, and a predetermined number of pixels are randomly selected, and the corresponding pixel representation is filled into the tail of the corresponding pixel representation memory queue according to the category of the pixel to be labeled;

[0015] In the process of selecting difficult positive and negative samples, the inner product of the pixel representation of pixel i and each representation in the pixel representation memory bank is calculated, and a plurality of representations with the largest inner product and a plurality of representations with the smallest inner product are selected as difficult positive and negative samples.

[0016] Further improvement of the present application is that: in the process of randomly selecting ordinary positive and negative samples from the current batch feature map, a predetermined number of pixel representations are randomly selected from the feature map output from the current batch, the inner product between each selected pixel representation and the pixel representation of pixel i is calculated, and the selected pixel representation with the inner product greater than 0 is selected as a simple positive sample, and the selected pixel representation with the inner product less than 0 is selected as a simple negative sample.

[0017] Further improvement of the present application is that: in step S2, the expression of the probability distribution entropy H(p ij ) of each pixel in the pseudo-label image is:

[0018]

[0019] Wherein: p ijThe segmentation head representing the teacher model generates a softmax probability value at pixel j in the i-th unlabeled image, C is the number of classes of coal maceral groups, p ij (c) is p ij The softmax probability value on the c-th class;

[0020] The expression for distinguishing reliable pixels from unreliable pixels is

[0021]

[0022] Where: alpha is a threshold, and the probability distribution entropy H(p ij ) is less than the threshold, then it is a reliable pixel.

[0023] A further improvement of the present application is that in step S2, the contrast learning process specifically includes:

[0024] For each pixel class c, select the corresponding pixel representation as the anchor pixel set A c ;

[0025] Calculate the mean of the pixel representation in the anchor pixel set A c of each pixel class c as the positive sample of the class

[0026] Select negative samples from the current batch of feature maps and update the negative sample library;

[0027] Calculate the loss function of contrast learning Its expression is:

[0028]

[0029] Where C is the number of classes of coal maceral groups, M is the total number of anchor pixels, and N is the number of negative samples extracted; z ci represents the i-th anchor of the c-th class, each anchor pixel has 1 positive sample and N negative samples for calculation, respectively denoted as and <·,·> is the cosine similarity between the features of two different pixels, ranging from -1 to 1; the temperature parameter tau = 0.5.

[0030] A further improvement of the present application is that in step S2, when updating the student model, the teacher model accepts parameter updates from the student model in the form of exponential moving average

[0031] The scheme provided by the present application has the following technical effects:

[0032] 1. Coal is the main body of energy consumption in our country, with the gradual promotion of carbon neutral strategy, optimizing the structure of coal supply, realizing low carbon and efficient use of coal has become the key to achieve the goal of carbon neutral. The technical solution solves the problems of high efficiency, high precision, high technical experience requirement of personnel, high time cost and other problems in the process of artificial analysis and identification of traditional coal rock microscopic image, realizes the automatic classification of coal rock microscopic component group. At the same time, the technical solution has the significance of improving the level of coal analysis process, can provide strong support for the automation transformation of coking, steel smelting, coal chemical industry and other industries, further promote the upgrading and transformation of China's coal industry, promote the development of coal rock microscopic component group analysis process industry, and contribute to the development of steel smelting, coal chemical industry.

[0033] 2. Coal rock microscopic component group analysis is an important link of further research and evaluation of coal quality. In production, this technology is often used to determine the grade of coal, thereby directly affecting the production cost and coal combustion efficiency. The use of the technical solution can improve the number of coal samples processed by manual analysis of existing coal rock microscopic component group from 2 per day (calculated at 8 hours) to 25, while the analysis accuracy is improved from 80% to 93%. According to the calculation of one coal sample per 50 tons, 1150 tons of coal can be processed per day. According to the calculation of 6000 yuan per month per person, 100 coal samples can be processed manually in a month, and the project can be completed in 4 days. The efficiency is improved by 1250%, and the cost is saved by 900,000 yuan per year. 1150*30*12 = 41.4 million tons of coal can be processed more per year. According to the effect analysis after using the invention patent, the economic benefit will be greatly improved.

[0034] 3. The technical solution can effectively improve the segmentation accuracy of the existing deep learning semantic segmentation model for coal rock microscopic component group, and can fully utilize a large number of unlabeled coal rock microscopic images for training under the condition of less labeled images, and reduce the training time. The invention patent can automatically segment the coal rock microscopic component group under the condition of small sample, only a small amount of labeled images are needed for training, and the accuracy rate similar to that of full supervision method requiring a large number of labeled images for training can be achieved.

[0035] The concept, specific structure and technical effects of the present application will be further described in conjunction with the drawings to fully understand the purpose, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is the flow chart of the construction method of the coal rock microscopic component group automatic segmentation model of the present application;

[0037] Figure 2is a schematic diagram of a method for constructing a coal maceral group automatic segmentation model. DETAILED DESCRIPTION

[0038] Other advantages and benefits of the present application will become apparent to those skilled in the art upon consideration of the disclosure or can be learned by practice of the application. The disclosure can be realized and attained by means not only the embodiments described herein but also in other different embodiments, and the particulars shown herein are by way of example and for purposes of illustrative. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed.

[0039] It should be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the drawings, rather than being drawn according to the number, shape and size of the components in actual implementation. The actual implementation of each component may be a random change, and the component layout pattern may be more complex.

[0040] Some exemplary embodiments of the present application are described for illustrative purposes. It should be understood that the present application can be implemented in other ways not specifically shown in the drawings.

[0041] As shown in Figure 1 , Figure 2 The present embodiment provides a method for constructing a coal maceral group automatic segmentation model. In the present embodiment, the data samples used are collected by a cooperative unit and include 60 coal samples with different metamorphic degrees, and the maximum reflectance of vitrinite is between 0.61% and 1.70%. The sample images are obtained by a Zeiss optical microscope Axioskop 40 with a magnification of 500 times, and the resolution is 2048x1536. All the sample images are in RGB format collected under a yellow filter. A total of 79 sample images are collected. Since the collected pictures are too large to train the network, the original picture resolution size is cropped to 512x512 in this paper. In order to avoid the occurrence of overfitting phenomenon, each picture will be randomly cropped, scaled, randomly flipped and brightness adjusted, wherein the scaling ratio is 0.8-1.2. If the scaled size is less than the fixed input size, the mirror image is completed, and if it is larger than the fixed input, it is randomly cropped, and the label is also subjected to the same operation. The probability of random flipping is 50%, and the brightness adjustment range is 0.8-1.2. In addition, the data enhancement method in this paper is only used in training, and data enhancement is not used in testing the model.

[0042] In this embodiment, 79 coal maceral images are used for experiments, each model is trained for 200 epochs, the batch size of training is 4, the Adam optimizer is used, and the initial learning rate is 10 -3To verify the performance of the model constructed by the method more accurately and objectively, a five-fold cross-validation method is used for evaluation.

[0043] As shown in Figure 1 , 2 After obtaining the training images, the first process is the full supervision learning process, which learns the semantic representation of each microscopic component group. In this process, the labeled images are fed into any convolutional neural network that can extract features, and the full convolutional neural network is used in this embodiment. The extracted feature maps will enter the segmentation head and the representation head respectively to obtain the corresponding results (the class probability of each pixel and the semantic segmentation map of the coal and rock microscopic component group), and the loss function is calculated. Since only using the cross-entropy loss function will cause the neural network to only consider a single pixel during training, ignoring the relationship between pixels, and unable to narrow the gap between each class in the coal and rock microscopic component group, therefore, in this embodiment, for each labeled image, the training target is to minimize the cross-entropy loss in formula 1 and the contrast loss at the representation head in formula 2 The two losses correspond to the and in the figure respectively.

[0044]

[0045] In the formula, y(c) represents the annotated true label, p(c) represents the prediction result, and C is the number of training samples.

[0046]

[0047] In the formula, P i and N i are the pixel representation sets of the positive and negative samples of pixel i, respectively, i + and i - are the selected positive and negative sample pixel representations in P i and N i respectively, '·' represents the inner product, and τ>0 is the temperature coefficient.

[0048] Using the NCE Loss of formula (2) requires positive and negative samples for contrast learning, so this embodiment proposes a pixel representation memory bank. Specifically, in the pixel representation memory bank, each pixel class c has a corresponding pixel representation memory queue, and the pixel representation memory bank also includes a regional mean representation memory bank. The regional mean representation memory bank stores the pixel representation mean of each pixel class c corresponding to the pixel representation in the feature map corresponding to the current batch of labeled images.

[0049] The total size of all pixel representation memory queues is CxNxD, where C is the total number of coal and rock microcomponent groups, N is a self-designated value (the length of the queue), and D is the dimension of the pixel representation (the size of the channel of the output feature map). The vector (c, n, d) represents the dth dimension of the nth pixel representation in the queue with a specific category c. The value range of c is 1 to C, which represents the specific category identifier; the value range of n is 1 to N, which represents the specific pixel representation; and the value range of d is 1 to D, which represents the specific dimension of the designated pixel representation.

[0050] Each batch of labeled images is input into the convolutional neural network, and the region mean representation memory bank is updated according to the output feature map. A predetermined number of pixels are randomly selected, and the corresponding pixel representation is filled into the tail of the corresponding pixel representation memory queue according to the category labeled by the pixel.

[0051] Considering that the randomly sampled pixel representations in the pixel representation memory bank are too sparse to represent the content of the entire image, a region mean representation memory bank is constructed to store the mean of the pixel representations belonging to the same category calculated in the current training batch of images. The size of the region mean representation memory bank is CxD, where C is the total number of coal and rock microcomponent groups, and D is the dimension of the pixel representation. The value of the region mean representation memory bank is updated in each batch, and the key point is that it calculates the mean of all pixel representations of the same category in this batch of images, which is representative. The method of establishing pixel representation and region mean representation memory bank not only better represents all pixels in the data set, but also reduces the training time and memory bank storage space.

[0052] For coal and rock, the characteristics within each microcomponent group are not only greatly different, but also easily confused with other coal and rock microcomponent groups. As the training progresses, more and more samples are too simple for the model to train, and a large number of easily confused microcomponent groups are mixed in simple samples, which cannot provide a large enough gradient to the model. Training with difficult samples has a significant effect on improving the performance of the model, because compared with simple samples, difficult samples provide a larger gradient, so the selection of key difficult training samples has a very important influence on contrastive representation learning. Therefore, this embodiment hopes that difficult samples are more involved in loss calculation when calculating the contrastive loss of the representation, but if the number of difficult samples is too large, it will cause the model to fall into a local optimum. In order to solve this problem, this embodiment uses the strategy of randomly sampling difficult samples and ordinary samples.

[0053] The positive and negative samples include difficult positive and negative samples selected from the pixel representation memory bank and ordinary positive and negative samples randomly selected from the current batch feature map. In the process of selecting the difficult positive and negative samples, the inner product of the pixel representation of pixel i and each representation in the pixel representation memory bank (including the pixel representations in the queue and each pixel representation mean in the region mean representation memory bank) is calculated, and the 10% of the representations with the largest and smallest inner products are selected as the difficult positive and negative samples.

[0054] In the process of randomly selecting the ordinary positive and negative samples from the current batch feature map, a predetermined number of pixel representations are randomly selected from the feature map corresponding to the input image of the current batch. In this embodiment, the number of selected pixel representations is equal to the total number of difficult positive and negative samples. After the selection is completed, the inner product between each selected pixel representation and the pixel representation of pixel i is calculated, and the selected pixel representations with the inner product greater than 0 are regarded as simple positive samples, and the selected pixel representations with the inner product less than 0 are regarded as simple negative samples.

[0055] The strategy of randomly sampling difficult samples and ordinary samples not only ensures that the model can learn ordinary samples in the entire database, but also avoids the model falling into local optimization by only learning the most difficult samples. At the same time, since the memory bank is constantly updated, each pixel representation has the opportunity to learn by comparison with the difficult samples in the entire data set. The model performance is improved, and the feature space is more structured. The neural network obtained by minimizing the two loss functions in the training process can predict higher quality pseudo-label images in the high-quality pseudo-label learning module, thereby enabling the finally constructed model to have better performance.

[0056] The teacher model can be obtained after the fully supervised learning stage. The pseudo-label learning stage uses the teacher model as the initial student model. The pseudo-label learning stage can effectively utilize the unlabeled data as a supplement to the labeled data to improve the model performance. In the training process, the conventional sample screening method is to leave only the high-confidence (reliable) prediction results, but this will exclude a large amount of low-confidence unlabeled data. For example, in the coal maceral segmentation task, directly ignoring the unreliable pseudo-label pixels will result in the loss of a large amount of available information, leading to insufficient model training. Therefore, in the high-quality pseudo-label learning stage of this embodiment, all unlabeled data are used, and reliable and unreliable pseudo-labels predicted are added to the training of the model to fully exploit their value.

[0057] The unlabeled image is first input into the teacher model for prediction to obtain the pseudo-label. In order to avoid adding unreliable pseudo-labels directly to the training, the probability distribution entropy of each pixel in the pseudo-label is used to distinguish reliable and unreliable pseudo-labels in this embodiment. The formula for calculating the entropy is shown in formula 3, and the method for judging reliable and unreliable labels is shown in formula 4.

[0058]

[0059] wherein: p ij represents the softmax probability value generated by the segmentation head of the teacher model at pixel j in the ith unlabeled image, C is the number of classes of coal maceral groups, p ij (c) is p ij the softmax probability value on the cth class.

[0060] The expression for distinguishing reliable pixels from unreliable pixels is

[0061]

[0062] wherein: α is a threshold value, and the probability distribution entropy H(p ij ) is less than the threshold value, then it is a reliable pixel.

[0063] For reliable pixels, the embodiment directly adds them to the training of the student model. The student model is optimized through the cross-entropy loss function (Formula 1). Unreliable pixels participate in the contrast learning process to optimize the student model.

[0064] The contrast learning process in the pseudo-label learning stage includes: for each pixel class c, selecting the corresponding pixel representation as the anchor pixel set A c ; calculating the mean of the pixel representation in the anchor pixel set A c of each pixel class c as the positive sample of the class. Selecting negative samples from the current batch of feature maps and updating the negative sample library; calculating the loss function of contrast learning

[0065] Anchor pixel selection. In the training process, a series of anchor pixels are selected in each batch. These anchors need to meet certain conditions, and the selection criteria are consistent for labeled samples and unlabeled samples, that is, the predicted probability of the sample on the true value label class or the pseudo-label class is greater than a threshold value, wherein the feature set of the cth anchor pixel of the labeled sample is represented as The feature set of the cth anchor pixel of the unlabeled sample is represented as Then for class c, all anchor pixel sets that can meet the conditions are A c , as shown in Formula 5.

[0066]

[0067] In the formula, y ij is the label value of the jth pixel of the ith labeled image, is the pseudo-label value of the jth pixel of the ith pseudo-labeled image, and δ is a threshold value, which is set to 0.3.

[0068] (2) Positive sample selection. For the pixel set A ij c , the embodiment calculates their mean value and takes the result as the feature center of the category. For all anchor pixels in the same category, they take the feature center as the positive sample, denoted as as shown in formula 6.

[0069]

[0070] (3) Negative sample selection.

[0071] First, the embodiment defines a binary (only 0 or 1) variable n ij (c) to determine whether the jth pixel of the ith image can be a qualified negative sample of category c, as shown in formula 7.

[0072]

[0073] In the formula, a and b respectively represent the indicators of whether the jth pixel of the ith image with and without labels can be a negative sample of category c, and the specific determination of the values is introduced below.

[0074] For the labeled image, the negative sample of category c needs to meet: a) not belonging to category c; b) its real category label is difficult to distinguish from category c. Therefore, the embodiment introduces the pixel-level category order O ij , which sorts the softmax probability value p ij of each pixel in the image according to the difficulty of distinguishing the category from its real category label, that is, O ij (c) = argsort(p ij ). It can be seen that for the case of the largest output probability value, it is most difficult to distinguish from category c, so its order is the first, which is defined as 0 in the order of the embodiment, that is, O ij (arg max p ij ) = 0. Similarly, for the case of the smallest output probability value, it is most easily distinguished from category c, and it is placed at the end, that is, O ij (arg min p ij ) = C-1, C is the total number of categories. Therefore, for the labeled image, the calculation formula of the indicator a is shown in formula 8.

[0075]

[0076] where x l ​is a threshold value, set to 2. The two terms on the right side of equation 8 correspond to the two conditions that a negative sample of class c needs to satisfy.

[0077] For unlabeled images, since the pseudo labels may be incorrect, the embodiment does not completely determine the correctness of the predicted label, and for a negative sample of class c, the following conditions need to be met: a) does not belong to class c; b) the predicted probability value is within a certain range; c) belongs to unreliable pseudo label pixels. The index The calculation formula is shown in equation 9.

[0078]

[0079] where the threshold value x h is set to 5. All negative samples that can be added to training satisfy n ij (c) = 1, as shown in equation 10.

[0080] N c = {z ij | n ij (c) = 1} (10)

[0081] In summary, for unreliable pixels, the loss function is shown in equation 11.

[0082]

[0083] where C is the number of classes of coal maceral groups, M is the total number of anchor pixels, and N is the number of negative samples extracted. z ci represents the i-th anchor of class c, and each anchor pixel has 1 positive sample and N negative samples for calculation, denoted as and <·,·> is the cosine similarity between the features of two different pixels, ranging from -1 to 1. The temperature parameter τ = 0.5.

[0084] Due to the long-tail problem in the dataset, using only one batch of samples as negative samples for contrastive learning is problematic, so the embodiment uses a memory bank to maintain a class-related negative sample bank, which is updated in a queue (first-in, first-out) manner. Note that the memory bank in the high-quality pseudo label learning stage is different from the memory bank in the full supervision module, and the memory bank in the high-quality pseudo label learning stage is only a normal memory bank.

[0085] In contrastive learning, the embodiment uses the screening method of anchor pixels, positive sample pixels and negative sample pixels to add unreliable pixels of pseudo labels to training, fully utilizing all pixel information to improve the performance of the model.

[0086] In the whole model, the optimization objectives of the labeled coal rock microscopic images are to minimize the cross-entropy loss functions at the segmentation head and the representation head, respectively (Formula 1) and the contrast loss function (Formula 2). As for the unlabeled coal rock microscopic images, the pseudo labels are obtained by putting them into the teacher model, and then the reliable pixels in the pseudo labels are screened out by the method of screening labels described above. The reliable pixels are added to the training of the student model by Formula 1, and the loss is recorded as The unreliable pseudo labels are added to the contrast learning, and the loss is recorded as The optimization objective function of the whole model is shown in Formula 12.

[0087]

[0088] In the formula, and respectively represent the losses applied to the segmentation head and the representation head of the labeled images, and respectively represent the losses applied to the reliable pseudo labels and the unreliable pseudo labels of the unlabeled images. λ r and λ c are the weights of the reliable pseudo label loss and the unreliable pseudo label loss. After the pseudo label learning stage is completed, the final student model is the coal rock microscopic component group automatic segmentation model constructed by the method.

[0089] In the experiment, the method is compared with the most advanced semi-supervised method under the condition that the proportion of labeled images in the whole dataset is 1 / 2, 1 / 4 and 1 / 8, as shown in Tables 1, 2 and 3, respectively. It can be found that the method of the embodiment achieves the best performance on different proportions of labeled image datasets. The three semantic segmentation indicators used to evaluate the performance of the model are pixel accuracy (Pixel Accuracy, PA), mean pixel accuracy (Mean Pixel Accuracy, MPA) and mean intersection over union (Mean Intersection over Union, MIoU). Union

[0090] Table 1 Comparison with advanced semi-supervised segmentation methods under the condition of using 1 / 2 proportion of labeled data for training

[0091]

[0092] Table 2 Comparison with advanced semi-supervised segmentation methods under the condition of using 1 / 4 proportion of labeled data for training

[0093]

[0094] ​Table 3 Comparison of the present embodiment with advanced semi-supervised segmentation methods under the condition of training with 1 / 8 of the labeled data

[0095]

[0096] Table 4 Comparison of the present embodiment with the most advanced fully supervised segmentation method under the condition of using 1 / 8, 1 / 4, 1 / 2 and all of the labeled images. The present embodiment is 11.2% higher than the fully supervised method in terms of pixel accuracy when the number of labeled images is less than half of the total number of images in the coal maceral segmentation task.

[0097] Table 4 Comparison of the present embodiment with the most advanced fully supervised segmentation method under the condition of using 1 / 8, 1 / 4, 1 / 2 and all of the labeled images. The present embodiment is 11.2% higher than the fully supervised method in terms of pixel accuracy when the number of labeled images is less than half of the total number of images in the coal maceral segmentation task.

[0098]

[0099] The above embodiments are only illustrative of the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A method for constructing a coal maceral group automatic segmentation model, the coal maceral group automatic segmentation model comprising a convolutional neural network for extracting a feature map from an input image, a segmentation head and a representation head for processing the feature map, the construction process comprising the following steps: S1, a full-supervised learning stage: inputting labeled images in batches into the convolutional neural network to optimize the coal maceral group automatic segmentation model, so as to obtain a teacher model, with the aim of minimizing the cross-entropy loss at the segmentation head and the contrastive loss at the representation head; S2, a pseudo-label learning stage: taking the teacher model as an initial student model; during the training process using unlabeled images in batches, the teacher model first performs pseudo-label inference on the unlabeled images to obtain pseudo-labeled images; the probability distribution entropy of each pixel in the pseudo-labeled images is calculated to distinguish reliable pixels from unreliable pixels; the reliable pixels are added to the student model for training, and the student model is optimized with the aim of minimizing the cross-entropy loss at the segmentation head and the contrastive loss at the representation head; the unreliable pixels participate in the contrastive learning process, and the student model is optimized; the labeled images and the unlabeled images are both coal maceral images; In step S1, the contrast loss of the pixel representation i is expressed as: wherein: P i and N i are the pixel representation sets of positive and negative samples of pixel i, respectively, i + and i - are the positive and negative sample pixel representations selected from P i and N i respectively, '·' denotes inner product, τ>0 is a temperature coefficient; the positive and negative samples include difficult positive and negative samples selected from the pixel representation memory bank and ordinary positive and negative samples randomly selected from the current batch feature map; in the pixel representation memory bank, each pixel class c has a corresponding pixel representation memory queue, and the pixel representation memory bank further comprises a regional mean representation memory bank; the regional mean representation memory bank stores the pixel representation mean of each pixel class c corresponding to each pixel representation in the feature map corresponding to the current batch of labeled images; after each batch of labeled images is input into the convolutional neural network, the regional mean representation memory bank is updated according to the output feature map, and a predetermined number of pixels are randomly selected, and the corresponding pixel representations are filled into the tail of the corresponding pixel representation memory queue according to the class of the pixel; in the process of selecting difficult positive samples and difficult negative samples, the inner product of the pixel representation of pixel i and each representation in the pixel representation memory bank is calculated, and a number of representations with the maximum and minimum inner products are selected as difficult positive samples and difficult negative samples; in the process of randomly selecting ordinary positive samples and ordinary negative samples from the current batch of feature maps, a predetermined number of pixel representations are randomly selected from the feature map output in the current batch, the inner product between each selected pixel representation and the pixel representation of pixel i is calculated, and the selected pixel representations with an inner product greater than 0 are selected as simple positive samples, and the selected pixel representations with an inner product less than 0 are selected as simple negative samples.

2. The method according to claim 1, wherein the method is characterized by: In step S2, the expression of the entropy H(p) of the probability distribution of each pixel in the pseudo-label image is as follows: ij ) the expression of the entropy H(p) of the probability distribution of each pixel in the pseudo-label image is as follows: wherein: p ij denotes the softmax probability value generated by the segmentation head of the teacher model at pixel j in the ith unlabeled image, C is the number of classes of coal maceral groups, p ij (c) is the softmax probability value on the cth class; and ij the softmax probability value on the cth class. The expression for distinguishing reliable pixels from unreliable pixels is Where: α is the threshold, and H(p) is the probability distribution entropy. ij A pixel smaller than this threshold is considered a reliable pixel.

3. The method according to claim 2, wherein the method is characterized by: In step S2, the contrastive learning process specifically comprises: For each pixel class c, select the corresponding pixel representation as the anchor pixel set A c ; Compute the anchor pixel set A for each pixel class c c Mean of the medium pixel representation as positive sample for this class selecting negative samples from the feature map in the current batch, and updating the negative sample bank; Loss function for contrastive learning The expression is: where C is the number of categories of coal maceral groups, M is the total number of anchor pixels, N is the number of negative samples extracted; z ci represents the i-th anchor point of the c category, each anchor pixel has 1 positive sample and N negative samples for calculation, respectively denoted as and <·,·> is the cosine similarity between the features of two different pixels, which ranges from -1 to 1; the temperature parameter τ = 0.

5.

4. The method according to claim 1, wherein the method is characterized by: In step S2, when updating and optimizing the student model, the teacher model accepts parameter updates from the student model in the form of exponential moving average.

Images