Automatic determination method and device for coal ash fusibility based on deep learning image segmentation

Through a semantic segmentation model based on deep learning, combined with semantic segmentation and instance segmentation, the adhesion problem in coal ash melting determination is solved, and the accuracy and reliability of automatic determination of coal ash melting determination is improved, reducing the computational complexity and cost .

CN118710913BActive Publication Date: 2024-11-29XIANGJIANG LAB
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411191114.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-29
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

The existing coal ash meltability measurement technology has complex manual observation and judgment methods and large errors. Traditional image segmentation algorithms cannot handle the adhesion problem, resulting in inaccurate recognition of melt feature temperature.

Method used

Using a semantic segmentation model based on deep learning, combining semantic segmentation and instance segmentation, through multi-scale feature extraction and attention mechanism, the coal ash melt image segmentation model can be trained, which can directly learn the edge features of the adhesion area and improve the model's ability to extract edges of the adhesion area. , realize accurate segmentation of gray cone area.

Benefits of technology

The accuracy and reliability of automatic determination of coal ash melting properties is achieved, which reduces the computational complexity and cost, avoids complex debonding strategies, and improves the identification accuracy of melting characteristic temperatures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118710913B_ABST
    Figure CN118710913B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for automatically measuring coal ash fusibility based on deep learning image segmentation. The method comprises the following steps: constructing a semantic segmentation model based on deep learning, including a feature encoding module and a feature decoding module, wherein the feature decoding module includes a semantic segmentation branch and an instance segmentation branch; using an ash cone melting image data set to train the semantic segmentation model according to a target loss function to obtain a coal ash melting image segmentation model, wherein the target loss function is calculated using the semantic segmentation loss function and the instance segmentation loss function; obtaining an ash cone melting image of the coal ash to be tested during the heating process, inputting the image into the coal ash melting image segmentation model, performing image segmentation to obtain a binary image of the melting area and extracting geometric features of the image; and identifying the moment when the ash cone is at the melting characteristic temperature during the melting process according to the geometric features. The present invention has the advantages of simple operation, low cost, high detection accuracy and reliability, and strong robustness.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of coal ash fusibility measurement, and in particular to a method and device for automatically measuring coal ash fusibility based on deep learning image segmentation. Background Art

[0002] The ash melting characteristics refer to the temperature characteristics of the ash when it deforms, softens and melts during the heating process. It is an important indicator of coal quality, especially an important indicator of power coal, and can reflect the changing state of minerals in coal in the boiler. The properties of ash during boiler operation directly affect the safety and stability of its operation.

[0003] At present, the melting point method is usually used to measure the melting characteristics of coal ash, that is, the coal ash is first made into a triangular pyramid or cylinder of a certain size to form an ash cone, and then heated to above 1500°C in a weakly reducing or oxidizing gas medium at a specified heating rate. The morphological changes of the ash cone during the heating process are manually observed, and the melting characteristic temperature of the ash cone is recorded. However, the manual observation and judgment method is not only complicated and time-consuming, but also has large identification errors.

[0004] In order to realize the automatic determination of the characteristic temperature of coal ash fusibility, some practitioners have proposed to obtain images of the ash cone during the melting process through a CCD (Charge-Coupled Device) camera, and then identify the characteristic melting temperature based on the geometric shape changes of the ash cone in the image. However, in the process of determining the melting characteristics of coal ash by the melting point method, the ash cone needs to be placed on a carrier, and the melting process is carried out at high temperature, which causes the bottom of the ash cone to merge with the carrier, resulting in interference from high-temperature molten bodies such as the cone carrier, resulting in a more serious adhesion problem in the cone melting area in the image. Traditional image segmentation algorithms usually rely on image grayscale values ​​to achieve image segmentation, such as image segmentation algorithms based on grayscale thresholds and grayscale edge detection. However, such algorithms cannot handle random adhesion problems and cannot directly and robustly segment the cone area accurately, resulting in the fact that the directly identified characteristic melting temperature is not accurate and reliable. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a method and device for automatically determining the fusibility of coal ash based on deep learning image segmentation, which has simple operation, low cost, high detection accuracy and reliability, and strong robustness, and can automatically and accurately realize the determination of the fusibility of coal ash.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0007] A method for automatically determining the fusibility of coal ash based on deep learning image segmentation, comprising the following steps:

[0008] Constructing a semantic segmentation model based on deep learning, wherein the semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module, wherein the feature decoding module includes a semantic segmentation branch and an instance segmentation branch, wherein the semantic segmentation branch is used to predict the category information of each pixel of the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances;

[0009] Using an ash cone melting image dataset to train the deep learning-based semantic segmentation model according to a target loss function, a coal ash melting image segmentation model is obtained, wherein the ash cone melting image dataset is an ash cone melting image of an ash cone during a heating process when different coal ashes are measured using a melting point method, and the target loss function is calculated using a semantic segmentation loss function and an instance segmentation loss function;

[0010] A ash cone melting image of the ash cone during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested is obtained as the ash cone melting image to be tested, and the obtained ash cone melting image to be tested is input into the coal ash melting image segmentation model. The coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area and extracts the geometric features of the image; and the moment when the ash cone is at the melting characteristic temperature during the melting process is identified according to the extracted geometric features.

[0011] Furthermore, the characteristic melting temperature includes a deformation temperature DT, a softening temperature ST, a hemispherical temperature HT and a flow temperature FT. The deformation temperature DT is the temperature at which the tip or edge of the ash cone begins to become rounded or bent, the softening temperature ST is the temperature when the ash cone is deformed to the tip of the cone or becomes spherical, the hemispherical temperature HT is the temperature when the ash cone is deformed to an approximately hemispherical shape, and the flow temperature FT is the thin layer temperature when the ash cone melts and expands to a height of less than 1.5 mm.

[0012] Furthermore, the geometric features include height, bottom width and area, the height is the vertical distance from the top of the gray cone to the bottom of the gray cone, the bottom width is the horizontal distance from the bottom of the gray cone, and the area is the number of pixels removed from the gray cone target. When the moment of the gray cone melting process at the melting characteristic temperature is identified according to the extracted geometric features, the following judgments are made in sequence: if it is judged that the height in the extracted geometric features is , bottom width is And the area is , it is identified as being at the deformation temperature DT; if it is determined that the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the softening temperature ST; if it is determined that the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the hemispherical temperature HT; if it is determined that the height of the extracted geometric features is , the bottom width is , then it is identified as being at the flow temperature FT, where , , are the height, width and area of ​​the gray cone before deformation.

[0013] Furthermore, the deep learning-based semantic segmentation model adopts deeplabv (Deep Label Semantic Image Segmentation) 3+ as the basic network, the feature encoding module includes a lightweight network and a multi-scale feature extraction sub-module, the multi-scale feature extraction sub-module adopts hole convolution based on different gap sizes, Conv 1x1 and avgpool (average pooling) to realize deep separable convolution, and the semantic segmentation branch adopts CBAM (Convolutional Block Attention Module) attention mechanism to make the model focus on foreground features.

[0014] Furthermore, the semantic segmentation loss function is calculated using the area attention factor, the shape complexity attention factor, and the difficult sample labeling factor, and the calculation expression is:

[0015]

[0016] in, represents the semantic segmentation loss function, It is calculated to take the value of N of the specified proportion, N is the total number of pixels in the image, Indicates i The area attention factor of the instance corresponding to the pixels is used to characterize the area size of the instance. Indicates i The shape complexity factor of the instance corresponding to the pixels is used to characterize the shape complexity of the instance. represents the difficult sample labeling factor, Indicates i The difference between the pixel prediction value and the true label, the difficult sample labeling factor is used to label the sample that satisfies The samples are the ones that need special attention. Indicates the threshold for distinguishing difficult and easy pixels.

[0017] Furthermore, the area concern factor, shape complexity concern factor and difficult sample labeling factor are calculated as follows:

[0018]

[0019]

[0020]

[0021]

[0022] in, Indicates the area of ​​the object to which the current pixel belongs. represents the shape factor threshold, Indicates the perimeter of the object described by the current pixel, s Indicates the area of ​​the object to which the current pixel belongs.

[0023] Furthermore, the instance detection loss function is calculated using the instance region center loss function and the instance region pixel offset loss function, and the calculation expression is:

[0024]

[0025] in, represents the instance detection loss function, is the instance region center loss function used to optimize the consistency between the instance region predicted by the model and the actual instance region. The instance region pixel offset loss function is used to optimize the pixel-level alignment between the instance region predicted by the model and the true instance region. , They are , The weight value of .

[0026] Furthermore, after the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, it also includes post-processing the binary image to obtain a final binary connected domain, and the post-processing includes sequentially performing corrosion operations, filtering out objects smaller than a specified size, dilating, and retaining the largest area binary connected domain.

[0027] A device for automatically measuring coal ash fusibility based on deep learning image segmentation, comprising:

[0028] A model building module, used to build a semantic segmentation model based on deep learning, wherein the semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module, wherein the feature decoding module includes a semantic segmentation branch and an instance segmentation branch, wherein the semantic segmentation branch is used to predict the category information of each pixel of the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances;

[0029] A model training module is used to train the semantic segmentation model based on deep learning according to the target loss function using the ash cone melting image dataset to obtain a coal ash melting image segmentation model, wherein the ash cone melting image dataset is an ash cone melting image of the ash cone during the heating process when the melting point method is used to measure the melting properties of different coal ashes, and the target loss function is calculated using the semantic segmentation loss function and the instance segmentation loss function. When the melting point method is used to measure the melting properties of the coal ash to be tested, the coal ash to be tested is made into a triangular pyramid to form an ash cone, and is heated at a specified heating rate in a weakly reducing or oxidizing gas medium;

[0030] The melting measurement module is used to obtain the ash cone melting image of the ash cone during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested, as the ash cone melting image to be tested, and input the obtained ash cone melting image to be tested into the trained coal ash melting image segmentation model, and the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, and extracts the geometric features of the binary image; the melting feature temperature is identified according to the extracted geometric features until the coal ash melting feature identification is completed.

[0031] A computer device comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above method.

[0032] Compared with the prior art, the advantages of the present invention are: the present invention trains the coal ash melting image segmentation model through a semantic segmentation model based on deep learning, and adopts a panoramic segmentation training framework, so that the model can not only directly learn the edge features of the adhesion area between the bottom of the ash cone and the surface of the carrier, but also pay more attention to instance distinction, improve the model's ability to extract the edge of the adhesion area, so that the ash cone area can be accurately and robustly segmented, and the edge of the ash cone melting image can be accurately and robustly extracted, thereby improving the accuracy and reliability of the automatic determination of coal ash fusibility, and can also effectively reduce the computational complexity and implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the principle of ash cone melting characteristics.

[0034] Figure 2 It is a schematic diagram of the effect of gray cone melting image.

[0035] Figure 3 It is a schematic diagram of the implementation process of the method for automatically determining the fusibility of coal ash based on deep learning image segmentation in this embodiment.

[0036] Figure 4 It is a schematic diagram of the structural principle of the feature extraction module in the semantic segmentation training framework based on panoptic segmentation adopted in this embodiment.

[0037] Figure 5 It is a schematic diagram of the structural principle of the feature decoding module in the semantic segmentation training framework based on panoramic segmentation adopted in this embodiment.

[0038] Figure 6 It is a schematic diagram of the process for realizing the identification of melting characteristic temperature.

[0039] Figure 7 It is a schematic diagram of the coal ash melting image segmentation effect obtained in a specific application embodiment of the present invention.

[0040] Figure 8 Schematic diagram of the process of characteristic temperature identification in this embodiment. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0042] The core content of ash cone fusibility determination is to determine the time when the four melting characteristic temperatures occur during the ash cone melting process and record the relevant characteristic temperatures as an important indicator for coal quality analysis. Figure 1 As shown, the four characteristic melting temperatures of the ash cone in the melting process specifically include deformation temperature DT, softening temperature ST, hemisphere temperature HT and flow temperature FT, wherein the deformation temperature DT is the temperature at which the tip or edge of the ash cone begins to become rounded or bent, the softening temperature ST is the temperature when the ash cone is deformed to the tip or the ash cone becomes spherical, the hemisphere temperature HT is the temperature when the ash cone is deformed to an approximate hemispherical shape, that is, the height is approximately equal to half the length of the base, and the flow temperature FT is the thin layer temperature when the ash cone melts and expands to a height of less than 1.5 mm.

[0043] The gray cone melting image corresponding to each melting characteristic temperature has its own geometric characteristics. The specific geometric characteristics can be taken as the height H, bottom width D and area S of the gray cone melting image. The height H is the vertical distance from the gray cone vertex to the gray cone bottom, the bottom width D is the horizontal distance of the gray cone bottom, and the area S is the number of gray cone target pixels. The geometric characteristic relationship between the target characteristic temperature and the gray cone melting image is shown in Table 1.

[0044] Table 1 Relationship between target geometric characteristics and characteristic temperature

[0045]

[0046] It can be seen from Table 1 that each characteristic melting temperature has specific geometric features, so the time when each characteristic melting temperature occurs can be identified based on the geometric features in the image.

[0047] However, the gray cone melting image is composed of the gray cone area and the carrier area, such as Figure 2As shown in the figure, since the melting process is carried out at high temperature, the melting area of ​​the ash cone and the carrier area will merge together, so that there is a large adhesion between the bottom of the ash cone and the surface of the carrier, and finally there is no edge boundary between the two. At the beginning of melting, there are strong straight line features in the adhesion edge area of ​​the two, but as the melting process deepens, the edge geometric features of the adhesion area have certain random changes. If the traditional segmentation algorithm is used for segmentation, it is necessary to adopt a complex de-adhesion strategy before segmentation, and the accuracy and robustness of the ash cone edge detection cannot be guaranteed, which leads to inaccurate subsequent geometric feature extraction, and finally leads to low accuracy and poor robustness of the automatic feature temperature recognition equipment.

[0048] Taking into account the powerful feature representation ability of the deep network, the present invention trains the coal ash melting image segmentation model through a semantic segmentation model based on deep learning, and adopts a panoramic segmentation training framework, combining semantic segmentation and instance segmentation to respectively predict the category information of each pixel of the image and distinguish different instances of each category. During the model training process, a loss function composed of a semantic segmentation loss function and an instance segmentation loss function is used to guide the training, that is, the loss function of the instance detection branch will participate in the parameter update of the feature decoding module, which can play a strong supervisory role in the training of the semantic segmentation model, forcing the semantic segmentation model to better learn the image features, so that the model can not only directly learn the edge features of the adhesion area between the bottom of the ash cone and the surface of the vehicle, but also pay more attention to instance distinction, improve the model's ability to extract the edge of the adhesion area, so that the ash cone area can be accurately and robustly segmented, and the edge of the ash cone melting image can be accurately and robustly extracted, and the accuracy and reliability of the automatic determination of coal ash fusibility can be improved, and there is no need to rely on complex de-adhesion strategies, avoid complex calculation processes, and effectively reduce the calculation complexity and implementation cost, so that it can be flexibly applied to ordinary CPU computing power platforms.

[0049] like Figure 3 As shown, the steps of the method for automatically determining the fusibility of coal ash based on deep learning image segmentation in this embodiment include:

[0050] Step S01. Construct a semantic segmentation model based on deep learning. The semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module. The feature decoding module includes a semantic segmentation branch and an instance segmentation branch. The semantic segmentation branch is used to predict the category information of each pixel in the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances.

[0051] In order to solve the problem that the adhesion area between the bottom of the gray cone and the surface of the vehicle is large and the edge features of the adhesion area have a certain randomness, the deep network is designed in this embodiment. The first consideration is that the network needs to be able to better obtain the overall structural features of the gray cone melting image area. Image segmentation based on deep learning includes three types of segmentation: semantic segmentation, instance segmentation and panoramic segmentation. Semantic segmentation is to assign each pixel in the image to a predefined category, which does not care about the differences between different individuals in the same category; instance segmentation is a more detailed segmentation based on semantic segmentation, by detecting the target in the input image and assigning a category label to each pixel of the target; panoramic segmentation is a further upgrade of instance segmentation. In fact, a semantic label and a unique instance identifier are assigned to each pixel in the image. The goal of panoramic segmentation is to provide semantic understanding of objects and backgrounds in the image and the positioning and segmentation of object instances at the same time. That is, panoramic segmentation is the most accurate and complete for image segmentation. It can achieve the most accurate segmentation for adhesion images. The use of panoramic image segmentation can better solve the problem of adhesion image segmentation. However, panoramic segmentation belongs to a multi-task image recognition problem. The structure of the model itself and related post-processing are relatively complex, and it is difficult to achieve a certain real-time performance on a limited algorithm platform.

[0052] Taking into account the limitations of implementation cost and limited computing power in actual engineering applications, this embodiment sets a semantic segmentation branch and an instance segmentation branch in the model to predict the category information of each pixel and distinguish the instance respectively. Combining the semantic segmentation branch and the instance segmentation branch in the subsequent model training process, a semantic segmentation training framework based on panoramic segmentation can be implemented. By adopting the panoramic segmentation architecture to train a supervised semantic segmentation model, the instance segmentation branch can achieve strong supervision of the semantic segmentation branch, forcing the model to further pay attention to adhesion edge learning, which can improve the semantic segmentation model's ability to segment adhesion images and balance accuracy and real-time performance.

[0053] The semantic segmentation model based on deep learning in this embodiment includes a feature encoding module and a feature decoding module, respectively. Figure 4 , 5 As shown, Figure 4 Corresponding to the feature encoding module, Figure 5 Corresponding to the feature decoding module, Figure 4 The A, B, and C nodes correspond to Figure 5The A, B, and C nodes in the model are connected, and rate represents the rate. The model can specifically use a lightweight semantic segmentation module such as deeplabv3+ as the basic network. The feature encoding module includes a lightweight network and a multi-scale feature extraction submodule. The multi-lightweight network can use mobilenet2, etc. The lightweight network is used as the backbone main network. The multi-scale feature extraction submodule uses a hole convolution based on different gap sizes, Conv 1x1 and avgpool to form a structure similar to a deep separable convolution, to achieve semantic multi-scale feature extraction and fast extraction and fusion of instance multi-scale features, so that the target pixel and the surrounding pixels in the image can establish the optimal semantic information connection, which is conducive to enhancing the semantic information between pixels and improving the model's feature extraction and generalization capabilities. The feature decoding module mainly includes two branches: semantic decoding and instance decoding. The semantic segmentation branch (semantic decoder) is used to predict the category information of each pixel in the image. Only the semantic segmentation branch is used during reasoning to ensure the real-time and accuracy of the model running on the ordinary algorithm platform. The instance segmentation branch (instance decoder) is used to assign instance labels to each pixel to distinguish different instances.

[0054] In order to enable the model to learn the adhesion area features more effectively, this embodiment further combines the attention mechanism and adopts the CBAM attention mechanism in the semantic segmentation branch. By combining the channel attention mechanism with the spatial attention mechanism, the model can focus on the foreground features, thereby effectively learning the edge features of the adhesion area between the bottom of the gray cone and the vehicle surface, improving the model's ability to detect the edge features of the adhesion area, and further improving the segmentation accuracy and reliability. Further combined with the use of a 5x5 convolution module, the semantic connection between the target pixel and the surrounding pixels can be expanded. By using a hole convolution in the encoding part and a 5×5 large convolution in the decoding part, the image pixels can be associated with a larger surrounding area.

[0055] Step S02. Use the ash cone melting image dataset to train a deep learning-based semantic segmentation model according to the target loss function to obtain a coal ash melting image segmentation model. The ash cone melting image dataset is an ash cone melting image of the ash cone during the heating process when the melting point method is used to measure the fusibility of different coal ashes. The target loss function is calculated using the semantic segmentation loss function and the instance segmentation loss function. The semantic segmentation loss function is the loss function of the corresponding semantic segmentation branch, and the instance segmentation loss function is the loss function of the corresponding instance segmentation branch.

[0056] When the melting point method is used to determine the fusibility of the coal ash to be tested, the coal ash to be tested is made into a triangular pyramid to form an ash cone, and is heated at a specified heating rate in a weakly reducing or oxidizing gas medium.

[0057] like Figure 4 , 5As shown, in order to improve the ability of the semantic segmentation model to segment adhesion images, the present embodiment adopts a training framework based on panoramic segmentation to train the semantic segmentation model, and the instance segmentation branch is used to implement strong supervision of the semantic segmentation branch, so that the model focuses on the learning of adhesion edge areas to improve the model's representation ability for adhesion images. The feature decoding module includes a semantic segmentation branch and an instance segmentation branch. The instance segmentation branch is used as a supervision branch to supervise the training of the semantic segmentation module from another perspective (instance extraction). The feature decoding module adopts a shared parameter strategy. The target loss function used in the training process uses the semantic segmentation loss function and the instance segmentation loss function, so that the loss function of the instance detection branch will also participate in the feature decoding module parameter update. Compared with the traditional semantic segmentation training method, the model can pay more attention to instance distinction, thereby effectively improving the model's ability to extract the edges of the adhesion area.

[0058] In order to construct a semantic segmentation training framework for the adhesion problem based on foreground segmentation, this embodiment constructs the target loss function L according to the following formula:

[0059] (1)

[0060] in, represents the segmentation loss function, Instance segmentation loss function.

[0061] That is, the target loss function L consists of two parts: semantic segmentation loss function And instance segmentation loss function , the training process is jointly supervised by comprehensive semantic segmentation loss and instance segmentation loss, which effectively improves the model's ability to extract the edges of adhesion areas.

[0062] Since the edge features of the adhesion region are weak, it is difficult for difficult samples with small areas and complex shapes to accurately learn the edge features of the adhesion region. When constructing the semantic segmentation loss function, this embodiment uses the area attention factor, shape complexity attention factor and difficult sample labeling factor to calculate, so that it can pay attention to the area, shape complexity and difficult samples at the same time, so that the model pays more attention to the learning of difficult samples with small areas and complex shapes, thereby effectively improving the model's regional segmentation ability for the foreground.

[0063] In a specific application embodiment, the semantic segmentation loss function The calculation expression can be:

[0064] (2)

[0065] in, represents the semantic segmentation loss function, It is calculated to take the value of N of the specified proportion, Nis the total number of pixels in the image, Represents the area attention factor of the instance corresponding to the i-th pixel. The area attention factor is used to characterize the area size of the instance. Represents the shape complexity attention factor of the instance corresponding to the i-th pixel. The shape complexity factor is used to characterize the shape complexity of the instance. represents the difficult sample labeling factor, Represents the difference between the predicted value of the i-th pixel and the true label. The difficult sample labeling factor is used to label the sample that satisfies The samples are the ones that need special attention. Indicates the threshold for distinguishing difficult and easy pixels.

[0066] In a specific application embodiment, the area concern factor, the shape complexity concern factor, and the difficult sample labeling factor can be calculated as follows:

[0067] (3)

[0068] (4)

[0069] (5)

[0070] (6)

[0071] in, Indicates the area of ​​the object to which the current pixel belongs. represents the shape factor threshold, Indicates the perimeter of the object described by the current pixel, s Indicates the area of ​​the object to which the current pixel belongs.

[0072] In this embodiment, the instance detection loss function is specifically calculated using the instance region center loss function (HeatmapLoss) and the instance region pixel offset loss function (Offset Loss), and the calculation expression can be:

[0073] (7)

[0074] in, represents the instance detection loss function, is the instance region center loss function used to optimize the consistency between the instance region predicted by the model and the actual instance region. The instance region pixel offset loss function is used to optimize the pixel-level alignment between the instance region predicted by the model and the true instance region. , They are , The weight value of .

[0075] In a specific application embodiment, the instance region center loss function The mean square error (MSE) loss can be used to minimize the distance between the predicted heatmap and the 2D Gaussian encoded Ground Truth heatmap. An L1 loss can be employed for offset prediction, which activates only at pixels belonging to object instances.

[0076] Step S03. Obtain an image of the ash cone melting during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested, as the ash cone melting image to be tested; input the acquired ash cone melting image to be tested into a coal ash melting image segmentation model; the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area and extracts the geometric features of the image; and identify the moment when the ash cone is at the melting characteristic temperature during the melting process according to the extracted geometric features.

[0077] Specifically, the melting characteristic temperature includes deformation temperature DT, softening temperature ST, hemispherical temperature HT and flow temperature FT. The deformation temperature DT is the temperature at which the tip or edge of the ash cone begins to become round or curved. The softening temperature ST is the temperature at which the ash cone is deformed to the tip of the cone or becomes spherical. The hemispherical temperature HT is the temperature at which the ash cone is deformed to an approximate hemispherical shape. The flow temperature FT is the thin layer temperature when the height of the ash cone melts and is less than 1.5 mm. The geometric features include height, bottom width and area. When the extracted geometric features are used to identify the time when the ash cone is at the melting characteristic temperature during the melting process, the following judgments are made in sequence: if the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the deformation temperature DT; if the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the softening temperature ST; if the extracted geometric features are judged to be highly , bottom width is And the area is , it is identified as being at hemispherical temperature HT; if the height of the extracted geometric features is , the bottom width is , then it is identified as being at the flow temperature FT, where , , are the height, width and area of ​​the gray cone before deformation.

[0078] The main purpose of melting characteristic temperature recognition is to identify the video frames of the four characteristic temperatures of DT, ST, HT and FT states when they just appear in a complete coal ash melting video, and record the temperatures corresponding to the four video frames at this time, and use these four temperatures as the melting characteristic temperatures of the coal ash to be tested. The above-mentioned DT, ST, HT and FT states occur in sequence, and various states can be judged in sequence through the geometric features extracted from the image. At the same time, the time when the four characteristic temperatures occur can be recorded accordingly, so that the automatic measurement of coal ash melting characteristics can be realized.

[0079] In this embodiment, after the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, it also includes post-processing the binary image to obtain a final binary connected domain. The post-processing includes sequentially performing corrosion operations, filtering out objects smaller than a specified size, dilating, and retaining the largest area binary connected domain, which can effectively filter out interference and further ensure detection accuracy.

[0080] like Figure 6 As shown, the detailed process of identifying the characteristic melting temperature in this embodiment is as follows:

[0081] Step S301. Image segmentation: The coal ash melting picture is fed into a coal ash melting image segmentation model to obtain a binary image of the melting area.

[0082] Step S302. Binary image post-processing: After corrosion, filtering out small objects, and dilation operations, the largest binary connected domain is retained to obtain the final binary connected domain. The final edge detection effect diagram obtained in the specific application embodiment is shown in FIG. Figure 7 shown.

[0083] Step S303. Characteristic temperature identification: Extract geometric features such as area, height, bottom width, etc. from the binary connected domain detected in step S302, and identify four characteristic temperatures based on the geometric feature relationship between characteristic temperature and melting image. Specifically, Figure 8 As shown, first, identify the first video frame in the DT state and record the corresponding temperature; second, after the DT state is identified, identify the first video frame in the ST state and record the corresponding temperature; third, after the ST state is identified, identify the first video frame in the HT state and record the corresponding temperature; finally, after the HT state is identified, identify the first video frame in the FT state and record the corresponding temperature. So far, the coal ash melting feature identification process is completed.

[0084] This embodiment also provides an automatic coal ash fusibility determination device based on deep learning image segmentation, including:

[0085] A model building module is used to build a semantic segmentation model based on deep learning. The semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module. The feature decoding module includes a semantic segmentation branch and an instance segmentation branch. The semantic segmentation branch is used to predict the category information of each pixel in the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances.

[0086] A model training module is used to train a semantic segmentation model based on deep learning according to a target loss function using an ash cone melting image dataset to obtain a coal ash melting image segmentation model. The ash cone melting image dataset is an ash cone melting image of an ash cone during heating when the melting properties of different coal ashes are measured using a melting point method. The target loss function is calculated using a semantic segmentation loss function and an instance segmentation loss function.

[0087] The melting measurement module is used to obtain the ash cone melting image of the ash cone during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested, as the ash cone melting image to be tested, and input the obtained ash cone melting image to be tested into the trained coal ash melting image segmentation model. The coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, and extracts the geometric features of the binary image; the melting feature temperature is identified according to the extracted geometric features until the coal ash melting feature identification is completed.

[0088] The automatic measurement device for coal ash fusibility based on deep learning image segmentation in this embodiment corresponds one to one with the above-mentioned automatic measurement method for coal ash fusibility based on deep learning image segmentation, and will not be described one by one here.

[0089] This embodiment further provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above method.

[0090] It is understandable that the above method of this embodiment can be executed by a single device, such as a computer or server, etc., and can also be applied to a distributed scenario and completed by multiple devices in cooperation with each other. In the case of a distributed scenario, one of the multiple devices can only execute one or more steps in the above method of this embodiment, and multiple devices interact to complete the above method. The processor can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, etc., for executing related programs to implement the above method of this embodiment. The memory can be implemented in the form of a read-only memory ROM, a random access memory RAM, a static storage device, and a dynamic storage device. The memory can store an operating system and other applications. When the above method of this embodiment is implemented by software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0091] Those skilled in the art should understand that the above-mentioned embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for automatically determining the fusibility of coal ash based on deep learning image segmentation, characterized in that the steps include: Constructing a semantic segmentation model based on deep learning, wherein the semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module, wherein the feature decoding module includes a semantic segmentation branch and an instance segmentation branch, wherein the semantic segmentation branch is used to predict the category information of each pixel of the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances; Using the ash cone melting image dataset to train the deep learning-based semantic segmentation model according to the target loss function, a coal ash melting image segmentation model is obtained, wherein the ash cone melting image dataset is an ash cone melting image of an ash cone during heating when the melting properties of different coal ashes are measured by a melting point method, and the target loss function is calculated using a semantic segmentation loss function and an instance segmentation loss function, wherein the semantic segmentation loss function is a loss function corresponding to a semantic segmentation branch, and the instance segmentation loss function is a loss function corresponding to an instance segmentation branch; The semantic segmentation loss function is calculated using the area attention factor, shape complexity attention factor and difficult sample labeling factor, and the calculation expression is: in, represents the semantic segmentation loss function, It is calculated to take the value of N of the specified proportion, N is the total number of pixels in the image, Indicates i The area attention factor of the instance corresponding to the pixels is used to characterize the area size of the instance. Indicates i The shape complexity factor of the instance corresponding to the pixels is used to characterize the shape complexity of the instance. represents the difficult sample labeling factor, Indicates i The difference between the pixel prediction value and the true label, the difficult sample labeling factor is used to label the sample that satisfies The samples are the ones that need special attention. represents the threshold for distinguishing between difficult and easy pixels; The instance segmentation loss function is calculated using the instance region center loss function and the instance region pixel offset loss function, and the calculation expression is: in, represents the instance segmentation loss function, is the instance region center loss function used to optimize the consistency between the instance region predicted by the model and the actual instance region. The instance region pixel offset loss function is used to optimize the pixel-level alignment between the instance region predicted by the model and the true instance region. , They are , The weight value of A ash cone melting image of the ash cone during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested is obtained as the ash cone melting image to be tested, and the obtained ash cone melting image to be tested is input into the coal ash melting image segmentation model. The coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area and extracts the geometric features of the image; and the moment when the ash cone is at the melting characteristic temperature during the melting process is identified according to the extracted geometric features.

2. The method for automatic determination of coal ash fusibility based on deep learning image segmentation according to claim 1, characterized in that: The characteristic melting temperatures include deformation temperature DT, softening temperature ST, hemispherical temperature HT and flow temperature FT. The deformation temperature DT is the temperature at which the tip or edge of the ash cone begins to become rounded or bent, the softening temperature ST is the temperature when the ash cone is deformed to the tip of the cone or becomes spherical, the hemispherical temperature HT is the temperature when the ash cone is deformed to an approximately hemispherical shape, and the flow temperature FT is the thin layer temperature when the ash cone melts and expands to a height of less than 1.5 mm.

3. The method for automatic determination of coal ash fusibility based on deep learning image segmentation according to claim 2, characterized in that: The geometric features include height, bottom width and area, wherein the height is the vertical distance from the top of the gray cone to the bottom of the gray cone, the bottom width is the horizontal distance from the bottom of the gray cone, and the area is the number of pixels removed from the gray cone target. When the moment of the gray cone melting process at the melting characteristic temperature is identified according to the extracted geometric features, the following judgments are made in sequence: if it is judged that the height in the extracted geometric features is , bottom width is And the area is , it is identified as being at the deformation temperature DT; if it is determined that the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the softening temperature ST; if it is determined that the height of the extracted geometric features is , bottom width is And the area is , it is identified as being at the hemispherical temperature HT; if it is determined that the height of the extracted geometric features is , the bottom width is , then it is identified as being at the flow temperature FT, where , , are the height, width and area of ​​the gray cone before deformation.

4. The method for automatically determining coal ash fusibility based on deep learning image segmentation according to claim 1, characterized in that: The deep learning-based semantic segmentation model adopts deeplabv3+ as the basic network. The feature encoding module includes a lightweight network and a multi-scale feature extraction submodule. The multi-scale feature extraction submodule adopts hole convolution, Conv 1x1 and avgpool based on different gap sizes to realize deep separable convolution. The semantic segmentation branch adopts CBAM attention mechanism to make the model focus on foreground features.

5. The method for automatic determination of coal ash fusibility based on deep learning image segmentation according to claim 1, characterized in that: The area concern factor, shape complexity concern factor and difficult sample labeling factor are calculated as follows: in, Indicates the area of ​​the object to which the current pixel belongs. represents the shape factor threshold, Indicates the perimeter of the object described by the current pixel, s Indicates the area of ​​the object to which the current pixel belongs.

6. The method for automatically measuring coal ash fusibility based on deep learning image segmentation according to any one of claims 1 to 5, characterized in that: After the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, it also includes post-processing the binary image to obtain a final binary connected domain, and the post-processing includes sequentially performing corrosion operations, filtering out objects smaller than a specified size, dilating, and retaining the largest area binary connected domain.

7. An automatic measurement device for coal ash fusibility based on deep learning image segmentation, characterized in that: include: A model building module, used to build a semantic segmentation model based on deep learning, wherein the semantic segmentation model based on deep learning includes a feature encoding module and a feature decoding module, wherein the feature decoding module includes a semantic segmentation branch and an instance segmentation branch, wherein the semantic segmentation branch is used to predict the category information of each pixel of the image, and the instance segmentation branch is used to assign an instance label to each pixel to distinguish different instances; A model training module is used to train the semantic segmentation model based on deep learning according to the target loss function using the ash cone melting image dataset to obtain a coal ash melting image segmentation model, wherein the ash cone melting image dataset is an ash cone melting image of the ash cone during the heating process when the melting point method is used to measure the melting properties of different coal ashes, and the target loss function is calculated using the semantic segmentation loss function and the instance segmentation loss function. When the melting point method is used to measure the melting properties of the coal ash to be tested, the coal ash to be tested is made into a triangular pyramid to form an ash cone, and is heated at a specified heating rate in a weakly reducing or oxidizing gas medium; The semantic segmentation loss function is calculated using the area attention factor, shape complexity attention factor and difficult sample labeling factor, and the calculation expression is: in, represents the semantic segmentation loss function, It is calculated to take the value of N of the specified proportion, N is the total number of pixels in the image, Indicates i The area attention factor of the instance corresponding to the pixels is used to characterize the area size of the instance. Indicates i The shape complexity factor of the instance corresponding to the pixels is used to characterize the shape complexity of the instance. represents the difficult sample labeling factor, Indicates i The difference between the pixel prediction value and the true label, the difficult sample labeling factor is used to label the sample that satisfies The samples are the ones that need special attention. represents the threshold for distinguishing between difficult and easy pixels; The instance segmentation loss function is calculated using the instance region center loss function and the instance region pixel offset loss function, and the calculation expression is: in, represents the instance segmentation loss function, is the instance region center loss function used to optimize the consistency between the instance region predicted by the model and the actual instance region. The instance region pixel offset loss function is used to optimize the pixel-level alignment between the instance region predicted by the model and the true instance region. , They are , The weight value of The melting measurement module is used to obtain the ash cone melting image of the ash cone during the heating process when the melting point method is used to determine the fusibility of the coal ash to be tested, as the ash cone melting image to be tested, and input the obtained ash cone melting image to be tested into the trained coal ash melting image segmentation model, and the coal ash melting image segmentation model performs image segmentation on the ash cone melting image to be tested to obtain a binary image of the melting area, and extracts the geometric features of the binary image; the melting feature temperature is identified according to the extracted geometric features until the coal ash melting feature identification is completed.

8. A computer device comprising a processor and a memory, wherein the memory is used to store a computer program, wherein: The processor is configured to execute the computer program to perform the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Multi-scale coal gangue image rapid detection method, system and device

    CN114494696A

  • Molybdenum-zirconium-tungsten-hafnium-based ash cone tray for coal ash fusibility test

    CN117623778A