A lightweight semantic segmentation system and method for mineral sintered surfaces
By implementing lightweight design and multi-scale information extraction for the backbone network, the problem of ignoring texture and semantic features in mineral sintering surface segmentation is solved, achieving efficient and accurate material surface segmentation and supporting smart smelting and industrial internet applications.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2023-10-27
- Publication Date
- 2026-07-14
Smart Images
Figure CN117237642B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mineral sintering technology, and more specifically to a lightweight semantic segmentation system and method for mineral sintering surfaces. Background Technology
[0002] Currently, to achieve the goals of high-quality, high-yield, and low-consumption steel manufacturing, ore sintering provides raw materials for subsequent refining processes, making it crucial for metal smelting. With the increasing maturity and application of technologies such as artificial intelligence, cyber-physical systems, big data, and cloud computing, their integration with the industrial internet supply chain will inevitably drive the upgrading of sintering technology, making intelligent sintering systems a key focus for future development. A crucial aspect of intelligent sintering systems is the identification of sintered surface defects. Effective segmentation of the mineral sintering surface is essential for effectively identifying defect types and reducing the impact of obstructions in actual working conditions.
[0003] Current methods for segmenting sintered surfaces can be broadly categorized into two types: methods based on gray-level abrupt changes and methods based on predefined similarity morphological processing. Methods based on gray-level abrupt changes utilize point, line, and edge detection, calculating pixel thresholds based on gray-level gradients to identify the edges of the detected surface. Methods based on predefined similarity morphological processing use thresholds based on pixel property distributions to achieve segmentation. However, these methods neglect the inherent texture and semantic features of the sintered surface, making them ineffective when dealing with heavily occluded sintered images. Traditional semantic segmentation methods based on deep neural networks neglect model lightweighting, which is crucial for real-time segmentation on mobile and small devices. Furthermore, traditional methods are not optimized for the characteristics of the sintering process, resulting in poor segmentation performance and limiting their widespread application in sintered surface extraction and segmentation.
[0004] Therefore, how to efficiently and accurately obtain the sintering material surface segmentation results based on a lightweight model, taking into account both texture and semantic features, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a lightweight semantic segmentation system and method for mineral sintering surfaces. By redesigning the backbone network to be lightweight and using hybrid sampling technology to extract multi-scale information from images, the present invention achieves the goal of efficiently and accurately obtaining sintering surface segmentation results based on a lightweight model that comprehensively considers texture features and semantic features.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] A lightweight semantic segmentation system for mineral sintering surfaces includes: a dataset acquisition module, an encoder, and a decoder;
[0008] The dataset acquisition module is used to collect and label images of mineral sintering surfaces to obtain a dataset.
[0009] The encoder includes: a backbone network, a pyramid module, and a first convolution;
[0010] The backbone network is used to extract features from the input dataset to obtain shallow features;
[0011] The pyramid module is used to extract features from the shallow features to obtain deep features;
[0012] The first convolution is used to count channels in the deep features to obtain deep high semantic features;
[0013] The decoder is used to merge the shallow features and the deep high semantic features and then upsample them to obtain the sintering surface segmentation result.
[0014] Preferably, the backbone network comprises, in sequence, a convolutional layer, a first max pooling layer, a first fire unit, a second fire unit, and a third fire unit;
[0015] The 7×7 convolution and the first max pooling layer are used together to extract features from the input dataset to obtain the first feature;
[0016] The first fire unit includes a first fire module, a second fire module, a third fire module, and a second max pooling layer connected in sequence, which are used together to extract features from the first feature to obtain the second feature;
[0017] The second fire unit includes a fourth fire module, a fifth fire module, a sixth fire module, a seventh fire module, and a third max pooling layer connected in sequence, which are used together to extract features from the second feature to obtain the third feature;
[0018] The third fire unit comprises a sequentially connected eighth fire module and a second convolution, which together are used to extract features from the third feature to obtain shallow features.
[0019] Preferably, all fire modules have the same structure, including extrusion modules and expansion modules;
[0020] The compression module includes a third convolution, used to reduce the dimensionality of the input data to the expansion module;
[0021] The extended module includes a fourth convolution and a first 3×3 convolution connected in sequence, which are used together to extract features from the input data.
[0022] Preferably, the pyramid module includes parallel layers: a fifth convolution, a first dilated convolution, a second dilated convolution, a third dilated convolution, and a global average pooling layer, which are used to extract features from the shallow features to obtain five different multi-scale features.
[0023] The pyramid module also includes a splicing module, which is used to splice and fuse the multi-scale features to obtain deep features.
[0024] Preferably, the decoder includes: a sixth convolution, a first processing module, a second 3×3 convolution, and a second processing module;
[0025] The sixth convolution is used to reduce the number of channels in the shallow features to obtain the processed shallow features.
[0026] The first processing module is used to upsample the deep high semantic features and merge them with the processed shallow features to obtain fused features;
[0027] The second 3×3 convolution is used to extract features from the fused features to obtain the final features;
[0028] The second processing module is used to upsample the final features to obtain the sintered surface segmentation result.
[0029] A lightweight semantic segmentation method for mineral sintering surfaces includes:
[0030] Images of the sintered surfaces of minerals are collected and labeled to obtain a dataset, which is then input into the backbone network.
[0031] The backbone network is used to extract features from the input dataset to obtain shallow features;
[0032] The deep features are obtained by extracting features from the shallow features using the pyramid module;
[0033] Deep semantic features are obtained by counting channels in the deep features through the first convolution.
[0034] The shallow features and the deep semantic features are merged by the decoder and then upsampled to obtain the sintering surface segmentation result.
[0035] Preferably, the backbone network performs feature extraction on the input dataset, specifically including:
[0036] The first feature is obtained by extracting features from the input dataset through a convolutional layer and a first max pooling layer.
[0037] The first feature is extracted by sequentially passing through the first fire module, the second fire module, the third fire module, and the second max pooling layer to obtain the second feature;
[0038] The second feature is extracted sequentially through the fourth fire module, the fifth fire module, the sixth fire module, the seventh fire module, and the third max pooling layer to obtain the third feature;
[0039] The third feature is extracted sequentially through the eighth fire module and the second convolution to obtain shallow features.
[0040] Preferably, all fire modules have the same data processing procedure, which includes:
[0041] The dimensionality of the input data to the expansion module is reduced by using a third convolution.
[0042] The input data is processed by sequentially passing through the fourth convolution and the first 3×3 convolution to extract features.
[0043] Preferably, feature extraction of the shallow features is performed using the pyramid module, specifically including:
[0044] The shallow features are extracted by using the fifth convolution, the first dilated convolution, the second dilated convolution, the third dilated convolution, and the global average pooling layer, respectively, to obtain five different multi-scale features;
[0045] The multi-scale features are spliced and fused using a splicing module to obtain deep features.
[0046] Preferably, the sintered surface segmentation result is obtained, and the specific process includes:
[0047] The number of channels in the shallow features is reduced by the sixth convolution to obtain the processed shallow features;
[0048] The deep high-semantic features are upsampled by the first processing module and then merged with the processed shallow features to obtain fused features;
[0049] The fused features are extracted using a second 3×3 convolution to obtain the final features;
[0050] The final features are upsampled by the second processing module to obtain the sintered surface segmentation result.
[0051] As can be seen from the above technical solution, compared with the prior art, this invention discloses a lightweight semantic segmentation system and method for mineral sintering surfaces. By redesigning the backbone network to be lightweight and using hybrid sampling technology to extract multi-scale information from images, it achieves the goal of efficiently and accurately obtaining sintering surface segmentation results based on a lightweight model, comprehensively considering texture features and semantic features. It has the following beneficial effects:
[0052] 1. The semantic segmentation model based on deep neural networks in this invention fully explores multi-scale information, utilizes the rich features obtained by the encoder, and applies dilated convolution to extract global and local features of arbitrary resolution according to available computing resources, so as to obtain material surface segmentation results efficiently and accurately.
[0053] 2. This invention features a lightweight design for the semantic segmentation backbone network, which simplifies the model structure, avoids consuming large amounts of computing and storage resources, and facilitates application configuration on small and medium-sized devices and mobile terminals.
[0054] 3. The proposed lightweight semantic segmentation method further improves the construction of smart smelting under the industrial internet system, and provides reliable image analysis methods and detection technologies for the metallurgical industry.
[0055] 4. The method of this invention provides engineers with real-time feedback on process effects, facilitating timely adjustment of relevant control parameters to achieve a more efficient and energy-saving sintering process.
[0056] 5. The semantic segmentation technology of sintering material surface based on deep neural networks also provides an effective case for the automation, intelligentization and green development of traditional industrial processes. Attached Figure Description
[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0058] Figure 1 This is a schematic diagram of the lightweight semantic segmentation system structure provided by the present invention.
[0059] Figure 2 This is a schematic diagram illustrating the annotation of a mineral sintering surface provided by the present invention.
[0060] Figure 3 This is a schematic diagram of the encoder structure provided by the present invention.
[0061] Figure 4 This is a schematic diagram of the lightweight backbone network structure provided by the present invention.
[0062] Figure 5 This is a schematic diagram of the fire module structure provided by the present invention.
[0063] Figure 6 This is a schematic diagram of different spans of dilated convolution provided by the present invention.
[0064] Figure 7 This is a schematic diagram of the decoder structure provided by the present invention.
[0065] Figure 8 The flowchart of the lightweight semantic segmentation method provided by the present invention is shown. Detailed Implementation
[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0067] Mineral sintering has its own unique characteristics. Sintering images are characterized by high noise, low quality, and high dimensionality, leading to complex automated processing algorithms. Furthermore, interference from moving probes and workers at the sintering image capture site often hinders accurate segmentation of the material surface. Manual segmentation is costly, and grayscale-based and morphological processing methods neglect the processing of texture features and semantic information, failing to solve the problem of accurate and efficient material surface segmentation. To address this, this invention discloses a lightweight semantic segmentation system and method for mineral sintering surfaces:
[0068] Example 1
[0069] like Figure 1 As shown, this embodiment of the invention discloses a lightweight semantic segmentation system for mineral sintering surfaces, including: a dataset acquisition module, an encoder, and a decoder;
[0070] The dataset acquisition module is used to collect and label images of mineral sintering surfaces to obtain a dataset.
[0071] The encoder consists of: a backbone network, a pyramid module, and a first convolution;
[0072] The backbone network is used to extract features from the input dataset to obtain shallow features;
[0073] The pyramid module is used to extract features from shallow features to obtain deep features;
[0074] The first convolution is used to count channels in deep features to obtain deep semantic features;
[0075] The decoder is used to merge shallow features and deep semantic features and then upsample them to obtain the sintered surface segmentation result.
[0076] Example 2
[0077] This invention discloses a lightweight semantic segmentation system for mineral sintering surfaces, comprising: a dataset acquisition module, an encoder, and a decoder.
[0078] Preferably, the encoder and decoder together constitute the semantic segmentation model.
[0079] The dataset acquisition module is used to collect and label images of mineral sintering surfaces to obtain a dataset.
[0080] Preferably, the acquired mineral sintering surface images are labeled to separate irrelevant backgrounds and other occlusions in the mineral sintering surface images, so as to avoid interfering with the segmentation results.
[0081] Preferably, considering the environment at the sintering site, image acquisition requires the use of a high-temperature resistant industrial camera module. The acquired complete mineral sintering surface image is annotated using the labelme component, with annotation examples as follows: Figure 2 As shown, the irrelevant background includes at least: visual image information, workshop floor and exhaust vents; other obstructions include at least: construction personnel, sampling equipment and lifting equipment.
[0082] like Figure 3 As shown, the encoder includes: a backbone network, a pyramid module, and a first convolution.
[0083] Preferably, the encoder acts as a feature extractor for the entire network, used to compress the original image and progressively extract feature maps rich in high-level semantic information.
[0084] Preferably, introducing pooling and stride convolution into the encoder to gradually reduce the spatial resolution of the input feature map can balance lightweight design and high performance. On the one hand, it can reduce the consumption of computational resources, and on the other hand, it can expand the receptive field to a certain extent. In addition, besides the translation invariance inherent in convolution operations, introducing pooling operations can also bring a certain degree of translation invariance to the network, making the network less sensitive to the target location, indirectly enhancing the network's reasoning ability for unknown data, while also controlling the model complexity by sharing convolution kernels.
[0085] Preferably, the encoder in this embodiment is designed based on DeepLabV3+, which can be used to extract shallow features and deep semantic features simultaneously, balancing lightweight and high performance.
[0086] The backbone network is used to extract features from the input dataset to obtain shallow features.
[0087] Preferred, such as Figure 4 As shown, the backbone network consists of the following sequentially connected layers: convolutional layer, first max pooling layer, first fire unit, second fire unit, and third fire unit;
[0088] The convolutional layer and the first max pooling layer are used together to extract features from the input dataset to obtain the first feature;
[0089] The first fire unit comprises a first fire module, a second fire module, a third fire module, and a second max pooling layer connected in sequence, which are used together to extract features from the first feature to obtain the second feature;
[0090] The second fire unit consists of the fourth fire module, the fifth fire module, the sixth fire module, the seventh fire module, and the third max pooling layer, which are connected in sequence and used together to extract features from the second feature to obtain the third feature.
[0091] The third fire unit consists of a sequentially connected eighth fire module and a second convolution, which together are used to extract features from the third feature to obtain shallow features.
[0092] Preferably, both the first and second convolutions are 1×1 convolutions, and the convolutional layers are 7×7 convolutions.
[0093] Preferably, the shallow features extracted by the backbone network are fed into the encoder and the pyramid module, respectively.
[0094] Preferably, the backbone network is designed with a lightweight approach based on the SqueezeNet model, which minimizes storage and computational complexity while maintaining high accuracy, making it suitable for mobile devices.
[0095] Preferably, a fire module is set in the backbone network, and 1×1 convolutions are used instead of traditional 3×3 convolutions in the squeezing and expanding modules, reducing the corresponding computational cost to 1 / 9 of the original. The number of input features to the 3×3 convolutions is reduced by decomposing the convolutional layers using the fire module, which consists of several 1×1 and 3×3 convolutions. Downsampling is used at the end of the network to expand the receptive field. The above design simplifies the complexity of the backbone network.
[0096] Preferably, the fire module is used instead of traditional convolution operations for feature extraction, thereby reducing model parameters.
[0097] Preferably, the backbone network has eight fire layers, interspersed with three max pooling layers, thereby reducing the number of parameters; at the same time, the backbone network retains convolutional structures at both the top and bottom layers, thus ensuring that the input and output of the model can be adjusted.
[0098] Preferred, such as Figure 5 As shown, the first fire module, second fire module, third fire module, fourth fire module, fifth fire module, sixth fire module, seventh fire module and eighth fire module have the same structure, and all include extrusion module and expansion module;
[0099] The squeezing module includes a third convolution to reduce the dimensionality of the input data to the expansion module;
[0100] The extension module consists of a fourth convolution and a first 3×3 convolution, which are connected sequentially and used together to extract features from the input data, thereby improving the accuracy of the network.
[0101] The extrusion module and the expansion module correspond to the first RELU and the second RELU.
[0102] RELU stands for Rectified Linear Function. In image processing tasks, the Rectified Linear Function is used as the activation function of Convolutional Neural Networks (CNNs) to extract feature information from images and perform tasks such as image classification, object detection, and image semantic segmentation.
[0103] Preferably, both the third and fourth convolutions use 1×1 convolutions.
[0104] Preferably, the fire module uses a large number of 1×1 convolutions instead of the 3×3 or 5×5 convolutions used in most models, significantly reducing network layer parameters and computational complexity. At the same time, the number of internal convolutional kernels is controllable, improving the algorithm's processing performance for sintering images of different specifications.
[0105] The pyramid module is used to extract features from shallow features to obtain deep features.
[0106] Preferably, the pyramid module includes parallel layers: a fifth convolution, a first dilated convolution, a second dilated convolution, a third dilated convolution, and a global average pooling layer, which are used to extract features from shallow features to obtain five different multi-scale features.
[0107] The pyramid module also includes a stitching module, which is used to stitch and fuse multi-scale features to obtain deep features.
[0108] Preferably, the fifth convolution is a 1×1 convolution.
[0109] Preferably, the fifth convolution, the first dilated convolution, the second dilated convolution, the third dilated convolution, and the global average pooling layer simultaneously extract features from the shallow features, resulting in five different multi-scale feature maps. Padding operations are then used to ensure that the obtained multi-scale feature maps are of the same size.
[0110] Preferably, multi-scale features are extracted using a pyramid module, which overcomes the problem that general semantic segmentation algorithms are only suitable for fixed scales. At the same time, unlike multi-stage segmentation algorithms, it does not require candidate region generation and classification. Instead, it directly outputs the location and category information of the target through a convolutional neural network, which simplifies the complexity of the algorithm and thus improves the segmentation speed.
[0111] Preferably, the pyramid module can generate feature information at different scales and achieve feature fusion, and then input the deep features into the decoder. This module effectively solves the feature extraction problem of sintered images of different specifications; at the same time, this module can avoid losing contextual information that represents the relationship between different sub-regions.
[0112] Preferred, such as Figure 6 As shown, dilated convolution can increase the receptive field without compressing information or increasing the number of parameters. The receptive field can be controlled by adjusting the span.
[0113] Preferably, in this embodiment, the first dilated convolution uses a 3×3 dilated convolution with a span of 6, the second dilated convolution uses a 3×3 dilated convolution with a span of 12, and the third dilated convolution uses a 3×3 dilated convolution with a span of 18.
[0114] Preferably, the first dilated convolution, the second dilated convolution, the third dilated convolution, and the first convolution together constitute a depth-separable dilated convolution. The dilated convolutions of different spans independently perform spatial convolution on each input channel, while the first convolution is used to merge the outputs of the dilated convolutions, which greatly reduces the computational complexity.
[0115] The first convolution is used to count channels in deep features to obtain deep high-semantic features.
[0116] The decoder is used to merge shallow features and deep semantic features and then upsample them by 4 times to obtain the sintered surface segmentation result.
[0117] Preferred, such as Figure 7 As shown, the decoder includes: a sixth convolution, a first processing module, a second 3×3 convolution, and a second processing module;
[0118] The sixth convolution is used to reduce the number of channels in shallow features to reduce computational complexity and obtain the processed shallow features; the sixth convolution uses a 1×1 convolution.
[0119] The first processing module is used to upsample the deep high semantic features by bilinear interpolation by 4 times and then merge them with the processed shallow features to obtain fused features;
[0120] The second 3×3 convolution is used to extract features from the fused features to obtain the final features;
[0121] The second processing module is used to upsample the final features by 4 times to ensure that the decoder output image is consistent with the input image, thus obtaining the sintered surface segmentation result.
[0122] Example 3
[0123] If the input format of the sintering image is RGB and the resolution is 224×224, the data processing flow of the lightweight backbone network in this embodiment is as follows:
[0124] After a 7×7 convolutional layer and a first max pooling layer, a first feature with a size of 55×55×96 is obtained.
[0125] After passing through 3 fire modules and a second max pooling layer, a second feature with a size of 27×27×256 is obtained;
[0126] After four fire modules and a third max pooling layer, a third feature with a size of 13×13×512 is obtained.
[0127] After passing through one fire module and a second convolution, shallow features with an output size of 13×13×1000 are obtained.
[0128] The parameters of the lightweight backbone network are shown in Table 1:
[0129] Table 1
[0130] Network layer name / type (Output) Size kernel size depth Input image 224×224×3 / / 7×7 convolutional layer 111×111×96 7×7 / 2(×96) 1 First maximum pooling layer 55×55×96 3×3 / 2 0 First fire module 55×55×128 / 2 Second fire module 55×55×128 / 2 Third Fire Module 55×55×256 / 2 Second maximum pooling layer 27×27×256 3×3 / 2 0 Fourth Fire Module 27×27×256 / 2 Fifth Fire Module 27×27×384 / 2 The sixth fire module 27×27×384 / 2 The seventh fire module 27×27×512 / 2 Third maximum pooling layer 13×13×512 3×3 / 2 0 Eighth Fire Module 13×13×512 / 2 Second 1×1 convolution 13×13×1000 1×1 / 1(×1000) 1
[0131] Example 4
[0132] Training and prediction of semantic segmentation models
[0133] A semantic segmentation model is constructed based on the encoder and decoder described above.
[0134] The dataset obtained through the dataset acquisition module is divided into training set, validation set and test set. When there is sufficient data, the division ratio is set to 7:2:1.
[0135] The semantic segmentation model is trained using a training set. During the training process, the model is monitored for overfitting using a validation set, which is then used to adjust the model's hyperparameters, thereby obtaining alternative semantic segmentation models.
[0136] The segmentation accuracy of the candidate semantic segmentation model is verified by testing the test set. If the accuracy is higher than the preset value, it is determined as the optimal semantic segmentation model and applied to specific sintering scenarios.
[0137] Preferably, the accuracy rate in this embodiment is preset to 95%.
[0138] In actual testing, the semantic segmentation model designed in this invention achieved an accuracy of 98% in the sintering surface segmentation task using only 5% of labeled data for training. Therefore, the semantic segmentation model of this invention can be used when the amount of labeled data is insufficient, while taking into account both high performance and lightweight design, which facilitates the deployment of sintering intelligent monitoring systems.
[0139] Example 5
[0140] like Figure 8 As shown, a lightweight semantic segmentation method for mineral sintering surfaces includes:
[0141] Images of mineral sintering surfaces were collected and labeled to obtain a dataset, which was then input into the backbone network.
[0142] The backbone network is used to extract features from the input dataset to obtain shallow features;
[0143] The pyramid module is used to extract features from shallow features to obtain deep features;
[0144] Deep semantic features are obtained by counting channels in the deep features through the first convolution.
[0145] The shallow features and deep semantic features are merged by the decoder and then upsampled by 4 times to obtain the sintering surface segmentation result.
[0146] Preferably, the backbone network extracts features from the input dataset, specifically including:
[0147] The first feature is obtained by extracting features from the input dataset through a combination of convolutional layers and a first max pooling layer.
[0148] The first feature is extracted by sequentially passing through the first fire module, the second fire module, the third fire module, and the second max pooling layer to obtain the second feature;
[0149] The second feature is extracted sequentially through the fourth fire module, the fifth fire module, the sixth fire module, the seventh fire module, and the third max pooling layer to obtain the third feature;
[0150] The third feature is extracted by sequentially passing through the eighth fire module and the second convolution to obtain shallow features.
[0151] Preferably, all fire modules have the same data processing procedure, which includes:
[0152] The dimensionality of the input data to the expansion module is reduced by using a third convolution.
[0153] The input data is processed by sequentially passing through the fourth convolution and the first 3×3 convolution to extract features.
[0154] Preferably, feature extraction of shallow features is performed using the pyramid module, specifically including:
[0155] The shallow features were extracted by using the fifth convolution, the first dilated convolution, the second dilated convolution, the third dilated convolution, and the global average pooling layer, respectively, resulting in five different multi-scale features;
[0156] The deep features are obtained by stitching and fusing multi-scale features through the stitching module.
[0157] Preferably, the sintered surface segmentation result is obtained, and the specific process includes:
[0158] The number of channels in the shallow features is reduced by the sixth convolution to obtain the processed shallow features;
[0159] The deep, high-semantic features are upsampled by a factor of 4 by the first processing module and then merged with the processed shallow features to obtain the fused features.
[0160] The final features are obtained by extracting features from the fused features through a second 3×3 convolution.
[0161] The final features are upsampled by 4 times by the second processing module to obtain the sintered surface segmentation result.
[0162] As can be seen from the above technical solution, compared with the prior art, this invention discloses a lightweight semantic segmentation system and method for mineral sintering surfaces. By redesigning the backbone network to be lightweight and using hybrid sampling technology to extract multi-scale information from images, it achieves the goal of efficiently and accurately obtaining sintering surface segmentation results based on a lightweight model, comprehensively considering texture features and semantic features. It has the following beneficial effects:
[0163] 1. The semantic segmentation model based on deep neural networks in this invention fully explores multi-scale information, utilizes the rich features obtained by the encoder, and applies dilated convolution to extract global and local features of arbitrary resolution according to available computing resources, so as to obtain material surface segmentation results efficiently and accurately.
[0164] 2. This invention features a lightweight design for the semantic segmentation backbone network, which simplifies the model structure, avoids consuming large amounts of computing and storage resources, and facilitates application configuration on small and medium-sized devices and mobile terminals.
[0165] 3. The proposed lightweight semantic segmentation method further improves the construction of smart smelting under the industrial internet system, and provides reliable image analysis methods and detection technologies for the metallurgical industry.
[0166] 4. The method of this invention provides engineers with real-time feedback on process effects, facilitating timely adjustment of relevant control parameters to achieve a more efficient and energy-saving sintering process.
[0167] 5. The semantic segmentation technology of sintering material surface based on deep neural networks also provides an effective case for the automation, intelligentization and green development of traditional industrial processes.
[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0169] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A lightweight semantic segmentation system for mineral sintering surfaces, characterized in that, include: Dataset acquisition module, encoder, and decoder; The dataset acquisition module is used to collect and label images of mineral sintering surfaces to obtain a dataset. The encoder includes: a backbone network, a pyramid module, and a first convolution; The backbone network is used to extract features from the input dataset to obtain shallow features; The backbone network is designed with a lightweight approach based on the SqueezeNet model. The backbone network comprises, in sequence, a convolutional layer, a first max pooling layer, a first fire unit, a second fire unit, and a third fire unit; The convolutional layer and the first max pooling layer are used together to extract features from the input dataset to obtain the first feature; The first fire unit includes a first fire module, a second fire module, a third fire module, and a second max pooling layer connected in sequence, which are used together to extract features from the first feature to obtain the second feature; The second fire unit includes a fourth fire module, a fifth fire module, a sixth fire module, a seventh fire module, and a third max pooling layer connected in sequence, which are used together to extract features from the second feature to obtain the third feature; The third fire unit includes a sequentially connected eighth fire module and second convolution, which are used together to extract features from the third feature to obtain shallow features. All fire modules have the same structure, including extrusion modules and expansion modules; The compression module includes a third convolution, used to reduce the dimensionality of the input data to the expansion module; The extended module includes a fourth convolution and a first 3×3 convolution connected in sequence, which are used together to extract features from the input data; The pyramid module is used to extract features from the shallow features to obtain deep features; The first convolution is used to count channels in the deep features to obtain deep high semantic features; The decoder is used to merge the shallow features and the deep high semantic features and then upsample them to obtain the sintering surface segmentation result.
2. The lightweight semantic segmentation system for mineral sintering surfaces according to claim 1, characterized in that, The pyramid module includes parallel layers: a fifth convolution, a first dilated convolution, a second dilated convolution, a third dilated convolution, and a global average pooling layer, which are used to extract features from the shallow features to obtain five different multi-scale features. The pyramid module also includes a splicing module, which is used to splice and fuse the multi-scale features to obtain deep features.
3. A lightweight semantic segmentation system for mineral sintering surfaces according to claim 1, characterized in that, The decoder includes: a sixth convolution, a first processing module, a second 3×3 convolution, and a second processing module; The sixth convolution is used to reduce the number of channels in the shallow features to obtain the processed shallow features. The first processing module is used to upsample the deep high semantic features and merge them with the processed shallow features to obtain fused features; The second 3×3 convolution is used to extract features from the fused features to obtain the final features; The second processing module is used to upsample the final features to obtain the sintered surface segmentation result.
4. A lightweight semantic segmentation method for mineral sintering surfaces, characterized in that, include: Images of the sintered surfaces of minerals are collected and labeled to obtain a dataset, which is then input into the backbone network. The backbone network is used to extract features from the input dataset to obtain shallow features; The backbone network is designed with a lightweight approach based on the SqueezeNet model. The backbone network extracts features from the input dataset, specifically including: The first feature is obtained by extracting features from the input dataset through a convolutional layer and a first max pooling layer. The first feature is extracted by sequentially passing through the first fire module, the second fire module, the third fire module, and the second max pooling layer to obtain the second feature; The second feature is extracted sequentially through the fourth fire module, the fifth fire module, the sixth fire module, the seventh fire module, and the third max pooling layer to obtain the third feature; The third feature is extracted sequentially through the eighth fire module and the second convolution to obtain shallow features; All fire modules have the same data processing procedure, which includes: The dimensionality of the input data to the expansion module is reduced by using a third convolution. The input data is processed sequentially through the fourth convolution and the first 3×3 convolution to extract features. The deep features are obtained by extracting features from the shallow features using the pyramid module; Deep semantic features are obtained by counting channels in the deep features through the first convolution. The shallow features and the deep semantic features are merged and then upsampled to obtain the sintering surface segmentation result.
5. A lightweight semantic segmentation method for mineral sintering surfaces according to claim 4, characterized in that, Feature extraction of the shallow features is performed using the pyramid module, specifically including: The shallow features are extracted by using the fifth convolution, the first dilated convolution, the second dilated convolution, the third dilated convolution, and the global average pooling layer, respectively, to obtain five different multi-scale features; The multi-scale features are spliced and fused using a splicing module to obtain deep features.
6. A lightweight semantic segmentation method for mineral sintering surfaces according to claim 5, characterized in that, The specific process for obtaining the sintered surface segmentation results includes: The number of channels in the shallow features is reduced by the sixth convolution to obtain the processed shallow features; The deep high-semantic features are upsampled by the first processing module and then merged with the processed shallow features to obtain fused features; The fused features are extracted using a second 3×3 convolution to obtain the final features; The final features are upsampled by the second processing module to obtain the sintered surface segmentation result.