Blast furnace sintered ore particle size detection method and system based on RGB and laser feature fusion

Abstract

The present invention discloses a blast furnace sinter ore particle size detection method and system based on RGB and laser feature fusion. By acquiring an RGB data set and a laser data set, a deep learning network of an encoder-decoder structure is constructed to obtain an RGB initial feature tensor and a laser initial feature tensor. A multi-source feature weighted fusion subnetwork is constructed at the last layer of a fusion branch. Based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are adaptively weighted fused to obtain a fused feature tensor, and the fused feature tensor is input into a decoding process to obtain a segmented image. The particle size of the blast furnace sinter ore is detected according to the segmented image, and the technical problem of low detection accuracy of the existing blast furnace sinter ore particle size is solved. By adding a multi-source feature weighted fusion subnetwork to the deep learning network of the encoder-decoder structure, the adaptive weighted fusion of multi-source features can be achieved, and the supplementary role of the laser is fully utilized, which is beneficial to improving the detection accuracy of the blast furnace sinter ore particle size.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of blast furnace ironmaking, and in particular to a blast furnace sintered ore particle size detection method and system based on RGB and laser feature fusion. Background Art

[0002] The blast furnace is a key equipment used for ironmaking in the steel industry, and its stable operation determines the quality of ironmaking. The blast furnace production process mainly includes four major systems: air supply system, charging system, slag making system and thermal system. The four basic systems are interdependent and influence each other, directly affecting the operation and smooth operation of the blast furnace and thus affecting the quality of ironmaking. Optimization of the charging system can make the gas distribution in the furnace reasonable, improve the contact conditions between ore and gas, reduce the resistance of gas to the descent of charge, avoid blast furnace air blocking and hanging material, and at the same time improve gas utilization and indirect reduction degree of ore, reduce coke ratio, and promote stable and smooth production of blast furnace. How to reasonably optimize the charging system depends largely on the quality of charge transported to the blast furnace, such as charge particle size.

[0003] Sintered ore is an important type of blast furnace ore. In the current blast furnace charging process, sintered ore is tested every 4 hours by manual sampling. This manual screening method not only has low detection frequency, low accuracy and slow speed, but the sampling results cannot fully represent the actual situation of the raw fuel particle size, and it is impossible to effectively track the raw fuel particle size on site, and it is difficult to evaluate the impact on the blast furnace condition. Low accuracy, low sampling frequency and slow speed. Therefore, a new method is needed to increase the frequency of particle size detection and improve the accuracy of particle size detection, so as to grasp the particle size, distribution and trend of the raw fuel entering the furnace in real time.

[0004] At present, the particle size detection method for charge ore is mainly based on image segmentation method. Image segmentation method mainly detects the particle size of ore by distinguishing the edge and non-edge area of ​​ore through features such as color, brightness, and grayscale. At present, it mainly includes traditional image segmentation and image segmentation based on deep learning. Traditional image segmentation algorithms mainly include threshold-based, clustering-based, and region growth-based methods. Due to the complexity of the surface texture of blast furnace charge and the noise caused by objective environments such as uneven lighting. The low-level features of noise have local drastic changes, which leads to the inability of traditional image segmentation methods to effectively distinguish edges from noise in the edge detection of blast furnace charge images, resulting in over-segmentation.

[0005] Image segmentation algorithms based on deep learning are relatively rarely used in blast furnace charge image segmentation, but they are widely used in image edge detection. In recent years, with the introduction of the FCN framework, more and more deep learning networks with encoder-decoder structures have been widely used in the field of semantic segmentation. For example, SegNet, U-Net, deeplab, etc. have better application effects in medical images. However, due to the uneven lighting environment of the blast furnace charging system and the interference of complex ore textures, a single RGB image can no longer meet the demand for high precision. Single RGB information is difficult to distinguish the complex texture and edges of the ore surface. Therefore, UNet, PSPNet, etc. only train the network by learning the information in the ore RGB image, and cannot accurately segment the ore image.

[0006] Learning effective information from depth images is one of the important means of conducting multi-source information fusion research. In the research on fusing RGB and depth information, early attempts simply cascaded the RGB and depth channels into four-channel inputs and sent them to the traditional RGB modality network. This method does not consider the different contributions of the two input data to the segmentation results during the fusion process. These RGB-D semantic segmentation methods can obtain better segmentation results than single RGB-based methods. The main reason is that compared with single RGB images, the introduction of accurate dense depth maps contains more position and contour information, which is conducive to contextual key semantic segmentation and will inevitably achieve better segmentation results. When processing two input data, these methods do not quantitatively consider the different contributions of the two input data to the segmentation results, but make unified or equivalent treatments, which may cause the additional information to have a negative impact (negative contribution) on the segmentation results.

[0007] Most of these methods are based on dense depth maps to achieve segmentation, and are not applicable to sparse depth maps (such as laser and radar images). In the harsh production process of blast furnaces, it is almost impossible to directly obtain dense depth maps, and only sparse depth maps can be obtained.

[0008] Difficulty 1: The surface texture of the ore image is too complex, and it is difficult to achieve high-precision segmentation with a single information. Due to the uneven lighting in the environment where the blast furnace feeding system is located and the interference of the complex texture of the ore, a single RGB image has been unable to meet the demand for high precision. Single RGB information is difficult to distinguish the complex texture and edge of the ore surface. Therefore, the present invention innovatively chooses to fuse RGB information with laser information (L) to enhance the effective features for high-precision segmentation.

[0009] Difficulty 2: Laser information is obviously sparse, which makes its contribution to the fusion process of RGB very limited. If the sparse depth map is converted into a dense depth map through the interpolation algorithm, some error information will be generated in the interpolation process (this is an inherent limitation of the interpolation algorithm and calibration algorithm). These error information may have a negative effect in the fusion process, and may cause the segmentation effect of the fusion of laser information to be even worse than that of the single information. The final table and segmentation results are compared.

[0010] Public document CN113052826A discloses a pancreatic image segmentation method with selective fusion of multi-scale features. The invention proposes a pancreatic image segmentation method with selective fusion of multi-scale features, constructs a two-stage coarse-to-fine pancreatic segmentation framework, firstly performs coarse segmentation and segmentation optimization on the preprocessed whole image to obtain a coarse segmentation result, determines the bounding box of the pancreatic segmentation area, crops the preprocessed image, and then performs fine segmentation and segmentation optimization on the cropped image to obtain a fine segmentation result of the pancreatic image; introduces SE mechanism and Inception idea into feature fusion at different levels, and constructs a multi-scale feature selective fusion module MSSFM; based on the Unet model, the encoding part uses residual convolution blocks to make the network deeper and prevent model degradation, thereby improving the learning of effective features; the decoding part uses the selective fusion module of multi-scale features to fuse multi-scale features from bottom to top, increases the adaptability of the network to multi-scale targets and focuses on the target area, thereby improving the accuracy of model segmentation.

[0011] This invention only uses RGB information, which is difficult to apply to sintered ore images with complex surface textures. Moreover, the SE mechanism selectively fuses features of different scales using artificial prior knowledge, and does not reflect the difference in the contribution of different features to the segmentation results.

[0012] Public document CN107766794B discloses a method for image semantic segmentation with learnable feature fusion coefficients. The invention relates to a method for image semantic segmentation with learnable feature fusion coefficients. The method mainly includes the following steps: firstly training a deep convolutional network classification model from image to category label on an image classification data set; converting the fully connected layer type in the classification model to a convolutional layer type to obtain a fully convolutional deep neural network model to perform pixel-level category prediction; then expanding the convolutional layer branches, setting a coefficient for each branch, and fusing the feature fusion layer according to the coefficient ratio, setting the coefficient to a learnable state; secondly, fine-tuning training is performed on the image semantic segmentation data set, and coefficient learning is performed at the same time to obtain a semantic segmentation model; after fine-tuning training and fusion coefficient learning, 1 to 20 groups of fusion coefficients can be obtained; finally, the branch with the largest coefficient is selected in each group, and the final combination is performed, and fine-tuning training and coefficient learning are performed again to obtain the final semantic segmentation model. The present invention enables the feature fusion effect to reach the best state.

[0013] This invention also only utilizes RGB information and assigns weights to different branches for training through artificial prior knowledge. It is also unable to reflect the difference in contribution of different features to the segmentation results, and is difficult to apply when faced with sintered ore images with complex surface textures. Summary of the invention

[0014] The blast furnace sinter ore particle size detection method and system based on RGB and laser feature fusion provided by the present invention solve the technical problem of low accuracy in the existing blast furnace sinter ore particle size detection.

[0015] In order to solve the above technical problems, the present invention proposes a blast furnace sinter ore particle size detection method based on RGB and laser feature fusion, which includes:

[0016] Collect RGB images of blast furnace sintered ore and single-line laser detection data on the blast furnace charge conveyor belt to obtain RGB data sets and laser data sets;

[0017] Construct a deep learning network with an encoder-decoder structure, which includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process;

[0018] According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain the RGB initial feature tensor and the laser initial feature tensor;

[0019] A multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused to obtain a fused feature tensor;

[0020] The fused feature tensor is input into the decoding process to obtain a segmented image, and the particle size of the blast furnace sintered ore is detected based on the segmented image.

[0021] Furthermore, a multi-source feature weighted fusion subnetwork is constructed in the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weighted fused to obtain the fused feature tensor including:

[0022] Performing feature compression on the RGB initial feature tensor and the laser initial feature tensor respectively to obtain an RGB compressed feature tensor and a laser compressed feature tensor;

[0023] Perform feature expansion on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain an RGB expanded feature tensor and a laser expanded feature tensor;

[0024] Convolution calculation and linear transformation are performed on the RGB expanded feature tensor and the laser expanded feature tensor respectively to obtain the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor;

[0025] Based on the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor, the RGB initial feature tensor and the laser initial feature tensor are weighted fused to obtain a fused feature tensor.

[0026] Furthermore, convolution calculation and linear transformation are performed on the RGB expanded feature tensor and the laser expanded feature tensor respectively, and the feature weights of each feature channel in the RGB initial feature tensor and the laser initial feature tensor are obtained, including:

[0027] Each feature channel of the RGB expanded feature tensor and the laser expanded feature tensor is regarded as a neuron, and the channel coefficient of each feature channel is regarded as the weight of the neuron, and a neural network for training RGB channel coefficients and a neural network for training laser channel coefficients are established respectively;

[0028] The channel coefficient of each feature channel of the RGB expanded feature tensor and the laser expanded feature tensor is trained to obtain the RGB channel coefficient and the laser channel coefficient;

[0029] Based on the RGB expansion feature tensor and the RGB channel coefficient, an RGB convolution feature tensor is obtained; based on the laser expansion feature tensor and the laser channel coefficient, a laser convolution feature tensor is obtained;

[0030] The RGB convolution feature tensor and the laser convolution feature tensor are linearly transformed to obtain the feature weights of each feature channel in the RGB initial feature tensor and the laser initial feature tensor.

[0031] Furthermore, based on the RGB convolution feature tensor and the RGB channel coefficient, the RGB convolution feature tensor is obtained, and based on the laser convolution feature tensor and the laser channel coefficient, the calculation formula for obtaining the laser convolution feature tensor is:

[0032]

[0033] in, and Respectively represent the RGB convolution feature tensor and the laser convolution feature tensor, w c,0 and w c,1 They represent the channel coefficients of the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, and w c+j,0 and w c+j,1 They represent the channel coefficients of the c+jth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, and w c,0 、w c,1 、w c+j,0 and w c+j,1 The initial value of is 1, d represents the neighborhood size of the feature channel, which is 1, and D represents the number of feature channels involved in the calculation of a feature tensor in a T-shaped convolution operation, D = 2d + 1.

[0034] Furthermore, the RGB convolution feature tensor and the laser convolution feature tensor are linearly transformed to obtain the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor:

[0035]

[0036] in, and They represent the feature weights of the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, and They represent the RGB convolution feature tensor and the laser convolution feature tensor respectively.

[0037] Further, feature expansion is performed on the RGB compressed feature tensor and the laser compressed feature tensor respectively, and obtaining the RGB expanded feature tensor and the laser expanded feature tensor includes:

[0038] Performing the first fully connected layer operation on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain the reduced-dimensional RGB compressed feature tensor and the laser compressed feature tensor;

[0039] A second fully connected layer operation is performed on the RGB compressed feature tensor and the laser compressed feature tensor after dimension reduction, and the RGB expanded feature tensor and the laser expanded feature tensor are obtained through the sigmoid function.

[0040] Furthermore, the RGB initial feature tensor and the laser initial feature tensor are respectively subjected to feature compression, and the calculation formulas for obtaining the RGB compressed feature tensor and the laser compressed feature tensor are as follows:

[0041]

[0042] Among them, u c,0 and u c,1 represents the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, c,0 and z c,1 Respectively represent u c,0 and u c,1 The result after feature compression is a real number, F sq represents the feature compression operation, h and w represent the dimensions of the feature channels in the RGB initial feature tensor and the laser initial feature tensor, and i and j represent the element positions in the two-dimensional matrix corresponding to the feature channel;

[0043] The RGB compressed feature tensor and the laser compressed feature tensor are respectively expanded, and the specific formulas for obtaining the RGB expanded feature tensor and the laser expanded feature tensor are as follows:

[0044]

[0045] Among them, F ex represents the feature expansion operation, s c,0 and c,1 They represent the cth feature channel in the RGB expanded feature tensor and the laser expanded feature tensor respectively, σ and δ represent the sigmoid activation function and the ReLU activation function respectively, and W1 and W2 represent the network parameters of the first fully connected layer and the second fully connected layer respectively.

[0046] Furthermore, according to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively, and the RGB initial feature tensor and the laser initial feature tensor are obtained, including:

[0047] According to the RGB encoding branch, pooling and convolution operations are performed on the RGB data set to obtain the RGB initial feature tensor;

[0048] Interpolate the laser data set to obtain dense laser data;

[0049] According to the laser encoding branch, pooling and convolution operations are performed on the dense laser data to obtain the initial laser feature tensor.

[0050] Furthermore, the fused feature tensor is input into the decoding process to obtain the segmented image including:

[0051] Concatenate the fused feature tensor with the upsampled result of the previous layer output of the multi-source feature weighted fusion subnetwork in the deep learning network;

[0052] The spliced ​​results are input into the decoding process of the deep learning network, and the segmented image is obtained through four bottom-up upsampling processes.

[0053] The blast furnace sintered ore particle size detection system based on RGB and laser feature fusion provided by the present invention comprises:

[0054] A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the blast furnace sintered ore particle size detection method based on RGB and laser feature fusion provided by the present invention are implemented.

[0055] Compared with the prior art, the advantages of the present invention are:

[0056] The blast furnace sinter ore particle size detection method and system based on RGB and laser feature fusion provided by the present invention obtain an RGB data set and a laser data set by collecting RGB images of blast furnace sinter ore on a blast furnace charge conveyor belt and single-line laser detection data, and construct a deep learning network with an encoder-decoder structure. The deep learning network includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process. According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor. A multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor is extracted. The laser initial feature tensor is weightedly fused to obtain a fused feature tensor, and the fused feature tensor is input into the decoding process to obtain a segmented image, and the particle size of blast furnace sinter is detected according to the segmented image, which solves the technical problem of low accuracy of existing blast furnace sinter particle size detection. By adding a multi-source feature weighted fusion sub-network to the deep learning network of the encoder-decoder structure, the feature weight of each feature channel in the feature tensor can be obtained in real time, and the complementary role of the laser is fully exerted, which is beneficial to improving the detection accuracy of blast furnace sinter particle size. The contribution of different features to the segmentation results in the fusion process is quantitatively demonstrated through visualization, which is of great significance for subsequent research related to fusion and interdisciplinary studies.

[0057] The present invention aims to propose an image segmentation network FNet based on weighted fusion of RGB and laser depth features for segmenting blast furnace ore images.

[0058] On the one hand, due to the uneven lighting environment of the blast furnace feeding system and the interference of complex ore textures, a single RGB image can no longer meet the demand for high precision. Single RGB information is difficult to distinguish the complex texture and edges of the ore surface. Therefore, in order to achieve higher segmentation accuracy, the present invention innovatively chooses to fuse RGB information with laser information to enhance effective features for high-precision segmentation.

[0059] On the other hand, due to the obvious sparsity of laser information, its contribution to the fusion process of RGB is very limited, and it is difficult to achieve better results with conventional fusion methods. Therefore, in order to better fuse RGB and laser information and give full play to the complementary role of laser information, the present invention proposes a new multi-source feature weighted fusion subnetwork WF-SENet, and designs a T-shaped convolution template that spans features and channels. It is used to learn the relationship between feature maps and channels, and can assign more reasonable weights to each channel of each feature map, so that a fused feature tensor that is beneficial to improving the detection accuracy of blast furnace sintered ore particle size can be obtained based on the weighted weight. Reduce the negative impact of inaccurate information. Based on the codec framework, RGB, laser and fusion branches are designed to encode different input information, and finally the final segmented image is obtained by decoding.

[0060] Compared with the existing methods, the image segmentation method based on weighted fusion of RGB and laser deep features proposed in the present invention innovatively integrates RGB and laser information, and fuses multi-source features through a new multi-source feature weighted fusion sub-network, which gives full play to the complementary role of laser, is beneficial to improving the segmentation accuracy of the deep network, thereby improving the detection accuracy of the particle size of blast furnace sintered ore, and quantitatively demonstrates the difference in contribution of different features to the segmentation results in the fusion process through visualization, which is of great significance to the subsequent research related to fusion and interdisciplinary studies. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic diagram of a sintered ore image and laser detection system according to a second embodiment of the present invention;

[0062] Figure 2 This is an overall framework diagram of the segmentation network of the second embodiment of the present invention;

[0063] Figure 3 This is a structural diagram of a multi-source feature weighted fusion sub-network according to the second embodiment of the present invention;

[0064] Figure 4 Schematic diagram of the calculation process of the T-shaped convolution template of the second embodiment of the present invention;

[0065] Figure 5 This is a comparison diagram of the segmentation results of the sintered ore image according to the second embodiment of the present invention;

[0066] Figure 6 It is a structural block diagram of a blast furnace sintered ore particle size detection system based on RGB and laser feature fusion according to an embodiment of the present invention.

[0067] Reference numerals:

[0068] 10. Memory; 20. Processor. DETAILED DESCRIPTION

[0069] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively and meticulously below in conjunction with the accompanying drawings and preferred embodiments of the present invention, but the protection scope of the present invention is not limited to the following specific embodiments.

[0070] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0071] Embodiment 1

[0072] The method for detecting particle size of blast furnace sintered ore based on RGB and laser feature fusion provided in the first embodiment of the present invention includes:

[0073] Step S101, collecting RGB images and single-line laser detection data of blast furnace sintered ore on a blast furnace charge conveyor belt to obtain an RGB data set and a laser data set;

[0074] Step S102, constructing a deep learning network of an encoder-decoder structure, wherein the deep learning network includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process;

[0075] Step S103, according to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor;

[0076] Step S104, constructing a multi-source feature weighted fusion subnetwork at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, performing weighted fusion on the RGB initial feature tensor and the laser initial feature tensor to obtain a fused feature tensor;

[0077] Step S105, inputting the fused feature tensor into a decoding process to obtain a segmented image, and detecting the particle size of the blast furnace sintered ore according to the segmented image.

[0078] The blast furnace sinter ore particle size detection method based on RGB and laser feature fusion provided by the embodiment of the present invention obtains RGB data sets and laser data sets by collecting RGB images of blast furnace sinter ore on a blast furnace charge conveyor belt and single-line laser detection data, and constructs a deep learning network with an encoder-decoder structure. The deep learning network includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process. According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor. A multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor is extracted. The weighted fusion of the laser initial feature tensor is performed to obtain a fused feature tensor, and the fused feature tensor is input into the decoding process to obtain a segmented image, and the particle size of the blast furnace sinter is detected according to the segmented image, which solves the technical problem of low accuracy of the existing blast furnace sinter particle size detection. By adding a multi-source feature weighted fusion sub-network to the deep learning network of the encoder-decoder structure, the feature weight of each feature channel in the feature tensor can be obtained in real time, thereby realizing the adaptive weighted fusion of multi-source features, giving full play to the complementary role of the laser, and quantitatively demonstrating the difference in contribution of different features to the segmentation results in the fusion process through visualization, which is of great significance for subsequent research related to fusion and interdisciplinary studies.

[0079] Embodiment 2

[0080] The blast furnace sintered ore particle size detection method based on RGB and laser feature fusion in an embodiment of the present invention includes:

[0081] Step S201 , collecting RGB images and single-line laser detection data of blast furnace sintered ore on a blast furnace charge conveyor belt to obtain an RGB data set and a laser data set.

[0082] Specifically, in order to obtain high-resolution RGB images and laser detection data, RGB image information and laser depth information are collected through hardware devices such as industrial cameras and laser scanners. Among them, the laser detection equipment used is the LMS4111R-13000 laser line scanner produced by SICK, and the data acquisition software is SOPAS Engineering Tool. The industrial camera used is the German Basler acA2500-14gm Basler industrial array camera data acquisition software is pylon Viewer. Both use Ethernet for communication. The acquisition system is as follows: Figure 1 As shown, Figure 1 The device for detecting the particle size of blast furnace sintered ore based on RGB and laser feature fusion in the embodiment of the present invention includes an industrial camera, a single-line laser scanner, a bracket, a light source, a computer, etc.

[0083] The initial data of this embodiment is stored in the form of .txt messages. First, the ranging information in the message is extracted, and a little preprocessing is performed before it can be used as depth information. Parse the message and locate the location of key information such as depth information, acquisition time, and number of frames by looking for the flag. And extract the ranging information. The message data type is divided into received and connected. Connect only indicates the status of the data connection. Received is the received information. It is usually necessary to exclude the connect data. At the same time, the data is judged whether it is abnormal according to the number of frames (the normal number of frames is between 6000-9000). The laser sequence of the corresponding image is selected according to the image acquisition time. According to the acquisition time of the laser sequence and the image, the nearest match is performed, and the time interval does not exceed 0.01s. The laser data was originally a polar coordinate system with the emission point as the center of the circle. In order to facilitate intuitive observation and calculation, it is converted into a rectangular coordinate system. Then, the RGB data and laser information are aligned by a joint calibration method based on the camera and the single-line laser, and finally a data set is made through data enhancement, that is, an RGB data set and a laser data set.

[0084] Step S202, constructing a deep learning network of an encoder-decoder structure, wherein the deep learning network includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process.

[0085] Step S203: According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor.

[0086] Specifically, this embodiment first builds an image segmentation network FNet based on RGB and laser on the basis of UNet. The overall network architecture is as follows: Figure 2 As shown in the figure, the network consists of two parts: the main module and the multi-source feature weighted fusion sub-network. The main module uses the encoder-decoder structure deep learning network as the basic skeleton to ensure that the model can retain more details during the training process.

[0087] The main module of this embodiment includes an RGB encoding branch (including four convolutional layers), a Laser encoding branch (including four convolutional layers) and a decoding process on the right (including four convolutional layers). Figure 2 As shown in the figure. In the two encoding branches on the left, each layer consists of a feature extraction module consisting of pooling, two groups of 3×3 convolutions, BN (Batch Normalization), and ReLU activation functions. Maximum pooling is used between the two layers for dimensionality reduction. The decoding process on the right adopts a similar strategy to the compression branch on the left. Each layer has upsampling, two groups of convolutions, BN and ReLU activation functions, with a total of four layers.

[0088] In this embodiment, an RGB image I=(i0, i1, i2) of size 256*256*3 is first input into the RGB encoding branch, and a laser image L=(l0) of size 256*256*1 is converted into a dense depth map S=(s0, s1, s2) of size 256*256*3 through a bilinear interpolation algorithm, and then input into the Laser encoding branch.

[0089] In these two encoding branches, four pooling dimension reduction (maximum pooling) and 3*3 convolution operations are performed at the same time, and finally the RGB feature tensors U0=(u 0,0 ,u 1,0 ,...,u c,0 ) and c*h*w laser characteristic tensor U1=(u 0,1 ,u 1,1 ,...,u c,1 ).

[0090] Step S204, constructing a multi-source feature weighted fusion sub-network at the last layer of the fusion branch, and based on the multi-source feature weighted fusion sub-network, performing weighted fusion on the RGB initial feature tensor and the laser initial feature tensor to obtain a fused feature tensor.

[0091] The embodiment of the present invention constructs a multi-source feature weighted fusion subnetwork at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, performs weighted fusion on the RGB initial feature tensor and the laser initial feature tensor to obtain the fused feature tensor, which specifically includes:

[0092] A multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused to obtain the fused feature tensor, including: performing feature compression on the RGB initial feature tensor and the laser initial feature tensor respectively to obtain the RGB compressed feature tensor and the laser compressed feature tensor; performing feature expansion on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain the RGB expanded feature tensor and the laser expanded feature tensor; performing convolution calculation and linear transformation on the RGB expanded feature tensor and the laser expanded feature tensor respectively to obtain the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor; based on the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused to obtain the fused feature tensor.

[0093] Specifically, in order to more efficiently fuse the information of the RGB encoding branch and the Laser encoding branch, a fusion branch is designed between the RGB branch and the Laser branch. In the last layer of the fusion branch, the embodiment of the present invention adds a multi-source feature weighted fusion sub-network (WF-SENET), such as Figure 2 The "WF" module in the Figure 3 shown.

[0094] In this embodiment, a multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused. The specific process of obtaining the fused feature tensor includes:

[0095] (1) Given an RGB feature tensor U0 and a laser feature tensor U1, let k = 0, 1. First, perform feature compression along the spatial dimension and convert the feature tensor U0 into k =(u 0,k ,u 1,k ,...,u c,k ) becomes the feature tensor Z k =(z 0,k ,z 1,k ,...,z c,k ), z c,k is a real number. k =(z 0,k ,z 1,k ,...,z c,k ) has a global receptive field to some extent, and the dimension of the output matches the number of input feature channels c. It represents the global distribution of responses on the feature channels, and enables the layers close to the input to also obtain the global receptive field. The specific operation is global flat pooling. Formula 1 converts the input feature U of size c*h*w into k =(u 0,k ,u 1,k ,...,u c,k ) is converted into c*1*1 output Z k =(z 0,k ,z 1,k ,...,z c,k ). The result of this step shows the numerical distribution of the c feature channels of this layer.

[0096]

[0097] u c,k Represents the cth feature channel in the kth feature tensor. sq Represents a feature compression operation. c,k Indicates u c,kThe result obtained after feature compression is a real number. i, j represents the element position in the two-dimensional matrix corresponding to the feature channel.

[0098] (2) The second operation is feature expansion. The result of the previous feature compression is Z k =(z 0,k ,z 1,k ,...,z c,k ) (Calculated by Formula 1), firstly, a fully connected layer is used to transform the Z of size c*1*1 k =(z 0,k ,z 1,k ,...,z c,k ) is transformed into a tensor Z′ of size c / r*1*1 k =(z′ 0,k ,z′ 1,k ,...,z′ c / r,k ), r is a scaling parameter, which is 16 in the embodiment of the present invention. The purpose of this parameter is to reduce the number of feature channels and thus reduce the amount of calculation. Then, through a similar fully connected layer operation, Z′ with a size of c / r*1*1 is converted k =(z′ 0,k ,z′ 1,k ,...,z′ c / r,k )Restore to Z″ of size c*1*1 k =(z″ 0,k ,z″ 1,k ,...,z″ c,k ), and finally pass through the sigmoid function to obtain the feature tensor S k =(s 0,k ,s 1,k ,...,s c,k )

[0099] s c,k =F ex (z c,k ,W)=σ(W2δ(W1z c,k )) (2)

[0100] F ex represents the feature expansion operation. σ and δ represent the activation function, S k =(s 0,k ,s 1,k ,...,s c,k ) represents the feature tensor obtained after feature expansion operation, with a size of c*1*1, recorded as feature weight. c,k W1 represents the feature weight of the cth channel in the kth feature tensor obtained by feature compression and feature expansion operations above, and W2 represents the parameters of the two fully connected layers.

[0101] (3) Next is the fusion operation, the input of which is the RGB feature weight S0=(s 0,0 ,s 1,0 ,...,s c,0 ) and Laser feature weight S1=(s 0,1 ,s 1,1 ,...,s c,1 ), the output is the fusion feature weight after adaptive weighted fusion Its size is c*1*1. First, a T-shaped convolutional template is designed to learn the feature weights S of multi-source information. k =(s 0,k ,s 1,k ,...,s c,k ) to adjust the weight relationship between feature tensors and feature channels, and obtain like Figure 4 As shown in the figure, the T-shaped convolution template can be regarded as a convolution of D×1 and a convolution of 1×W, where D depends on the neighborhood size d of the current channel (D=2d+1) and W represents the number of feature maps involved in feature fusion. The neighborhood size d is set to 1 in the embodiment of the present invention, that is, D=3, indicating that the current channel and two adjacent channels participate in the calculation. Finally, a linear transformation is performed through formula 4 to obtain

[0102]

[0103]

[0104] where s c,k Represents the feature weight of the cth feature channel in the kth feature tensor obtained by feature compression and feature expansion operations. Represents the result of the T-shaped convolution operation. express The result after linear transformation represents the final fusion feature weight of the cth feature channel. c,i and w c+j,k The initial value of is 1, W=2.

[0105] Among them, when performing T-shaped convolution operations, each feature channel involved in the calculation is regarded as a neuron, and the coefficient of each feature channel is regarded as the weight of the neuron. The specific training process is similar to the parameter training process of a general neural network:

[0106] 1. Initialize the weights;

[0107] 2. The input data is calculated through T-shaped convolution to obtain the output value;

[0108] s T =F T (s,w) (5)

[0109] Among them, F T Indicates T-shaped convolution operation, s T It represents the result of T-shaped convolution operation on s, and w represents the coefficient corresponding to each two-dimensional matrix in the feature tensor.

[0110] 3. Find the error between the network output value and the target value. The target value is the feature weight s obtained by concatenating the RGB feature tensor and the Laser feature tensor and inputting them into SENet. g ;

[0111]

[0112] s g It represents the feature weight output after the RGB feature tensor and the Laser feature tensor are concatenated and input into SENet. Loss represents the loss function during the training process of w.

[0113] 4. When the error is greater than our expected value, the error is passed back to the network, and the errors of the fully connected layer, downsampling layer, and convolutional layer are obtained in turn. The error of each layer can be understood as how much the network should bear for the total error of the network; when the error is equal to or less than our expected value, the training ends. The weights are updated according to the obtained error, and then the second step is entered. The weight update formula is:

[0114]

[0115] Among them, lr represents the learning rate, and Loss represents the loss function during w training.

[0116] Optionally, in the back propagation of the neural network, a loss function is needed to measure the error between the predicted output and the label value, so as to optimize the network parameters in the iterative process of training. The segmentation problem of blast furnace charge images is essentially a three-classification problem, that is, there are K = 3 label values, and the probability that the i-th sample is predicted to be the k-th label value is p i,k =Pr(t i,k =1), k = 0, 1, 2. That is, there are N samples in total, then the loss function of the data set is:

[0117]

[0118] where y i,k It represents the kth label value of sample i, the extracted target is 1, and the non-extracted target is 0; p i,kIt represents the probability that the kth label value of sample i is predicted as the target. The cross entropy function is non-negative. When the error between the actual output result and the expected output result is large, the value of the cross entropy loss function increases accordingly, the network weights are updated faster, and the model can converge quickly. When the error is small, the model network update rate can be reduced to improve the accuracy of the network model. However, due to the large difference in the number of pixels between the edge part and the non-edge part in the blast furnace charge image label, the positive and negative samples of the model training are unevenly distributed. When the number of samples of a certain category is too large, the category has a large proportion in the calculation of the loss function, which causes the model to tilt towards the category with a large number of samples during training, and the loss function cannot provide an effective learning signal. Therefore, based on the imbalance problem between the non-edge part and the edge part, the embodiment of the present invention, on the basis of the cross entropy loss function, the loss function of the image segmentation network is designed as:

[0119]

[0120]

[0121] where α i,k Represents the weight information of the kth label value of sample i, which is used to solve the sample imbalance problem in multi-classification problems. i,k It represents the number of pixels with label value k in the sample i image. The hyperparameter γ is used to solve the problem of imbalance between difficult and easy samples. In the embodiment of the present invention, γ is 2.

[0122] (4) The last step is to re-weight the feature weights output by the feature fusion operation. It is regarded as the importance of the feature tensor after feature selection, and then weighted to the previous feature tensor U by multiplication channel by channel. k =(u 0,k ,u 1,k ,...,u c,k ) to complete the recalibration of the original feature tensor in the channel dimension. It is learned through the previous fully connected layer and nonlinear layer, so it can be trained end-to-end. The function of these two fully connected layers is to fuse the feature tensor information of each channel. Feature tensor U k =(u 0,k ,u 1,k ,...,u c,k ) and fusion feature weights Multiply to get the weighted feature tensor U′ k =(u′ 0,k ,u′ 1,k ,...,u′ c,k ). Then all weighted feature maps U′ k =(u′ 0,k ,u′1,k ,...,u′ c,k ) are added to obtain the weighted fusion feature tensor U f =(u 0,f ,u 1,f ,...,u c,f ), as shown in Formula 5:

[0123]

[0124] u c,i Represents the original feature tensor U of the input k =(u 0,k ,u 1,k ,...,u c,k ) is the two-dimensional matrix of the c-th feature channel in .

[0125] Step S206, concatenating the fused feature tensor with the up-sampled result of the previous layer output of the multi-source feature weighted fusion subnetwork in the deep learning network.

[0126] Step S207, input the spliced ​​results into the decoding process of the deep learning network, obtain the segmented image through four bottom-up upsampling processes, and detect the particle size of the blast furnace sintered ore based on the segmented image.

[0127] Specifically, the fused feature tensor U0 and U1 are input into the multi-source feature weighted fusion subnetwork WF-SENet to obtain the fused feature tensor U f =(u 0,f ,u 1,f ,...,u c,f ), and then the fused feature tensor U of size c*h*w f =(u 0,f ,u 1,f ,...,u c,f ) is input to the first layer of the FNet decoding process, and through four upsampling and 3*3 convolution operations, and finally through 1*1 convolution operations to output the segmentation image U of size 256*256*3 s =(u 0,s ,u 1,s ,u 2,s ).

[0128] Key points of the embodiments of the present invention

[0129] (1) We innovatively integrated RGB information and single-line laser information (L) for blast furnace ore image segmentation and produced an RGB laser dataset. Based on UNet, we proposed a multi-scale feature fusion framework Fnet based on RGB-L, which integrated multi-scale features and weighted fusion features, and finally realized an end-to-end multi-classification semantic segmentation network based on RGB-L.

[0130] (2) A WF-SENet weight fusion module is proposed, and a T-shaped convolutional template is designed. By learning the relationship between features and channels, the weights of each feature map are recalibrated, the weight relationship between feature maps is adaptively adjusted, and the different contributions of input data to the segmentation results are quantitatively displayed. It can also greatly reduce the problem of negative contribution.

[0131] (3) To address the imbalance problem between the non-edge part and the edge part, based on the cross entropy loss function, the number of pixels in each category is used as a weight to adjust the sample ratio, and a loss function that can balance the sample ratio is designed.

[0132] The embodiment of the present invention takes blast furnace sintered ore as the research object and proposes a blast furnace ore image segmentation method based on weighted fusion of RGB and laser features. Compared with the existing methods, the method proposed in the present invention innovatively integrates RGB and laser information, fuses multi-source features through a new multi-source feature weighted fusion sub-network, gives full play to the supplementary role of laser, and quantitatively displays the difference in contribution of different features to the segmentation results during the fusion process through visualization, which is of great significance for subsequent research related to fusion and interdisciplinary studies.

[0133] Embodiment 3

[0134] This example uses a 2650m 3 A large blast furnace was used for verification, and the specific steps of the blast furnace ore image segmentation method based on weighted fusion of RGB and laser features are as follows:

[0135] 1. Data set preparation. In order to obtain RGB images and Laser detection data of blast furnace sintered ore, the present invention collects RGB image information and Laser depth information on the blast furnace feeding belt of a steel plant through hardware devices such as industrial cameras and laser scanners. Among them, the laser detection equipment is the LMS4111R-13000 laser line scanner produced by SICK, and the data acquisition software is SOPAS Engineering Tool. The industrial camera is the German Basler acA2500-14gmBasler industrial array camera data acquisition software is pylon Viewer. Both use Ethernet to communicate. A total of 8 groups of data are collected in a concentrated time period by industrial cameras and laser scanners (each time a group of data is collected, a batch of ore is replaced), each group of data includes 1000-2000 pictures (the acquisition time is different, the acquisition frequency is 60ms, which is only comparable to the laser scanning frequency. The acquisition frequency of laser scanning data will float between 55ms-65ms due to hardware reasons), and the picture size is 2592*2048. The laser scanning data contains 1000-2000 lines of laser messages, including some hardware information, ranging data and configuration data. The ranging data is the depth information we need. Then the registered data set is obtained through the joint calibration algorithm.

[0136] 2. The image segmentation network based on the weighted fusion of RGB and laser features is supervised training. In the two encoding branches, each layer consists of a feature extraction module consisting of 2*2 pooling, two groups of 3×3 convolutions, BN (Batch Normalization), and ReLU activation functions. The maximum pooling is used between the two layers for dimensionality reduction. The decoding process on the right adopts a strategy similar to the compression branch on the left. Each layer has upsampling, two groups of convolutions, BN and ReLU activation functions. During the training process, Adam is used as the optimization algorithm. The learning rate is set to 10^-4, and the cosine annealing learning rate scheduling strategy is used to adjust the learning rate, with the minimum value of 10^-6. The weight decay is set to 10^-8. The model capable of three-classification is obtained through training, aiming to divide the pixels in the blast furnace ore image into background, ore edge and ore internal area.

[0137] 3. Experimental results and analysis. Using the same data set on the same experimental platform, experiments were conducted based on U-Net, RFNet, the method of this embodiment (sparse input) and the method of this embodiment (dense input). U-Net, RFNet and the method of this embodiment all use the spatial information in the encoding process during upsampling. The difference is that UNet only uses RGB image information. The method of this embodiment proposes a multi-feature weight fusion module, which is applied to the weight fusion of RGB and Laser data, and proposes a multi-branch direct fusion module based on UNET. Therefore, this embodiment will compare and verify the relevant indicators of UNet, RFNet and the method of this embodiment, and evaluate the general accuracy of the segmentation effects of these methods as well as the accuracy of the ore area and quantity. This embodiment gives the segmentation results of the edge detection performance indicators mIoU and pwIoU of RFNet, U-Net, the method of this embodiment (sparse) and the method of this embodiment (dense) on eight samples in the test set, as shown below. Figure 5 As shown, Figure 5 From left to right are the original RGB image, laser image, dense depth map, UNet, RFNet, the method according to an embodiment of the present invention (sparse input), the method according to an embodiment of the present invention (dense input), and the true value.

[0138] The results show that the model proposed in this embodiment can effectively segment the blast furnace charge image, which is closer to the label result and has better effect. It can be seen from the two general segmentation indicators in Table 1 that the average IoU and pwIoU of the prediction results of the blast furnace charge image segmentation deep learning model (ILF-UNet) with multi-scale feature fusion proposed in this embodiment are 0.6419 and 0.8113 respectively, which are better than other deep learning models. Depth input is improved by 2.03% and 1.67% in mIoU compared with UNet and RFNet respectively.

[0139] Table 1 Segmentation performance of different methods

[0140]

[0141]

[0142] Reference Figure 6 The blast furnace sintered ore particle size detection system based on RGB and laser feature fusion proposed in the embodiment of the present invention includes:

[0143] A memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the processor 20 implements the steps of the blast furnace sintered ore particle size detection method based on RGB and laser feature fusion proposed in this embodiment when executing the computer program.

[0144] The specific working process and working principle of the blast furnace sinter ore particle size detection system based on RGB and laser feature fusion in this embodiment can refer to the working process and working principle of the blast furnace sinter ore particle size detection method based on RGB and laser feature fusion in this embodiment.

[0145] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting particle size of blast furnace sintered ore based on RGB and laser feature fusion, characterized in that: The method comprises: Collect RGB images of blast furnace sintered ore and single-line laser detection data on the blast furnace charge conveyor belt to obtain RGB data sets and laser data sets; Constructing a deep learning network of an encoder-decoder structure, wherein the deep learning network includes an RGB encoding branch, a laser encoding branch, a fusion branch, and a decoding process; According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor; A multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused to obtain a fused feature tensor, wherein a multi-source feature weighted fusion subnetwork is constructed at the last layer of the fusion branch, and based on the multi-source feature weighted fusion subnetwork, the RGB initial feature tensor and the laser initial feature tensor are weightedly fused to obtain a fused feature tensor, comprising: Performing feature compression on the RGB initial feature tensor and the laser initial feature tensor respectively to obtain an RGB compressed feature tensor and a laser compressed feature tensor; Performing feature expansion on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain an RGB expanded feature tensor and a laser expanded feature tensor; Performing convolution calculation and linear transformation on the RGB expanded feature tensor and the laser expanded feature tensor respectively to obtain a feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor; Based on the feature weights of each feature channel in the RGB initial feature tensor and the laser initial feature tensor, weighted fusion is performed on the RGB initial feature tensor and the laser initial feature tensor to obtain a fused feature tensor; The fused feature tensor is input into a decoding process to obtain a segmented image, and the particle size of the blast furnace sintered ore is detected according to the segmented image.

2. The blast furnace sintered ore particle size detection method based on RGB and laser feature fusion according to claim 1 is characterized in that: The RGB expanded feature tensor and the laser expanded feature tensor are respectively subjected to convolution calculation and linear transformation to obtain the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor, including: Each feature channel of the RGB expanded feature tensor and the laser expanded feature tensor is regarded as a neuron, and the channel coefficient of each feature channel is regarded as the weight of the neuron, and a neural network for training RGB channel coefficients and a neural network for training laser channel coefficients are established respectively; The channel coefficient of each feature channel of the RGB expanded feature tensor and the laser expanded feature tensor is trained to obtain the RGB channel coefficient and the laser channel coefficient; Based on the RGB expansion feature tensor and the RGB channel coefficient, an RGB convolution feature tensor is obtained; based on the laser expansion feature tensor and the laser channel coefficient, a laser convolution feature tensor is obtained; The RGB convolution feature tensor and the laser convolution feature tensor are linearly transformed to obtain the feature weights of each feature channel in the RGB initial feature tensor and the laser initial feature tensor.

3. The blast furnace sintered ore particle size detection method based on RGB and laser feature fusion according to claim 2 is characterized in that: Based on the RGB convolution feature tensor and the RGB channel coefficient, the RGB convolution feature tensor is obtained. Based on the laser convolution feature tensor and the laser channel coefficient, the calculation formula for obtaining the laser convolution feature tensor is: in, and Respectively represent the RGB convolution feature tensor and the laser convolution feature tensor, w c,0 and w c,1 They represent the channel coefficients of the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, and w c+j,0 and w c+j,1 They represent the channel coefficients of the c+jth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, and w c,0 、w c,1 、w c+j,0 and w c+j,1 The initial value of is 1, d represents the neighborhood size of the feature channel, which is 1, and D represents the number of feature channels involved in the calculation of a feature tensor in a T-shaped convolution operation, D = 2d + 1.

4. The blast furnace sintered ore particle size detection method based on RGB and laser feature fusion according to claim 3 is characterized in that: The RGB convolution feature tensor and the laser convolution feature tensor are linearly transformed to obtain the feature weight of each feature channel in the RGB initial feature tensor and the laser initial feature tensor: in, and They represent the feature weights of the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, and They represent the RGB convolution feature tensor and the laser convolution feature tensor respectively.

5. The blast furnace sintered ore particle size detection method based on RGB and laser feature fusion according to claim 1 or 4, characterized in that: Performing feature expansion on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain the RGB expanded feature tensor and the laser expanded feature tensor comprises: Performing a first fully connected layer operation on the RGB compressed feature tensor and the laser compressed feature tensor respectively to obtain a reduced-dimensional RGB compressed feature tensor and a reduced-dimensional laser compressed feature tensor; A second fully connected layer operation is performed on the RGB compressed feature tensor and the laser compressed feature tensor after dimension reduction, and the RGB expanded feature tensor and the laser expanded feature tensor are obtained through the sigmoid function.

6. The method for detecting particle size of blast furnace sintered ore based on RGB and laser feature fusion according to claim 5, characterized in that: The RGB initial feature tensor and the laser initial feature tensor are respectively subjected to feature compression, and the calculation formulas for obtaining the RGB compressed feature tensor and the laser compressed feature tensor are as follows: Among them, u c,0 and u c,1 represents the cth feature channel in the RGB initial feature tensor and the laser initial feature tensor, respectively, c,0 and z c,1 Respectively represent u c,0 and u c,1 The result after feature compression is a real number, F sq represents the feature compression operation, h and w represent the dimensions of the feature channels in the RGB initial feature tensor and the laser initial feature tensor, and i and j represent the element positions in the two-dimensional matrix corresponding to the feature channel; The RGB compressed feature tensor and the laser compressed feature tensor are respectively expanded to obtain the specific formulas of the RGB expanded feature tensor and the laser expanded feature tensor: s c,0 =F ex (z c,0 ,W)=σ(W2δ(W1z c,0 )) s c,1 =F ex (z c,1 ,W)=σ(W2δ(W1z c,1 )), Among them, F ex represents the feature expansion operation, s c,0 and c,1 They represent the cth feature channel in the RGB expanded feature tensor and the laser expanded feature tensor respectively, σ and δ represent the sigmoid activation function and the ReLU activation function respectively, and W1 and W2 represent the network parameters of the first fully connected layer and the second fully connected layer respectively.

7. The method for detecting particle size of blast furnace sintered ore based on RGB and laser feature fusion according to claim 6, characterized in that: According to the RGB encoding branch and the laser encoding branch, feature extraction is performed on the RGB data set and the laser data set respectively to obtain an RGB initial feature tensor and a laser initial feature tensor, including: According to the RGB encoding branch, performing pooling and convolution operations on the RGB data set to obtain an RGB initial feature tensor; interpolating the laser data set to obtain dense laser data; According to the laser encoding branch, pooling and convolution operations are performed on the dense laser data to obtain an initial laser feature tensor.

8. The method for detecting particle size of blast furnace sintered ore based on RGB and laser feature fusion according to claim 7, characterized in that: The fused feature tensor is input into the decoding process to obtain the segmented image, including: The fused feature tensor is concatenated with the up-sampled result of the previous layer output of the multi-source feature weighted fusion sub-network in the deep learning network; The spliced ​​results are input into the decoding process of the deep learning network, and the segmented image is obtained through four bottom-up upsampling processes.

9. A blast furnace sintered ore particle size detection system based on RGB and laser feature fusion, the system comprising: A memory (10), a processor (20), and a computer program stored in the memory (10) and executable on the processor (20), characterized in that the processor (20) implements the steps of any of the methods described in claims 1 to 8 when executing the computer program.

Images