A method, device and system for identifying blast furnace material types based on multivariate features of images

Abstract

The present invention discloses a blast furnace material type recognition method, device and system based on image multivariate features. The method collects high frame rate material transportation videos and converts the high frame rate material transportation videos into frame images. The entire area of ​​the frame image including the lateral area of ​​the material is selected as the material area image. The multivariate features of the material area image are extracted. The multivariate features include color features, texture features, grayscale features and material density distribution features. According to the multivariate features of the material area image, a classifier is trained. The blast furnace material type is recognized according to the trained classifier. The technical problem of low accuracy in the existing blast furnace material type recognition is solved. By using multivariate features to recognize the material type, the material type currently transported by the blast furnace charging belt can be accurately recognized. The recognition accuracy is high and the real-time performance is good. The system has great application value.

Description

Technical Field

[0001] The present invention mainly relates to the field of blast furnace material type detection, and in particular to a blast furnace material type recognition method and system based on multivariate image features. Background Art

[0002] The blast furnace consumes "reducing agents" to convert iron ore in the form of lump ore, sintered ore and pellets into molten iron, and the gangue and coke in the ore charge and the ash in the coal form slag. The charge and coke are transported to the silo, where the raw materials are stored, screened and weighed before finally being added to the blast furnace. The charge and coke are transported by independent belts, and after passing through the vibrating screen, the required charge and coke are obtained, and then fall into the weighing hopper. Once the set weight is reached, the screening is suspended. Once there is charge or coke in the weighing hopper and the valve of the weighing hopper is opened, it falls directly to the main feeding belt, and then is transported to the top of the blast furnace through the feeding belt, and then added to the blast furnace in sequence to form the iron ore layer and coke layer. The feeding method is layered mixed feeding, which is to prevent the pellets from slipping during the feeding process and cover the sintered ore on the pellets. However, this feeding method will cause the phenomenon of chasing materials, that is, the valves of sintered ore, pellets, and lump ore are closed soon, and the valve of coke is opened, and the interval time is too short. There are drawbacks in solving the problem of chasing materials on site. Use images to detect the type of materials on the current belt, judge whether it meets the type of materials currently loaded, check whether the wrong materials are loaded, and realize real-time detection of materials on the belt; detect whether there is mixing caused by chasing materials on the current belt, roughly calculate the quantity of each material, and get the approximate mixing ratio, so as to facilitate the timely discovery of chasing materials, play an early warning role, reduce manpower consumption, and provide guiding suggestions for feeding; at the same time, it provides a reliable basis for raw material balance and cost accounting, which is of great significance to reducing blast furnace energy consumption. Therefore, it is of great significance to timely and accurately identify the types of materials in the blast furnace feeding process.

[0003] The identification object of the present invention is the material transported by the feeding belt in the blast furnace feeding system. There are strong interference factors such as inevitable vibration and unevenly distributed dust at the detection site, and the ore is in a stacked state, which brings great challenges to the detection. At present, the existing methods for identifying the type of materials on the blast furnace feeding belt mainly include: traditional subjective identification methods and image recognition methods. Image recognition methods include: X-ray imaging recognition methods, laser imaging recognition methods, high-definition camera imaging recognition methods, etc.

[0004] The traditional subjective identification method is that the on-site staff judge the type of the current material by observing the materials transported by the blast furnace feeding belt. The accuracy of this identification method is affected by the amount of relevant knowledge and experience, and is easily disturbed by subjective consciousness, resulting in unstable accuracy. At the same time, the belt is in motion during transportation, and it is difficult for the staff to identify whether the current material is mixed. This method requires high time and labor costs, and is difficult to be widely used in the feeding process.

[0005] The X-ray imaging recognition method is to emit X-rays to the ore on the conveyor belt, then use the detector to transmit the signal to the computer, and then use the computer to obtain its grayscale image, combine the grayscale value and R value method, and finally use the machine learning algorithm to identify the ore. However, due to the harsh on-site environment and the radioactivity of X-rays, accurate recognition results cannot be obtained, and the cost is increased.

[0006] The laser imaging recognition method is to irradiate the laser beam on the ore to be identified on the conveyor belt, and obtain the return signal of the ore to be identified, process it to obtain the data information of the ore, and then use this data information to obtain a three-dimensional image of the ore, and then extract the texture and shape characteristics of the ore, and finally use a deep learning algorithm to identify the ore. However, the processing process of the three-dimensional image is slow and the real-time performance is poor, making it difficult to apply in practice.

[0007] The high-definition camera imaging recognition method uses a camera to shoot the ore on the conveyor belt, then pre-processes the ore image, extracts the shallow features of the ore such as grayscale, texture, and color, and uses classification algorithms or convolutional neural networks to achieve recognition. However, due to the large amount of dust and ore stacking on site, the accuracy of recognition is seriously affected.

[0008] Identifying the type of material transported by the feeding belt in the blast furnace feeding system is extremely challenging. There are few related patents and the existing patents have major defects.

[0009] Patent announcement number CN 203540946 U invention patent is an online ore sorting machine based on visual recognition technology, including a feeding system, a visual recognition system, a rejection and separation system and a control system. Its working principle is that the ore and gangue produced in the ore production are fed into the slide plate through the feeding conveyor, and the ore and gangue fall freely from the slide plate to the optical detection area, and then the detection camera of the visual recognition system is used to collect images of the ore and gangue, and the collected image signal is sent to the signal processing unit, and the signal processing unit processes and calculates the image signal, determines the difference between the ore image signal and the gangue image signal, and then determines the gangue to be rejected according to the parameters set by the control system. However, the sorting machine can only identify and classify two types of ores, and the number of detection cameras used is large, the cost is high, and the nozzle is prone to misoperation, resulting in low accuracy.

[0010] Patent publication number CN 106269564 A is an ore sorting process and equipment based on image recognition technology and a mechanical arm. Its working steps are: after the various ores are initially screened, a part of them enter the belt for gangue picking, and the image recognition system distinguishes the gangue or valuable ore in the gangue picking belt, and transmits the information to the controller. The controller controls the mechanical arm to perform the picking action. The mechanical arm picks out the gangue or valuable ore that needs to be separated according to the recognition result, and discharges it through the chute set on the outside of the mechanical arm. The ore sorting equipment described in this patent is targeted at gangue and coal, which has certain limitations, making the versatility of the equipment not high. Summary of the invention

[0011] The blast furnace material type recognition method and system based on multi-element image features provided by the present invention solve the technical problem of low accuracy in the existing blast furnace material type recognition.

[0012] In order to solve the above technical problems, the present invention proposes a blast furnace material type recognition method based on multivariate features of images, which includes:

[0013] Collect high frame rate material transportation videos and convert them into frame images;

[0014] Selecting an area in the frame image that includes the lateral area of ​​the material as a material area image;

[0015] Extract multivariate features of material region images, including color features, texture features, grayscale features, and material density distribution features;

[0016] According to the multivariate features of the material area image, the classifier is trained, and the blast furnace material types are identified based on the trained classifier.

[0017] Further, selecting an area in the frame image that includes the lateral area of ​​the material as the material area image includes:

[0018] The Canny operator is used to calculate the gradient information of each pixel in the frame image to obtain the material gradient image;

[0019] Based on the material gradient image, the image projection method is used to select the area in the frame image that includes the lateral area of ​​the material as the material area image.

[0020] Furthermore, based on the material gradient image, the image projection method is used to select the region in the frame image that includes the lateral region of the material as the material region image, including:

[0021] Calculate the pixel difference from the left and right edges of the material gradient image to the edge of the material respectively;

[0022] Obtain left and right interception coordinates of a material region image of the material image according to a preset number of pixel differences from the left and right edges of the material gradient image to the material edge, and obtain pixel width and pixel height of the material region image according to the left and right interception coordinates;

[0023] According to the material gradient image and the pixel width and pixel height of the material region image, the bottom edge line and the top edge line of the material region image on the material gradient image are calculated, and the calculation formulas of the bottom edge line and the top edge line are:

[0024]

[0025] y top =y bottom +h

[0026] Among them, y bottom and top They respectively represent the equations of the bottom line and the top line of the material area image on the material gradient image, H represents the height of the material gradient image, h represents the height of the material area image, k represents the slope of the bottom line, b represents the intercept of the bottom line, x1 and x2 represent the left and right intercept coordinates respectively.

[0027] The material area image is obtained according to the left and right interception coordinates, the bottom edge line straight line and the top edge line straight line of the material area image on the material gradient image.

[0028] Furthermore, after obtaining the material region image according to the left and right interception coordinates, the bottom edge line straight line and the top edge line straight line of the material region image on the material gradient image, the method further includes:

[0029] Perform multi-scale adaptive median filtering on the material area image.

[0030] Furthermore, the multivariate features of the extracted material region image include:

[0031] Extract the color features, texture features and grayscale features of the material area image;

[0032] The Unet network is used to train on medical cell images to obtain a pre-trained model;

[0033] The preprocessed material region image is used to produce a material segmentation label image to obtain a segmentation label data set;

[0034] The prepared segmentation label dataset is used to fine-tune the transfer learning parameters of the pre-trained Unet network to obtain a segmentation model suitable for blast furnace material images;

[0035] The segmentation model is used as an image segmentor to perform material segmentation on the acquired material region image to obtain a material segmentation image;

[0036] The number of material edge pixels in the material segmentation image with consistent scale is counted to obtain the density distribution characteristics of the material area image.

[0037] Furthermore, the color features, texture features and grayscale features of the extracted material region image include:

[0038] The material area image in the RGB color space is converted into the material area image in the HIS color space. According to the color characteristics between material categories, the first-order moment and second-order moment of the H color component, the ratio of the R component to the B component, and the average value of the distance between the pixel point and the black, white and gray line in the RGB color space are selected to form a color feature matrix;

[0039] Based on the material area image of HIS color space, the I component image is a grayscale image. The local binary pattern texture description method is used to extract the texture of the local adjacent area in the grayscale image. The gray level co-occurrence matrix is ​​used to extract the feature parameters of the obtained texture image, and contrast, energy, and entropy are selected to form a texture feature matrix.

[0040] Based on the grayscale image, the grayscale information of the grayscale image is extracted, and the first-order moment, second-order moment and entropy of the I component are selected to form a grayscale feature matrix.

[0041] Furthermore, according to the multivariate features of the material area image, a classifier is trained, and the blast furnace material type is identified according to the trained classifier, including:

[0042] Construct a multivariate feature set based on the color features, texture features, grayscale features and density distribution features of the material area image;

[0043] Perform feature dimensionality reduction on multivariate feature sets;

[0044] According to the multivariate feature set after dimension reduction, a fuzzy support vector machine classifier is trained;

[0045] The blast furnace material types are identified based on the trained fuzzy support vector machine classifier.

[0046] The blast furnace material type identification device based on multivariate image features provided by the present invention comprises a high-speed camera video capture module and a video preprocessing module, an image preprocessing module, a labeling module, a model training module and a result output module connected in sequence to the high-speed camera video capture module, wherein:

[0047] High-speed camera video capture module for collecting high frame rate material transportation videos;

[0048] The video preprocessing module is used to convert the high frame rate material transportation video into frame images and extract the region of interest of the frame images to obtain the material area image;

[0049] Image preprocessing module, used for multi-scale adaptive median filtering preprocessing of material area images;

[0050] The labeling module is used to classify and label the material area images preprocessed by multi-scale adaptive median filtering to obtain training sets and test sets;

[0051] A model training module is used to extract the color features, texture features, grayscale features and density distribution features of the images in the training set, and train the classifier based on the color features, texture features, grayscale features and density distribution features of the images in the training set;

[0052] The result output module is used to identify the images in the test set according to the model training module.

[0053] The blast furnace material type recognition system based on multi-element image features provided by the present invention comprises:

[0054] A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the blast furnace material type identification method based on multivariate image features provided by the present invention are implemented.

[0055] Compared with the prior art, the advantages of the present invention are:

[0056] The present invention provides a blast furnace material type recognition method, device and system based on image multivariate features, which collect high-frame rate material transportation videos and convert the high-frame rate material transportation videos into frame images, select the area in the frame image that includes the lateral area of ​​the material as the material area image, extract the multivariate features of the material area image, and the multivariate features include color features, texture features, grayscale features, and material density distribution features. According to the multivariate features of the material area image, a classifier is trained, and the blast furnace material type is recognized according to the trained classifier, which solves the technical problem of low accuracy in the existing blast furnace material type recognition. By using multivariate features to recognize the material type, the material type currently transported by the blast furnace charging belt can be accurately recognized, and the recognition accuracy is high and the real-time performance is good, which has great application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a structural block diagram of a blast furnace material type identification device based on multi-element features of an image according to an embodiment of the present invention;

[0058] Figure 2 This is a block diagram of the FPGA hardware platform structure of the second embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0060] Figure 4 It is a schematic diagram of the on-site installation of the blast furnace material type identification device according to an embodiment of the present invention;

[0061] Figure 5 The structure block diagram of the blast furnace material type recognition system based on multi-dimensional image features according to an embodiment of the present invention.

[0062] Reference numerals:

[0063] 10. Memory; 20. Processor. DETAILED DESCRIPTION

[0064] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively and meticulously below in conjunction with the accompanying drawings and preferred embodiments of the present invention, but the protection scope of the present invention is not limited to the following specific embodiments.

[0065] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0066] Embodiment 1

[0067] The first embodiment of the present invention provides a method for identifying blast furnace material types based on multivariate features of an image, comprising:

[0068] Step S101, collecting high frame rate material transportation videos and converting the high frame rate material transportation videos into frame images;

[0069] Step S102, selecting an area in the frame image that includes the lateral area of ​​the material as a material area image;

[0070] Step S103, extracting multivariate features of the material region image, where the multivariate features include color features, texture features, grayscale features, and material density distribution features;

[0071] Step S104, training a classifier according to the multivariate features of the material region image, and identifying the blast furnace material type according to the trained classifier.

[0072] The method for identifying blast furnace material types based on multivariate features of images provided in an embodiment of the present invention collects high-frame rate material transportation videos and converts the high-frame rate material transportation videos into frame images, selects an area in the frame image that includes the lateral area of ​​the material as a material area image, extracts multivariate features of the material area image, and the multivariate features include color features, texture features, grayscale features, and material density distribution features. According to the multivariate features of the material area image, a classifier is trained, and the blast furnace material types are identified according to the trained classifier, thereby solving the technical problem of low accuracy in the existing blast furnace material type identification. By using multivariate features to identify the material types, the material type currently transported by the blast furnace charging belt can be accurately identified, and the identification accuracy is high and the real-time performance is good, which has great application value.

[0073] Embodiment 2

[0074] The second embodiment of the present invention provides a method for identifying blast furnace species based on multivariate features of an image, comprising:

[0075] S1: Use a high-speed camera to capture the material transportation status of the blast furnace feeding belt and obtain a high-frame rate material transportation video;

[0076] S2: Convert the high frame rate material transportation video into a frame image, extract the ROI image area of ​​the frame image, and perform multi-scale adaptive median filtering on the ROI image area to obtain a preprocessed material area image;

[0077] S3: classify, identify and annotate the preprocessed material area images to obtain label images, and divide the label images into a training set and a test set;

[0078] S4: extracting the color features, texture features, grayscale features, and density distribution features of the images in the training set, and training the classifier based on the color features, texture features, grayscale features, and density distribution features of the images in the training set;

[0079] S5: According to the model training module, the images in the test set are recognized.

[0080] Specifically, step S1 is to install the high-speed camera perpendicular to the feeding belt using a "cross" type fixing device according to the specified high-speed camera parameters, and the high-speed camera lens is parallel to the feeding belt.

[0081] Step S2 specifically includes:

[0082] S21: Based on Opencv, convert the compressed video frames of the source video storage unit into .jpg image output. The width of the belt used by the feeding system to transport materials is D, which is used as a calibration object, and the pixel-level width of the belt in the video is D c , the pixel-level length of the belt in the video is L c, then the actual length of the belt L is:

[0083]

[0084] Then, according to the actual conveying speed V of the current belt in the video, the time T when the new scene appears in the video is calculated, that is:

[0085]

[0086] Therefore, the interval of converting the video frame into an image is set to T, thereby obtaining a frame image group to be processed.

[0087] S22: To achieve real-time measurement, it is necessary to increase the data processing speed, select the region of interest (ROI) in the frame image during the loading process, and reduce the data throughput. Based on the difference in surface smoothness between the material and the belt area, the Canny operator is used to calculate the gradient information of each pixel in the image; then, using the material gradient image, in order to ensure that the material region of interest is the entire lateral area containing the material, the image projection method is used to determine the coordinate position on both sides of the area, and obtain the pixel width and height of the ROI image. Assuming that the image size is H×W, calculate the pixel differences P1 and P2 from the left and right edges of the image to the edge of the material respectively. According to the size of P1 and P2 of most images, set the threshold X. When P≥X is satisfied, record the horizontal coordinates x1 and x2 at this time, which are the left and right interception coordinates of the material image ROI, so that the width w of the material image to be processed is obtained as follows:

[0088] w=|x1-x2| (3)

[0089] According to the needs of subsequent steps, set the height of the ROI image to:

[0090] h=w (4)

[0091] In the horizontal direction of the acquired material image, the center line of the image is used as the reference point, the ROI bottom edge line straight line is y=kx+b, and the selected material area is a horizontal area (k=0), so the bottom edge line straight line formula is as follows:

[0092]

[0093] Therefore, according to the height of the ROI image and its bottom line, its top line can be obtained as:

[0094] y top =y bottom +h (6)

[0095] Through the above steps, the material area image is obtained;

[0096] S23: Multi-scale adaptive median filtering is performed on the material area image to reduce noise and retain details. The adaptive filtering algorithm has a high computing speed and low energy consumption. It is used for online measurement and processing of blast furnace charging material images to facilitate real-time filtering of images. Considering that a small filter kernel has a better ability to retain details and a large filter kernel has a better ability to filter out noise, the present invention adopts small, medium and large scale adaptive median filtering to obtain images I after adaptive filtering at three scales. i (i=1,2,3), and then use the input image I0 to obtain three detail images D with different degrees of detail. i (i=1,2,3), as shown in formula (7):

[0097] D i =I i-1 -I i ,i=1,2,3 (7)

[0098] Then add the detail information to the original image I0 to get three new images I detaili (i=1,2,3). The above three images are decomposed into three channel images of R, G, and B respectively, and the variance of the image corresponding to each channel is calculated respectively. The corresponding weights of different channel images of each image are calculated according to formula (8), and then the images are fused into one image according to formula (9) to obtain the preprocessed material area image.

[0099]

[0100]

[0101] In some embodiments, before step S3, it also includes: performing data enhancement on the material area image to obtain a sufficient and effective data set through data enhancement; dividing the data set into a training set and a test set in step S3 using the StratifiedKFold method to ensure that the proportion of each category of material images in the training set and the test set is the same as that in the original data set.

[0102] In a specific embodiment, step S4 specifically includes:

[0103] S41: Build an algorithm to extract color features of material area images. Based on Opencv, convert the RGB color space image into the HIS color space. According to the color features between material categories, select the first-order moment and second-order moment of the H color component, the ratio of the R component to the B component, and the average value of the distance between the pixel point and the black, white and gray line in the RGB color space to form the color feature matrix F_Color;

[0104] S42: Construct an algorithm for extracting texture features of material area images. Based on the image HIS color space described in S41, where the I component image is a grayscale image, the texture of the local adjacent area in the grayscale image is extracted using the local binary pattern (LBP) texture description method, and then the grayscale co-occurrence matrix is ​​used to extract feature parameters from the obtained texture image, and contrast, energy, and entropy are selected to form a texture feature matrix F_Texture;

[0105] S43: Build an algorithm for extracting grayscale features of material area images. Based on the grayscale image described in S42, extract the grayscale information of the image, and select the first-order moment, second-order moment, and entropy of the I component to form a grayscale feature matrix F_Gray;

[0106] S44: Build an algorithm to extract the material density distribution features of regional images. Material density distribution is defined as the number of particles in the regional material image. For the materials transported on the blast furnace feeding belt, the material density distribution has certain differences, so the material density distribution is selected as one of the features for material identification and classification. The key to calculating the material density distribution is whether the material image is effectively segmented. Due to the characteristics of material ore stacking and adhesion and multiple surface textures in the current image, the existing edge segmentation algorithms such as the watershed algorithm, Sobel operator and other traditional methods have serious problems of over-segmentation and under-segmentation. Firstly, according to the similarity between the cell segmentation task of medical images and the characteristics of materials in the blast furnace charging process, the Unet network is used to train on the medical cell images to obtain a pre-trained model; secondly, the material image of the region of interest after preprocessing is used to produce the material segmentation label image; again, the pre-trained Unet network is used to fine-tune the transfer learning parameters using the prepared data set to obtain a segmentation model suitable for blast furnace material images; then the fine-tuned segmentation model is used as an image segmentor to perform material segmentation on the acquired material images to obtain segmented images of various materials; finally, the number of material edge pixels in the segmented images of different materials with the same scale is counted to obtain its density distribution feature F_Density as shown in Formula 3.

[0107]

[0108] Where M×N is the area of ​​the material image in the region of interest, and number is the total number of connected domains counted on the material segmentation image.

[0109] In a specific embodiment, step S5 specifically includes:

[0110] S51: Input the training set image into the above-mentioned feature extraction algorithm system to obtain a feature set with a dimension of m, that is, a feature set consisting of F_Color, F_Texture, F_Gray, and F_Density;

[0111] S52: In order to speed up the processing of valuable information of the image, avoid useless and redundant features, use easy-to-understand features, and improve the efficiency of the model, XGBoost is used to calculate the importance of m-dimensional features, and n-dimensional features are selected based on the difference in importance.

[0112] S53: Input the above n-dimensional feature set into the fuzzy support vector machine (FSVM) classifier, train and save the model.

[0113] In a specific embodiment, step S6 specifically inputs the test set into the above-mentioned feature extraction algorithm system to obtain the corresponding m-dimensional feature set composed of F_Color, F_Texture, F_Gray, and F_Density, and uses XGBoost to calculate the feature importance and filter the features to reduce its dimension to n dimensions, input the trained model, output the recognition and classification results, and calculate the two features with the highest scores based on the SelectKBest univariate feature selection method in Sklearn, and then visualize the features selected by the univariate feature selection method and the corresponding recognition and classification results based on matplotlib.

[0114] Reference Figure 1 The blast furnace material type identification device based on multi-element image features provided by the present invention comprises:

[0115] High-speed camera video capture module: including high-speed camera lens, video acquisition unit, real-time display unit and source video storage unit. The high-speed camera lens is equipped with a protective shell, which is a sealed box with glass, and a cleaning device is installed on the outside of the sealed box glass. During the feeding process, there is a lot of dust on the site, which affects the clarity of the high-speed camera lens and the service life of the high-speed camera body. For this reason, the high-speed camera must be installed with a sealed box with glass, and a cleaning device is installed on the outside of the glass to clean the dust on the glass surface in time. The cleaning device is similar to the wiper of a car. The material transportation status information of the feeding belt captured by the high-speed camera lens is transmitted to the video acquisition unit. The unit converts the analog signal of the high-frequency video source into a digital signal through an A / D converter, and then transmits it to the temporary storage on the board card. Then, the video compression chip on the video acquisition card is used to execute the compression algorithm to compress the huge video signal. Finally, the compressed video signal is transmitted to the local real-time display unit and the source video storage unit through the PCI bridge chip. When the video is transferred to the source video storage unit, the storage space of the temporary memory will be released.

[0116] Video preprocessing module: Based on Opencv, the compressed video frames of the source video storage unit are converted into .jpg images for output. According to the actual transportation speed of the current belt in the video and the pixel-level length of the belt in the video, the time T when a new scene appears in the video is calculated, so the interval frame of converting the video frame to the image is set to T, thereby obtaining the frame image group to be processed. Afterwards, in order to reduce data throughput and improve operating efficiency, the area in the image that includes the lateral area of ​​the material is extracted as the area of ​​interest, thereby obtaining the material area image, and the area image is transmitted to the image preprocessing module.

[0117] Image preprocessing module: In order to retain details and filter out noise, multi-scale adaptive median filtering is performed on the transmitted material area image, thereby retaining image details while reducing noise.

[0118] Labeling module: Label the preprocessed material area images for image classification and recognition to obtain training sets and test sets, and transfer the training sets to the model training module.

[0119] Model training module: including an algorithm system for extracting material image features, namely, a color feature extraction method, a texture feature extraction method, a grayscale feature extraction method, a material particle size segmentation algorithm, and a classification algorithm. The images in the training set are input into the algorithm system for extracting material image features to obtain a feature set corresponding to the image, and the dimension of the feature set is reduced to two dimensions by dimensionality reduction, so that one image corresponds to one feature set and one label, and the one-to-one corresponding features and labels are input into the classifier, the classifier is trained, and the model is saved.

[0120] Result output module: Input the images of the test set into the algorithm system for extracting material image features to obtain its feature set, and reduce the dimension of the feature set to two dimensions through dimensionality reduction. Input the feature set of the test set into the trained model to obtain the classification result; at the same time, visualize the reduced feature set and the corresponding classification result to obtain the distribution of each image feature, thereby displaying the current recognition and classification results in real time.

[0121] The video processing module in this embodiment is completely embedded in the FPGA hardware platform. Figure 2 As shown in the figure, the video captured by the high-speed camera is transmitted to the FPGA hardware platform through the CameraLink interface, and then the analog signal is converted into a digital signal through the A / D converter, and then the digital signal is temporarily stored in the FPGA hardware platform using the memory, and then it is compressed using the compression algorithm and transmitted to the LCD display unit and storage device through the PCI bridge chip. The power conversion module supplies power to the entire FPGA hardware platform; the fill light control module realizes the control of the light source, making the obtained video clearer.

[0122] Optionally, the present invention also provides an electronic device, comprising: one or more servers; a storage device for storing high frame rate video captured by a high-speed camera video capture module and its converted frame image data; a computer readable and writable storage medium for storing one or more programs, when the one or more programs are run by one or more servers, the one or more servers implement the method described in the first aspect.

[0123] The high-speed camera video capture module needs to be installed according to the determined installation parameters; and the video acquisition unit it contains includes an A / D converter, a board temporary memory, a video compression chip, and a PCI bridge chip. The video acquisition unit receives the video stream information captured by the high-speed camera, and converts the analog signal into a digital signal through the A / D converter. The board temporary memory is used to store the converted digital signal. The video compression chip compresses the digital signal in the memory, and the PCI bridge chip transmits the compressed digital signal to the real-time display unit and the source video storage unit; the real-time display unit it contains displays the material transportation status currently photographed by the high-speed camera on the screen; the source video storage unit it contains is a large-capacity storage device used to store high frame rate video.

[0124] The video acquisition unit is integrated on the FPGA (Programmable Gate Array) hardware platform. The FPGA module transmits and processes the real-time high frame rate video stream acquired by the high-speed camera through the high-speed video acquisition interface Camera Link. The compressed video stream is transmitted to the local real-time display unit and the source video storage unit through the PCI bridge chip. The real-time display unit is an LCD display; the source video storage unit is a storage device, which is a hard disk that stores information using magnetic energy.

[0125] The video preprocessing module, the labeling module, the image preprocessing module, the model training module, and the result output module are all implemented by programming, and one or more programming languages ​​can be used to write program codes that implement the functions of the above modules, and the program codes are embedded in the computer-readable and writable storage medium in the electronic device. Once the video stream is converted into a frame image by the video preprocessing module, the high frame rate video stream in the source video storage unit is cleared to prepare for subsequent video storage.

[0126] Reference Figure 3, a schematic diagram of the structure of the computer device (terminal device) of the electronic device proposed in the present invention, which includes an arithmetic unit (central processing unit CPU and graphics processing unit GPU), which can perform various appropriate operations and processes according to the program stored in the readable and writable storage medium, and can use the program to process the image data in the storage device. The input, output, arithmetic unit, readable and writable storage medium, and storage device are all controlled by the control device, wherein the input part includes high frame rate video data transmitted by the keyboard, mouse, PCI bridge chip, etc.; the output part includes speakers, computer display screen, etc.; the storage device includes a hard disk, etc.; the communication device includes a LAN card, a network interface card, etc.; the computer readable and writable storage medium is used to store the blast furnace material identification method described in the present invention, and the method can be fine-tuned according to the real-time application situation.

[0127] The blast furnace material identification method of the present invention is implemented by a computer software program, and the program can be implemented by one or more programming languages, such as Java, Python, C++, C, etc. The program is stored in a computer readable and writable storage medium, and a computer is used to read the program, and then read the data set in the storage device, and the program is executed on a local computer or a remote server connected to a network, and the output result is returned to the terminal device.

[0128] The method and device for identifying the type of materials in a blast furnace charging system based on multivariate features of images of the present invention use a high-speed camera to collect a video of the transportation status of materials on a charging belt in the blast furnace charging system; then convert the high frame rate video into a frame image array in .jpg format based on Opencv; then extract ROI, and use multi-scale adaptive median filtering to filter out noise and enhance details; then annotate the material area image to obtain an annotated image, and use the StratifiedKFold method to divide it into a training set and a test set; input the training set into a feature extraction algorithm system to obtain an m feature set consisting of F_Color, F_Texture, F_Gray, and F_Density, and use XGBoost to reduce the dimension of the feature set to n dimensions; input the reduced-dimensional features into FSVM, train and save the model; input the test set into the feature extraction algorithm to obtain the same feature set, and then reduce its dimension, input the reduced-dimensional features into the trained model, output the recognition and classification results, and use a univariate feature selection method to select the two-dimensional features with the highest scores and the corresponding labels for visualization using matplotlib. Compared with the method of using a single feature to identify the type of material, the present invention uses multiple features to identify the type of material, retains more comprehensive features, and achieves a higher recognition accuracy rate; compared with the separate operations of data acquisition, processing and recognition, the present invention integrates the acquisition, preprocessing and recognition of material images into the same model, with a faster processing speed and good real-time performance; compared with the laboratory environment, the present invention uses the material image data obtained on-site for model training and testing, which has stronger robustness. Therefore, this method can accurately identify the type of material currently transported by the blast furnace charging belt, with high recognition accuracy and good real-time performance, and has great application value.

[0129] Embodiment 3

[0130] The specific implementation scheme of the present invention is further described in conjunction with the accompanying drawings. The embodiment of the present invention is applied to a 2650m 3 On the feeding system of the blast furnace, on the feeding belt of the feeding system, Figure 4 Install high-speed cameras and other devices. In camera selection, the most important parameters are frame rate and resolution. A suitable frame rate can capture the transportation status of materials, and sufficient resolution can provide more detailed information on the image. To meet the above conditions, the industrial camera has a resolution of 1280×720 and a frame rate of 240.

[0131] Depend on Figure 4 It can be seen that the device includes a high-speed camera bracket, a high-speed camera protective shell, a high-speed camera cleaning device, a high-speed camera, a conveyor belt, materials to be identified, a video acquisition unit, an LCD display, a terminal device, and a server.

[0132] The high-speed camera is fixed on a "cross" device according to the installation parameters and field data, and the camera is calibrated to determine the relationship between the image coordinate system and the world coordinate system; the video acquisition unit uses the Camera Link interface to obtain the video information captured by the high-speed camera, and transmits it to the LCD display and the source video storage unit through the PCI bridge chip; the source video storage unit is the storage device in the electronic device; the terminal device reads the high frame rate video information stored in the source video storage unit through the storage area network (SAN); the terminal device and the server communicate through the network, wherein the network connection type can be wired, wireless or optical fiber cable, etc.

[0133] The user realizes the interaction between the terminal device and the server through the network to send or receive information, etc. The terminal device is hardware, which can be various electronic devices, including tablet computers, laptops, desktop computers, smart phones, etc. Various applications can be installed on the terminal device, such as image processing applications, file processing applications, etc.; the server refers to a server that can provide various services, such as: a file server that runs file processing software, a data server that processes image data uploaded by the terminal device, etc. The blast furnace material type identification method based on multi-dimensional image features is implemented by programming. The program is executed by a server, and each module in the blast furnace material type identification device is embedded in the terminal device. The terminal device is connected to the server through the network, and the server completes the execution and then transmits the result back to the terminal device.

[0134] Reference Figure 5 The blast furnace material type recognition system based on multi-element image features proposed in the embodiment of the present invention includes:

[0135] A memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the processor 20 implements the steps of the blast furnace material identification method based on multivariate image features proposed in this embodiment when executing the computer program.

[0136] The specific working process and working principle of the blast furnace material type recognition system based on multi-image features in this embodiment can refer to the working process and working principle of the blast furnace material type recognition method based on multi-image features in this embodiment.

[0137] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying blast furnace material types based on multivariate features of images, characterized in that: The method comprises: Collecting high frame rate material transportation videos and converting the high frame rate material transportation videos into frame images; Selecting an area in the frame image that includes the lateral area of ​​the material as the material area image, wherein selecting an area in the frame image that includes the lateral area of ​​the material as the material area image includes: The Canny operator is used to calculate the gradient information of each pixel in the frame image to obtain the material gradient image; Based on the material gradient image, an area in the frame image including the lateral area of ​​the material is selected as the material area image by using an image projection method, wherein based on the material gradient image, an area in the frame image including the lateral area of ​​the material is selected as the material area image by using an image projection method comprises: Calculate the pixel difference from the left and right edges of the material gradient image to the edge of the material respectively; Obtain left and right interception coordinates of a material region image of the material image according to a preset number of pixel differences from the left and right edges of the material gradient image to the material edge, and obtain pixel width and pixel height of the material region image according to the left and right interception coordinates; According to the material gradient image and the pixel width and pixel height of the material region image, the bottom edge line and the top edge line of the material region image on the material gradient image are calculated, and the calculation formulas of the bottom edge line and the top edge line are: y top =y bottom +h Among them, y bottom and top They respectively represent the bottom line and top line equations of the material area image on the material gradient image, H represents the height of the material gradient image, h represents the height of the material area image, k represents the slope of the bottom line, b represents the intercept of the bottom line, x1 and x2 represent the left and right intercept coordinates respectively; Obtaining a material region image according to the left and right interception coordinates, a bottom edge line straight line and a top edge line straight line of the material region image on the material gradient image; Extracting multivariate features of the material region image, wherein the multivariate features include color features, texture features, grayscale features, and material density distribution features; According to the multivariate features of the material area image, a classifier is trained, and the blast furnace material type is identified according to the trained classifier.

2. The blast furnace material identification method based on multivariate image features according to claim 1 is characterized in that: After obtaining the material area image according to the left and right interception coordinates, the bottom edge line straight line and the top edge line straight line of the material area image on the material gradient image, the method further includes: Multi-scale adaptive median filtering is performed on the material area image.

3. The blast furnace material identification method based on multivariate image features according to claim 1 or 2, characterized in that: Extracting the multivariate features of the material area image includes: Extracting color features, texture features and grayscale features of the material area image; The Unet network is used to train on medical cell images to obtain a pre-trained model; The preprocessed material region image is used to produce a material segmentation label image to obtain a segmentation label data set; The prepared segmentation label dataset is used to fine-tune the transfer learning parameters of the pre-trained Unet network to obtain a segmentation model suitable for blast furnace material images; The segmentation model is used as an image segmentor to perform material segmentation on the acquired material region image to obtain a material segmentation image; The number of material edge pixels in the material segmentation image with consistent scale is counted to obtain the density distribution characteristics of the material area image.

4. The blast furnace material identification method based on multivariate image features according to claim 3 is characterized in that: Extracting the color features, texture features and grayscale features of the material area image includes: The material area image in the RGB color space is converted into the material area image in the HIS color space. According to the color characteristics between material categories, the first-order moment and second-order moment of the H color component, the ratio of the R component to the B component, and the average value of the distance between the pixel point and the black, white and gray line in the RGB color space are selected to form a color feature matrix; Based on the material area image of HIS color space, the I component image is a grayscale image. The local binary pattern texture description method is used to extract the texture of the local adjacent area in the grayscale image. The gray level co-occurrence matrix is ​​used to extract the feature parameters of the obtained texture image, and contrast, energy, and entropy are selected to form a texture feature matrix. Based on the grayscale image, the grayscale information of the grayscale image is extracted, and the first-order moment, second-order moment and entropy of the I component are selected to form a grayscale feature matrix.

5. The blast furnace material identification method based on multivariate image features according to claim 4 is characterized in that: According to the multivariate features of the material area image, a classifier is trained, and the blast furnace material type is identified according to the trained classifier, including: Constructing a multivariate feature set according to the color features, texture features, grayscale features of the material region image and the density distribution features of the material region image; Performing feature dimensionality reduction on the multivariate feature set; According to the multivariate feature set after dimension reduction, a fuzzy support vector machine classifier is trained; The blast furnace material types are identified based on the trained fuzzy support vector machine classifier.

6. A blast furnace material identification system based on multivariate image features, the system comprising: A memory (10), a processor (20), and a computer program stored in the memory (10) and executable on the processor (20), characterized in that the processor (20) implements the steps of any of the methods described in claims 1 to 5 when executing the computer program.

Images