Iron ore grade recognition method based on improved YOLOv4-Tiny algorithm

Abstract

The present invention discloses an iron ore grade identification method and device based on an improved YOLOv4‑Tiny algorithm, and relates to the technical field of grade identification and positioning. First, pictures of ores of different grades are collected, and data enhancement and expansion are performed on the collected ore pictures to obtain a sample set; an ore grade identification model is generated based on the improved YOLOv4‑Tiny network structure, and training is performed; the collected ore pictures of different grades are subjected to noise reduction processing and then input into the trained ore grade identification model, the category, coordinates, confidence and offset information of the ore grade target are extracted, and the detected image or video is output in real time to realize the identification of the ore grade. At the same time, the trained ore grade identification model is commercialized through the Lattepanda development board to obtain an ore grade identification device. The method and device solve the early problems of the inability to roughly judge the ore grade on site and automated mineral processing, and improve the accuracy of identification.

Description

Technical Field

[0001] The present invention relates to the technical field of grade identification and positioning, and in particular to an iron ore grade identification method based on an improved YOLOv4-Tiny algorithm. Background Art

[0002] The grade of iron ore has always been an important basis for iron ore companies to reduce production costs and increase profits. At present, the judgment of iron ore grade cannot be carried out in a timely and effective manner in both open-pit and underground mining. Experienced miners can roughly judge the approximate grade of iron ore, but this requires time to accumulate experience. In addition, manual judgment has a certain degree of subjectivity, which makes the judgment inaccurate. With the proposal of the 14th Five-Year Plan, deep learning combined with robotic arms and embedded devices to achieve automated mineral processing is also in line with the current development trend of mining automation and intelligence, and it also plays a role in promoting the Internet of Things chip industry. Therefore, a method that can roughly judge the grade of ore is needed to solve the above problems.

[0003] The Chinese patent "CN201910859933.9 A coal mine dynamic face recognition attendance method and system based on deep learning" provides a face recognition attendance method based on the Res50 residual network. This patent uses Res50 as the backbone feature extraction network, and the generated feature layer uses 8 scales and 3 ratios of anchor windows to identify the features of the face; a threshold is set during detection, and the detection of blurry images is skipped, and the features are processed using a normalized algorithm to improve the accuracy of the model. However, this method uses Res50 as the backbone feature extraction network, and the model's operation speed and parameter volume are relatively large, so it is unlikely to be used for embedded devices and there is no way to prevent explosions. For coal mines, it is impossible to use it underground, and the performance of Res50 as the backbone feature extraction network is relatively backward. In video processing, directly skipping blurry images will further slow down the running speed. Using image processing algorithms to enhance images is a better choice.

[0004] The Chinese patent "CN202011590566.6 A motorcycle helmet wearing detection method based on YOLOv4" uses the YOLOv4 algorithm and improves the non-maximum suppression algorithm to detect motorcycles and riders. This patent trains the algorithm through a specific data set, performs helmet ROI on the motorcycle and rider area, and uses a custom module for detection. It can count and record whether the motorcycle is overloaded and whether a helmet is worn. This method also has the above-mentioned problems, and the computing equipment required for YOLOv4 training is not available in general university laboratories. The update of the later model is a big problem, and this patent has calculation problems for statistics. The same motorcycle and rider may cause over-counting or under-counting. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an iron ore grade identification method based on an improved YOLOv4-Tiny algorithm in view of the deficiencies of the above-mentioned prior art. According to the characteristics of the grade of the iron ore (mainly texture and color, etc.), the backbone feature extraction network and the feature extraction network are structurally improved, the recognition effect of the neural network for low-level characteristics (texture color, etc.) is enhanced, the training of the network is accelerated by using Leaky-Relu and Early Stopping functions, and the keras model is converted into a tensorflow-lite model using a custom operator for embedded devices. The situation that artificial intelligence in the existing mining industry is limited to traditional computers is opened up, so that artificial intelligence can be productized, miniaturized, and generalized in the mining industry, and combined with the Internet of Things, embedded devices, single-chip microcomputers, etc., is more in line with the development of modern equipment.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: an iron ore grade identification method based on an improved YOLOv4-Tiny algorithm, comprising the following steps:

[0007] Collect pictures of ores of different grades, and perform data enhancement and expansion on the collected ore pictures to obtain a sample set;

[0008] Generate an ore grade recognition model based on the improved YOLOv4-Tiny network structure, and use the sample set for model training;

[0009] Perform noise reduction on the collected pictures of ores of different grades;

[0010] The trained ore grade recognition model is used to identify the ore grade.

[0011] Furthermore, the specific method of collecting pictures of ores of different grades and performing data enhancement and expansion on the collected ore pictures to obtain a sample set is:

[0012] Collect m pictures of iron ore and surrounding rocks of different grades from open-pit and underground iron ore mines, and label the pictures according to the iron ore category;

[0013] Use annotation software to annotate the collected images;

[0014] Flip, crop, and scale the annotated images to achieve data enhancement;

[0015] The data-augmented images are reduced to images with random length and width ratios, and the n reduced images are spliced ​​together, n<4, to form new images to obtain a sample set.

[0016] Furthermore, the ore grade recognition model generated based on the improved YOLOv4-Tiny network structure includes a CSPdarknet21_tiny backbone feature extraction network and an improved PANet simple bidirectional feature fusion network;

[0017] The CSPdarknet21_tiny backbone feature extraction network is used to identify the color and contour of the ore. The input image size is 832*832 and the structure is:

[0018] DarknetConv2D_BN_Leaky+Resblock_body*3+Resblock_body*2+Resblock_body*1+Resblock_body*1+Resblock_body*1

[0019] Among them, DarknetConv2D is a two-dimensional convolution module, BN stands for normalization, Leaky stands for the activation function is Leaky_relu, Resblock_body stands for the residual block, and Leaky_Relu is selected as the activation function of the network;

[0020] The improved PANet simple bidirectional feature fusion network selects the sizes of the three fused feature layers to be 104*104, 52*52 and 26*26 respectively, and the output feature layers are 52*52 and 26*26 respectively; the feature layer with a size of 104*104 is referred to as P1, the feature layer with a size of 52*52 is referred to as P2, and the feature layer with a size of 26*26 is referred to as P3. The feature fusion process of the improved PANet simple bidirectional feature fusion network is:

[0021] P3 is fused with P2 to obtain P4, which is then upsampled and fused with P1 to obtain P5. P5 is downsampled and then downsampled with P4 to obtain the first output feature layer P6 with a size of 52*52. P6 is downsampled again and fused with P3 to obtain the second output feature layer P7 with a size of 26*26. Finally, two output feature layers P6 and P7 are obtained.

[0022] Furthermore, the specific method of using the sample set for model training is:

[0023] Based on the idea of ​​transfer learning and loading the trained model weights, iterative training of the ore grade recognition model is performed:

[0024] (1) Early Stopping is used to control whether the iteration is terminated early. The model is saved and converted at the end of training, and finally two sizes of feature layers are generated for object detection.

[0025] (2) The data-enhanced and annotated training set is scaled to 832*832 in size with the aspect ratio unchanged and input into the ore grade recognition model for iterative calculation;

[0026] (3) Set the number of iterations, train the training set through the GPU, and generate a trained ore grade recognition model file.

[0027] Furthermore, the specific method for performing noise reduction processing on the collected pictures of ores of different grades is:

[0028] The ore images of different grades in the test set are subjected to noise reduction processing using mean filtering, Gaussian filtering or median filtering.

[0029] Furthermore, the specific method of using the trained ore grade recognition model to identify the ore grade is:

[0030] The images in the test set that have been processed by noise reduction are detected in real time to extract the category, coordinates, confidence and offset information of the ore grade target, and the detected images or videos are output in real time to realize the identification of ore grade.

[0031] Furthermore, the method also commercializes the trained ore grade identification model through the Lattepanda development board, which specifically includes two parts: model transplantation and platform testing;

[0032] The specific method of the model transplantation is:

[0033] The iron ore grade identification model of the present invention is not suitable for embedded devices, and the tensorflow library does not have the necessary operators for converting the model of the present invention. Therefore, in the conversion stage of the present invention, three custom operators are designed according to the requirements to convert the iron ore grade identification model of the present invention.

[0034] 1. Design three custom operators to convert the ore grade identification model according to the requirements; the first operator is used to compile various types of functions and data structures required by the improved YOLOv4-Tiny algorithm in Python and tensorflow environment: including the backend, backbone network structure, feature extraction network structure, activation function, loss function, data structure, input and output processing information used by the improved YOLOv4-Tiny algorithm;

[0035] The second operator is used to convert the ore grade recognition model generated under the Win10 system. The ore grade recognition models generated by training are all .h5 format models under the keras library. When using tensorflow-lite for conversion, even if the first operator is used, the conversion still cannot be performed. Therefore, the second operator is used to convert the keras .h5 model into a tensorflow model file;

[0036] The third operator is used to convert the model. It uses the function under the tensorflow-lite library to convert the tensorflow model file generated by the second operator, including the range of custom operators, the size of the input image of the ore grade recognition model, and conversion storage; finally, the keras .h5 model generated in the Win10 system is converted into a .tflite model file that can be used by embedded devices;

[0037] 2. Configure the Lattepanda development board system and the development environment, load the ore grade identification model program and the converted .tflite model file, and use the CPU computing power to identify the ore grade;

[0038] The specific method of the platform test is:

[0039] PYQT5 is used to design the device UI. Three buttons are set for image recognition, video recognition and camera real-time recognition. At the same time, display frames for ore sample images and recognized ore grade images are set. The image recognition and video recognition buttons are used to select ore pictures or video files and input the file path into the ore grade recognition program. The camera real-time recognition button connects the camera for real-time ore grade recognition.

[0040] Open the Lattepanda development board and click the three buttons on the UI interface to test the three functions and get the ore grade identification results.

[0041] The beneficial effects of the above technical solution are: the iron ore grade identification method based on the improved YOLOv4-Tiny algorithm provided by the present invention improves the structure of the backbone feature extraction network and feature extraction network of the YOLOv4-Tiny network according to the characteristics of the grade of the iron ore (mainly texture and color, etc.), enhances the recognition effect of the network for low-level characteristics (texture color, etc.), uses Leaky-Relu and Early Stopping functions to accelerate the training of the network, and converts the keras model into a tensorflow-lite model for embedded devices. It opens up the situation that artificial intelligence in the existing mining industry is limited to traditional computers, allowing artificial intelligence to be productized, miniaturized, and universalized in the mining industry, and is more in line with the development of modern equipment in combination with the Internet of Things, embedded devices, single-chip microcomputers, etc.

[0042] By using the improved YOLOv4-Tiny algorithm and optimizing the training process, the model training and prediction issues for computer performance are reduced, and the portability of the model is improved so that the model can be run on the Lattepanda development board, realizing the practical application of the model. The model solves the problem of being unable to roughly judge the ore grade and the early stage of automated beneficiation on site. At the same time, the image recognition accuracy for images with only one category or multiple categories can reach more than 90%. In addition, for ore stacking, mutual occlusion can also be well identified. The optimized YOLOv4-tiny deep learning network framework can also be ported to small portable devices. After testing, the video recognition efficiency on the Lattepanda development board exceeds 20FPS / S, further solving the problem that the mining area cannot perform cloud computing due to network problems. During the model training process, the shooting problems that can be encountered on site, such as light and noise, are removed and the brightness is balanced during testing and use, further enhancing the robustness of the model. The ore grade identification model of the present invention has good robustness and generalization for identifying different ore grades, and thanks to the miniaturization of the model, it can also be installed on small equipment for productization. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of an iron ore grade identification method based on an improved YOLOv4-Tiny algorithm provided in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the improved YOLOv4-Tiny network structure provided in an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the fusion process of the improved PANet simple bidirectional feature fusion network provided in an embodiment of the present invention;

[0046] Figure 4 The image preprocessing effect diagrams using different filtering methods provided in the embodiments of the present invention, wherein (a) is the original iron ore image, (b) is the mean filtering result, (c) is the Gaussian filtering result, and (d) is the median filtering result;

[0047] Figure 5 This is a diagram of the recognition results of different types of iron ore images in the test set provided by the embodiment of the present invention, wherein (a) is a rich magnetite ore, (b) is a poor magnetite ore, (c) is a rich hematite ore, and (d) is a poor hematite ore;

[0048] Figure 6 This is a UI interface diagram of the iron ore grade identification device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0050] In this embodiment, a method for identifying iron ore grade based on an improved YOLOv4-Tiny algorithm is provided. Figure 1 As shown, the following steps are included:

[0051] Step 1: Collect pictures of ores of different grades, perform data enhancement and expansion on the collected ore pictures to obtain a sample set; collect pictures of ores of different grades from open-pit and underground iron ore mines, and label the pictures according to the iron ore category, and then perform data enhancement and expansion on the collected ore pictures according to the actual situation to obtain a sample set, and divide the sample set into a training set, a validation set, and a test set according to a certain ratio. The specific method is as follows:

[0052] Step 1.1: Collect pictures of iron ore and surrounding rocks of different grades from open-pit and underground iron ore mines, and label the pictures by category;

[0053] In this embodiment, pictures of iron ore and surrounding rocks of two grades, magnetite and hematite, are collected. Each iron ore is divided into three categories: lean ore, rich ore and surrounding rock. There are 150-200 pictures for each category, and a total of 1131 pictures are obtained. The pictures are then classified, and the category numbers are represented by 01-06, specifically: magnetite: lean ore 01, rich ore 02, surrounding rock 03, hematite: lean ore 04, rich ore 05, surrounding rock 06;

[0054] Step 1.2: Use annotation software to annotate the collected images;

[0055] Step 1.3: Flip, crop and scale the annotated image to achieve data enhancement;

[0056] In this embodiment, the probability of image flipping is 50%, and the direction is uncertain (left, right, diagonal); 60%-80% of the image is randomly placed, the image size is kept unchanged, and the blank is filled with white; each image is scaled 0.2-0.8 times, and the image is kept unchanged and the remaining blank is filled with white.

[0057] Step 1.4: Image stitching: Reduce the data-enhanced image to an image with a random length and width ratio, and stitch the n reduced images together, n≤4, to form a new image and obtain a sample set;

[0058] In this embodiment, the original ore images are expanded from 1131 to 7756 images after data expansion, that is, the sample set includes 7756 ore images;

[0059] Step 1.5: Divide the sample set into training set, validation set and test set according to a certain ratio;

[0060] In this embodiment, 85% of the sample set is used as a training set, 5% is used as a validation set, and 10% is used as a test set.

[0061] Step 2: Generate an ore grade recognition model based on the improved YOLOv4-Tiny network structure and use the sample set for model training;

[0062] Step 2.1: Generate an ore grade recognition model based on the improved YOLOv4-Tiny network structure, which specifically includes the CSPdarknet21_tiny backbone feature extraction network and the improved PANet simple bidirectional feature fusion network, such as Figure 2 As shown;

[0063] The CSPdarknet21_tiny backbone feature extraction network is used to identify the color and contour of the ore. It uses a large resolution to enhance the extraction of shallow feature information (large resolution is also for detail information). That is, the overall structure focuses on the front end and simplifies the back end. Specifically, the input image size is 832*832, and the structure is:

[0064] DarknetConv2D_BN_Leaky+Resblock_body*3+Resblock_body*2+Resblock_body*1+Resblock_body*1+Resblock_body*1

[0065] Among them, DarknetConv2D is a two-dimensional convolution module, BN represents normalization, Leaky represents the activation function is Leaky_relu, Resblock_body represents the residual block, Leaky_Relu is selected as the activation function of the network, and the structural parameters are shown in Table 1;

[0066] Table 1 CSPdarknet21_tiny backbone feature extraction network structure parameters

[0067]

[0068]

[0069] The improved PANet simple bidirectional feature fusion network selects the sizes of the three fused feature layers to be 104*104, 52*52 and 26*26 respectively, and the output feature layers are 52*52 and 26*26 respectively; the feature layer with a size of 104*104 is referred to as P1, the feature layer with a size of 52*52 is referred to as P2, and the feature layer with a size of 26*26 is referred to as P3. The feature fusion process of the improved PANet simple bidirectional feature fusion network is as follows: Figure 3 As shown, specifically:

[0070] P3 and P2 are feature fused to obtain P4, which is then fused with P1 after upsampling to obtain P5. P5 is downsampled and then downsampled with P4 to obtain the first output feature layer P6 with a size of 52*52. P6 is downsampled again and fused with P3 to obtain the second output feature layer P7 with a size of 26*26. Finally, two output feature layers P6 and P7 are obtained.

[0071] Step 2.2: Based on the idea of ​​transfer learning and loading the trained model weights, iterative training of the ore grade recognition model is performed;

[0072] Disassemble and improve the shallow weights in the YOLOv4-Tiny network weight file to train the ore grade recognition model, and use Early Stopping to control whether the iteration is terminated early. Save the model and convert the model format at the end of training;

[0073] Because the small unit structure of the improved YOLOv4-Tiny network is the same as that of YOLO, the shallow weights in the weight file can be disassembled for use, and Early Stopping is used to control whether the iteration is terminated early. At the end of training, the save_model function in the code library tensorflow.keras.model is used to save the entire model (including network structure, activation function, weight information, iteration function, and various optimizers);

[0074] Step 2.3: The ore grade recognition model finally generates two sizes of feature layers for target detection, that is, detection at 26*26 and 52*52 sizes;

[0075] In this embodiment, six anchor boxes (preset boxes) are obtained by K-Means clustering at the feature layer of 26*26 and 52*52 sizes. The accuracy of the comparison between the final anchor box (preset box) and the grand truth (real box) is 79.8%;

[0076] Step 2.4: Scale the data-enhanced and annotated training set to 832*832 in size according to the unchanged aspect ratio and input it into the ore grade recognition model for iterative calculation;

[0077] In this embodiment, the iteration is set to 50 epochs, and the training set is trained through the GPU to generate a trained ore grade recognition model file. The final model stops training at 44 epochs, and the final loss value is 5.9;

[0078] In this embodiment, the final ore grade recognition model obtained has an accuracy of 94% and a loss value of 6.1 in the validation set test;

[0079] Step 3: Perform noise reduction on the collected ore images of different grades;

[0080] Image preprocessing is performed to solve the problem of image noise caused by insufficient fill light of the Lattepanda development board camera during field use. Mean filtering, Gaussian filtering or median filtering are used to reduce noise on images of different grades of ores collected in the test set or on site.

[0081] The three noise removal methods are compared: mean filtering, Gaussian filtering (Gaussian) and median filtering (median);

[0082] Among them, mean filtering (blur): use a 3*3 window for mean filtering. The principle is to add the eight points around the center point of the 3*3 window to get the average value, and then bring it into the center point;

[0083] Gaussian filtering (Gaussian): using Gaussian formula Generate a 3*3 Gaussian filter Filter the image.

[0084] Median filter: Also use a 3*3 window and take the median of the eight points near the center point instead of the value of the center point.

[0085] In this embodiment, by Figure 4 The filter effect comparison of the test set shown above shows that median filtering is selected as the image preprocessing method for recognition.

[0086] Step 4: Ore grade identification: Use the ore grade identification model trained in step 2 to perform real-time detection and identification on the images or videos in the test set or taken on site, extract the category, coordinates, confidence and offset information of the ore grade target, and output the detected image or video in real time to realize ore grade identification;

[0087] In this embodiment, 10% of the test set allocated in proportion in step 1 is extracted; the test set is tested using the ore grade recognition model generated in step 2, and the MPA value is calculated; MAP is the average value of multiple categories of ap, which is one of the key indicators for evaluating the model, ap is the area enclosed by the coordinate system of the x-axis coordinate of recall and the y-axis coordinate of precision, wherein the recall and precision are shown in the following formulas:

[0088] Precision=TP / (TP+FP)

[0089] Recall=TP / (TP+FN)

[0090] Among them, TP is the number of correctly classified positive samples; TN is the number of correctly classified negative samples; FP is the number of incorrectly classified positive samples (in fact, this sample is a negative sample); FN is the number of incorrectly classified negative samples (in fact, this sample is a positive sample).

[0091] In this embodiment, the recognition results for the ore images in the test set are as follows: Figure 5 shown.

[0092] The present invention also commercializes the trained ore grade recognition model through the Lattepanda development board, including two parts: model transplantation and platform testing;

[0093] The specific method of the model transplantation is:

[0094] The improved model of the present invention is not suitable for embedded devices, and the tensorflow library does not have the necessary operators for converting the model of the present invention. Therefore, in the final conversion stage, the present invention designs three custom operators according to the requirements to convert the model of the present invention.

[0095] The specific method of the model transplantation is:

[0096] 1. Design three custom operators to convert the ore grade identification model according to the requirements; the first operator is used to compile various types of functions and data structures required by the improved YOLOv4-Tiny algorithm in Python and tensorflow environment: including the backend, backbone network structure, feature extraction network structure, activation function, loss function, data structure, input and output processing information used by the improved YOLOv4-Tiny algorithm;

[0097] The second operator is used to convert the model generated under the Win10 system. The ore grade recognition models generated by training are all .h5 format models under the keras library. When using tensorflow-lite for conversion, even if the first operator is used, the conversion still cannot be performed. Therefore, the second operator is used to convert the keras .h5 model into a tensorflow model file;

[0098] The third operator is used to convert the model. It uses the function under the tensorflow-lite library to convert the model file generated by the second operator, including the range of the custom operator, the size of the input image of the ore grade recognition model, and the conversion storage. Finally, the keras .h5 model generated in the Win10 system is converted into a .tflite model file that can be used by embedded devices.

[0099] In this embodiment, the program codes of the three custom operators are specifically as follows:

[0100] #Import the required code library

[0101] import tensorflow as tf

[0102] import tensorflow.keras as keras

[0103] import h5py

[0104] import os

[0105] from OGIV1 import model

[0106] import functools

[0107] from tensorflow.python.keras.utils import CustomObjectScope,get_custom_objects

[0108] #Write a custom function for the keras (.h5) model (also called an operator, this part is the custom operator)

[0109] The first function is to inject the backbone network structure, feature extraction network structure, activation function, loss function, data format and processing method of the OGIV1 method compiled under tensorflow.keras.

[0110] #The second function is used to initialize the custom object. The above is the description of the structure. The second function is used to initialize this structure. The two functions correspond to the first operator.

[0111]

[0112]

[0113] #The following function officially starts the conversion work. First, the format of the model is converted, corresponding to the second operator;

[0114] #The first line of code calls the second function (that is, the custom operator), the second line gets the name of the model to be converted, the third line is the path when it is not converted, the fourth line loads the unconverted model, the fifth and sixth lines save the unconverted model in different formats, and the last line loads the model again

[0115] #tensorflow-lite library does not support directly using the model saved by keras. Although there are examples in the library, it still reports an error. Therefore, the keras model is loaded and then converted into a tensorflow model and then loaded for subsequent conversion and use, and the necessary operators are initialized at the same time.

[0116] init_keras_custom_objects()

[0117] keras_model_name = 'OGIV1.h5'

[0118] keras_model_path=os.path.join('keras_models',keras_model_name)

[0119] save_model=tf.keras.models.load_model(keras_model_path)

[0120] export_dir='save'

[0121] tf.saved_model.save(save_model,export_dir)

[0122] new_model=tf.saved_model.load(export_dir)

[0123] #The following function is the formal conversion model, corresponding to the third operator;

[0124] #Different from the functions provided by the official library of tensorflow-lite, the third operator defined in this invention adds a range demarcation, otherwise the conversion model will report an error;

[0125] #Convert within the scope where the operator is defined:

[0126]

[0127]

[0128] #Call the function to convert and then save

[0129] converter.optimizations=[tf.lite.Optimize.OPTIMIZE_FOR_SIZE]

[0130] converter.allow_custom_ops=True

[0131] tflite_model=converter.convert()

[0132] open("OGIV1.tflite","wb").write(tflite_model)

[0133] 2. Configure the Lattepanda development board system and the development environment, load the ore grade identification model program and the converted .tflite model file, and use the CPU computing power to identify the ore grade;

[0134] The specific method of the platform test is:

[0135] Use PYQT5 for device UI design, such as Figure 6 As shown, three buttons are specifically set, which are used for: image recognition, video recognition and camera real-time recognition. At the same time, display frames of ore sample images and recognized ore grade images are set; among them, the image recognition and video recognition buttons are used to select ore pictures or video files and input the file path into the ore grade recognition program, and the camera real-time recognition button connects the camera for real-time ore grade recognition;

[0136] Open the Lattepanda development board and click the three buttons on the UI interface to test the three functions and get the ore grade identification results.

[0137] In this embodiment, two prepared videos are used for video detection. It is detected that the recognition speed of the ore grade recognition model on the Lattepanda development board can reach more than 20FPS / S, and the MAP value is 73.11.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for iron ore grade identification based on an improved YOLOv4-Tiny algorithm, characterized in that: Collect pictures of ores of different grades, and perform data enhancement and expansion on the collected ore pictures to obtain a sample set; Generate an ore grade recognition model based on the improved YOLOv4-Tiny network structure, and use the sample set for model training; Perform noise reduction on the collected pictures of ores of different grades; Use the trained ore grade recognition model to identify the ore grade; The ore grade recognition model generated based on the improved YOLOv4-Tiny network structure includes a CSPdarknet21_tiny backbone feature extraction network and an improved PANet simple bidirectional feature fusion network; The CSPdarknet21_tiny backbone feature extraction network is used to identify the color and contour of the ore. The input image size is 832*832 and the structure is: DarknetConv2D_BN_Leaky+Resblock_body*3+Resblock_body*2+Resblock_body*1+Resblock_body*1+Resblock_body*1 Among them, DarknetConv2D is a two-dimensional convolution module, BN stands for normalization, Leaky stands for the activation function is Leaky_relu, Resblock_body stands for the residual block, and Leaky_Relu is selected as the activation function of the network; The improved PANet simple bidirectional feature fusion network selects the sizes of the three fused feature layers to be 104*104, 52*52 and 26*26 respectively, and the output feature layers are 52*52 and 26*26 respectively; the feature layer with a size of 104*104 is referred to as P1, the feature layer with a size of 52*52 is referred to as P2, and the feature layer with a size of 26*26 is referred to as P3. The feature fusion process of the improved PANet simple bidirectional feature fusion network is: P3 and P2 are feature fused to obtain P4, which is then fused with P1 after upsampling to obtain P5. P5 is downsampled and then downsampled with P4 to obtain the first output feature layer P6 with a size of 52*52. P6 is downsampled again and fused with P3 to obtain the second output feature layer P7 with a size of 26*26. Finally, two output feature layers P6 and P7 are obtained. The trained ore grade recognition model is commercialized through the Lattepanda development board, including model transplantation and platform testing; The specific method of the model transplantation is:

1. Design three custom operators to convert the ore grade identification model according to the requirements; the first operator is used to compile various types of functions and data structures required by the improved YOLOv4-Tiny algorithm in Python and tensorflow environment: including the backend, backbone network structure, feature extraction network structure, activation function, loss function, data structure, input and output processing information used by the improved YOLOv4-Tiny algorithm; The second operator is used to convert the ore grade recognition model generated under the Win10 system, converting the keras .h5 model into a tensorflow model file; The third operator is used to convert the model. It uses the function under the tensorflow-lite library to convert the model file generated by the second operator, including the range of the custom operator, the size of the input image of the ore grade recognition model, and the conversion storage. Finally, the keras .h5 model generated in the Win10 system is converted into a .tflite model file that can be used by embedded devices.

2. Configure the Lattepanda development board system and the development environment, load the ore grade identification model program and the converted .tflite model file, and use the CPU computing power to identify the ore grade; The specific method of the platform test is: PYQT5 is used to design the device UI. Three buttons are set for image recognition, video recognition and camera real-time recognition. At the same time, display frames for ore sample images and recognized ore grade images are set. The image recognition and video recognition buttons are used to select ore pictures or video files and input the file path into the ore grade recognition program. The camera real-time recognition button connects the camera for real-time ore grade recognition. Open the Lattepanda development board and click the three buttons on the UI interface to test the three functions and get the ore grade identification results.

2. The iron ore grade identification method based on the improved YOLOv4-Tiny algorithm according to claim 1 is characterized in that: The specific method of collecting pictures of ores of different grades and performing data enhancement and expansion on the collected ore pictures to obtain a sample set is: Collect m pictures of iron ore and surrounding rocks of different grades from open-pit and underground iron ore mines, and label the pictures according to the iron ore category; Use annotation software to annotate the collected images; Flip, crop, and scale the annotated images to achieve data enhancement; The data-augmented images are reduced to images with random length and width ratios, and the n reduced images are spliced ​​together, n ≤ 4, to form new images to obtain a sample set.

3. The iron ore grade identification method based on the improved YOLOv4-Tiny algorithm according to claim 2 is characterized in that: The specific method of using the sample set for model training is: Based on the idea of ​​transfer learning and loading the trained model weights, iterative training of the ore grade recognition model is performed: (1) Early Stopping is used to control whether the iteration is terminated early. The model is saved and converted at the end of training, and finally two sizes of feature layers are generated for object detection. (2) The data-enhanced and annotated training set is scaled to 832*832 in size with the aspect ratio unchanged and input into the ore grade recognition model for iterative calculation; (3) Set the number of iterations, train the training set through the GPU, and generate a trained ore grade recognition model file.

4. The iron ore grade identification method based on the improved YOLOv4-Tiny algorithm according to claim 3 is characterized in that: The specific method for performing noise reduction processing on the collected ore images of different grades is as follows: The images of ore with different grades collected in the test set or on site are subjected to noise reduction processing using mean filtering, Gaussian filtering or median filtering.

5. The iron ore grade identification method based on the improved YOLOv4-Tiny algorithm according to claim 3 is characterized in that: The specific method of using the trained ore grade recognition model to identify the ore grade is as follows: The images collected in the test center or on-site after noise reduction are tested in real time to extract the category, coordinates, confidence and offset information of the ore grade target, and the tested images or videos are output in real time to realize the identification of ore grade.

Images