Ore area measurement method based on rotating ellipse target detection and related device
By adding a rotation angle prediction module in the YOLOv3 model and using the OpenCV library for elliptic calculation, the accuracy and safety of ore block detection in traditional methods are solved, and automated ore area measurement is achieved.
Patent Information
- Application Number
- CN202410976891.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional methods have strong randomness, low accuracy and safety risks in the detection of ore blocks, and it is difficult to accurately estimate the ore area based on semantic segmentation and target detection methods, especially in dense scenarios.
Using a method based on rotation ellipse object detection, the modified YOLOv3 model is constructed, and the rotation angle prediction module is added, combined with the OpenCV library for image processing and ellipse calculation, to achieve accurate estimation of ore area.
Automatic and accurate measurement of large-block ore area is achieved, improving the accuracy and safety of detection and reducing the risk of manual detection.
Smart Images

Figure CN118968002A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an ore area measurement method based on rotating ellipse target detection and a related device. Background Art
[0002] Ore is an important basic industrial raw material. The mining and mineral processing processes include blasting, crushing and flotation. Among them, the tool that connects different processes is the conveyor belt, which can transport the ore after the previous process to the initial position of the next process. It can be seen that belt conveying plays a vital role in the entire mining and mineral processing industry. However, there is a risk of blockage accidents due to the large size of the ore in this link. Blockage will damage the conveying equipment, bring safety hazards to the surrounding workers, and cause the entire conveying process to stop operating, which seriously affects the product quality, safety and efficiency of the mineral processing process.
[0003] The traditional method of manually detecting large pieces of material on a belt with a fast transmission speed has problems such as strong randomness, low accuracy and high risk factor, while the method based on machine vision can make up for the above shortcomings. In recent years, deep learning has gradually replaced the dominant position of traditional image algorithms in many fields including ore image analysis with its powerful automatic feature extraction capabilities. Among them, the technologies used for the identification and positioning of large pieces of ore are mostly semantic segmentation and target detection. However, when facing dense scenes such as ore, semantic segmentation is difficult to achieve good separation effects when predicting mutually adhered targets, and the horizontal box predicted by conventional target detection cannot accurately estimate the ore area because it contains more background. The ore area obtained based on these two visual technologies is larger than the true value. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides an ore area measurement method and related devices based on rotating ellipse target detection to achieve automatic and accurate area estimation of large ore blocks.
[0005] In order to solve the above technical problems, an embodiment of the present invention provides a method for measuring ore area based on rotating ellipse target detection, the method comprising:
[0006] Collect ore images above the conveyor belt at the mine site, and build a training dataset and a validation dataset based on the collected ore images;
[0007] A classic YOLOv3 model is constructed, and a prediction module for a rotation angle is added to a detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model;
[0008] The modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using the verification data set, and after the verification passes, a converged YOLOv3 model is obtained;
[0009] Obtaining image data of the ore to be predicted, inputting the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtaining predicted position parameter data;
[0010] Visualization processing is performed based on the OpenCV library and the predicted position parameter data, and the ore area is estimated and calculated using an ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
[0011] Optionally, the collecting of ore images located above a conveyor belt at a mine site, and constructing a training data set and a verification data set based on the collected ore images, includes:
[0012] Performing video acquisition and processing on the conveyed ore based on a high-definition camera device arranged on the conveyor belt to obtain an ore video stream;
[0013] Performing image frame extraction processing on the ore video stream according to a preset frame interval to obtain extracted image frames;
[0014] Based on the point prompt mechanism of the visual communication model, the extracted image frame is subjected to semantic segmentation processing, and the fitEllipse function in OpenCV is called to perform ore target detection marking processing on the semantic segmentation extracted image frame to generate a marked extracted image frame;
[0015] Filling the mark-extracted image frame into a square and scaling the original resolution to a preset resolution to obtain a mark-processed image frame;
[0016] Performing data enhancement processing on the marked processed image frame to obtain a marked enhanced image frame, wherein the data enhancement processing includes horizontal rotation and random rotation in an interval of 0 to 15 degrees;
[0017] The marked enhanced image frames are divided into a training data set and a verification data set according to a preset ratio.
[0018] Optionally, the classic YOLOv3 model includes a backbone network part for feature extraction, a neck network part for aggregating multi-scale features, and a detection head part for identifying and locating targets at multiple scales;
[0019] The backbone network is a DarkNet53 network, which can be divided into five stages from top to bottom according to the resolution, and is downsampled 32 times in total; the neck network is a Feature Pyramid Network structure, which is used to transmit deep semantic information upward step by step and fuse it with shallow detail information; each level of the Feature Pyramid Network structure includes upsampling, feature concatenation and feature fusion; the detection head part includes three detection heads, and the downsampling ratios of the three detection heads are 8, 16, and 32 respectively; each detection head consists of a CBL3 and a Convolution1 with a kernel size of 1.
[0020] Optionally, the result of modifying the output of the YOLOv3 model includes confidence, category, coordinates of the center point of the object, lengths of the minor axis and major axis of the object, and rotation angle of the object.
[0021] Optionally, the modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using a verification data set, and after the verification is passed, a converged YOLOv3 model is obtained, including:
[0022] The images in the training data set are sequentially input into the modified YOLOv3 model for training processing. During the training processing, the confidence loss and the category loss are calculated using a binary cross entropy function, the position regression loss is calculated using a mean square error loss function, and the node parameters in the modified YOLOv3 model are updated according to the confidence loss, the category loss and the position regression loss to obtain a trained YOLOv3 model;
[0023] The images in the verification data set are sequentially input into the trained YOLOv3 model for verification processing, and when the verification result is within a preset range, a converged YOLOv3 model is obtained;
[0024] When the verification result is not within the preset range, the back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and the training is returned until a converged YOLOv3 model is obtained.
[0025] Optionally, the step of inputting the ore image data to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain predicted position parameter data includes:
[0026] Filling the image data of the ore to be predicted according to preset squares to obtain the filled image data of the ore to be predicted;
[0027] The resolution scaling value of the filled ore image data to be predicted is preset for resolution processing to obtain the scaled ore image data to be predicted;
[0028] The scaled ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain predicted position parameter data, which includes the center point coordinates of the ore, the short axis length of the ore, the long axis length of the ore and the rotation angle of the ore.
[0029] Optionally, the visualization processing based on the OpenCV library and the predicted position parameter data, and the estimated calculation processing of the ore area using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore, includes:
[0030] The predicted position parameter data is characterized by an ellipse using the ellipse function in the OpenCV library, and the ore shape is approximately fitted according to the characterization result to generate a visible ore fitting ellipse;
[0031] The ellipse area calculation formula is used to perform estimated calculation processing on the visible ore fitting ellipse to obtain the estimated calculated area of the ore.
[0032] In addition, an embodiment of the present invention further provides an ore area measurement device based on rotating ellipse target detection, the device comprising:
[0033] Training data acquisition module: used to collect ore images located above the conveyor belt at the mine site, and build training data sets and verification data sets based on the collected ore images;
[0034] Model building module: used to build a classic YOLOv3 model, and add a prediction module for the rotation angle in the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model;
[0035] Model training module: used to train the modified YOLOv3 model using the training data set to generate a trained YOLOv3 model, and to verify the trained YOLOv3 model using the verification data set, and obtain a converged YOLOv3 model after the verification passes;
[0036] Parameter prediction module: used to obtain the image data of the ore to be predicted, input the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtain the predicted position parameter data;
[0037] Area estimation module: used for performing visualization processing based on the OpenCV library and the predicted position parameter data, and performing estimated calculation processing on the ore area using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
[0038] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement the ore area measurement method as described in any one of the above.
[0039] In addition, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program or code. When the computer program or code is executed by a processor, the ore area measurement method as described in any one of the above is implemented.
[0040] In an embodiment of the present invention, an image of ore located above a conveyor belt at a mine site is collected, and a training data set and a verification data set are constructed based on the collected ore images; a classic YOLOv3 model is constructed, and a prediction module for a rotation angle is added to the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model; the modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using the verification data set, and after the verification passes, a converged YOLOv3 model is obtained; ore image data to be predicted is obtained, and the ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on ore position parameters to obtain predicted position parameter data; visualization processing is performed based on the OpenCV library and the predicted position parameter data, and an estimated calculation processing is performed on the ore area using an ellipse calculation formula according to the visualization result to obtain an estimated calculated area of the ore; automatic and accurate area estimation of large pieces of ore is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 It is a flow chart of a method for measuring ore area based on rotating ellipse target detection in an embodiment of the present invention;
[0043] Figure 2 Schematic diagram of the structure of an ore area measurement device based on rotating ellipse target detection in an embodiment of the present invention;
[0044] Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] For example, see Figure 1 , Figure 1 It is a flow chart of a method for measuring ore area based on rotating ellipse target detection in an embodiment of the present invention.
[0047] like Figure 1 As shown, a method for measuring ore area based on rotating ellipse target detection comprises:
[0048] S11: Collect ore images located above the conveyor belt at the mine site, and construct a training data set and a verification data set based on the collected ore images;
[0049] In the specific implementation process of the present invention, the ore image located above the conveyor belt at the mine site is collected, and a training data set and a verification data set are constructed based on the collected ore image, including: based on the high-definition camera device arranged on the conveyor belt, a video acquisition process is performed on the conveyed ore to obtain an ore video stream; image frame extraction processing is performed in the ore video stream according to a preset frame interval to obtain an extracted image frame; semantic segmentation processing is performed on the extracted image frame based on the point prompt mechanism of the visual communication model, and the fitEllipse function in OpenCV is called to perform ore target detection marking processing on the semantically segmented extracted image frame to generate a marked extracted image frame; the marked extracted image frame is filled with a square and the original resolution is scaled to a preset resolution to obtain a marked processed image frame; the marked processed image frame is data enhanced to obtain a marked enhanced image frame, and the data enhancement processing includes horizontal rotation and random rotation in the interval of 0 to 15 degrees; the marked enhanced image frame is divided into a training data set and a verification data set according to a preset ratio.
[0050] Specifically, a high-definition camera is set on the conveyor belt, and real-time video of the ore transported on the conveyor belt is captured by the high-definition camera to obtain an ore video stream; image frame extraction and processing are performed in the ore video stream at a preset frame interval to obtain corresponding extracted image frames; it should be noted that the high-definition camera can be arbitrarily set at any position on the conveyor belt, and the number of frames between which an image is captured can be arbitrary, and the setting criterion is: select an appropriate interval frame number so that the captured two adjacent frames of images have a large difference, thereby ensuring the diversity of the acquired images.
[0051] Then, the semantic segmentation labels of the large pieces of ore in the extracted image frame are semi-automatically constructed through the point prompt mechanism of the Segment Anything Model (SAM), and then the "boundingRect", "minAreaRect" and "fitEllipse" functions in OpenCV are called to obtain the horizontal box / ellipse, rotated box and rotated ellipse labels surrounding the segmentation mask to form a marked extracted image frame; and the marked extracted image frame is filled with a square and the original resolution is scaled to a preset resolution to obtain a marked processed image frame, and the resolution is generally scaled from 1920×1920 to 256×256 to improve efficiency; then the marked processed image frame is data enhanced to obtain a marked enhanced image frame, and horizontal flipping and random rotation in the range of 0° to 15° are used for data enhancement to improve the overfitting problem caused by the small amount of data; finally, the marked enhanced image frame is divided into a training data set and a verification data set according to a preset ratio.
[0052] Among them, it should be noted that the data processing is not limited to the specific forms mentioned above. The data division method and proportion are arbitrary, the image preprocessing method is arbitrary, the annotation tools and data enhancement methods used are arbitrary, and the overall principle is: the choice of tools is mainly based on convenience and efficiency, and the preprocessing method and data enhancement method should be selected according to the characteristics of the data set, with the aim of improving the prediction accuracy of the model.
[0053] S12: constructing a classic YOLOv3 model, and adding a prediction module for the rotation angle to the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model;
[0054] In the specific implementation process of the present invention, the classic YOLOv3 model includes a backbone network part for feature extraction, a neck network part for aggregating multi-scale features, and a detection head part for identifying and locating targets at multiple scales;
[0055] The backbone network is a DarkNet53 network, which can be divided into five stages from top to bottom according to the resolution, and is downsampled 32 times in total; the neck network is a Feature Pyramid Network structure, which is used to transmit deep semantic information upward step by step and fuse it with shallow detail information; each level of the Feature Pyramid Network structure includes upsampling, feature concatenation and feature fusion; the detection head part includes three detection heads, and the downsampling ratios of the three detection heads are 8, 16, and 32 respectively; each detection head consists of a CBL3 and a Convolution1 with a kernel size of 1.
[0056] Furthermore, the result of modifying the output of the YOLOv3 model includes confidence, category, object center point coordinates, object minor axis and major axis lengths, and object rotation angle.
[0057] Specifically, the constructed classic YOLOv3 model mainly includes three parts, namely the backbone network for extracting features, the neck network for aggregating multi-scale features, and the detection head for identifying and locating targets at multiple scales. The backbone network is DarkNet53, which can be divided into 5 stages from top to bottom according to the resolution, with a total downsampling of 32 times. First, the network adjusts the number of input channels through a CBL3 (Convolution3, Batch Normalization, Leaky_ReLU) module with a kernel size of 3, and then extracts features through five different residual-like structures Rex_X. Each Rex_X includes a downsampling module CBL with a kernel size of 3 and a step size of 2. 3 / 2and a residual-like module Rex stacked X times. The Rex module contains a main path and a skip connection branch. The main path contains CBL1 for halving the number of feature channels and CBL3 for extracting features and restoring the number of channels. From top to bottom, the number of residual-like modules stacked after downsampling is 1, 2, 8, 8, and 4 at different stages. The neck network is a FPN (Feature Pyramid Network) structure, which can pass deep semantic information upward step by step and merge it with shallow detail information. Each level of the FPN structure contains three steps, namely upsampling, feature concatenation, and feature fusion. The deconvolution module TCBL2 / 2 (Transposed Convolution2 / 2, BatchNormalization, Leaky_ReLU) with a kernel size and stride of 2 is used to double the resolution of the deep feature map and halve the number of channels, and then it is concatenated with the feature map in the adjacent shallow encoder in the channel dimension. Finally, the feature fusion is performed through the ConvolutionSet composed of CBL1, CBL3, CBL1, CBL3, and CBL1. YOLOv3 includes three detection heads in total, with downsampling ratios of 8, 16, and 32 respectively. Each detection head consists of a CBL3 and a Convolution1 (Conv1) with a kernel size of 1. The output of the modified detection head of the YOLOv3 model includes confidence (Confidence, Con), category (Class, Cls), object center coordinates (X, Y), object short axis and long axis length (W, H), object rotation angle (Angel, A).
[0058] S13: Using the training data set to train the modified YOLOv3 model to generate a trained YOLOv3 model, and using the verification data set to verify the trained YOLOv3 model, and after the verification passes, obtaining a converged YOLOv3 model;
[0059] In the specific implementation process of the present invention, the modified YOLOv3 model is trained by using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified by using the verification data set, and after the verification is passed, a converged YOLOv3 model is obtained, including: inputting the images in the training data set into the modified YOLOv3 model in sequence for training processing, during the training processing, using a binary cross entropy function to calculate the confidence loss and the category loss, using a mean square loss error function to calculate the position regression loss, and updating the node parameters in the modified YOLOv3 model according to the confidence loss, the category loss and the position regression loss to obtain the trained YOLOv3 model; inputting the images in the verification data set into the trained YOLOv3 model in sequence for verification processing, and when the verification result is within a preset range, a converged YOLOv3 model is obtained; when the verification result is not within the preset range, back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and returning to train until a converged YOLOv3 model is obtained.
[0060] Specifically, the images in the training data set are sequentially input into the modified YOLOv3 model for training processing. During the training process, the confidence loss and the category loss are calculated by using the binary cross entropy function, and the position regression loss is calculated by using the mean square loss error function. The node parameters in the YOLOv3 model are updated and modified according to the confidence loss, the category loss and the position regression loss to obtain the trained YOLOv3 model; then the images in the verification data set are sequentially input into the trained YOLOv3 model for verification processing. When the verification result is within a preset range, a converged YOLOv3 model is obtained; when the verification result is not within the preset range, the back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and the training is returned until a converged YOLOv3 model is obtained.
[0061] The above training data set and test set and the same method are used to train U-Net, horizontal box YOLOv3, horizontal ellipse YOLOv3 and rotated box YOLOv3; it should be noted that the semantic segmentation model and target detection model used are not limited to the above specific algorithms, and the final adopted or improved model is arbitrary.
[0062] Use the trained models to predict the validation set and calculate the AP value. M (AveragePrecision for Mask / %) and AA(Average Area / pixel 2 ). Among them, AP MIt refers to the average accuracy of the mask. It means that no matter what form the prediction is, the label is a mask that can represent the complete shape of the ore. This indicator can reflect the separation of the adhering ore and the degree of consistency between the prediction result and the ore itself. AA refers to the average area of the ore obtained. AP M The calculation formula is as follows:
[0063]
[0064] The prediction results of various forms are compared with the mask labels by calculating the intersection-over-union ratio and comparing them with the set threshold to determine whether it is a true positive (TP) or a false positive (FP). The missed objects are false negatives (FN), where N is the number. TP 、N FP and N FN After the statistics are completed, the precision (Precision for Mask, Pre M / %) and recall rate (Recall for Mask, Re M / %), and then use the former to integrate the latter to get AP M The quantitative comparison results of the semantic segmentation model U-Net and four different forms of YOLOv3 on the validation set are shown in Table 1.
[0065] Table 1 Quantitative comparison results of different fitting methods in the validation set
[0066]
[0067] As shown in Table 1, whether from the perspective of APM or AApre, the object detection model YOLOv3 based on the rotated ellipse form achieved the best results, and its performance is significantly better than other expressions. In the APM indicator, it is 11.45% higher than the second-place semantic segmentation model U-Net, and in the AApre indicator, it is 4320.99pixel2 lower than the second-place rotated box YOLOv3.
[0068] S14: obtaining image data of the ore to be predicted, inputting the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtaining predicted position parameter data;
[0069] In the specific implementation process of the present invention, the ore image data to be predicted is input into the convergent YOLOv3 model for prediction processing of ore position parameters to obtain predicted position parameter data, including: filling the ore image data to be predicted according to a preset square to obtain the filled ore image data to be predicted; performing resolution scaling processing on the filled ore image data to be predicted according to a preset resolution value to obtain the scaled ore image data to be predicted; inputting the scaled ore image data to be predicted into the convergent YOLOv3 model for prediction processing of ore position parameters to obtain predicted position parameter data, wherein the predicted position parameter data includes the coordinates of the center point of the ore, the length of the short axis of the ore, the length of the long axis of the ore and the rotation angle of the ore.
[0070] Specifically, after obtaining the ore image data to be predicted, it needs to be processed according to corresponding rules, that is, first, the ore image data to be predicted is filled according to a preset square to obtain the filled ore image data to be predicted; then the resolution scaling value of the filled ore image data to be predicted is preset for resolution processing to obtain the scaled ore image data to be predicted; finally, the scaled ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain the predicted position parameter data, which includes the center point coordinates of the ore, the short axis length of the ore, the long axis length of the ore and the rotation angle of the ore.
[0071] S15: Performing visualization processing based on the OpenCV library and the predicted position parameter data, and performing estimated calculation processing on the ore area using an ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
[0072] In the specific implementation process of the present invention, the OpenCV library and the predicted position parameter data are visualized, and the ore area is estimated and calculated using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore, including: using the ellipse function in the OpenCV library to characterize the predicted position parameter data using an ellipse, and approximating the ore shape according to the characterization result to generate a visible ore fitting ellipse; using the ellipse area calculation formula to estimate the calculation of the visible ore fitting ellipse to obtain the estimated calculated area of the ore.
[0073] Specifically, the "ellipse" function in the OpenCV library is used to represent the predicted ore position parameters with an ellipse to approximately fit the ore shape and generate a visible ore fitting ellipse; the visible ore fitting ellipse is estimated and calculated using the ellipse area calculation formula to obtain the estimated calculated area of the ore, as follows: The ellipse area calculation formula S = π × W × H / 4 estimates the ore area; W is the length of the minor axis of the ore, and H is the length of the major axis of the ore.
[0074] In an embodiment of the present invention, an image of ore located above a conveyor belt at a mine site is collected, and a training data set and a verification data set are constructed based on the collected ore images; a classic YOLOv3 model is constructed, and a prediction module for a rotation angle is added to the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model; the modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using the verification data set, and after the verification passes, a converged YOLOv3 model is obtained; ore image data to be predicted is obtained, and the ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on ore position parameters to obtain predicted position parameter data; visualization processing is performed based on the OpenCV library and the predicted position parameter data, and an estimated calculation processing is performed on the ore area using an ellipse calculation formula according to the visualization result to obtain an estimated calculated area of the ore; automatic and accurate area estimation of large pieces of ore is achieved.
[0075] For example 2, please refer to Figure 2 , Figure 2 It is a schematic diagram of the structural composition of an ore area measurement device based on rotating ellipse target detection in an embodiment of the present invention.
[0076] like Figure 2 As shown, a device for measuring the area of ore based on rotating ellipse target detection, the device comprising:
[0077] Training data acquisition module 21: used to acquire ore images located above the conveyor belt at the mine site, and construct a training data set and a verification data set based on the acquired ore images;
[0078] In the specific implementation process of the present invention, the ore image located above the conveyor belt at the mine site is collected, and a training data set and a verification data set are constructed based on the collected ore image, including: based on the high-definition camera device arranged on the conveyor belt, a video acquisition process is performed on the conveyed ore to obtain an ore video stream; image frame extraction processing is performed in the ore video stream according to a preset frame interval to obtain an extracted image frame; semantic segmentation processing is performed on the extracted image frame based on the point prompt mechanism of the visual communication model, and the fitEllipse function in OpenCV is called to perform ore target detection marking processing on the semantically segmented extracted image frame to generate a marked extracted image frame; the marked extracted image frame is filled with a square and the original resolution is scaled to a preset resolution to obtain a marked processed image frame; the marked processed image frame is data enhanced to obtain a marked enhanced image frame, and the data enhancement processing includes horizontal rotation and random rotation in the interval of 0 to 15 degrees; the marked enhanced image frame is divided into a training data set and a verification data set according to a preset ratio.
[0079] Specifically, a high-definition camera is set on the conveyor belt, and real-time video of the ore transported on the conveyor belt is captured by the high-definition camera to obtain an ore video stream; image frame extraction and processing are performed in the ore video stream at a preset frame interval to obtain corresponding extracted image frames; it should be noted that the high-definition camera can be arbitrarily set at any position on the conveyor belt, and the number of frames between which an image is captured can be arbitrary, and the setting criterion is: select an appropriate interval frame number so that the captured two adjacent frames of images have a large difference, thereby ensuring the diversity of the acquired images.
[0080] Then, the semantic segmentation labels of the large pieces of ore in the extracted image frame are semi-automatically constructed through the point prompt mechanism of the Segment Anything Model (SAM), and then the "boundingRect", "minAreaRect" and "fitEllipse" functions in OpenCV are called to obtain the horizontal box / ellipse, rotated box and rotated ellipse labels surrounding the segmentation mask to form a marked extracted image frame; and the marked extracted image frame is filled with a square and the original resolution is scaled to a preset resolution to obtain a marked processed image frame, and the resolution is generally scaled from 1920×1920 to 256×256 to improve efficiency; then the marked processed image frame is data enhanced to obtain a marked enhanced image frame, and horizontal flipping and random rotation in the range of 0° to 15° are used for data enhancement to improve the overfitting problem caused by the small amount of data; finally, the marked enhanced image frame is divided into a training data set and a verification data set according to a preset ratio.
[0081] Among them, it should be noted that the data processing is not limited to the specific forms mentioned above. The data division method and proportion are arbitrary, the image preprocessing method is arbitrary, the annotation tools and data enhancement methods used are arbitrary, and the overall principle is: the choice of tools is mainly based on convenience and efficiency, and the preprocessing method and data enhancement method should be selected according to the characteristics of the data set, with the aim of improving the prediction accuracy of the model.
[0082] Model construction module 22: used to construct a classic YOLOv3 model, and add a prediction module for the rotation angle in the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model;
[0083] In the specific implementation process of the present invention, the classic YOLOv3 model includes a backbone network part for feature extraction, a neck network part for aggregating multi-scale features, and a detection head part for identifying and locating targets at multiple scales;
[0084] The backbone network is a DarkNet53 network, which can be divided into five stages from top to bottom according to the resolution, and is downsampled 32 times in total; the neck network is a Feature Pyramid Network structure, which is used to transmit deep semantic information upward step by step and fuse it with shallow detail information; each level of the Feature Pyramid Network structure includes upsampling, feature concatenation and feature fusion; the detection head part includes three detection heads, and the downsampling ratios of the three detection heads are 8, 16, and 32 respectively; each detection head consists of a CBL3 and a Convolution1 with a kernel size of 1.
[0085] Furthermore, the result of modifying the output of the YOLOv3 model includes confidence, category, object center point coordinates, object minor axis and major axis lengths, and object rotation angle.
[0086] Specifically, the constructed classic YOLOv3 model mainly includes three parts, namely the backbone network for extracting features, the neck network for aggregating multi-scale features, and the detection head for identifying and locating targets at multiple scales. The backbone network is DarkNet53, which can be divided into 5 stages from top to bottom according to the resolution, with a total downsampling of 32 times. First, the network adjusts the number of input channels through a CBL3 (Convolution3, Batch Normalization, Leaky_ReLU) module with a kernel size of 3, and then extracts features through five different residual-like structures Rex_X. Each Rex_X includes a downsampling module CBL with a kernel size of 3 and a step size of 2. 3 / 2and a residual-like module Rex stacked X times. The Rex module contains a main path and a skip connection branch. The main path contains CBL1 for halving the number of feature channels and CBL3 for extracting features and restoring the number of channels. From top to bottom, the number of residual-like modules stacked after downsampling is 1, 2, 8, 8, and 4 at different stages. The neck network is a FPN (Feature Pyramid Network) structure, which can pass deep semantic information upward step by step and merge it with shallow detail information. Each level of the FPN structure contains three steps, namely upsampling, feature concatenation, and feature fusion. The deconvolution module TCBL2 / 2 (Transposed Convolution2 / 2, BatchNormalization, Leaky_ReLU) with a kernel size and stride of 2 is used to double the resolution of the deep feature map and halve the number of channels, and then it is concatenated with the feature map in the adjacent shallow encoder in the channel dimension. Finally, the feature fusion is performed through the ConvolutionSet composed of CBL1, CBL3, CBL1, CBL3, and CBL1. YOLOv3 includes three detection heads in total, with downsampling ratios of 8, 16, and 32 respectively. Each detection head consists of a CBL3 and a Convolution1 (Conv1) with a kernel size of 1. The output of the modified detection head of the YOLOv3 model includes confidence (Confidence, Con), category (Class, Cls), object center coordinates (X, Y), object short axis and long axis length (W, H), object rotation angle (Angel, A).
[0087] Model training module 23: used to train the modified YOLOv3 model using the training data set to generate a trained YOLOv3 model, and to verify the trained YOLOv3 model using the verification data set, and obtain a converged YOLOv3 model after the verification passes;
[0088] In the specific implementation process of the present invention, the modified YOLOv3 model is trained by using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified by using the verification data set, and after the verification is passed, a converged YOLOv3 model is obtained, including: inputting the images in the training data set into the modified YOLOv3 model in sequence for training processing, during the training processing, using a binary cross entropy function to calculate the confidence loss and the category loss, using a mean square loss error function to calculate the position regression loss, and updating the node parameters in the modified YOLOv3 model according to the confidence loss, the category loss and the position regression loss to obtain the trained YOLOv3 model; inputting the images in the verification data set into the trained YOLOv3 model in sequence for verification processing, and when the verification result is within a preset range, a converged YOLOv3 model is obtained; when the verification result is not within the preset range, back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and returning to train until a converged YOLOv3 model is obtained.
[0089] Specifically, the images in the training data set are sequentially input into the modified YOLOv3 model for training processing. During the training process, the confidence loss and the category loss are calculated by using the binary cross entropy function, and the position regression loss is calculated by using the mean square loss error function. The node parameters in the YOLOv3 model are updated and modified according to the confidence loss, the category loss and the position regression loss to obtain the trained YOLOv3 model; then the images in the verification data set are sequentially input into the trained YOLOv3 model for verification processing. When the verification result is within a preset range, a converged YOLOv3 model is obtained; when the verification result is not within the preset range, the back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and the training is returned until a converged YOLOv3 model is obtained.
[0090] The above training data set and test set and the same method are used to train U-Net, horizontal box YOLOv3, horizontal ellipse YOLOv3 and rotated box YOLOv3; it should be noted that the semantic segmentation model and target detection model used are not limited to the above specific algorithms, and the final adopted or improved model is arbitrary.
[0091] Use the trained models to predict the validation set and calculate the AP value. M (AveragePrecision for Mask / %) and AA(Average Area / pixel 2 ). Among them, AP MIt refers to the average accuracy of the mask. It means that no matter what form the prediction is, the label is a mask that can represent the complete shape of the ore. This indicator can reflect the separation of the adhering ore and the degree of consistency between the prediction result and the ore itself. AA refers to the average area of the ore obtained. AP M The calculation formula is as follows:
[0092]
[0093] The prediction results of various forms are compared with the mask labels by calculating the intersection-over-union ratio and comparing them with the set threshold to determine whether it is a true positive (TP) or a false positive (FP). The missed objects are false negatives (FN), where N is the number. TP 、N FP and N FN After the statistics are completed, the precision (Precision for Mask, Pre M / %) and recall rate (Recall for Mask, Re M / %), and then use the former to integrate the latter to get AP M The quantitative comparison results of the semantic segmentation model U-Net and four different forms of YOLOv3 on the validation set are shown in Table 1.
[0094] Table 1 Quantitative comparison results of different fitting methods in the validation set
[0095]
[0096] As shown in Table 1, whether from the perspective of APM or AApre, the object detection model YOLOv3 based on the rotated ellipse form achieved the best results, and its performance is significantly better than other expressions. In the APM indicator, it is 11.45% higher than the second-place semantic segmentation model U-Net, and in the AApre indicator, it is 4320.99pixel2 lower than the second-place rotated box YOLOv3.
[0097] Parameter prediction module 24: used to obtain the image data of the ore to be predicted, input the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtain the predicted position parameter data;
[0098] In the specific implementation process of the present invention, the ore image data to be predicted is input into the convergent YOLOv3 model for prediction processing of ore position parameters to obtain predicted position parameter data, including: filling the ore image data to be predicted according to a preset square to obtain the filled ore image data to be predicted; performing resolution scaling processing on the filled ore image data to be predicted according to a preset resolution value to obtain the scaled ore image data to be predicted; inputting the scaled ore image data to be predicted into the convergent YOLOv3 model for prediction processing of ore position parameters to obtain predicted position parameter data, wherein the predicted position parameter data includes the coordinates of the center point of the ore, the length of the short axis of the ore, the length of the long axis of the ore and the rotation angle of the ore.
[0099] Specifically, after obtaining the ore image data to be predicted, it needs to be processed according to corresponding rules, that is, first, the ore image data to be predicted is filled according to a preset square to obtain the filled ore image data to be predicted; then the resolution scaling value of the filled ore image data to be predicted is preset for resolution processing to obtain the scaled ore image data to be predicted; finally, the scaled ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain the predicted position parameter data, which includes the center point coordinates of the ore, the short axis length of the ore, the long axis length of the ore and the rotation angle of the ore.
[0100] The area estimation module 25 is used to perform visualization processing based on the OpenCV library and the predicted position parameter data, and to perform estimated calculation processing on the ore area using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
[0101] In the specific implementation process of the present invention, the OpenCV library and the predicted position parameter data are visualized, and the ore area is estimated and calculated using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore, including: using the ellipse function in the OpenCV library to characterize the predicted position parameter data using an ellipse, and approximating the ore shape according to the characterization result to generate a visible ore fitting ellipse; using the ellipse area calculation formula to estimate the calculation of the visible ore fitting ellipse to obtain the estimated calculated area of the ore.
[0102] Specifically, the "ellipse" function in the OpenCV library is used to represent the predicted ore position parameters with an ellipse to approximately fit the ore shape and generate a visible ore fitting ellipse; the visible ore fitting ellipse is estimated and calculated using the ellipse area calculation formula to obtain the estimated calculated area of the ore, as follows: The ellipse area calculation formula S = π × W × H / 4 estimates the ore area; W is the length of the minor axis of the ore, and H is the length of the major axis of the ore.
[0103] In an embodiment of the present invention, an image of ore located above a conveyor belt at a mine site is collected, and a training data set and a verification data set are constructed based on the collected ore images; a classic YOLOv3 model is constructed, and a prediction module for a rotation angle is added to the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model; the modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using the verification data set, and after the verification passes, a converged YOLOv3 model is obtained; ore image data to be predicted is obtained, and the ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on ore position parameters to obtain predicted position parameter data; visualization processing is performed based on the OpenCV library and the predicted position parameter data, and an estimated calculation processing is performed on the ore area using an ellipse calculation formula according to the visualization result to obtain an estimated calculated area of the ore; automatic and accurate area estimation of large pieces of ore is achieved.
[0104] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for measuring the ore area of any one of the above embodiments is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.
[0105] An embodiment of the present invention further provides a computer application program, which runs on a computer and is used to execute the ore area measurement method of any one of the above embodiments.
[0106] also, Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.
[0107] The embodiment of the present invention further provides an electronic device, such as Figure 3 The electronic device includes a processor 302, a memory 303, an input unit 304, a display unit 305 and other devices. Those skilled in the art will understand that Figure 3 The structural components of the electronic device shown do not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 303 can be used to store the application 301 and various functional modules, and the processor 302 runs the application 301 stored in the memory 303, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both internal and external memories. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory disclosed in the present invention includes but is not limited to these types of memories. The memory disclosed in the present invention is only used as an example and not as a limitation.
[0108] The input unit 304 is used to receive the input of the signal and the keyword input by the user. The input unit 304 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 305 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 305 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 302 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 303, and calling the data stored in the memory.
[0109] As an embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more applications 301, wherein the one or more applications 301 are stored in the memory 303 and are configured to be executed by the one or more processors 302, and the one or more applications 301 are configured to execute the ore area measurement method in any one of the above embodiments.
[0110] In addition, the above provides a detailed introduction to the embodiments of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for measuring ore area based on rotating ellipse target detection, characterized in that: The method comprises: Collect ore images above the conveyor belt at the mine site, and build a training dataset and a validation dataset based on the collected ore images; A classic YOLOv3 model is constructed, and a prediction module for a rotation angle is added to a detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model; The modified YOLOv3 model is trained using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified using the verification data set, and after the verification passes, a converged YOLOv3 model is obtained; Obtaining image data of the ore to be predicted, inputting the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtaining predicted position parameter data; Visualization processing is performed based on the OpenCV library and the predicted position parameter data, and the ore area is estimated and calculated using an ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
2. The method for measuring the ore area according to claim 1, characterized in that: The method includes collecting ore images located above a conveyor belt at a mine site, and constructing a training data set and a verification data set based on the collected ore images, including: Performing video acquisition and processing on the conveyed ore based on a high-definition camera device arranged on the conveyor belt to obtain an ore video stream; Performing image frame extraction processing on the ore video stream according to a preset frame interval to obtain extracted image frames; Based on the point prompt mechanism of the visual communication model, the extracted image frame is subjected to semantic segmentation processing, and the fitEllipse function in OpenCV is called to perform ore target detection marking processing on the semantic segmentation extracted image frame to generate a marked extracted image frame; Filling the mark-extracted image frame into a square and scaling the original resolution to a preset resolution to obtain a mark-processed image frame; Performing data enhancement processing on the marked processed image frame to obtain a marked enhanced image frame, wherein the data enhancement processing includes horizontal rotation and random rotation in an interval of 0 to 15 degrees; The marked enhanced image frames are divided into a training data set and a verification data set according to a preset ratio.
3. The method for measuring ore area according to claim 1, characterized in that: The classic YOLOv3 model includes a backbone network part for feature extraction, a neck network part for aggregating multi-scale features, and a detection head part for identifying and locating targets at multiple scales; The backbone network is a DarkNet53 network, which can be divided into five stages from top to bottom according to the resolution, and is downsampled 32 times in total; the neck network is a Feature Pyramid Network structure, which is used to transmit deep semantic information upward step by step and fuse it with shallow detail information; each level of the Feature Pyramid Network structure includes upsampling, feature concatenation and feature fusion; the detection head part includes three detection heads, and the downsampling ratios of the three detection heads are 8, 16, and 32 respectively; each detection head consists of a CBL3 and a Convolution1 with a kernel size of 1.
4. The method for measuring ore area according to claim 1, characterized in that: The modified YOLOv3 model output includes the confidence level, category, object center coordinates, object minor axis and major axis lengths, and object rotation angle.
5. The method for measuring ore area according to claim 1, characterized in that: The modified YOLOv3 model is trained by using the training data set to generate a trained YOLOv3 model, and the trained YOLOv3 model is verified by using the verification data set, and after the verification is passed, a converged YOLOv3 model is obtained, including: The images in the training data set are sequentially input into the modified YOLOv3 model for training processing. During the training processing, the confidence loss and the category loss are calculated using a binary cross entropy function, the position regression loss is calculated using a mean square error loss function, and the node parameters in the modified YOLOv3 model are updated according to the confidence loss, the category loss and the position regression loss to obtain a trained YOLOv3 model; The images in the verification data set are sequentially input into the trained YOLOv3 model for verification processing, and when the verification result is within a preset range, a converged YOLOv3 model is obtained; When the verification result is not within the preset range, the back propagation and gradient descent algorithms are used to update the parameter weights in the trained YOLOv3 model, and the training is returned until a converged YOLOv3 model is obtained.
6. The method for measuring the ore area according to claim 1, characterized in that: The step of inputting the ore image data to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain the predicted position parameter data includes: Filling the image data of the ore to be predicted according to preset squares to obtain the filled image data of the ore to be predicted; The resolution scaling value of the filled ore image data to be predicted is preset for resolution processing to obtain the scaled ore image data to be predicted; The scaled ore image data to be predicted is input into the converged YOLOv3 model to perform prediction processing on the ore position parameters to obtain predicted position parameter data, which includes the center point coordinates of the ore, the short axis length of the ore, the long axis length of the ore and the rotation angle of the ore.
7. The method for measuring the ore area according to claim 1, characterized in that: The method of performing visualization processing based on the OpenCV library and the predicted position parameter data, and performing estimated calculation processing on the ore area using an ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore includes: The predicted position parameter data is characterized by an ellipse using the ellipse function in the OpenCV library, and the ore shape is approximately fitted according to the characterization result to generate a visible ore fitting ellipse; The ellipse area calculation formula is used to perform estimated calculation processing on the visible ore fitting ellipse to obtain the estimated calculated area of the ore.
8. A device for measuring the area of ore based on rotating ellipse target detection, characterized in that: The device comprises: Training data acquisition module: used to collect ore images located above the conveyor belt at the mine site, and build training data sets and verification data sets based on the collected ore images; Model building module: used to build a classic YOLOv3 model, and add a prediction module for the rotation angle in the detection head part of the classic YOLOv3 model to generate a modified YOLOv3 model; Model training module: used to train the modified YOLOv3 model using the training data set to generate a trained YOLOv3 model, and to verify the trained YOLOv3 model using the verification data set, and obtain a converged YOLOv3 model after the verification passes; Parameter prediction module: used to obtain the image data of the ore to be predicted, input the image data of the ore to be predicted into the converged YOLOv3 model to perform prediction processing on the ore position parameters, and obtain the predicted position parameter data; Area estimation module: used for performing visualization processing based on the OpenCV library and the predicted position parameter data, and performing estimated calculation processing on the ore area using the ellipse calculation formula according to the visualization result to obtain the estimated calculated area of the ore.
9. An electronic device comprising a processor and a memory, characterized in that: The processor runs the computer program or code stored in the memory to implement the ore area measurement method according to any one of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program or code, characterized in that: When the computer program or code is executed by a processor, the ore area measurement method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Remote sensing image small target detection method based on improved YOLOv3
CN113971764A
3D target detection method based on image and 4D millimeter wave radar fusion
CN117542010A
Infrared image insulator rotation detection method based on IoU prediction
CN117935089A
Cited By
Intelligent detection method and system for section ovality of medical wire
CN119600086A
Intelligent foreign matter detection system and dynamic threshold adaptive alarm method
CN120411844A
