Modal instance segmentation method and device for accumulated ore rock particles based on occlusion perception

Abstract

The invention relates to the technical field of image recognition and prediction, and provides a modal instance segmentation method and device for accumulated ore and rock particles based on occlusion perception. The method comprises the following steps: building ore rock particle accumulation models in different accumulation states, and acquiring an original image database by using a camera; performing a series of processing on the original image data; building an AGG-Mask2Former network by means of a Pytorch framework, and defining a multi-task loss function to optimize a neural network training process; the database is divided into multiple classes of sub-training sets, a neural network is trained to obtain an optimal model, and cross validation experiments are carried out to research recognition and prediction results of the model in different shielding states; and extracting the ore rock particles identified and predicted by the model, and predicting the grading of the ore rock particles in a stacking state. According to the method, the efficiency and precision of the neural network in the training, recognition and prediction process in the field of particle recognition of the ore rock particle accumulation state are greatly improved, and a more reliable data basis is provided for subsequent parameter extraction and gradation statistics.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and prediction, and in particular to a modal instance segmentation method and device for accumulated ore particles under occlusion perception. Background Art

[0002] In mining engineering and related engineering fields, the system of accumulated ore and rock particles can be seen everywhere. These ore and rock particles play a vital role in their flow, transportation, separation and flotation. In addition, their geometric parameters directly affect the blasting parameters selected in the blasting process such as ore crushing and secondary crushing. Therefore, the intelligent identification and geometric feature detection of ore and rock particles are of great significance in optimizing engineering processes, improving production efficiency and improving ore processing processes.

[0003] Traditionally, the geometric detection method of rock particles is to randomly select test samples for manual or machine screening tests. Although the manual measurement method may have high accuracy, it is inefficient and costly. With the rapid development of computer vision technology, particle recognition methods based on image segmentation technology cover a variety of methods, including threshold segmentation, watershed algorithm, region growing method, edge detection method, machine learning method and deep learning method. However, the rock particles at the engineering site have complex occurrence conditions and are limited by equipment, resulting in poor image quality captured by the camera. This leads to certain challenges for existing traditional algorithms and machine learning methods, which are prone to over-segmentation and under-segmentation problems. In addition, it is difficult to achieve large-scale automation and batch processing, resulting in low overall recognition and geometric detection efficiency. In contrast, instance segmentation in deep learning methods has shown good performance in the field of rock particle recognition. It can not only identify rock particles of different types and individuals, but also segment each rock particle and extract its relevant geometric dimensions.

[0004] Since rock particles have various colors and textures, and often occlude each other in a stacked state, neural networks face great challenges. Not only is the accuracy of identifying rock particles always low, but the occluded parts cannot be accurately processed, which reduces the accuracy of parameter extraction results after identification. However, there are few studies on the complete identification of stacked rock particles under occlusion perception, and there are problems such as insufficient and in-depth research, which makes it difficult to meet the requirements of accurate, efficient and complete identification and prediction of rock particles. Summary of the invention

[0005] In order to solve the existing problems that there are few studies on the complete identification of accumulated rock particles under occlusion perception, and there are problems such as insufficient and in-depth research, which makes it difficult to meet the technical problems of accurately, efficiently and completely identifying and predicting rock particles, the embodiment of the present invention provides a modal instance segmentation method and device for accumulated rock particles under occlusion perception. The technical solution is as follows:

[0006] On the one hand, a modal instance segmentation method based on accumulated mineral particles under occlusion perception is provided, and the method is implemented by a modal instance segmentation device based on accumulated mineral particles under occlusion perception, and the method includes:

[0007] S1. Building a rock particle accumulation model under different accumulation states, and obtaining an original image database through the rock particle accumulation model;

[0008] S2, performing image data processing on the original image database to obtain processed image data;

[0009] S3. Build the AGG-Mask2Former network based on the Pytorch framework; define a multi-task loss function to optimize the AGG-Mask2Former network training process;

[0010] S4, dividing the processed image data into multiple sub-training sets; training the AGG-Mask2Former network through multiple sub-training sets to obtain the optimal model; obtaining the recognition and prediction results of the model under different occlusion states through cross-validation experiments;

[0011] S5. Perform feature extraction on the mineral rock particles identified and predicted by the model, predict the gradation of the mineral rock particles in the stacked state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

[0012] Optionally, S1, building a rock particle accumulation model under different accumulation states, and obtaining original image data through the rock particle accumulation model, including:

[0013] Arrange the ore and rock particles in a preset stacking state under uniform illumination to obtain a stacking model of ore and rock particles in different stacking states;

[0014] The original image database is obtained by shooting with a camera, and the original image database is divided into three categories of databases, including: the first category of database used for neural networks to learn the basic information of mineral rock particles; the second category of database used for neural networks to learn and predict the occluded part of mineral rock particles; the third category of database used for neural network training, testing, and subsequent in-depth exploration of the model's recognition and prediction results of mineral rock particles under different occlusion states.

[0015] Optionally, S2, performing image data processing on the multi-class sub-training sets to obtain processed image data, includes:

[0016] Obtain image data from the second and third databases, and provide multiple subsidiary images based on each image data; wherein the subsidiary images are images obtained by shooting at the same height and angle after removing particles above the obscured mineral particles visible in the original image;

[0017] Data processing is performed on the image data in the three types of databases. The data processing includes: data preprocessing, data labeling, data enhancement and data post-processing to obtain processed image data.

[0018] Optionally, data labeling includes: completely labeling the blocked parts in the mineral particles; using the SAM segmentation large model or Labelme software to label the first type of database through SAM; and manually labeling the second and third types of databases through Labelme software.

[0019] Optionally, S3 builds the AGG-Mask2Former network based on the Pytorch framework, including:

[0020] Based on the Mask2Former neural network, graph neural network (GNN) and generative adversarial network (GAN) are introduced; including: introducing GNN after the backbone network extracts image features to capture the relationship between features; adding GNN after the Transformer decoder outputs the detection results to analyze the occlusion relationship (occlusion and occlusion) between mineral rock particles; adding GAN and GNN after Mask2Former outputs the detection and segmentation results to generate a mask of the occluded part for each occluded mineral rock particle.

[0021] Optionally, in S3, a multi-task loss function is defined to optimize the neural network training process, including:

[0022] The calculation formula of the multi-task loss function MLoss is defined as follows:

[0023] MLoss=L cls +L box +L mask +L amodal (1)

[0024] Where: L cls is the classification loss for target monitoring, L box is the regression loss of the bounding box, L mask To predict the mask loss, L amodal To predict the occlusion mask loss.

[0025] Optionally, S4, dividing the processed image data into multiple sub-training sets; training the AGG-Mask2Former network through the multiple sub-training sets to obtain an optimal model; and obtaining recognition and prediction results of the model under different occlusion states through a cross-validation experiment, including:

[0026] The processed image data is divided into multiple sub-training sets, and the neural network is trained by the multiple sub-training sets to obtain the optimal model; among them, the three types of databases are randomly divided into training sets and test sets in a ratio of 7:3;

[0027] The three types of databases are divided into multiple sub-training sets for cross-validation according to different research significances;

[0028] Among them, the multiple sub-training sets include: class a sub-training set, class b sub-training set and class c sub-training set; the class a sub-training set contains all the training set data in the first, second and third categories of databases, which is used to train and obtain the final model; the class b sub-training set contains all the training set data in the first and second categories of databases, and is tested with the test set of the third category of database, which is used to study the recognition of different occlusions by the verification model; the class c sub-training set contains all the training set data in the first and third categories of databases, and is tested with the test set of the second category of database, which is used to study the recognition of real accumulation by the verification model.

[0029] Optionally, S5, extracting features of the ore and rock particles identified and predicted by the model, predicting the gradation of the ore and rock particles in the stacked state based on the feature extraction results, and completing the modal instance segmentation of the stacked ore and rock particles based on occlusion perception, including:

[0030] Extract features of the mineral rock particles identified and predicted by the model, and extract relevant geometric parameters including particle size and area;

[0031] The function FindContours in the OpenCV library is used to obtain the contour of the mask image, the area of ​​the contour is calibrated using the library function ContourArea, and the basic parameters such as the minimum circumscribed oblique rectangle are obtained through the function MinAreaRect. The Feret shortest diameter of the mask contour is defined as the equivalent particle size L of the ore particle according to the following formula (2);

[0032]

[0033] Where: S is the area of ​​the contour, L max is the longest diameter of Ferre;

[0034] The volume of the rock particles is used to replace the mass of the rock particles to calculate the gradation of the rock particles; the gradation is the mass fraction of the rock particles with different particle sizes in the whole rock particles; the equivalent volume V of the rock particles is calculated according to the following formula (3), different particle size intervals are divided according to different equivalent particle sizes, and the cumulative rock particle volumes in different intervals are calculated to obtain the gradation prediction of the accumulated rock particles under occlusion perception by the optimized and trained AGG-Mask2Former network:

[0035]

[0036] On the other hand, a modal instance segmentation device based on accumulated mineral particles under occlusion perception is provided, and the device is applied to a modal instance segmentation method based on accumulated mineral particles under occlusion perception, and the device includes:

[0037] The original image acquisition module is used to build a rock particle accumulation model under different accumulation states and obtain the original image database through the rock particle accumulation model;

[0038] An image processing module is used to process the original image database to obtain processed image data;

[0039] Model building module, used to build AGG-Mask2Former network based on Pytorch framework; define multi-task loss function to optimize the AGG-Mask2Former network training process;

[0040] The model training and verification module is used to divide the processed image data into multiple sub-training sets; train the AGG-Mask2Former network through multiple sub-training sets to obtain the optimal model; and obtain the recognition and prediction results of the model under different occlusion states through cross-validation experiments;

[0041] The modal instance segmentation module is used to extract features from the mineral rock particles identified and predicted by the model, predict the grading of the mineral rock particles in the stacking state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

[0042] On the other hand, a modal instance segmentation device based on accumulated mineral rock particles under occlusion perception is provided, and the modal instance segmentation device based on accumulated mineral rock particles under occlusion perception comprises: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned modal instance segmentation methods based on accumulated mineral rock particles under occlusion perception is implemented.

[0043] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned modal instance segmentation methods for accumulated mineral rock particles under occlusion perception.

[0044] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0045] In the embodiment of the present invention, image recognition and prediction technology is used to shoot the independently constructed ore rock particle accumulation model under different accumulation states with a camera, obtain the original image database, use the deep learning algorithm, build the AGG-Mask2Former network based on the Pytorch framework, divide the database into multiple sub-training sets and train the neural network to obtain the optimal model, and extract the geometric parameters of the ore rock particles identified and predicted by the model, so as to predict the gradation of the ore rock particles under the accumulation state. It is worth noting that the present invention starts from the segmentation of modal instances, not only focusing on the segmentation problem of the unobstructed part of the ore rock particles, but also predicting the occlusion interval of the ore rock particles by connecting the relationship between pixels and particles. The research ideas and research results of the present invention can provide effective technical means for the efficient and accurate identification and gradation statistics of accumulation particles under different types of occlusion perception such as rocks and soils, and have certain guiding significance for subsequent optimization of engineering processes, improvement of production efficiency, and improvement of ore processing processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 It is a flow chart of a modal instance segmentation method for accumulated mineral rock particles based on occlusion perception provided by an embodiment of the present invention;

[0048] Figure 2 This is an unobstructed and piled-up ore rock particle image provided by an embodiment of the present invention;

[0049] Figure 3 The real occluded and accumulated ore rock particle image provided by the embodiment of the present invention;

[0050] Figure 4 The embodiment of the present invention provides Figure 3 The recognition result graph of

[0051] Figure 5 An image of mineral rock particles with simple occlusion and accumulation provided by an embodiment of the present invention;

[0052] Figure 6 The embodiment of the present invention provides Figure 5 The recognition result graph of

[0053] Figure 7 It is a block diagram of a modal instance segmentation device based on accumulated mineral rock particles under occlusion perception provided by an embodiment of the present invention;

[0054] Figure 8 It is a structural schematic diagram of a modal instance segmentation device based on accumulated mineral rock particles under occlusion perception provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0056] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0057] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0058] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0060] The embodiment of the present invention provides a modal instance segmentation method based on the accumulation of mineral rock particles under occlusion perception, which can be implemented by a modal instance segmentation device based on the accumulation of mineral rock particles under occlusion perception, and the modal instance segmentation device based on the accumulation of mineral rock particles under occlusion perception can be a terminal or a server. Figure 1The flow chart of the modal instance segmentation method based on the accumulation of mineral rock particles under occlusion perception is shown. The present invention uses a camera to shoot the independently constructed mineral rock particle accumulation model under different accumulation states, obtains the original image database to establish an image database of mineral rock particles accumulated under occlusion perception, builds an AGG-Mask2Former neural network, trains the neural network with the training set to obtain the optimal model, and finally realizes efficient, fast, accurate and complete recognition of mineral rock particles accumulated under occlusion perception. The present invention significantly improves the efficiency and accuracy of model training, recognition and prediction, and provides a more reliable data basis for the subsequent extraction of geometric parameters such as particle size and area of ​​mineral rock particles and grading prediction.

[0061] Based on the fact that the existing neural network has always had low accuracy in the process of identifying mineral rock particles, and cannot accurately handle the occluded part, thereby reducing the accuracy of the parameter extraction results after identification, the present invention deeply explores the recognition and prediction results of mineral rock particles by the model under different occlusion states, builds the AGG-Mask2Former neural network and trains it to the optimal model, so as to realize the intelligent recognition of mineral rock particles accumulated under occluded perception in an efficient, fast, accurate and complete manner.

[0062] like Figure 1 The processing flow of the method shown may include the following steps:

[0063] S1. Building a rock particle accumulation model under different accumulation states, and obtaining an original image database through the rock particle accumulation model;

[0064] In a feasible implementation, S1, building a rock particle accumulation model under different accumulation states, and obtaining original image data through the rock particle accumulation model, including:

[0065] Arrange the ore and rock particles in a preset stacking state under uniform illumination to obtain a stacking model of ore and rock particles in different stacking states;

[0066] The original image database is obtained by shooting with a camera, and the original image database is divided into three categories of databases, including: the first category of database used for neural networks to learn the basic information of mineral rock particles; the second category of database used for neural networks to learn and predict the occluded part of mineral rock particles; the third category of database used for neural network training, testing, and subsequent in-depth exploration of the model's recognition and prediction results of mineral rock particles under different occlusion states.

[0067] In a feasible implementation, mineral rock particles with a particle size distribution within the range of 10-20 mm are selected, and arranged in a specific stacking state under uniform lighting conditions. The original image database is obtained by photographing with a camera and is divided into three types of databases.

[0068] First, the first type of database is unobstructed piled ore particles. This type of database is mainly used for neural networks to learn the basic information of ore particles, such as Figure 2 As shown, by randomly changing the positions of the ore particles, a total of 200 original image data were acquired.

[0069] The second type of database is the real occluded accumulated mineral rock particles. This type of database is mainly used for neural network learning and predicting the occluded part of mineral rock particles, such as Figure 3 As shown, by randomly changing the positions of the ore particles, a total of 200 original image data were acquired.

[0070] The third type of database is simple occlusion and accumulation of mineral rock particles. This type of database is used for neural network training, testing, and subsequent in-depth exploration of the recognition and prediction results of the model for mineral rock particles under different occlusion states. In order to facilitate statistics and evaluation of the recognition and prediction results, only two mineral rock particles are placed, such as Figure 5 As shown, this study defines the ratio of the blocked area of ​​the obscured mineral rock particles to the total area as a variable, and sets 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, and 90% of the mineral rock particles in different stacking states to be photographed. Each group is photographed 10 times, and a total of 100 original image data are obtained.

[0071] In addition, since the mineral rock particles in the second and third categories of databases contain occluded parts, in order to facilitate the subsequent acquisition of more accurate label files, each original image data has corresponding multiple subsidiary images, which are taken at the same height and angle after removing the particles above the obscured mineral rock particles visible in the original image. For real occluded and accumulated mineral rock particles, the obscured particles visible in the sample image may be accumulated by multiple layers of particles. Therefore, different sample images may contain multiple subsidiary images, which ensures that each visible particle in the sample image has a complete contour description.

[0072] These three types of databases constitute the image database of this study, providing basic data for subsequent neural network training, testing, prediction, and evaluation.

[0073] S2, performing image data processing on the original image database to obtain processed image data;

[0074] In a feasible implementation, S2, performing image data processing on multiple sub-training sets to obtain processed image data, includes:

[0075] Obtain image data from the second and third databases, and provide multiple subsidiary images based on each image data to facilitate subsequent acquisition of more accurate label files; wherein the subsidiary images are images obtained by taking pictures at the same height and angle after removing particles above the obscured mineral particles visible in the original image;

[0076] The original image data needs to go through a series of processing steps before it can be used to train the neural network. Data processing is performed on the image data in the three types of databases. The data processing includes: data preprocessing, data labeling, data enhancement and data post-processing to obtain processed image data.

[0077] In a feasible implementation, data preprocessing mainly cuts out irrelevant information in the original image data, retains only the mineral-bearing rock particle portion, and generates a sample image.

[0078] In a feasible implementation, data labeling includes: completely labeling the obstructed parts of the mineral particles; using the SAM segmentation model or Labelme software to label the first type of database through SAM; and manually labeling the second and third types of databases through Labelme software.

[0079] In a feasible implementation mode, Labelme annotation refers to annotating the sample image and the auxiliary image separately when using Labelme, directly annotating the unobstructed mineral rock particles in the sample image, and annotating the obstructed mineral rock particles visible in the sample image through the auxiliary image to ensure that each mineral rock particle visible in the sample image has a corresponding complete mask, and finally merge multiple label files to form a complete annotation of all mineral rock particles in the sample image.

[0080] This is because the segmentation problem of mineral rock particles in the first type of database is relatively simple, while the mineral rock particles in the second and third types of databases are in a piled state, and the edge information between particles is not obvious enough, so SAM cannot handle it well. In addition, SAM cannot mark the obscured parts of mineral rock particles. When using Labelme, the second and third types of databases respectively annotate the sample image and the auxiliary image, directly annotate the unobstructed mineral rock particles in the sample image, and annotate the obscured mineral rock particles visible in the sample image through multiple auxiliary images to ensure that each mineral rock particle visible in the sample image has a corresponding complete mask, and merge multiple label files to finally form a complete annotation of the sample image containing obscured mineral rock particles.

[0081] In a feasible implementation, data enhancement mainly includes multiple methods such as synchronously flipping the sample image and the label file horizontally, vertically and mirroring, changing the brightness, adding noise and random points, and finally the first, second and third databases produce 1200, 1200 and 600 image data respectively.

[0082] Data post-processing refers to converting data into COCO data format for neural network training.

[0083] S3. Build the AGG-Mask2Former network based on the Pytorch framework; define a multi-task loss function to optimize the AGG-Mask2Former network training process;

[0084] In a feasible implementation, S3 builds the AGG-Mask2Former (Amodal GNN GAN Masked-attention MaskTransformer) network based on the Pytorch framework with strong flexibility, dynamic computational graphs, and powerful GPU acceleration capabilities, including:

[0085] On the basis of Mask2Former neural network, GNN (Graph Neural Network) and GAN (Generative Adversarial Network) are introduced; including: GNN is introduced after the backbone network extracts image features to capture the relationship between features; GNN is added after the Transformer decoder outputs the detection results to analyze the occlusion relationship (occlusion and occlusion) between mineral particles; GAN and GNN are added after Mask2Former outputs the detection and segmentation results to generate a mask of the occluded part for each occluded mineral particle. AGG-Mask2Former connects a predicted occlusion task in series after the two parallel tasks of detection and segmentation.

[0086] GNN enables the network to understand the relationship between rock particles and pixels, thereby improving the segmentation performance and prediction results of the model. Specifically, the relationship between rock particles is analyzed during detection to determine the relationship between occlusion and obstruction; the relationship between pixels is analyzed during segmentation to better process the edge information of rock particles; the relationship between pixels is analyzed during prediction to assist GAN in generating mask pixels.

[0087] In a feasible implementation, GNN and GAN are introduced on the basis of Mask2Forme, wherein the role of introducing GNN includes: analyzing the relationship between mineral rock particles during detection to determine the relationship between occlusion and occlusion; analyzing the relationship between pixels during segmentation to better process the edge information of mineral rock particles; analyzing the relationship between pixels during prediction to assist GAN in generating mask pixels; the role of introducing GAN is to generate corresponding masks for the occluded parts of mineral rock particles to achieve occlusion prediction.

[0088] In a feasible implementation, the intelligent recognition of deposited mineral particles and occlusion prediction under occlusion perception belongs to the field of instance segmentation, and its loss function should comprehensively consider the performance of the three major tasks of target detection, semantic segmentation, and occlusion prediction. Therefore, a multi-task loss function (MLoss) is defined to optimize the neural network training process.

[0089] The calculation formula of the multi-task loss function MLoss is defined as follows:

[0090] MLoss=L cls +L box +L mask +L amodal (1)

[0091] Where: L cls is the classification loss for target monitoring, L box is the regression loss of the bounding box, L mask To predict the mask loss, L amodal To predict the occlusion mask loss.

[0092] IoU calculates the ratio of the overlapping area between the predicted mask and the true mask of a single instance to the total area. When dealing with multi-target segmentation tasks, the average IoU value of all target instances, i.e. MIoU, is usually used to measure the accuracy of the model in the instance segmentation task.

[0093] The calculation formula of IoU is as follows:

[0094]

[0095] Among them: A is the predicted mask area, and B is the real mask area.

[0096] AP is the integral of the precision-recall curve. Precision (P) is the percentage of real rock particle pixels in the predicted pixels to all pixels predicted as rock particles, reflecting the correct probability of the results predicted as rock particles. Mean Average Precision (mAP) represents the arithmetic mean of Average Precision (AP) of each category (rock particles and background). As the main comprehensive evaluation index of the model, it is divided into Box_mAP and Segm_mAP. Box_mAP is the accuracy evaluation index of detection, and Segm_mAP is the accuracy evaluation index of the overall segmentation of rock particles.

[0097] The calculation formula of P is as follows:

[0098]

[0099] Among them: TP represents the total number of pixels marked and predicted as mineral particles; FP represents the total number of pixels marked as background but incorrectly predicted as mineral particles.

[0100] The calculation formula of mAP is as follows:

[0101]

[0102] Where: K is the category number.

[0103] S4, dividing the processed image data into multiple sub-training sets; training the AGG-Mask2Former network through multiple sub-training sets to obtain the optimal model; obtaining the recognition and prediction results of the model under different occlusion states through cross-validation experiments;

[0104] In a feasible implementation, S4, the processed image data is divided into multiple sub-training sets; the AGG-Mask2Former network is trained by the multiple sub-training sets to obtain the optimal model; and the recognition and prediction results of the model under different occlusion states are obtained by cross-validation experiments, including:

[0105] The processed image data is divided into multiple sub-training sets, and the neural network is trained by the multiple sub-training sets to obtain the optimal model; among them, the three types of databases are randomly divided into training sets and test sets in a ratio of 7:3;

[0106] The three types of databases are divided into multiple sub-training sets for cross-validation according to different research significances;

[0107] Among them, the multiple sub-training sets include: class a sub-training set, class b sub-training set and class c sub-training set; the class a sub-training set contains all the training set data in the first, second and third categories of databases, which is used to train and obtain the final model; the class b sub-training set contains all the training set data in the first and second categories of databases, and is tested with the test set of the third category of database, which is used to study the recognition of different occlusions by the verification model; the class c sub-training set contains all the training set data in the first and third categories of databases, and is tested with the test set of the second category of database, which is used to study the recognition of real accumulation by the verification model.

[0108] The three sub-training sets a, b, and c are put into the AGG-Mask2Former neural network for training to obtain the optimal model. Then the test set in the database is put into the three models for cross-validation, and the recognition and prediction results of the model are evaluated and analyzed by indicators such as Box_mAP and Segm_mAP. Figure 4 and Figure 6 The best model pairs trained separately Figure 3 and Figure 5 The recognition result.

[0109] S5. Perform feature extraction on the mineral rock particles identified and predicted by the model, predict the gradation of the mineral rock particles in the stacked state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

[0110] In a feasible implementation, S5, feature extraction is performed on the ore and rock particles identified and predicted by the model, and the gradation of the ore and rock particles in the stacking state is predicted based on the feature extraction results to obtain a complete ore and rock particle outline, and complete the modal instance segmentation of the stacked ore and rock particles based on occlusion perception, including:

[0111] Extract features of the mineral rock particles identified and predicted by the model, and extract relevant geometric parameters including particle size and area;

[0112] Considering that the output result of the AGG-Mask2Former network is a combined mask composed of all overlapping mineral particles, it makes it inconvenient to extract the parameters of the mineral particles. Therefore, an independent mask image is generated for each mineral particle, and the OpenCV library is used to extract geometric parameters such as area and particle size for each mineral particle, so as to predict the gradation of the mineral particles in the stacking state. The function FindContours in the OpenCV library is used to obtain the contour of the mask image, the area of ​​the contour is calibrated using the library function ContourArea, and the basic parameters such as the minimum circumscribed oblique rectangle are obtained through the function MinAreaRect, and the Feret shortest diameter of the mask contour is defined as the equivalent particle size L of the mineral particle according to the following formula (2);

[0113]

[0114] Where: S is the area of ​​the contour, L max is the longest diameter of Ferre;

[0115] The volume of the ore rock particles replaces the mass of the rock particles to calculate the gradation of the rock particles; the gradation is the mass fraction of the ore rock particles with different particle sizes in the whole ore rock particles;

[0116] The equivalent volume V of the ore rock particles is calculated according to the following formula (3), and different particle size intervals are divided according to different equivalent particle sizes. The cumulative volume of ore rock particles in different intervals is calculated to obtain the gradation prediction of the accumulated ore rock particles under occlusion perception by the optimized and trained AGG-Mask2Former network:

[0117]

[0118] In the embodiment of the present invention, gradation refers to the mass fraction of mineral rock particles with different particle sizes in the entire mineral rock particles; the method proposed in the present invention can identify the complete mineral rock particle outline, which includes the recognition result of the non-occluded part and the prediction result of the occluded part. On the basis of this result, the particle size and area of ​​the mineral rock particles are extracted by OpenCV, and the equivalent volume is used instead of the mass to calculate the mass fraction of mineral rock particles with different particle sizes, thereby realizing the prediction of the gradation of mineral rock particles.

[0119] In the embodiment of the present invention, a camera is used to shoot a self-constructed accumulation model of mineral rock particles in different accumulation states, and original image data is obtained to establish an image database of mineral rock particles accumulated under occlusion perception. An AGG-Mask2Former neural network is built, and the database is used to train the neural network to obtain an optimal model, ultimately achieving efficient, rapid, accurate and complete identification of mineral rock particles accumulated under occlusion perception. The present invention significantly improves the efficiency and accuracy of model training, identification and prediction, and provides a more reliable data basis for the subsequent extraction of geometric parameters such as mineral rock particle size and area, and grading prediction.

[0120] Figure 7 The block diagram of a modal instance segmentation device based on occlusion-aware accumulation of mineral particles according to an exemplary embodiment is shown. The device is used for a modal instance segmentation method based on occlusion-aware accumulation of mineral particles. Figure 7 The device includes an original image acquisition module 310, an image processing module 320, a model construction module 330, a model training and verification module 340, and a modality instance segmentation module 350. Among them:

[0121] The original image acquisition module 310 is used to build a rock particle accumulation model under different accumulation states and obtain an original image database through the rock particle accumulation model;

[0122] The image processing module 320 is used to process the original image database to obtain processed image data;

[0123] The model building module 330 is used to build the AGG-Mask2Former network based on the Pytorch framework; define a multi-task loss function to optimize the AGG-Mask2Former network training process;

[0124] The model training and verification module 340 is used to divide the processed image data into multiple sub-training sets; train the AGG-Mask2Former network through the multiple sub-training sets to obtain the optimal model; and obtain the recognition and prediction results of the model under different occlusion states through cross-validation experiments;

[0125] The modal instance segmentation module 350 is used to extract features from the mineral rock particles identified and predicted by the model, predict the gradation of the mineral rock particles in the stacking state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

[0126] Optionally, the original image acquisition module 310 is used to arrange the mineral rock particles according to a preset stacking state under the condition of uniform illumination to obtain the mineral rock particle stacking model under different stacking states;

[0127] The original image database is obtained by shooting with a camera, and the original image database is divided into three categories of databases, including: the first category of database used for neural networks to learn the basic information of mineral rock particles; the second category of database used for neural networks to learn and predict the occluded part of mineral rock particles; the third category of database used for neural network training, testing, and subsequent in-depth exploration of the model's recognition and prediction results of mineral rock particles under different occlusion states.

[0128] Optionally, the image processing module 320 is used to obtain image data in the second database and the third database, and provide multiple subsidiary images based on each image data; wherein the subsidiary images are images obtained by shooting at the same height and angle after removing particles above the obscured mineral particles visible in the original image;

[0129] Data processing is performed on the image data in the three types of databases. The data processing includes: data preprocessing, data labeling, data enhancement and data post-processing to obtain processed image data.

[0130] Optionally, data labeling includes: completely labeling the blocked parts in the mineral particles; using the SAM segmentation large model or Labelme software to label the first type of database through SAM; and manually labeling the second and third types of databases through Labelme software.

[0131] Optionally, the model building module 330 is used to introduce a graph neural network (GNN) and a generative adversarial network (GAN) based on the Mask2Former neural network; which includes: introducing a GNN after the backbone network extracts image features to capture the relationship between features; adding a GNN after the Transformer decoder outputs the detection results to analyze the occlusion relationship (occlusion and occlusion) between mineral rock particles; adding a GAN and a GNN after the Mask2Former outputs the detection and segmentation results to generate a mask of the occluded part for each occluded mineral rock particle.

[0132] Optionally, the model building module 330 is used to define a calculation formula of the multi-task loss function MLoss as follows:

[0133] MLoss=L cls +L box +L mask +L amodal (1)

[0134] Where: L cls is the classification loss for target monitoring, L box is the regression loss of the bounding box, L mask To predict the mask loss, L amodal To predict the occlusion mask loss.

[0135] Optionally, the model training and verification module 340 is used to divide the processed image data into multiple sub-training sets, and train the v network through the multiple sub-training sets to obtain the optimal model; wherein the three types of databases are randomly divided into training sets and test sets in a ratio of 7:3;

[0136] The three types of databases are divided into multiple sub-training sets for cross-validation according to different research significances;

[0137] Among them, the multiple sub-training sets include: class a sub-training set, class b sub-training set and class c sub-training set; the class a sub-training set contains all the training set data in the first, second and third categories of databases, which is used to train and obtain the final model; the class b sub-training set contains all the training set data in the first and second categories of databases, and is tested with the test set of the third category of database, which is used to study the recognition of different occlusions by the verification model; the class c sub-training set contains all the training set data in the first and third categories of databases, and is tested with the test set of the second category of database, which is used to study the recognition of real accumulation by the verification model.

[0138] Optionally, the modal instance segmentation module 350 is used to extract features of the mineral rock particles identified and predicted by the model, and extract relevant geometric parameters including particle size and area;

[0139] The function FindContours in the OpenCV library is used to obtain the contour of the mask image, the area of ​​the contour is calibrated using the library function ContourArea, and the basic parameters such as the minimum circumscribed oblique rectangle are obtained through the function MinAreaRect. The Feret shortest diameter of the mask contour is defined as the equivalent particle size L of the ore particle according to the following formula (2);

[0140]

[0141] Where: S is the area of ​​the contour, L max is the longest diameter of Ferre;

[0142] The volume of the ore rock particles replaces the mass of the rock particles to calculate the gradation of the rock particles; the gradation is the mass fraction of the ore rock particles with different particle sizes in the whole ore rock particles;

[0143] The equivalent volume V of the ore rock particles is calculated according to the following formula (3), and different particle size intervals are divided according to different equivalent particle sizes. The cumulative volume of ore rock particles in different intervals is calculated to obtain the gradation prediction of the accumulated ore rock particles under occlusion perception by the optimized and trained AGG-Mask2Former network:

[0144]

[0145] In the embodiment of the present invention, a camera is used to shoot a self-constructed accumulation model of mineral rock particles in different accumulation states, and original image data is obtained to establish an image database of mineral rock particles accumulated under occlusion perception. An AGG-Mask2Former neural network is built, and the database is used to train the neural network to obtain an optimal model, ultimately achieving efficient, rapid, accurate and complete identification of mineral rock particles accumulated under occlusion perception. The present invention significantly improves the efficiency and accuracy of model training, identification and prediction, and provides a more reliable data basis for the subsequent extraction of geometric parameters such as mineral rock particle size and area, and grading prediction.

[0146] like Figure 8 is a structural schematic diagram of a modal instance segmentation device for accumulated mineral rock particles based on occlusion perception provided by an embodiment of the present invention, such as Figure 8 As shown, the modal instance segmentation device based on the accumulation of mineral rock particles under occlusion perception may include the above Figure 7 The modal instance segmentation device based on the accumulation of mineral particles under occlusion perception is shown. Optionally, the modal instance segmentation device based on the accumulation of mineral particles under occlusion perception may include a first processor 2001.

[0147] Optionally, the modal instance segmentation device based on accumulated mineral particles under occlusion perception may also include a memory 2002 and a transceiver 2003 .

[0148] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0149] Combine the following Figure 8 The components of the modal instance segmentation device for accumulated mineral particles under occlusion perception are introduced in detail:

[0150] The first processor 2001 is the control center of the modal instance segmentation device for accumulated mineral particles under occlusion perception, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0151] Optionally, the first processor 2001 can perform various functions of the modal instance segmentation device based on accumulated mineral rock particles under occlusion perception by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0152] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 are shown in FIG.

[0153] In a specific implementation, as an embodiment, the modal instance segmentation device based on the accumulation of mineral particles under occlusion perception may also include multiple processors, such as Figure 8 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0154] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0155] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0156] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0157] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0158] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be implemented through an interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0159] It should be noted that Figure 8 The structure of the modal instance segmentation device based on accumulated mineral rock particles under occlusion perception shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0160] In addition, the technical effects of the modal instance segmentation device based on accumulated mineral rock particles under occlusion perception can refer to the technical effects of the modal instance segmentation method based on accumulated mineral rock particles under occlusion perception described in the above method embodiment, and will not be repeated here.

[0161] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0162] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0163] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0164] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0165] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0166] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0167] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0172] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A modal instance segmentation method for accumulated mineral particles based on occlusion perception, characterized in that: The method comprises: S1. Building a rock particle accumulation model under different accumulation states, and obtaining an original image database through the rock particle accumulation model; S2, performing image data processing on the original image database to obtain processed image data; S3. Build the AGG-Mask2Former network based on the Pytorch framework; define a multi-task loss function to optimize the training process of the AGG-Mask2Former network; S4, dividing the processed image data into multiple sub-training sets; training the AGG-Mask2Former network with the multiple sub-training sets to obtain an optimal model; and obtaining recognition and prediction results of the model under different occlusion states through a cross-validation experiment; S5. Perform feature extraction on the mineral rock particles identified and predicted by the model, predict the gradation of the mineral rock particles in the stacked state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

2. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 1 is characterized in that: The S1, building a rock particle accumulation model under different accumulation states, and obtaining original image data through the rock particle accumulation model, includes: Arrange the ore and rock particles in a preset stacking state under uniform illumination to obtain a stacking model of ore and rock particles in different stacking states; The original image database is obtained by shooting with a camera, and the original image database is divided into three categories of databases, including: a first category of databases used for neural networks to learn basic information of mineral rock particles; a second category of databases used for neural networks to learn and predict the occluded parts of mineral rock particles; and a third category of databases used for neural network training, testing, and subsequent in-depth exploration of the model's recognition and prediction results of mineral rock particles under different occlusion states.

3. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 1 is characterized in that: The step S2, performing image data processing on the multi-class sub-training sets to obtain processed image data, includes: Acquire image data from the second and third databases, and provide multiple subsidiary images based on each image data; wherein the subsidiary images are images obtained by shooting at the same height and angle after removing particles above the obscured mineral particles visible in the original image; Data processing is performed on the image data in the three types of databases, and the data processing includes: data preprocessing, data labeling, data enhancement and data post-processing to obtain processed image data.

4. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 3 is characterized in that: The data labeling includes: completely labeling the blocked parts in the ore particles; using the SAM segmentation large model or Labelme software to label the first type of database through SAM; and manually labeling the second and third types of databases through Labelme software.

5. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 4 is characterized in that: The S3, based on the Pytorch framework, builds the AGG-Mask2Former network, including: Based on the Mask2Former neural network, graph neural network (GNN) and generative adversarial network (GAN) are introduced; including: introducing GNN after the backbone network extracts image features to capture the relationship between features; adding GNN after the Transformer decoder outputs the detection results to analyze the occlusion relationship between mineral rock particles; adding GAN and GNN after Mask2Former outputs the detection and segmentation results to generate a mask of the occluded part for each occluded mineral rock particle.

6. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 5 is characterized in that: In S3, a multi-task loss function is defined, including: The calculation formula for defining the multi-task loss function MLoss is as follows (1): (1) Where: L cls is the classification loss for target monitoring, L box is the regression loss of the bounding box, L mask To predict the mask loss, L amodal To predict the occlusion mask loss.

7. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 6 is characterized in that: S4, dividing the original image database into multiple sub-training sets; The optimal model is obtained by training the neural network through the multi-class sub-training sets; the recognition and prediction results of the model under different occlusion states are obtained through cross-validation experiments, including: The original image database is divided into multiple sub-training sets, and the optimal model is obtained by training the neural network through the multiple sub-training sets; wherein the three types of databases are randomly divided into training sets and test sets in a ratio of 7:3; The three types of databases are divided into multiple sub-training sets for cross-validation according to different research significances; Among them, the multiple categories of sub-training sets include: category A sub-training set, category B sub-training set and category C sub-training set; category A sub-training set contains all the training set data in the first, second and third categories of databases, which is used to train and obtain the final model; category B sub-training set contains all the training set data in the first and second categories of databases, and is tested with the test set of the third category of database, which is used to study the recognition of different occlusions by the verification model; category C sub-training set contains all the training set data in the first and third categories of databases, and is tested with the test set of the second category of database, which is used to study the recognition of real accumulation by the verification model.

8. The modal instance segmentation method for accumulated mineral particles based on occlusion perception according to claim 7 is characterized in that: S5, extracting features of the ore and rock particles identified and predicted by the model, predicting the gradation of the ore and rock particles in the stacking state based on the feature extraction results, and completing the modal instance segmentation of the stacked ore and rock particles based on occlusion perception, includes: Extract features of the mineral rock particles identified and predicted by the model, and extract relevant geometric parameters including particle size and area; The FindContours function in the OpenCV library is used to obtain the contour of the mask image, the area of ​​the contour is calibrated using the library function ContourArea, and the basic parameters such as the minimum circumscribed oblique rectangle are obtained through the MinAreaRect function. The Feret shortest diameter of the mask contour is defined as the equivalent particle size L of the ore particle according to the following formula (2); (2) Where: S is the area of ​​the contour, L max is the longest diameter of Ferre; The volume of the ore rock particles replaces the mass of the rock particles to calculate the gradation of the rock particles; the gradation is the mass fraction of the ore rock particles with different particle sizes in the whole ore rock particles; The equivalent volume V of the ore rock particles is calculated according to the following formula (3), and different particle size intervals are divided according to different equivalent particle sizes. The cumulative volume of ore rock particles in different intervals is calculated to obtain the gradation prediction of the accumulated ore rock particles under occlusion perception by the optimized and trained AGG-Mask2Former network: (3)。 9. A modal instance segmentation device for accumulated mineral particles under occlusion perception, the modal instance segmentation device for accumulated mineral particles under occlusion perception is used to implement the modal instance segmentation method for accumulated mineral particles under occlusion perception as claimed in any one of claims 1 to 8, characterized in that: The device comprises: The original image acquisition module is used to build a rock particle accumulation model under different accumulation states, and obtain an original image database through the rock particle accumulation model; An image processing module, used for performing image data processing on the original image database to obtain processed image data; A model building module is used to build an AGG-Mask2Former network based on the Pytorch framework; a multi-task loss function is defined to optimize the training process of the AGG-Mask2Former network; A model training and verification module is used to divide the processed image data into multiple sub-training sets; train the AGG-Mask2Former network through the multiple sub-training sets to obtain an optimal model; and obtain recognition and prediction results of the model under different occlusion states through cross-validation experiments; The modal instance segmentation module is used to extract features from the mineral rock particles identified and predicted by the model, predict the grading of the mineral rock particles in the stacking state based on the feature extraction results, obtain the complete mineral rock particle contour, and complete the modal instance segmentation of the stacked mineral rock particles based on occlusion perception.

10. A modal instance segmentation device based on accumulated mineral particles under occlusion perception, characterized in that: The modal instance segmentation device based on accumulated mineral rock particles under occlusion perception includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images