Intelligent coal gangue identification method based on selective state-space equation

Through the intelligent identification method of coal gangue based on selective state space equations, the problem of low efficiency and accuracy of coal gangue recognition in the existing technology is solved, and the rapid and accurate identification of lightweight deep learning architecture is achieved, reducing the computing resource requirements , support the comprehensive utilization of coal resources and environmental protection.

CN119048740APending Publication Date: 2024-11-29HENAN UNIVERSITY
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411168933.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing coal gangue identification methods are inefficient and have low accuracy, are subjective, and require high-performance computing platforms. The real-time and deployment costs are high in scenarios with limited resources.

Method used

Using the intelligent identification method of coal gangue based on selective state space equations, the detection heads of state space network, SimSPPF, feature fusion layer and branch decoupling are constructed, and feature extraction and fusion are used for discrete selective state space equations to realize coal Quick and accurate identification of gangue.

Benefits of technology

While ensuring identification accuracy, it significantly reduces the demand for computing resources, and realizes fast and accurate identification of coal gangue. It does not require relying on expensive high-performance computing equipment. It can identify coal gangue in real time, improving work efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048740A_ABST
    Figure CN119048740A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of intelligent mining industry, in particular to a coal gangue intelligent identification method based on a selective state-space equation, which comprises the following steps: acquiring a training set and a verification set in a coal gangue data set, and constructing a coal gangue identification model; training the coal gangue recognition model by using the training set, and after each round of training is finished, calculating a model loss function value by using the verification set until the set round number is finished, so as to obtain the trained coal gangue recognition model; and obtaining a to-be-detected preprocessed coal gangue image, and inputting the to-be-detected preprocessed coal gangue image into the trained coal gangue recognition model to obtain an intelligent coal gangue recognition result. According to the method, by introducing the selective state-space equation, modeling and analysis are performed on the image features of the coal gangue, so that more effective and robust feature representation is extracted, and the accuracy of an intelligent coal gangue recognition result is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart mining technology, and in particular to a method for intelligently identifying coal gangue based on a selective state space equation. Background Art

[0002] Coal gangue is solid waste generated during coal mining and washing. Its large-scale accumulation not only occupies land resources, but also may have a negative impact on the environment and ecology. Therefore, in order to improve the comprehensive utilization rate of coal resources, promote environmental protection and sustainable development, it is necessary to quickly and accurately identify and classify coal gangue.

[0003] Traditional gangue identification methods mainly rely on manual visual discrimination or simple physical and chemical tests, which are not only inefficient, but also have problems such as strong subjectivity and low accuracy. With the rapid development of computer vision and artificial intelligence technology, intelligent identification methods based on image processing and deep learning have gradually been applied in the field of gangue identification. In the existing intelligent identification methods of gangue, deep learning models such as convolutional neural networks classify gangue images by learning the appearance features of gangue. However, in practical applications, the appearance features of gangue are complex and diverse, including color, texture, shape and other aspects, making model training and learning difficult; the image acquisition environment of gangue is diverse, and factors such as lighting and occlusion will affect the image quality, thereby affecting the recognition effect of the model; most of the existing intelligent identification methods of gangue need to be run on high-performance computing platforms. For scenarios with limited resources, their real-time and deployment costs are relatively high. Summary of the invention

[0004] In order to solve the technical problem of low recognition accuracy of the above-mentioned existing gangue recognition methods, the purpose of the present invention is to provide a gangue intelligent recognition method based on selective state space equations, and the technical scheme adopted is as follows:

[0005] An embodiment of the present invention provides a method for intelligently identifying coal gangue based on a selective state space equation, the method comprising the following steps:

[0006] Acquire a coal gangue data set consisting of a preset number of preprocessed coal gangue images, wherein the coal gangue data set is randomly divided into a training set and a validation set according to a preset ratio;

[0007] Constructing a gangue recognition model, the gangue recognition model includes a state space network, SimSPPF, a feature fusion layer and a branch decoupled detection head, the state space network applies a selective state space equation;

[0008] The gangue recognition model is trained using the training set, and after each round of training, the model loss function value is calculated using the validation set, and the gangue recognition model corresponding to the minimum value of the current model loss function is saved until the set number of rounds is completed to obtain the trained gangue recognition model;

[0009] The pre-processed gangue image to be detected is obtained, and the pre-processed gangue image to be detected is input into the trained gangue recognition model to obtain the gangue intelligent recognition result.

[0010] Furthermore, after obtaining the coal gangue intelligent identification result, it also includes:

[0011] If the intelligent recognition result of coal gangue is satisfactory, the intelligent recognition result of coal gangue will be used as the final recognition result; if the intelligent recognition result of coal gangue is not satisfactory, the target confidence threshold and the intersection-over-union ratio threshold of maximum suppression will be adjusted according to the model recognition situation, and the adjusted trained coal gangue recognition model will be returned to obtain the intelligent recognition result of coal gangue until the standard model recognition situation is reached; wherein, the intelligent recognition result of coal gangue includes the number and location information of coal and gangue.

[0012] Furthermore, the step of acquiring a coal gangue data set consisting of a preset number of preprocessed coal gangue images includes:

[0013] A preset number of gangue images are collected using a photographing device; coal and gangue in each gangue image are annotated using an annotation tool to obtain annotation information of each gangue image;

[0014] The annotation information of each gangue image is saved as a corresponding XML tag file, and then the XML tag file is converted into a TXT tag file in DOTA format to obtain a TXT tag file corresponding to each gangue image;

[0015] Use DOTA_devkit to segment each gangue image to obtain several regional images of preset resolutions corresponding to each gangue image;

[0016] The TXT tag file corresponding to each gangue image and the plurality of regional images with preset resolutions are combined to obtain a gangue data set consisting of all preprocessed gangue images.

[0017] Furthermore, the gangue recognition model is used to perform feature extraction, feature fusion and intelligent detection on the preprocessed gangue image, including:

[0018] Read the preprocessed gangue image and its annotation information, use the state space network to perform data processing on the read information, obtain the global field of view of the gangue image, allocate attention to the target area in the global field of view of the gangue image, and extract the spatial information and semantic information of the target area; wherein the target area is a pixel area corresponding to coal or gangue;

[0019] The SimSPPF of YOLOv6 is used after the state-space network to further extract feature information of the target area at different scales;

[0020] In a top-down and bottom-up manner, feature information at different scales is selected and input into the feature fusion layer for feature fusion to obtain a fused feature map;

[0021] The fused feature map is introduced into the branch decoupled detection head for prediction, and the outputs of the two branch decoupling heads are connected as the total output of the model, which includes the categories of coal and gangue and their corresponding position coordinates; wherein the two branches are: a prediction branch for the category and score of the target frame, and a prediction branch for the coordinate information of the target frame.

[0022] Furthermore, allocating attention to the target area in the global field of view of the gangue image and extracting spatial information and semantic information of the target area includes:

[0023] The state space network is composed of 9 layers of GSConv and SPSS modules. The SPSS module shuffles the n channels of the input global view and splits them into two feature maps. Both feature maps have channels; where n represents the number of channels in the global field of view;

[0024] For any feature map, linear embedding is first performed, and then the feature map is input into a 3×3 deep convolutional layer, and the output result of the deep convolutional layer is subsequently input into a two-dimensional selective image scanning module to obtain an updated feature map;

[0025] The updated feature map is normalized by the layer and then added to the original feature map to form a cascade The feature map of the channels; The feature maps of the channels are fused through a 1×1 convolutional layer to obtain the feature maps of n channels as the final output of the SPSS module.

[0026] Furthermore, the step of obtaining the updated feature map includes:

[0027] In the selective image scanning module, each channel of the feature map is expanded into a block sequence along the upper left-lower right and lower right-upper left in the horizontal and vertical directions, with a dimension of Among them, a represents the width of the expanded block sequence, and b represents the length of the expanded block sequence;

[0028] Each block sequence is input into the discretized selective state space equation for updating, and each updated block sequence is output; every 4 updated block sequences are fused and restored into a 2D channel, and a total of The feature map of the channels is restored The feature map of each channel is used as the updated feature map.

[0029] Furthermore, the derivation process of the discretized selective state space equation includes:

[0030] The general expression of the continuous-time state space equation is: ′ (t)=Ax(t)+Bu(t); y(t)=Cx(t)+Du(t); where, x ′ Represents state variables The derivative of , t represents the current moment, Indicates input, Indicates the output, represents the state transition moment, represents the matrix of input variables, represents the observation matrix, represents the feedforward matrix, A, B, C, and D are all learnable matrices, and n, m, and p represent the number of rows or columns of the matrix;

[0031] The state space equation is discretized by using the zero sequence holding method. For the differential equation x ′ The general solution of (t) = Ax(t) + Bu(t) is Let t0 = t k and t = t k+1 , and t k+1 -t k =Δ as the step size, then we have

[0032]

[0033] In the formula, e represents a natural constant, t0 represents the time t0, τ represents the integral variable, and t k Indicates t k Time, t k+1 Indicates t k+1 Time, x k+1 represents the state variable at time k+1, x k represents the state variable at time k, u krepresents the input at time k;

[0034] Then, the discretized linear state space equation is obtained, including: In the formula, I represents the identity matrix, represents the field of real numbers;

[0035] in, Use the first-order Taylor expansion to approximate, specifically:

[0036] When k = 0, Without considering the feedforward matrix D, the convolution operation is used to convert y k Expand to get: Where k represents the kth moment;

[0037] By determining the convolution kernel in the convolution operation, the sequence operation of the state space model is converted into a convolution operation, as shown below:

[0038] Among them, u=[u0,u1,....,u k ], u represents the input at all times, and * represents the convolution operation;

[0039] The HiPPO matrix initialization method is used to initialize A, specifically:

[0040] In the formula, A n′k′ represents the element in the n′th row and k′th column of matrix A, where n′ represents the number of rows and k′ represents the number of columns;

[0041] The linear state space equation is transformed into a selective state space equation, specifically:

[0042] The matrix A remains unchanged, B, C and Δ are functions of the input u, and the expression of the selectivity state space equation is:

[0043]

[0044] The output is represented as:

[0045]

[0046] Furthermore, the calculation formula for the size of the total output is:

[0047] N′=(N1×N1+N2×N2+N3×N3)×(cls+bbox reg ), where N′ represents the size of the total output, N1, N2, and N3 represent the different scales of the feature map, cls represents the number of categories, and bbox regIndicates the number of coordinate values ​​of the predicted sample box.

[0048] Furthermore, the calculation formula of the loss function value is:

[0049] Where, L total represents the loss function value, n positive Represents the number of positive sample frames of gangue images predicted by the model in the training set, n pred represents the number of all sample frames of coal gangue images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of (1-DIoU_i), w DFL_i represents the loss weight of DFL_i, w VFL express The loss weight of DIoU_i represents the distance intersection over union corresponding to the i-th positive sample box, DFL_i represents the distribution focal loss corresponding to the i-th positive sample box, and L VFL _i represents the varifocal loss of the category corresponding to the i-th prediction box, n pred Indicates the number of predicted sample frames of the gangue image predicted by the model in the training set;

[0050] Among them, the positive sample boxes in all detection boxes in the image are obtained by adopting the task aligned assigner method.

[0051] The present invention has the following beneficial effects:

[0052] The present invention provides an intelligent identification method for coal gangue based on selective state-space equations. The method is implemented based on a lightweight deep learning architecture. While ensuring the recognition accuracy, it significantly reduces the demand for computing resources and realizes the rapid and accurate identification of coal gangue. Researchers do not need to rely on expensive high-performance computing equipment, and can identify coal gangue in real time through ordinary computers or mobile devices, greatly improving work efficiency.

[0053] Specifically, the constructed gangue recognition model includes a state-space network, SimSPPF, a feature fusion layer and a branch-decoupled detection head. The state-space network and SimSPPF can be used to better extract the spatial and semantic information of related targets, thereby improving the accuracy and reliability of the model recognition results. The gangue recognition model is used to realize intelligent recognition of gangue, so that the present invention has the ability to detect and identify gangue in real time. Whether in a static image or in a video stream, gangue can be quickly and accurately identified, and its location, size and other information can be given, which is beneficial for researchers to monitor the distribution of gangue in real time and provide strong support for the comprehensive utilization of coal resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 This is a flow chart of the steps of a method for intelligently identifying coal gangue based on a selective state space equation according to an embodiment of the present invention;

[0056] Figure 2 Schematic diagram of the structure of GSConv in an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of the structure of the SPSS module in the embodiment of the present invention;

[0058] Figure 4 Schematic diagram of the overall structure of the gangue identification model in the embodiment of the present invention;

[0059] Figure 5 Schematic diagram of intelligent identification result of coal gangue in an embodiment of the present invention;

[0060] Figure 6 Schematic diagram of a precision-recall curve in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0062] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0063] Application scenarios targeted by the present invention:

[0064] The existing gangue identification process cannot be completed in real time and efficiently, and requires cumbersome manual operations or relies on expensive professional equipment. The accuracy and generalization ability of gangue identification are limited. In order to overcome the shortcomings of the above-mentioned existing technologies and promote the combination of artificial intelligence and mining development, this embodiment provides a gangue intelligent identification method based on selective state space equations. The method can identify and separate coal and gangue from the gangue mixture on the conveyor belt in real time, greatly reducing the workload of mining personnel and playing a positive role in promoting the development of smart mining.

[0065] like Figure 1 As shown, a coal gangue intelligent identification method based on selective state space equation may include the following steps:

[0066] S1, obtaining a coal gangue data set consisting of a preset number of preprocessed coal gangue images, and randomly dividing the coal gangue data set into a training set and a validation set according to a preset ratio.

[0067] In this embodiment, the number of images in the gangue data set can be set to 100. The larger the amount of training data, the more accurate the output result of the model. The implementer can set the number of images according to the actual situation, and no specific limitation is made here. In order to facilitate the subsequent extraction of image features of the gangue images, the gangue images need to be preprocessed. The ratio of the training set to the validation set can be set to 7:3. The implementer can set the ratio according to the actual situation, and no specific limitation is made here.

[0068] Among them, obtaining the coal gangue dataset can be achieved through the following steps:

[0069] In the first step, a preset number of coal gangue images are collected using a photographing device; coal and gangue in each coal gangue image are labeled using a labeling tool to obtain labeling information of each coal gangue image.

[0070] In this embodiment, the shooting device may be a portable camera, and the gangue image may be directly obtained from a relevant database, and the method for obtaining the gangue image is not specifically limited. The labeling tool may be LabelImg, which labels each collected gangue image, and the labeling information includes: the center point coordinates, width, and height of the coal, which are recorded as c x 、c y , w and h, and the category is coal; the center point coordinates, width and height of the gangue are recorded as c′x , c′ y 、w ′ and h ′ , and the category is gangue.

[0071] The second step is to save the annotation information of each gangue image as a corresponding XML tag file, and then convert the XML tag file into a TXT tag file in DOTA format to obtain the TXT tag file corresponding to each gangue image.

[0072] In this embodiment, the TXT tag file can be used in the subsequent training process of the coal gangue recognition model.

[0073] The third step is to use DOTA_devkit to segment each gangue image to obtain several regional images of preset resolutions corresponding to each gangue image.

[0074] In this embodiment, the preset resolution can be set to 640*640, and the corresponding TXT tag file is segmented so that the segmented region image and the region image in the TXT tag file present a one-to-one correspondence. The implementer can set the resolution size of the region image according to the actual situation, without specific limitation.

[0075] The fourth step is to combine the TXT tag file corresponding to each gangue image and several regional images with preset resolutions to obtain a gangue dataset consisting of all preprocessed gangue images.

[0076] So far, this embodiment has obtained the training set and the validation set in the coal gangue dataset.

[0077] S2, construct a coal gangue recognition model. The coal gangue recognition model includes a state space network, SimSPPF, a feature fusion layer and a branch decoupled detection head. The state space network applies a selective state space equation.

[0078] In this embodiment, the gangue recognition model is used to perform feature extraction, feature fusion and intelligent detection on the preprocessed gangue image. The specific implementation steps may include:

[0079] The first step is to construct a state space network as the core backbone network to extract coal and gangue image features.

[0080] In the first sub-step, the preprocessed gangue image and its annotation information are read, and the read information is processed using a state-space network to obtain a global view of the gangue image, allocate more attention to the target area in the global view of the gangue image, and extract the spatial information and semantic information of the target area.

[0081] In this embodiment, the state space network of the jump connection is composed of 9 layers of GSConv (Group ShuffleConvolution) and SPSS module (Shuffle-Partition State Space module). Among them, the target area is the pixel area corresponding to coal or gangue. The structural diagram of GSConv is as follows: Figure 2 , the structure diagram of SPSS module is as follows Figure 3 shown.

[0082] Among them, attention is allocated to the target area in the global field of view of the gangue image, and the spatial information and semantic information of the target area are extracted, including:

[0083] (1) The SPSS module shuffles the n channels of the global view input and splits them into two feature maps. Both feature maps have channels. Where n represents the number of channels in the global field of view.

[0084] (2) For any feature map, linear embedding is first performed, and then the feature map is input into a 3×3 deep convolutional layer. Subsequently, the output result of the deep convolutional layer is input into a two-dimensional selective image scanning module to obtain an updated feature map.

[0085] (2-1) In the selective image scanning module, each channel of the feature map is expanded into a block sequence along the upper left-lower right and lower right-upper left in the horizontal and vertical directions, with a dimension of Among them, a represents the width of the expanded block sequence, and b represents the length of the expanded block sequence.

[0086] (2-2) Each block sequence is successively input into the discretized selective state space equation for updating, and each updated block sequence is output.

[0087] The derivation process of the discretized selective state space equation includes:

[0088] (2-2-1) The general expression of the continuous-time state space equation is: ′ (t)=Ax(t)+Bu(t); y(t)=Cx(t)+Du(t); where, x ′ Represents state variables The derivative of , t represents the current moment, Indicates input, Indicates the output, represents the state transition moment, represents the matrix of input variables, represents the observation matrix, represents the feedforward matrix, A, B, C, and D are all learnable matrices, and n, m, and p represent the number of rows or columns of the matrix.

[0089] In this embodiment, D may also be referred to as a skip connection term coefficient, which may be understood as a transformed residual connection.

[0090] (2-2-2) Since the above state space equation is continuous and computers usually process the equation through sampling, it needs to be discretized and the zero-sequence hold method is used for discretization.

[0091] For the differential equation x ′ The general solution of (t) = Ax(t) + Bu(t) is:

[0092] Let t0 = t k and t = t k+1 , and t k+1 -t k =Δ as the step size, then:

[0093] In the formula, e represents a natural constant, t0 represents the time t0, τ represents the integral variable, and t k Indicates t k Time, t k+1 Indicates t k+1 Time, x k+1 represents the state variable at time k+1, x k represents the state variable at time k, u k Represents the input at time k.

[0094] Then, the discretized linear state space equation is obtained, including:

[0095] In the formula, I represents the identity matrix, Represents the field of real numbers.

[0096] in, Use the first-order Taylor expansion to approximate, specifically:

[0097] When k = 0, y0 represents y0 before it is converted into a selective state space equation. Without considering the feedforward matrix D, the convolution operation is used to convert y k Expand to get: Wherein, k represents the kth moment.

[0098] In the convolution operation, the sequence operation of the state space model can be converted into a convolution operation by designing an appropriate convolution kernel, as shown below:

[0099] Among them, u=[u0,u1,....,u k ], u represents the input at all times, and * represents the convolution operation.

[0100] The HiPPO (High-Order Polynomial Projection Operator) matrix initialization method is used to initialize A.

[0101] In the formula, A n′k′ Represents the element in the n′th row and k′th column of matrix A, where n′ represents the number of rows and k′ represents the number of columns.

[0102] It should be noted that the discretized linear state space equations belong to linear time-invariant equations, that is, A, B, C and Δ remain unchanged. From the perspective of recursion, the constant dynamics prevent them from selecting the correct information from the context or affecting the hidden state transmitted along the sequence in an input-dependent manner; from the perspective of convolution, although global convolution can solve standard copying tasks, it has difficulties in dealing with selective copying tasks that require content awareness because the spacing between input and output is variable and cannot be modeled with static convolution kernels.

[0103] (2-2-3) The linear state space equation is transformed into the selective state space equation.

[0104] In this embodiment, the selective state space equation abandons the dual attributes of convolution and recursion and relies only on loops, specifically time-varying parameterization. Specifically: the matrix A remains unchanged (note that will change), but B, C, and Δ now become functions of the input u, and the equation becomes a linear time-variable equation, that is, a selective state space equation, specifically:

[0105]

[0106] By x k and k Derived until there is no x term, the output is expressed as:

[0107]

[0108] It should be noted that the selective state space equation is similar to the dynamic parameters in the convolution layer. The parameters of the convolution kernel change every time a window is slid. It is not a convolution in essence. The selective state space equation indicates that each new state is the sum of the current input and the previous state, and the latter is the recursive sum of all previous states so far. The prefix sum operation on the array is also called a scan operation. In order to achieve fast parallel operation, a parallel scan method is adopted. Simply loop through the array and track the prefix sum. Each new sum will be the prefix sum plus the current input, so there is a time complexity of O(N). O(N) means that the running time of the algorithm is proportional to the input size N. Specifically, when the input size N increases, the running time of the algorithm will increase linearly.

[0109] (2-3) Every 4 updated block sequences are fused and restored into a 2D channel, and a total of The feature map of the channels is restored The feature map of each channel is used as the updated feature map.

[0110] (3) The updated feature map is normalized and then added to the original feature map to form a cascade The feature map of the channels; The feature maps of the n channels are fused through a 1×1 convolutional layer to obtain the feature maps of the n channels as the final output of the SPSS module, i.e., the spatial information and semantic information of the target area.

[0111] In the second sub-step, in order to further extract the information of coal and gangue of different sizes, SimSPPF of YOLOv6 is used after the state space network to further extract the feature information of the target area at different scales.

[0112] In this embodiment, the SimSPPF and the state space network of the jump connection together constitute the backbone network.

[0113] In the second step, features at different levels are fused in a top-down and bottom-up manner to obtain a fused feature map, which provides sufficient and effective information for bounding box regression and category prediction.

[0114] In this embodiment, the feature graphs obtained from the 4th, 6th and 9th layers of the backbone network are selected and input into the feature fusion layer to fuse the features at different levels. The feature fusion layer includes a first upsampling module, a first aggregation module, a first SPSS module, a second upsampling module, a second aggregation module, a second SPSS module, a first GSConv module, a third aggregation module, a third SPSS module, a second GSConv module, a fourth aggregation module and a fourth SPSS module, which are connected in sequence, and the first SPSS module is connected to the third aggregation module.

[0115] In the third step, the three fused feature maps are introduced into the branch decoupled detection head for prediction to determine the categories of coal and gangue and their corresponding position coordinates.

[0116] Specifically, the fused feature map is introduced into the branch decoupled detection head for prediction. The detection head adopts a prediction branch decoupling structure, that is, two prediction branches are formed; each bounding box predicts a group of categories, and the category with the largest score is obtained as the category of the bounding box, and the score is also the confidence of the bounding box; the outputs of the two branch decoupling heads are connected as the total output of the model, and the total output includes the categories of coal and gangue and their corresponding position coordinates; the two branches are: the prediction branch of the category and score of the target box, and the prediction branch of the coordinate information of the target box.

[0117] The calculation formula for the size of the total output of the model is:

[0118] N′=(N1×N1+N2×N2+N3×N3)×(cls+bbox reg ), where N′ represents the size of the total output, N1, N2, and N3 represent the different scales of the feature map, cls represents the number of categories, and bbox reg Indicates the number of coordinate values ​​of the predicted sample box.

[0119] So far, this embodiment has obtained a coal gangue identification model. The overall structural diagram of the coal gangue identification model is as follows: Figure 4 shown.

[0120] S3, use the training set to train the gangue recognition model, and after each round of training, use the verification set to calculate the model loss function value, save the gangue recognition model corresponding to the minimum loss function value of the current model, until the set number of rounds is completed, and obtain the trained gangue recognition model.

[0121] In this embodiment, the number of iterations required for training the entire training set is set to s, and the number of images read in at a time is q; dynamic sample allocation is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each coal gangue image is calculated. total , and update the network parameters using momentum stochastic gradient descent. Then, calculate the average of the model loss function on the validation set, that is, calculate the average of all the loss function values ​​calculated so far, save the network model that minimizes the average loss function so far, and repeat the training of the model until the set number of rounds is completed or the loss function does not decrease significantly.

[0122] Among them, the validation set is used to calculate the model loss function value, and the calculation formula can be:

[0123] Where, L totalrepresents the loss function value, n positive Represents the number of positive sample frames of gangue images predicted by the model in the training set, n pred represents the number of all sample frames of coal gangue images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of (1-DIoU_i), w DFL_i represents the loss weight of DFL_i, w VFL express The loss weight of DIoU_i represents the distance intersection over union corresponding to the i-th positive sample box, DFL_i represents the distribution focal loss corresponding to the i-th positive sample box, and L VFL _i represents the varifocal loss of the category corresponding to the i-th prediction box, n pred Represents the number of predicted sample boxes of gangue images predicted by the model in the training set.

[0124] Among them, the positive sample boxes in all detection boxes in the image are obtained by adopting the task aligned assigner method.

[0125] S4, obtaining the pre-processed gangue image to be detected, inputting the pre-processed gangue image to be detected into the trained gangue recognition model, and obtaining the gangue intelligent recognition result.

[0126] In this embodiment, the trained gangue recognition model and the pre-processed gangue image to be detected are loaded, that is, the pre-processed gangue image to be detected is input into the trained gangue recognition model, so that the coal and gangue in the mixture of the gangue image can be identified and screened to obtain the gangue intelligent recognition result. Among them, the schematic diagram of the gangue intelligent recognition result obtained by the present invention is as follows Figure 5 As shown, the precision-recall curve diagram is as follows Figure 6 As shown, in Figure 6 0 represents gangue and 1 represents coal.

[0127] The comparison results of the accuracy and parameter quantity between the present invention and other methods are shown in Table 1. The resolution of the input images used by each method in Table 1 is 640*640.

[0128] Table 1

[0129]

[0130] In Table 1, it can be seen that the accuracy (AP) of this embodiment is 0.5-1.5% higher than that of other lightweight models while maintaining a low number of parameters and floating-point operations (FLOPs). The coal gangue intelligent identification method based on the selective state space equation provided by this embodiment has higher recognition accuracy.

[0131] So far, this embodiment has obtained the coal gangue intelligent recognition result of the coal gangue image to be detected.

[0132] S5, based on the intelligent recognition result of coal gangue, determine whether the recognition result is satisfactory and obtain the final recognition result.

[0133] If you are satisfied with the intelligent recognition result of coal gangue, the intelligent recognition result of coal gangue will be used as the final recognition result; if you are not satisfied with the intelligent recognition result of coal gangue, adjust the target confidence threshold and the intersection-over-union ratio threshold of maximum suppression according to the model recognition situation, and return to the adjusted trained coal gangue recognition model to obtain the intelligent recognition result of coal gangue until the standard model recognition situation is reached.

[0134] Among them, the intelligent identification results of coal gangue include the number and location information of coal and gangue. The standard model recognition situation can be set by the implementer according to the specific recognition requirements without specific restrictions; the model recognition situation can be missed detection or multiple detection, and the same target corresponds to multiple detection frames or multiple targets that are very close to each other have only one detection frame or fewer detection frames; for missed detection, the target confidence threshold can be reduced, for multiple detections, the target confidence threshold can be increased, for the same target corresponding to multiple detection frames, the intersection-over-union ratio threshold can be increased, and for multiple targets that are very close to each other have only one detection frame or fewer detection frames, the intersection-over-union ratio threshold can be reduced. The target confidence threshold and the intersection-over-union ratio threshold of maximum suppression are both hyperparameters in the model, and the size can be set manually. Generally, the two thresholds can be set to 0.5, and the amplitude of adjusting the hyperparameters can also be set manually, and the empirical value can be taken as 0.05.

[0135] In summary, the present invention proposes an intelligent gangue identification method based on selective state space equations, which aims to solve the problems existing in the existing gangue identification methods, improve the accuracy and efficiency of gangue identification, and also provide strong support for the comprehensive utilization of coal resources and environmental protection.

[0136] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A coal gangue intelligent identification method based on selective state space equation, characterized in that: The following steps are involved: Acquire a coal gangue data set consisting of a preset number of preprocessed coal gangue images, wherein the coal gangue data set is randomly divided into a training set and a validation set according to a preset ratio; Constructing a gangue recognition model, the gangue recognition model includes a state space network, SimSPPF, a feature fusion layer and a branch decoupled detection head, the state space network applies a selective state space equation; The gangue recognition model is trained using the training set, and after each round of training, the model loss function value is calculated using the validation set, and the gangue recognition model corresponding to the minimum value of the current model loss function is saved until the set number of rounds is completed to obtain the trained gangue recognition model; The pre-processed gangue image to be detected is obtained, and the pre-processed gangue image to be detected is input into the trained gangue recognition model to obtain the gangue intelligent recognition result.

2. A coal gangue intelligent identification method based on selective state space equation, characterized in that: After obtaining the coal gangue intelligent identification result, it also includes: If the intelligent recognition result of coal gangue is satisfactory, the intelligent recognition result of coal gangue will be used as the final recognition result; if the intelligent recognition result of coal gangue is not satisfactory, the target confidence threshold and the intersection-over-union ratio threshold of maximum suppression will be adjusted according to the model recognition situation, and the adjusted trained coal gangue recognition model will be returned to obtain the intelligent recognition result of coal gangue until the standard model recognition situation is reached; wherein, the intelligent recognition result of coal gangue includes the number and location information of coal and gangue.

3. The method for intelligent identification of coal gangue based on selective state space equation according to claim 1, characterized in that: The step of obtaining a coal gangue data set consisting of a preset number of pre-processed coal gangue images comprises: A preset number of gangue images are collected using a photographing device; coal and gangue in each gangue image are annotated using an annotation tool to obtain annotation information of each gangue image; The annotation information of each gangue image is saved as a corresponding XML tag file, and then the XML tag file is converted into a TXT tag file in DOTA format to obtain a TXT tag file corresponding to each gangue image; Use DOTA_devkit to segment each gangue image to obtain several regional images of preset resolutions corresponding to each gangue image; The TXT tag file corresponding to each gangue image and the plurality of regional images with preset resolutions are combined to obtain a gangue data set consisting of all preprocessed gangue images.

4. The method for intelligent identification of coal gangue based on selective state space equation according to claim 3 is characterized in that: The gangue recognition model is used to perform feature extraction, feature fusion and intelligent detection on the preprocessed gangue image, including: Read the preprocessed gangue image and its annotation information, use the state space network to perform data processing on the read information, obtain the global field of view of the gangue image, allocate attention to the target area in the global field of view of the gangue image, and extract the spatial information and semantic information of the target area; wherein the target area is a pixel area corresponding to coal or gangue; The SimSPPF of YOLOv6 is used after the state-space network to further extract feature information of the target area at different scales; In a top-down and bottom-up manner, feature information at different scales is selected and input into the feature fusion layer for feature fusion to obtain a fused feature map; The fused feature map is introduced into the branch decoupled detection head for prediction, and the outputs of the two branch decoupling heads are connected as the total output of the model, which includes the categories of coal and gangue and their corresponding position coordinates; wherein the two branches are: a prediction branch for the category and score of the target frame, and a prediction branch for the coordinate information of the target frame.

5. The method for intelligent identification of coal gangue based on selective state space equation according to claim 4 is characterized in that: The method of allocating attention to the target area in the global field of view of the gangue image and extracting spatial information and semantic information of the target area includes: The state space network is composed of 9 layers of GSConv and SPSS modules. The SPSS module shuffles the n channels of the input global view and splits them into two feature maps. Both feature maps have channels; where n represents the number of channels in the global field of view; For any feature map, linear embedding is first performed, and then the feature map is input into a 3×3 deep convolutional layer, and the output result of the deep convolutional layer is subsequently input into a two-dimensional selective image scanning module to obtain an updated feature map; The updated feature map is normalized by the layer and then added to the original feature map to form a cascade The feature map of the channels; The feature maps of the channels are fused through a 1×1 convolutional layer to obtain the feature maps of n channels as the final output of the SPSS module.

6. The method for intelligent identification of coal gangue based on selective state space equation according to claim 5, characterized in that: The steps for obtaining the updated feature map include: In the selective image scanning module, each channel of the feature map is expanded into a block sequence along the upper left-lower right and lower right-upper left in the horizontal and vertical directions, with a dimension of Among them, a represents the width of the expanded block sequence, and b represents the length of the expanded block sequence; Each block sequence is input into the discretized selective state space equation for updating, and each updated block sequence is output; every 4 updated block sequences are fused and restored into a 2D channel, and a total of The feature map of the channels is restored The feature map of each channel is used as the updated feature map.

7. The method for intelligent identification of coal gangue based on selective state space equation according to claim 6, characterized in that: The derivation process of the discretized selective state space equation includes: The general expression of the continuous-time state space equation is: ′ (t)=Ax(t)+Bu(t); y(t)=Cx(t)+Du(t); where, x ′ Represents state variables The derivative of , t represents the current moment, Indicates input, Indicates the output, represents the state transition moment, represents the matrix of input variables, represents the observation matrix, represents the feedforward matrix, A, B, C, and D are all learnable matrices, and n, m, and p represent the number of rows or columns of the matrix; The state space equation is discretized by using the zero sequence holding method. For the differential equation x ′ The general solution of (t) = Ax(t) + Bu(t) is Let t0 = t k and t = t k+1 , and t k+1 -t k =Δ as the step size, then we have In the formula, e represents a natural constant, t0 represents the time t0, τ represents the integral variable, and t k Indicates t k Time, t k+1 Indicates t k+1 Time, x k+1 represents the state variable at time k+1, x k represents the state variable at time k, u k represents the input at time k; Then, the discretized linear state space equation is obtained, including: In the formula, I represents the identity matrix, represents the field of real numbers; in, Use the first-order Taylor expansion to approximate, specifically: When k = 0, Without considering the feedforward matrix D, the convolution operation is used to convert y k Expand to get: Where k represents the kth moment; By determining the convolution kernel in the convolution operation, the sequence operation of the state space model is converted into a convolution operation, as shown below: Among them, u=[u0,u1,....,u k ], u represents the input at all times, and * represents the convolution operation; The HiPPO matrix initialization method is used to initialize A, specifically: In the formula, A n′k′ represents the element in the n′th row and k′th column of matrix A, where n′ represents the number of rows and k′ represents the number of columns; The linear state space equation is transformed into a selective state space equation, specifically: The matrix A remains unchanged, B, C and Δ are functions of the input u, and the expression of the selectivity state space equation is: The output is represented as:

8. The method for intelligent identification of coal gangue based on selective state space equation according to claim 4, characterized in that: The calculation formula for the size of the total output is: N′=(N1×N1+N2×N2+N3×N3)×(cls+bbox reg ), where N′ represents the size of the total output, N1, N2, and N3 represent the different scales of the feature map, cls represents the number of categories, and bbox reg Indicates the number of coordinate values ​​of the predicted sample box.

9. The method for intelligent identification of coal gangue based on selective state space equation according to claim 1, characterized in that: The calculation formula of the loss function value is: Where, L total represents the loss function value, n positive Represents the number of positive sample frames of gangue images predicted by the model in the training set, n pred represents the number of all sample frames of coal gangue images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of (1-DIoU_i), w DFL_i represents the loss weight of DFL_i, w VFL express The loss weight of DIoU_i represents the distance intersection over union corresponding to the i-th positive sample box, DFL_i represents the distribution focal loss corresponding to the i-th positive sample box, and L VFL _i represents the varifocalloss of the category corresponding to the i-th prediction box, n pred Indicates the number of predicted sample frames of the gangue image predicted by the model in the training set; Among them, the positive sample boxes in all detection boxes in the image are obtained by adopting the task aligned assigner method.

Citation Information

Cited By

  • Ore conveying monitoring method and system based on computer images

    CN120339963A

  • Coal gangue tracking method based on hand-eye system of sorting robot

    CN121616851A

  • Coal gangue tracking method based on sorting robot hand-eye system

    CN121616851B