Coal gangue target detection neural network and coal gangue identification method and system

Abstract

The invention belongs to the technical field of coal processing, and particularly relates to a coal gangue target detection neural network and a coal gangue recognition method and system. Comprising a conv layer, a conv layer, a DC3 module I, a conv layer, a DC3 module II, a conv layer, a DC3 module III, a conv layer, a DC3 module IV, an SE attention mechanism module and an SPPF module which are connected in sequence; the multi-scale feature fusion network comprises a conv layer I, a UPsample layer, a first BiFPN module, a C3 module I, a conv layer II, a UPsample layer, a second BiFPN module and a C3 module II which are arranged from bottom to top; the device further comprises a conv layer III, a third BiFPN module, a C3 module III, a conv layer, a fourth BiFPN module and a C3 module IV which are arranged from top to bottom. And the detection head network comprises an upper-layer branch, a middle-layer branch and a lower-layer branch which are used for parallel processing and are all used for outputting a feature map containing coordinate information, confidence and category probability. According to the sorting system integrating dual-mode sensing and digital twinning, full-process intelligentization of coal gangue sorting is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal processing, and in particular relates to a gangue target detection neural network, a gangue identification method and a system. Background Art

[0002] In the coal production process, the separation of coal gangue has always been a key link. The traditional manual sorting method is inefficient, costly and has unstable accuracy.

[0003] With the development of automation technology, robot sorting has gradually been used, but the existing robot sorting systems are all serial robot sorting gangue systems. There are still many shortcomings in gangue recognition accuracy, robot collaboration efficiency, gangue grabbing speed and system intelligence.

[0004] Digital twin technology provides innovative ideas for solving these problems by building precise mapping and real-time interaction between physical entities and virtual models, as well as the rapid response and high precision characteristics of parallel robots themselves. Combined with advanced visual recognition technology, it can significantly improve the performance of coal gangue sorting systems. Summary of the Invention

[0005] In order to solve the existing problems, the present invention provides a gangue target detection neural network, a gangue identification method and a system.

[0006] The present invention adopts the following technical solution: a gangue target detection neural network, comprising: Feature extraction backbone network: It includes the conv layer, conv layer, DC3 module I, conv layer, DC3 module II, conv layer, DC3 module III, conv layer, DC3 module IV, SE attention mechanism module and SPPF module. Multi-scale feature fusion network: including the bottom-up arrangement of conv layer I, UPsample layer, the first BiFPN module, C3 module I, conv layer II, UPsample layer, the second BiFPN module, and C3 module II; It also includes the conv layer III, the third BiFPN module, the C3 module III, the conv layer, the fourth BiFPN module, and the C3 module IV arranged from top to bottom; Detection head network: This includes parallel processing of upper, middle, and lower branches, all of which are used to output feature maps containing coordinate information, confidence, and category probabilities. DC3 module II also outputs to the second BiFPN module; DC3 module III also outputs to the first BiFPN module; SPPF module outputs to the lowest conv layer of the multi-scale feature fusion network; conv layer I also outputs to the fourth BiFPN module; conv layer II also outputs to the third BiFPN module; C3 module II outputs to both conv layer III and the upper branch of the detection head network; C3 module III also outputs to the middle branch of the detection head network; C3 module IV outputs to the lower branch of the detection head network.

[0007] In some embodiments, the DC3 module includes: The first branch uses 1×1 convolution to reduce the dimension and stack multiple Bottleneck blocks to extract deep semantic features. The second branch is directly transferred through convolution dimensionality reduction; The outputs of the first and second branches are concatenated through Concat and then adjusted to the target output dimension through convolution.

[0008] In some embodiments, the SE attention mechanism module includes: The upper branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The lower branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The connection layer further performs a weighted operation on the new aggregation operation obtained by the two squeezing and activation operations on the upper branch layer and the lower branch layer, and finally outputs H×W×C features.

[0009] In some embodiments, the upper branches, middle branches, and lower branches include: Branch I, comprising two 3×3 conv layers and a 1×1 conv layer connected sequentially, outputting a feature tensor containing target classification information; Branch II, wherein the branch II includes connecting two 3×3 conv layers in sequence, and then connecting two parallel 1×1 conv layers, one for outputting a tensor containing the bounding box regression task feature, and the other for outputting a tensor containing the target object feature; The feature tensor containing target classification information, the feature tensor containing the bounding box regression task, and the feature tensor containing the target object are concatenated through Concat to output a feature map containing coordinate information, confidence, and category probability.

[0010] A method for identifying coal gangue, comprising: Collect images containing coal and coal gangue in various situations, and process the images to obtain training sets, validation sets, and test sets; The training set is input into the gangue target detection neural network for training. Evaluate, validate, and test the trained gangue target detection neural network; Collect image data and input the image data into the trained gangue target detection neural network for recognition.

[0011] In some embodiments, the image processing process includes: S100: Using cameras and other image acquisition equipment, we capture a large number of images of coal and gangue in various lighting conditions, including direct light, low light, backlight, and side light, and from multiple angles, including overhead, horizontal, and oblique shots. This data is collected in various scenarios, including underground coal mines, coal preparation plants, and coal piles. This ensures that all forms of gangue are covered and the integrity of the dataset is guaranteed. S200: labeling the collected images, creating category labels for coal and gangue respectively, accurately marking the locations of the two with rectangular boxes in each image, and recording the corresponding categories; S300: Perform random rotation, horizontal and vertical flipping, scaling, cropping, and adding salt and pepper or Gaussian noise to the original image to generate a diverse dataset while maintaining the integrity of the dataset; S400: Divide the labeled data into training set, validation set and test set according to scientific proportions.

[0012] In some embodiments, the process of training a gangue target detection neural network includes: S500: Set key training parameters, including the learning rate that determines the parameter update step size, the batch size that affects the training speed and memory usage, and the number of training rounds that determines the training depth; S600: The divided training and validation sets are efficiently batch processed and randomly shuffled using the data loader, and then loaded into the model training process; S700: Input the preprocessed data into the constructed model to start training. During training, the model calculates the loss function based on the difference between the annotation and prediction, and uses the backpropagation algorithm to reversely adjust the parameters to reduce the loss value, improve the prediction accuracy, and identify coal and coal gangue.

[0013] A system for sorting coal gangue, comprising: A feeding subsystem, wherein the feeding subsystem has a vibrating screen for vibrating the stacked coal and gangue apart so that the coal and gangue do not block each other's view; A material conveying system including a conveyor belt having a staggered stop block arrangement unit; A visual recognition subsystem, wherein the visual recognition subsystem is equipped with an adjustable light source, an industrial camera, and a recognition module for identifying gangue using a coal gangue recognition method; An X-ray identification subsystem, the X-ray identification subsystem comprising a collimator, an X-ray source, and a detector; Multi-parallel robot, with a spatially parallel structure of robotic arms and end effectors, used for sorting; Digital twin model construction system to achieve real-time data interaction between physical entities and virtual models; Among them, the visual recognition subsystem and the X-ray recognition subsystem constitute a two-level recognition system. The recognition results are optimized through the digital twin system to control the multi-parallel robots to perform sorting operations.

[0014] In some embodiments, in the X-ray identification subsystem, the X-ray source emits an X-ray beam to penetrate coal and gangue; the collimator limits the divergence angle of the X-ray beam; the X-ray detector detects the penetration intensity of the emitted X-rays and returns data, thereby identifying coal and gangue.

[0015] In some embodiments, the digital twin model construction system synchronizes the physical entity and the digital twin model through the virtual-real data interface to construct a digital twin model including a conveyor belt, a kinematic model of a multi-parallel robot sorting system, and the physical properties of coal gangue. The virtual-real data interface realizes two-way data interaction between the physical system and the digital twin model through the OPC UA / ROS protocol.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The backbone neural network utilizes the Deformable Convolution Enhanced CSP module (DCSP), introducing a deformable convolution kernel (Deformable Conv) in the C3 layer to adaptively adjust convolution sampling points, better capturing geometric deformations and enhancing target feature extraction. A channel-wise attention mechanism (SENetv2) is embedded after the C3 layer to automatically learn the importance of each channel and assign different weights to different channels. This allows for channel-level feature calibration of the feature maps output by the C3 layer, suppressing unimportant channels.

[0017] The feature fusion part adopts the Bifpn structure. Bifpn improves the trade-off between accuracy and efficiency through efficient bidirectional cross-scale connections and repeated block structures, obtains richer semantic features and position features, and improves the efficiency of feature fusion. At the same time, Bifpn adopts a weighted fusion mechanism, which can achieve comparable accuracy with other methods while minimizing computational costs.

[0018] Head - A new decoupled head model is used. After the initial 1×1 convolutional layer operation, it splits into two routes. The top path continues processing with two 3×3 convolutional layers, followed by a 1×1 convolutional layer, and outputs two feature tensors containing feature maps of category information, position (Reg) information, and object presence (Obj) information. This improves the overall performance of the model and significantly reduces the performance loss caused by task conflicts.

[0019] A sorting system that integrates dual-modal sensing and digital twins enables intelligent, full-process gangue sorting. A collaborative verification mechanism combining visual recognition and X-ray material analysis ensures high gangue identification accuracy, effectively overcoming the limitations of single-sensor technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the overall structure of the multi-parallel robot intelligent gangue sorting system of the present invention; Figure 2 This is the coal and gangue feeding subsystem of the present invention; Figure 3 It is the visual recognition and analysis subsystem of the present invention; Figure 4 The x-ray identification subsystem of the present invention; Figure 5 Flowchart for identifying coal and gangue; Figure 6 This is the neural network architecture diagram for gangue target detection; Figure 7 This is the DC3 module network architecture diagram; Figure 8 This is the attention mechanism network structure diagram of the SE attention mechanism module; Figure 9 This is the workflow diagram of the attention mechanism of the SE attention mechanism module; Figure 10 This is the network structure diagram of the decoupling head model; Figure 11 This is the workflow diagram of the decoupling head model; Figure 12 This is an experimental comparison chart; Figure 13This is the validation set image after training on the coal and gangue dataset; Figure 14 Training set images after training on coal and gangue datasets; In the figure, 1-coal and gangue loading subsystem, 2-conveyor belt, 3-arrangement unit, 4-visual recognition and analysis subsystem, 5-X-ray recognition subsystem, 6-robot truss, 7-parallel robot, 8-coal gangue sorting box, 9-camera truss, 10-adjustable light source, 11-industrial camera, 12-illuminance meter, 13-X-ray detector, 14-X-ray source, 15-collimator, 16-X-ray recognition system truss, 17-hopper, 18-hopper support truss, 19-vibrating screen, 20-support pin. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, a system for sorting coal gangue includes: The feeding subsystem 1 has a vibrating screen 19 for vibrating the stacked coal and gangue apart so that the coal and gangue do not block each other's view; A material conveying system comprising a conveyor belt 2 having an arrangement unit 3 with staggered stoppers; A visual recognition subsystem 4, which is equipped with an adjustable light source 10, an industrial camera 11 and a recognition module, and is used for recognition using a coal gangue recognition method; An X-ray identification subsystem 5, the X-ray identification subsystem 5 including a collimator 15, an X-ray source 14 and a detector 13; A multi-parallel robot 6, having a robotic arm 25 and an end effector in a spatial parallel structure, is used for sorting; Digital twin model construction system to achieve real-time data interaction between physical entities and virtual models; Among them, the visual recognition subsystem 4 and the X-ray recognition subsystem 5 constitute a two-level recognition system. The recognition results are optimized through the digital twin system and then the multi-parallel robot 6 is controlled to perform sorting operations.

[0023] The feeding subsystem 1 is connected to the vibrating screen 19, so that a certain angle is formed between the feeding hopper 17 and the vibrating screen 19. After external feeding, coal and gangue can slide along the feeding hopper 17 onto the vibrating screen 19. The vibrating screen can vibrate the stacked coal and gangue apart so that the coal and gangue will not block each other's sight. The vibrating screen 19 also has a certain angle, allowing the coal to slide onto the conveyor belt while vibrating.

[0024] When the coal and gangue pass through the vibrating screen 19 and slide onto the conveyor belt, the coal and gangue will continue to be carried forward by the conveyor belt, and the coal and gangue will move to the arrangement unit. A large amount of coal and gangue will pass through the limit blocks. The limit blocks are staggered and distributed in each arrangement unit, so that the coal and gangue can be more easily detected after finally passing through the arrangement unit.

[0025] like Figure 3 As shown, the figure shows the visual recognition and analysis subsystem 4, which is covered with a light shield to prevent the coal and gangue from being disturbed by external environmental factors during the visual recognition process. In the figure, the industrial camera 11, the illuminance meter 12, and the adjustable light source 10 are placed on the camera truss. The industrial camera 11 collects image information of the mixture of coal and gangue in all directions and angles, and feeds back information at all times. The adjustable light source 10 can change its own light intensity. Through deep learning, it can distinguish which light is easier to visually identify different coal and gangue, and provide the optimal light intensity for the recognition process of coal and gangue. The illuminance meter 12 monitors the light intensity at all times to ensure that the collected image is clear and accurate, and visually distinguishes coal and gangue during the recognition process.

[0026] like Figure 4 As shown, the figure shows the X-ray identification subsystem 5. After being identified by the visual identification and analysis subsystem 4, some surfaces covered with dust or black gangue, similar in color to coal, may be difficult to identify. The coal is then transported to the X-ray identification subsystem 5 via a conveyor belt 2. As shown in the figure, the X-ray source 14 can emit X-ray beams with a certain energy and intensity to penetrate the gangue sample. When the X-rays penetrate the coal and gangue, they are attenuated, and the degree of attenuation of the X-rays differs significantly. This achieves the purpose of secondary identification of the coal and gangue, increasing the accuracy of the entire device in identifying the coal and gangue. During the identification process, the collimator 15 can limit the divergence angle of the X-ray beam, so that the X-rays illuminate the gangue in a relatively parallel direction, reducing detection errors. The X-ray detector 13 detects the penetration intensity of the emitted X-rays and returns data, thus achieving the identification purpose.

[0027] Coal and gangue are accurately identified by the visual recognition and analysis subsystem 4 and the X-ray recognition subsystem 5. The two systems transmit the data to the multi-parallel robot control subsystem. The multi-parallel robot can plan the optimal sorting path for the multi-parallel robot based on the gangue information provided by the visual recognition and analysis subsystem 4 and the X-ray recognition subsystem 5, as well as the sorting scenario and robot model information in the digital twin model construction subsystem.

[0028] Throughout the sorting process, the digital twin model building subsystem uses data interfaces to achieve real-time data synchronization and bidirectional interaction between the physical entity and the virtual model, enabling the virtual model to accurately reflect the state changes of the physical system. Simultaneously, optimization strategies in the virtual model are fed back into the physical system in real time to guide its operation. The digital twin model building subsystem provides real-time feedback and control of the entire system, identifying potential problems throughout the sorting process and predicting impending issues to prevent potential dangers.

[0029] This system constructs a digital twin model that fully corresponds to the actual multi-parallel robot sorting coal gangue operation scenario. This model encompasses comprehensive information, from the coal conveyor belt, the parallel robot mechanical structure, the end effector, to the physical properties, geometry, kinematics, and dynamic parameters of the coal gangue material itself. The coal gangue material is modeled based on its different size, shape, density distribution, and other characteristics to accurately predict its motion trajectory on the conveyor belt and its interaction with the robot end effector in digital space. Simultaneously, the entire operating environment, such as lighting conditions and spatial layout, is digitally mapped to create a highly realistic virtual operation scene. Building a digital twin model for multi-parallel robots sorting gangue requires starting with system requirements analysis. First, the sorting efficiency, accuracy, and robot collaborative control goals are clarified. 3D scanning or CAD tools are used to build a physical environment model of the sorting scene, and the real-time data that the sensors need to collect (such as robot posture, gangue image, and mechanical parameters) is defined. Then, based on the kinematic and dynamic models of the parallel robots, the actual parameters are calibrated through encoders, force sensors, etc. At the same time, RGB-D cameras or lidars are used to extract the shape and texture features of gangue and build a classification data set. Industrial cameras and force sensor networks are deployed to achieve data synchronization. In the virtual model construction stage, a multi-robot collaborative simulation environment is built with the help of ROS-Gazebo or MATLAB / Simulink, an integrated physics engine is used to simulate the motion characteristics of gangue, an image recognition model is embedded to achieve dynamic classification, and the virtual-real data interface is opened through the OPC UA / MQTT protocol. Then, edge computing (such as NVIDIA Jetson) to achieve low-latency decision-making, adopt Kalman filtering to eliminate sensor noise, develop an adaptive control algorithm to dynamically adjust the robot trajectory, and build a visual monitoring interface through Unity; model verification requires offline simulation to simulate extreme working conditions, combined with hardware-in-the-loop testing (HIL) to optimize the control algorithm, and finally based on on-site sorting data, use reinforcement learning or historical data playback to iterate the sorting strategy; paper writing should focus on the innovation of multi-robot task allocation algorithm and virtual-reality closed-loop optimization logic, quantify efficiency improvement through comparative experiments, and supplement with trajectory heat map, confusion matrix and other visual results to verify the effectiveness of the model, forming a complete technical closed loop from physical perception to virtual deduction, and then to real-time optimization.

[0030] Before the actual system is installed and debugged, virtual debugging can be performed in the digital twin environment. By setting different coal gangue distribution scenarios, robot initial states, and other parameters, the entire sorting system's operational process can be fully tested. The random distribution of coal gangue of different sizes and shapes on the conveyor belt is simulated to verify the rationality of the parallel robot's grasping strategy and whether there is a risk of collision. Based on the optimization function of the digital twin model, the collaborative mode, sorting path, and action timing of multiple parallel robots are optimized. Optimization algorithms such as genetic algorithms are used to find the optimal robot path planning solution in digital space to minimize the robot's movement time and energy consumption while maximizing sorting efficiency. Furthermore, during the actual system operation, the digital twin model can continuously evaluate system performance based on real-time data. When system performance deteriorates or anomalies are detected, the optimization program is promptly launched to adjust relevant parameters and strategies. The optimized solution is then applied to the actual system, achieving self-optimization and adaptive operation of the system.

[0031] Digital twin technology enables extensive testing, optimization, and training in a virtual environment, reducing the trial-and-error costs and debugging time of the actual system. Furthermore, by optimizing the robot's operating parameters and collaborative strategies, it reduces energy consumption and equipment wear, thereby lowering the operating costs and resource consumption of the entire gangue sorting system.

[0032] like Figure 6 As shown, a gangue target detection neural network includes: Feature extraction backbone network: It includes the conv layer, conv layer, DC3 module I, conv layer, DC3 module II, conv layer, DC3 module III, conv layer, DC3 module IV, SE attention mechanism module and SPPF module. Multi-scale feature fusion network: including the bottom-up arrangement of conv layer I, UPsample layer, the first BiFPN module, C3 module I, conv layer II, UPsample layer, the second BiFPN module, and C3 module II; It also includes the conv layer III, the third BiFPN module, the C3 module III, the conv layer, the fourth BiFPN module, and the C3 module IV arranged from top to bottom; Detection head network: This includes parallel processing of upper, middle, and lower branches, all of which are used to output feature maps containing coordinate information, confidence, and category probabilities. DC3 module II also outputs to the second BiFPN module; DC3 module III also outputs to the first BiFPN module; SPPF module outputs to the lowest conv layer of the multi-scale feature fusion network; conv layer I also outputs to the fourth BiFPN module; conv layer II also outputs to the third BiFPN module; C3 module II outputs to both conv layer III and the upper branch of the detection head network; C3 module III also outputs to the middle branch of the detection head network; C3 module IV outputs to the lower branch of the detection head network.

[0033] First, a 640×640×3 image of coal and gangue is input. It passes through the first conv layer (Conv2D performs convolution operations on the input data to extract features. The BN layer then normalizes the Conv2D output to stabilize the data distribution, accelerate convergence, and alleviate gradient problems. Finally, the SiLU activation function introduces nonlinearity to enhance the model's expressiveness, enabling better fit to the data and task performance). This convolution operation extracts initial features from the image and outputs a 320×320×64 feature map, reducing the image size to half. After receiving the feature map output by the first conv layer, the second conv layer continues feature extraction, reducing the size of the feature map to 160×160×128, a quarter of the original image size. This is then transferred to the first DC3 module, which is divided into two branches. Branch 1 performs dimensionality reduction using 1×1 convolutions and stacks multiple Bottleneck blocks. This Bottleneck stacking extracts deep semantic features. Branch 2 performs dimensionality reduction using 1×1 convolutions and passes the data directly to the original image, preserving the original feature information and preventing information loss. Concat is used to concatenate the channel-dimensional outputs, combining deep and shallow features to enhance feature diversity. Finally, a 1×1 convolution is used to resize the output to the target dimension. After passing through the DC3 module, the feature map becomes a 160×160×128 feature map with rich semantic information and abstract features. The feature map output by the DC3 module passes through the third conv layer, resulting in an 80×80×256 feature map. This feature map is then passed to the second DC3 module. This smaller feature map focuses more on capturing high-level semantic features of the image than the output of the first DC3 module. The feature map output by the second DC3 module is first output to the BiFPN module and superimposed with the bottom-up interpolation data to output an 80×80×512 feature map after multi-scale feature fusion. After that, it is divided into two branches (branch 1 and branch 2) for further output after dimensionality reduction through a C3 module. Branch 1 outputs an 80×80×256 feature information map, which is passed to the Head part. Branch 2 continues to output to the conv layer for dimensionality reduction and is spliced with the content output by the subsequent BiFPN module. In the backbone part, the second DC3 module will continue to output downward and be sent to the fourth conv layer of the backbone part. The feature map becomes 40×40×512, which is 1 / 16 of the original image, and then output to the third DC3 module. At this time, the feature map size becomes smaller, and compared with the output of the second DC3, it is more focused on capturing the high-level semantic features of the image.The DC3 module does not change the image size, and outputs it to the BiFPN module. It is superimposed with the bottom-up interpolation data to output a 40×40×1024 feature map after multi-scale feature fusion. The output feature map is reduced in dimension by the C3 module to a 40×40×512 feature map, and then the conv layer outputs a 40×40×256 feature map. The output feature map and the output of branch 2 above enter the BiFPN module for splicing, and the output is a 40×40×512 multi-scale feature fusion feature map. The output content passes through a C3 module and is divided into two branches (branch 3 and branch 4). Branch 3 continues to output downward through a Con The v layer performs dimensionality reduction, converting the 40×40×512 multi-scale feature fusion feature map into a 20×20×512 feature map for splicing with subsequent content. Branch 4 inputs 40×40×512 to the Head part; the output of the third DC3 module in the backbone part will continue to be output downward and sent to the fifth conv layer. The feature map becomes 20×20×1024, which is 1 / 32 of the original image, and then sent to the fourth DC3 module. After the fourth DC3 module, it is output to the SE attention mechanism module, which assigns different weights to different channels, performs channel-level feature calibration on the feature map output by the C3 layer, and suppresses unimportant channels. Finally, it is sent to the SPPF module (the extracted features are further fused and enhanced to provide more representative feature maps for the subsequent Neck and Head parts, thereby improving the target detection performance of the model). After SPPF, the 20×20×1024 feature map is sent to the Neck part Conv module. This Conv module converts the original 20×20×1024 feature map into a 20×20×512 feature map, which is then divided into two branches (branch 5 and branch 6) for transmission. Branch 5 continues to transmit upward, forming the above-mentioned bottom-up interpolation and adding data for splicing with other data to form a bidirectional cross-scale connection and repeated block structure, improving the trade-off between accuracy and efficiency, obtaining richer semantic features and position features, and improving the efficiency of feature fusion. Branch 6 transmits the 20×20×1024 and the downward output part of the above branch 3 to the BiFPN module for splicing, and finally outputs a 20×20×1024 multi-scale feature fusion feature map. After a C3 module, the 20×20×1024 feature map is transmitted to the Head part.After the Head part receives the feature maps processed by the BiFPN module, the three feature maps are then transmitted to the initial 1×1 conv layer, and then split into two 3×3 conv layers for subsequent processing. The upper 3×3 conv layer has a subpath, which generates a feature tensor containing target classification information through a 1×1 conv layer; the lower layer is further divided into two layers after being processed by two 3×3 conv layers, and each layer is processed by a 1×1 conv layer to generate feature tensors of bounding box regression information and target object information.

[0034] like Figure 7 As shown, the DC3 module includes: The first branch uses 1×1 convolution to reduce the dimension and stack multiple Bottleneck blocks to extract deep semantic features. The second branch is directly transferred through convolution dimensionality reduction; The outputs of the first and second branches are concatenated through Concat and then adjusted to the target output dimension through convolution.

[0035] In the backbone neural network module, a deformable convolution-enhanced CSP module (DCSP) is inserted into each Bottleneck block to form a new module DC3. Deformable convolution means that the convolution kernel adds an additional direction parameter to each element, so that the convolution kernel can be expanded to a large range during training, thereby learning more features.

[0036] like Figure 8 As shown, the SE attention mechanism module includes: The upper branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The lower branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The connection layer further performs a weighted operation on the new aggregation operation obtained by the two squeezing and activation operations on the upper branch layer and the lower branch layer, and finally outputs H×W×C features.

[0037] like Figure 8As shown in this figure, the SE attention mechanism module first Figure X Perform a multi-branch squeeze operation (squeeze), and each channel of each branch generates a channel descriptor (directly accumulate the H*W of the channel descriptors of the two branches), and obtain a weight generated by the squeeze process. The squeeze operation is to compress the W×H×C feature map containing global information into a 1×1×C feature vector. This feature vector has a global receptive field, and the formula is: Where: H, W are the feature map sizes; Xc∈RH×W×C is the output feature map, C is the number of channels; Z is the weight generated by squeezing. After the squeezing operation, the conv layer output is input to the global average pooling layer to generate a channel-by-channel input. This input is sent to a fully connected layer with reduced size, followed by the excitation module, which first involves a fully connected layer without any reduction to restore the input to its original form. The fully connected layer is followed by a scaling operation, in which the output is multiplied channel-by-channel with the feature map. The final output is then rescaled to match its original shape. The calculation process can be expressed as: Where, is the Sigmoid function; ∈ Is the generated attention weight. Multiplying the original features with the learned channel weights, the calculation process can be expressed as Where, is the output feature map.

[0038] The integration of the SE attention mechanism module is shown in the formula: Where x represents the input; Sq function represents the squeeze operation; Ex operation represents the excitation operation; F() represents the operation to modify the input.

[0039] The deformable convolution Dconv module is used to enhance the ability to extract target features, and then the SE attention mechanism module is used to reasonably distribute the importance of feature channels, so as to achieve the purpose of improving the accuracy of coal and coal gangue features; the input of the backbone neural network is the image to be trained, and the output is the feature map of the P3-P7 layers.

[0040] like Figure 9The figure shows the feature fusion process. The BiFPN module introduces bidirectional connections and weighted fusion. Bidirectional connections allow BiFPN to introduce high-to-low and low-to-high resolution connections into the feature pyramid network, propagating information across different scales and promoting fusion. Weighted fusion can better learn the importance of feature information at different scales. Compared to the traditional approach of treating all input features equally, the BiFPN module adds a weight to adjust the contribution of important information. The network learns the importance of each feature, employing fast normalization fusion to constrain Wi to [0, 1] and avoid softmax operations. The formula is as follows: Where: e is the minimum learning rate, which constrains the numerical oscillation and is usually set to 0.0001; O is the output value; W i and W j is the learning weight; I i This method makes the weight values of different importance range from 0 to 1, thus achieving the purpose of efficient and fast normalized fusion of features.

[0041] Taking the P5 layer as an example, the feature fusion calculation formula is: Where, is the intermediate feature of the 5th layer of the top-down path, is the input feature of the 5th layer, Resize is used for resolution matching, is the output feature of the 5th layer of the bottom-up path.

[0042] Depend on Figure 9 You can see the network structure diagram of the BiFPN module, and you can see how features of different scales (P2 to P7) are fused through the upper and lower bidirectional paths. This structural design can maximize the effect of feature fusion while maintaining computational efficiency, thereby improving the overall performance of object detection. Figure 9 The network architecture diagram containing the BiFPN module can be seen in Figure 2. After receiving the P3-P5 feature maps from the backbone neural network, the feature fusion is performed to output a multi-scale feature fusion and optimized feature map.

[0043] BiFPN adaptively adjusts the fusion weights of multi-scale deformation features through bidirectional cross-scale connections and weighted feature fusion. The SE attention mechanism module emphasizes the material feature differences between coal and gangue through channel-level attention, but lacks attention to the spatial dimension. BiFPN's weighted fusion mechanism can be viewed as implicit spatial attention, forming a channel-spatial dual-domain attention synergy with the SE attention mechanism module. Uneven lighting in mines blurs the surface reflectance differences between coal and gangue. BiFPN suppresses the negative impact of low-quality feature layers through a feature reweighting mechanism. Coal and gangue often have material feature confusion due to the adhesion of coal dust on the surface. BiFPN's rapid normalization fusion automatically suppresses conflicting features through gradient descent, improving recognition accuracy.

[0044] like Figure 11 As shown, the upper branch, middle branch and lower branch have the same structure, including: Branch I, comprising two 3×3 conv layers and a 1×1 conv layer connected sequentially, outputting a feature tensor containing target classification information; Branch II, wherein the branch II includes connecting two 3×3 conv layers in sequence, and then connecting two parallel 1×1 conv layers, one for outputting a tensor containing the bounding box regression task feature, and the other for outputting a tensor containing the target object feature; The feature tensor containing target classification information, the feature tensor containing the bounding box regression task, and the feature tensor containing the target object are concatenated through Concat to output a feature map containing coordinate information, confidence, and category probability.

[0045] First, the input of the upper branch is the aforementioned branch 1 (80×80×256 feature map), which is the P3 feature map.

[0046] Initially input a 1×1conv layer, the 1×1conv layer reduces the number of channels of the 80×80×256 feature map and outputs a 80×80×128 feature map, which is then divided into two 3×3conv layers. The 3×3conv layer does not affect the number of channels. The upper branch passes through two 3×3conv layers and outputs a 80×80×128 feature map, which is input to the 1×1conv layer. The 1×1conv layer adjusts the 80×80×128 feature map to a 80×80×NC feature map (NC is the number of detected object categories) and outputs a feature tensor containing target classification information. The following two 3×3conv layers continue to split, and the middle branch passes through After two 3×3 conv layers output a feature map of 80×80×128, it is input to the 1×1 conv layer. The 1×1 conv layer adjusts the feature map of 80×80×128 to a feature map of 80×80×4 (4 represents the coordinate parameter) and outputs the feature tensor containing the bounding box regression task. The lower branch outputs a feature map of 80×80×128 after two 3×3 conv layers and inputs it to the 1×1 conv layer. The 1×1 convolution adjusts the feature map of 80×80×128 to 80×80×1 (1 represents the probability of the existence of the object) and outputs the feature tensor containing the target object. Finally, the results are integrated together to output a feature map containing coordinate information, confidence, and category probability.

[0047] The input of the middle branch is the 40×40×512 feature map mentioned above, which is the P4 feature map Initially input a 1×1conv layer, the 1×1conv layer conv layer reduces the number of channels of the 40×40×512 feature map and outputs a 40×40×256 feature map, which is then divided into two 3×3conv layers. The 3×3conv layer does not affect the number of channels. The upper branch passes through two 3×3conv layers and outputs a 40×40×256 feature map, which is input to the 1×1conv layer. The 1×1conv layer adjusts the 40×40×256 feature map to 40×40×N CThe feature map (NC is the number of detected object categories) is output, and the feature tensor containing target classification information is output. The following two 3×3conv layers continue to split, and the middle branch outputs a 40×40×256 feature map after two 3×3conv layers, which is input to the 1×1conv layer. The 1×1conv layer adjusts the 80×80×128 feature map to a 40×40×4 (4 represents the coordinate parameter) feature map, and outputs a feature tensor containing the bounding box regression task. The lower branch outputs a 40×40×256 feature map after two 3×3conv layers, which is input to the 1×1conv layer. The 1×1 convolution adjusts the 40×40×256 feature map to 40×40×1 (1 represents the probability of the existence of the object), and outputs a feature tensor containing the target object. Finally, the results are integrated together to output a feature map containing coordinate information, confidence, and category probability.

[0048] The input of the lower branch is the 20×20×1024 feature map mentioned above, which is the P5 feature map Initially input a 1×1conv layer, the 1×1conv layer conv layer reduces the number of channels of the 20×20×1024 feature map and outputs a 20×20×512 feature map, which is then divided into two 3×3conv layers. The 3×3conv layer does not affect the number of channels. The upper branch passes through two 3×3conv layers and outputs a 20×20×512 feature map, which is input to the 1×1conv layer. The 1×1conv layer adjusts the 20×20×512 feature map to 20×20×N C The feature map (NC is the number of detected object categories) is output, and the feature tensor containing target classification information is output. The following two 3×3conv layers continue to split, and the middle branch outputs a 20×20×512 feature map after two 3×3conv layers, which is input to the 1×1conv layer. The 1×1conv layer adjusts the 80×80×128 feature map to a 20×20×4 (4 represents the coordinate parameter) feature map, and outputs a feature tensor containing the bounding box regression task. The lower branch outputs a 20×20×512 feature map after two 3×3conv layers, which is input to the 1×1conv layer. The 1×1 convolution adjusts the 20×20×512 feature map to 20×20×1 (1 represents the probability of the existence of the object), and outputs a feature tensor containing the target object. Finally, the results are integrated together to output a feature map containing coordinate information, confidence, and category probability.

[0049] Figure 11The figure below illustrates the decoupled head workflow. After the initial 1×1 conv layer, it splits into two 3×3 conv layers for further processing. The upper 3×3 conv layer has a subpath that generates a feature tensor containing object classification information through a 1×1 conv layer. The lower layer, after processing by two 3×3 conv layers, is further split into two layers, each undergoing a 1×1 conv layer to generate feature tensors for bounding box regression and object information. In the resulting feature map, Nc represents the number of categories, object probability is represented by a value, and coordinates are represented by (x, y, w, h). The improved module inputs the multi-scale feature fusion and optimization feature map of the BiFPN module, and outputs feature tensors for object classification, bounding box regression, and object information.

[0050] like Figure 11 The figure shows the structure of the Decoupled head network model. The Decoupled head module can avoid the problem of focusing on certain tasks and ignoring other issues after receiving the feature information transmitted by the BiFPN module. When the model pays too much attention to the coal and gangue with obvious features and only identifies them, it will affect the accurate positioning of coal and gangue, resulting in inaccurate positioning of gangue and reduced convergence speed. The Decoupled head module can solve the above problems through decoupling operations.

[0051] like Figure 12 The following are the experimental results of the DBD model. The PR curve of the DBD model has a high degree of fitting, a wide platform area and a higher level of accuracy, indicating that it is less affected by data fluctuations and has strong robustness. The recall curve is very stable and smooth, indicating that the performance in different data subsets or detection scenarios is more stable and robust. The F1 curve of the DBD model has a high degree of fitting, the curve rises quickly, falls slowly, and has a high peak value. The validation set loss is stable, the accuracy is high, and the difference with the training set results is small. The generalization ability is strong, which effectively reduces the risk of overfitting, can adapt to complex scenarios, and greatly reduces false detections and missed detections. The recognition accuracy of the DBD model reaches 96.4%, the F1 score is 95.4%, and the mAP_0.5 is 95.4%. like Figure 13 、 Figure 14 As shown in the figure, the two groups are part of the pictures of the training set and the verification set respectively.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gangue target detection neural network, characterized in that: include: Feature extraction backbone network: It includes the conv layer, conv layer, DC3 module I, conv layer, DC3 module II, conv layer, DC3 module III, conv layer, DC3 module IV, SE attention mechanism module and SPPF module. Multi-scale feature fusion network: including the bottom-up arrangement of conv layer I, UPsample layer, the first BiFPN module, C3 module I, conv layer II, UPsample layer, the second BiFPN module, and C3 module II; It also includes the conv layer III, the third BiFPN module, the C3 module III, the conv layer, the fourth BiFPN module, and the C3 module IV arranged from top to bottom; Detection head network: This includes parallel processing of upper, middle, and lower branches, all of which are used to generate feature maps containing coordinate information, confidence, and class probabilities. DC3 module II also outputs to the second BiFPN module; DC3 module III also outputs to the first BiFPN module; the SPPF module outputs to the lowest conv layer of the multi-scale feature fusion network; conv layer I also outputs to the fourth BiFPN module; conv layer II also outputs to the third BiFPN module; C3 module II outputs to both conv layer III and the upper branch of the detection head network; C3 module III also outputs to the middle branch of the detection head network; C3 module IV outputs to the lower branch of the detection head network.

2. The gangue target detection neural network according to claim 1, characterized in that: The DC3 module includes: The first branch uses 1×1 convolution to reduce the dimension and stack multiple Bottleneck blocks to extract deep semantic features. The second branch is directly transferred through convolution dimensionality reduction; The outputs of the first and second branches are concatenated through Concat and then adjusted to the target output dimension through convolution.

3. The gangue target detection neural network according to claim 1, characterized in that: The SE attention mechanism module includes: The upper branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The lower branch layer includes the conv layer connected in sequence to input the H×W×C feature map, the squeeze operation compresses it into a 1×1×C feature vector, the first fully connected layer of the excitation operation reduces the dimension of 1×1×C, the second fully connected layer restores the dimension of 1×1×C, Scale recalibration, and outputs the H×W×C feature map in which the feature value of each spatial position of each channel is multiplied by a channel-specific weight; The connection layer further performs a weighted operation on the new aggregation operation obtained by the two squeezing and activation operations on the upper branch layer and the lower branch layer, and finally outputs H×W×C features.

4. The gangue target detection neural network according to claim 1, characterized in that: The upper branch, middle branch and lower branch have the same structure, including: Branch I, comprising two 3×3 conv layers and a 1×1 conv layer connected sequentially, outputting a feature tensor containing target classification information; Branch II, wherein the branch II includes connecting two 3×3 conv layers in sequence, and then connecting two parallel 1×1 conv layers, one for outputting a tensor containing the bounding box regression task feature, and the other for outputting a tensor containing the target object feature; The feature tensor containing target classification information, the feature tensor containing the bounding box regression task, and the feature tensor containing the target object are concatenated through Concat to output a feature map containing coordinate information, confidence, and category probability.

5. A method for identifying coal gangue, characterized in that: include: Collect images containing coal and coal gangue in various situations, and process the images to obtain training sets, validation sets, and test sets; The training set is input into the gangue target detection neural network for training. Evaluate, validate, and test the trained gangue target detection neural network; Collect image data and input the image data into the trained gangue target detection neural network for recognition.

6. The method for identifying coal gangue according to claim 5, characterized in that: The image processing process includes: S100: Using cameras and other image acquisition equipment, we capture a large number of images of coal and gangue in various lighting conditions, including direct light, low light, backlight, and side light, and from multiple angles, including overhead, horizontal, and oblique shots. This data is collected in various scenarios, including underground coal mines, coal preparation plants, and coal piles. This ensures that all forms of gangue are covered and the integrity of the dataset is guaranteed. S200: Labeling the collected images, creating category labels for coal and gangue respectively, accurately marking the locations of the two with rectangular boxes in each image, and recording the corresponding categories; S300: Perform random rotation, horizontal and vertical flipping, scaling, cropping, and adding salt and pepper or Gaussian noise to the original image to generate a diverse dataset while maintaining the integrity of the dataset; S400: Divide the labeled data into training set, validation set and test set according to scientific proportions.

7. The method for identifying coal gangue according to claim 5, characterized in that: The process of training the gangue target detection neural network includes: S500: Set key training parameters, including the learning rate that determines the parameter update step size, the batch size that affects the training speed and memory usage, and the number of training rounds that determines the training depth; S600: The divided training and validation sets are efficiently batch processed and randomly shuffled using the data loader, and then loaded into the model training process; S700: Input the preprocessed data into the constructed model to start training. During training, the model calculates the loss function based on the difference between the annotation and prediction, and uses the backpropagation algorithm to reversely adjust the parameters to reduce the loss value, improve the prediction accuracy, and identify coal and coal gangue.

8. A system for sorting coal gangue, characterized in that: include: A feeding subsystem (1), wherein the feeding subsystem (1) has a vibrating screen (19) for vibrating the stacked coal and gangue apart so that the coal and gangue do not block each other's view; A material conveying system comprising a conveyor belt (2) having a staggered limit block arrangement unit (3); A visual recognition subsystem (4), the visual recognition subsystem (4) being equipped with an adjustable light source (10), an industrial camera (11) and a recognition module, and being used for recognition using the coal gangue recognition method according to claim 5; An X-ray identification subsystem (5), the X-ray identification subsystem (5) comprising a collimator (15), an X-ray source (14) and a detector (13); A multi-parallel robot (6) having a spatially parallel structured robotic arm (25) and an end effector for sorting; Digital twin model construction system to achieve real-time data interaction between physical entities and virtual models; The visual recognition subsystem (4) and the X-ray recognition subsystem (5) form a two-level recognition system, and the recognition results are optimized through the digital twin system to control the multi-parallel robot (6) to perform the sorting operation.

9. The system for sorting coal gangue according to claim 8, characterized in that: In the X-ray identification subsystem (5), the X-ray source (14) emits an X-ray beam to penetrate coal and gangue; the collimator (15) limits the divergence angle of the X-ray beam; and the X-ray detector (13) detects the penetration intensity of the emitted X-rays and returns data, thereby identifying coal and gangue.

10. The system for sorting coal gangue according to claim 8, characterized in that: The digital twin model construction system synchronizes the physical entity with the digital twin model through the virtual-real data interface, and constructs a digital twin model including the kinematic model of the conveyor belt, the multi-parallel robot sorting system (6) and the physical characteristics of coal gangue. The virtual-real data interface realizes two-way data interaction between the physical system and the digital twin model through the OPC UA / ROS protocol.

Images