Multi-scale ore sorting method fusing multi-modal gradient representation and contrastive learning

Abstract

The application discloses a kind of ore multiscale sorting methods of fusing multimodal gradient representation and contrast learning, it is related to ore sorting technical field.The method first obtains the ore image to be detected, performs gradient convolution operation in horizontal and vertical directions and calculates gradient amplitude, generates double-channel composite tensor after original gray image and gradient amplitude are normalized and stacked.The tensor is input into multimodal gradient contrast network model, sequentially through residual convolution to extract multiscale feature, generates semantic enhancement feature by semantic embedding module, generates multiscale fusion feature by feature pyramid network, generates end fusion feature by spatial embedding module.At the same time, introduce contrast learning branch to carry out double-loss joint training, finally output ore class prediction probability distribution;The method effectively improves the recognition ability of model to difficult classification samples such as fine spot ore spot and weak ore spot, enhances the feature extraction robustness of model to different size ore particles, and the sorting precision is higher.

Description

Technical Field

[0001] This application relates to the field of ore sorting technology, and in particular to a multi-scale ore sorting method that integrates multimodal gradient representation and comparative learning. Background Technology

[0002] Ore sorting is a crucial step in mineral resource processing. Its core objective is to effectively classify raw ore based on its structural characteristics or inclusions before it enters subsequent crushing, grinding, and beneficiation processes, thereby achieving efficient resource utilization and reducing production costs. Sorting methods are based on physical properties, such as mechanical or physical separation based on differences in density, magnetism, electrical properties, or particle size. With the development of deep learning technology, its powerful capabilities in image processing and classification tasks have provided new solutions for ore sorting.

[0003] For tungsten ore, the specific method is to crush the raw ore to a suitable particle size and then irradiate it in an X-ray field. When the rays penetrate the ore particles, the intensity difference is produced due to the different absorption coefficients of the minerals and gangue. The detector receives and converts the radiation into a grayscale image. Mineral-rich areas appear as dark patches, while areas with low mineral content are brighter.

[0004] However, existing methods still have shortcomings in sorting. Tungsten ore X-ray images exhibit significant scale differences across multiple dimensions. In a single image, a complete tungsten ore grain simultaneously contains a macroscopic geometric outline and tiny mineral spots that require magnification to identify. Furthermore, the inconsistent mineral grain size caused by ore fragmentation leads to variations in image resolution and differences in the outline and proportion of mineral grains within the image. Adding to the challenge, the small area of ​​fine-spotted and weak-spotted mineral spots occupies only a minimal amount of pixel space in the image, making their features easily weakened or lost during multiple downsampling processes in convolutional networks, hindering the model's ability to fully learn their texture and boundary information. Summary of the Invention

[0005] Based on this, this application provides a multi-scale ore sorting method that integrates multimodal gradient representation and comparative learning, aiming to solve the technical problem of low recognition accuracy in tungsten ore sorting caused by the multi-scale nature of tungsten ore X-ray images, large differences in ore particle size, and easy loss of small ore features.

[0006] In a first aspect, embodiments of this application provide a multi-scale ore sorting method that integrates multimodal gradient representation and contrastive learning, including: Acquire an image of the ore to be detected, perform convolution operations and gradient magnitude calculations on the ore image respectively, and generate a dual-channel composite tensor; The dual-channel composite tensor is input into a pre-trained multimodal comparison network model to output ore sorting results; The data processing procedure for the multimodal contrastive network model includes: Residual features are extracted from the dual-channel composite tensor to generate multiple initial feature maps at different stages. The multiple initial feature maps are then input into a preset semantic embedding module in the model to generate semantically enhanced features. Multiple initial feature maps are subjected to pointwise convolution to obtain intermediate feature maps with a uniform number of channels. The intermediate feature maps are upsampled to match the corresponding bypass features and then fused pixel by pixel. After spatial convolution, a fused feature map is generated. The fused feature map and the semantically enhanced feature are input into a preset spatial embedding module in the model, and the target feature map is output. After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results with predicted probability distributions for each category are obtained.

[0007] In some embodiments, the step of acquiring the ore image to be detected, performing convolution operations and gradient magnitude calculations on the ore image respectively, and generating a dual-channel composite tensor includes: The ore image is subjected to gradient convolution operations in the horizontal and vertical directions to obtain a horizontal gradient response map and a vertical gradient response map. Based on the horizontal gradient response map and the vertical gradient response map, the fused gradient magnitude is calculated, and the gradient magnitude is normalized with the ore image and a channel stacking operation is performed to generate the dual-channel composite tensor.

[0008] In some embodiments, the steps of calculating the fused gradient magnitude based on the horizontal gradient response map and the vertical gradient response map, and uniformly normalizing the gradient magnitude with the ore image and performing a channel stacking operation to generate the dual-channel composite tensor include: The expression for calculating the horizontal gradient response plot is:

[0009] The formula for calculating the vertical gradient response map is:

[0010] The expression for calculating the gradient magnitude is:

[0011] The expression for calculating the two-channel composite tensor is:

[0012] In the formula, The image shows the ore to be inspected. This is a gradient convolution operation. ( () is for channel stacking operations.

[0013] In some embodiments, the steps of extracting residual features from the dual-channel composite tensor to generate multiple initial feature maps at different stages, and inputting the multiple initial feature maps into a preset semantic embedding module in the model to generate semantically enhanced features, include: The dual-channel composite tensor is subjected to four stages of residual convolution processing in sequence. Each stage is processed by multiple residual blocks in sequence, and each stage outputs the initial feature map with a different scale. The data processing procedure of the semantic embedding module includes: The initial feature maps output from the corresponding stages of the low-level and deep-level layers are used as low-level edge features and deep-level semantic features, respectively. The low-level edge features and the deep semantic features are aligned by convolution, and the channel-aligned deep semantic features are upsampled to make their spatial resolution consistent with that of the low-level edge features. The upsampled deep semantic features are processed through an activation function to generate a global attention weight map; The global attention weight map and the low-level edge features are subjected to a matrix dot product operation to generate semantically enhanced features.

[0014] In some embodiments, the step of inputting the fused feature map and the semantically enhanced feature into a preset spatial embedding module in the model and outputting a target feature map includes the following data processing procedure of the spatial embedding module: Alignment operations are performed on the semantic enhancement features and the fused feature map for both channel and spatial scale. The aligned fused feature map and the semantic enhancement features are then added pixel-by-pixel. The added features are then convolved and smoothed to generate the target feature map. The corresponding algorithm expression is as follows:

[0015] In the formula, For the target feature map, ( () represents the convolution operation. For semantic enhancement features, To fuse feature maps, ( ) represents an upsampling operation.

[0016] In some embodiments, the multimodal contrastive network model further includes a contrastive learning module for model optimization, wherein the data processing procedure of the contrastive learning module includes: Receive the high-dimensional feature map from the fused feature map, and perform spatial tensor conversion and spatial dimension compression operations on the high-dimensional feature map to obtain the global feature vector; The global feature vector is input into a projection head composed of a multilayer sensing mechanism and subjected to nonlinear transformation to obtain the projection feature vector. The projected feature vector is normalized, and a first loss value is calculated from the feature vectors of positive samples with the same class label as the current sample after normalization. The functional expression of the first loss value is:

[0017] In the formula, This is the normalized projected feature vector of the current sample; For the current batch of samples Feature vectors of positive samples with the same category label; For the current batch of samples A set of negative samples with different category labels; This refers to temperature hyperparameters.

[0018] In some embodiments, the training process of the multimodal contrastive network model includes: Based on the predicted probability distribution of the model output and the true labels, the second loss value is calculated, where the functional expression of the second loss value is:

[0019] Where N is the number of samples in the current batch. Let be the predicted probability of the i-th sample in the true class; The total loss value is determined based on the first loss value and the second loss value, and the parameters of the multimodal contrastive network model are updated according to the total loss value.

[0020] Compared with the prior art, the technical solution provided in the first aspect of this application includes at least the following beneficial effects or advantages: The sorting method provided in this application improves the feature representation of edge texture in ore images by constructing a dual-channel composite tensor containing original grayscale information and gradient magnitude information, laying the foundation for feature extraction of small mineral spots. Multi-stage residual feature extraction generates initial feature maps with different spatial resolutions, covering multi-dimensional visual information from macroscopic ore particle geometry to microscopic mineral spot details. The semantic embedding module, through cross-level feature interaction, uses deep semantic information to guide the selection of shallow edge features, preserving the boundary information of fine and weak mineral spots that are easily weakened by convolutional downsampling. After unifying the number of channels in each initial feature map through pointwise convolution, effective integration of features at different scales is achieved through top-down upsampling and pixel-by-pixel addition, adapting to changes in image resolution and ore particle proportion caused by differences in ore fragmentation particle size. The fused features are further deeply integrated with semantic enhancement features after spatial convolution, compensating for boundary details lost during feature extraction. The method improves the model's adaptability to the recognition of tungsten ore particles at different scales and enhances the recognition accuracy of difficult-to-classify samples such as fine and weak mineral spots.

[0021] Secondly, embodiments of this application provide a multi-scale ore sorting system that integrates multimodal gradient representation and contrastive learning, including: The acquisition module is configured to acquire an image of the ore to be detected, perform convolution operations and gradient magnitude calculations on the ore image, and generate a dual-channel composite tensor. The detection module is configured to input the dual-channel composite tensor into a pre-trained multimodal contrastive network model and output the ore sorting results; The data processing procedure for the multimodal contrastive network model includes: Residual features are extracted from the dual-channel composite tensor to generate multiple initial feature maps at different stages. The multiple initial feature maps are then input into a preset semantic embedding module in the model to generate semantically enhanced features. Multiple initial feature maps are subjected to pointwise convolution to obtain intermediate feature maps with a uniform number of channels. The intermediate feature maps are upsampled to match the corresponding bypass features and then fused pixel by pixel. After spatial convolution, a fused feature map is generated. The fused feature map and the semantically enhanced feature are input into a preset spatial embedding module in the model, and the target feature map is output. After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results with predicted probability distributions for each category are obtained.

[0022] Thirdly, this application also provides an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the ore multi-scale sorting method that integrates multimodal gradient representation and contrastive learning provided in the first aspect above.

[0023] Fourthly, this application also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the steps of the ore multi-scale sorting method integrating multimodal gradient representation and contrastive learning provided in the first aspect.

[0024] It is understood that the beneficial effects of the technical solutions provided in the second, third and fourth aspects above can be found in the relevant descriptions in the first aspect above, and will not be repeated here.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating the multi-scale ore sorting method integrating multimodal gradient representation and comparative learning provided in this application embodiment; Figure 2 This is a schematic diagram of the framework of the multimodal contrast network model provided in the embodiments of this application; Figure 3 This is a data processing flowchart of the multimodal contrast network model provided in the embodiments of this application; Figure 4 This is a block diagram of the data processing procedure of the semantic embedding module provided in the embodiments of this application; Figure 5 This is a block diagram of the data processing process of the comparative learning module provided in the embodiments of this application; Figure 6 This is a block diagram of the data processing procedure of the spatial embedding module provided in the embodiments of this application; Figure 7 This is a grayscale schematic diagram of various different mineral patches corresponding to different ores provided in the embodiments of this application; Figure 8A structural block diagram of a multi-scale ore sorting system that integrates multimodal gradient representation and comparative learning, provided in an embodiment of this application; Figure 9 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate several embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0029] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0030] Please see Figures 1 to 7 This embodiment provides a multi-scale ore sorting method that integrates multimodal gradient representation and comparative learning, specifically including steps S10 to S20.

[0031] Step S10: Obtain the image of the ore to be detected, perform convolution operation and gradient magnitude calculation on the ore image respectively, and generate a dual-channel composite tensor; In some embodiments, the ore image is a grayscale ore image, which is acquired by an industrial X-ray ore sorting device during the ore sorting process. The ore image can be classified into pure black ore, ore with obvious large mineral spots, ore with obvious small mineral spots, ore with fine spots, ore with weak mineral spots, waste rock with high grayscale, and waste rock with low grayscale based on the size, texture distribution, and grayscale characteristics of mineral spots on the ore surface. Furthermore, based on the gradient enhancement representation module, convolution operations and gradient magnitude calculations are performed on the ore images to generate a dual-channel composite tensor. The gradient enhancement representation module is placed at the input end of the multimodal contrast network model. The gradient enhancement representation module is set up by introducing spatial gradient components to construct a multimodal input representation with geometric priors, thereby solving the problem of low edge contrast of small ore spots and easy obscuring of details by noise in tungsten ore X-ray images.

[0032] Specifically, after receiving the ore image, the module performs the following data processing steps: performing gradient convolution operations on the ore image in both the horizontal and vertical directions to capture pixel-level intensity abrupt changes, resulting in a horizontal gradient response map and a vertical gradient response map; calculating the fused gradient magnitude based on the horizontal and vertical gradient response maps; extracting the edge energy distribution of the image by calculating the gradient magnitude; and normalizing the gradient magnitude with the ore image and performing channel stacking operations to generate a dual-channel composite tensor.

[0033] The expression for calculating the horizontal gradient response map is as follows:

[0034] The formula for calculating the vertical gradient response map is:

[0035] The expression for calculating the gradient magnitude is:

[0036] The expression for calculating the two-channel composite tensor is:

[0037] In the formula, The image shows the ore to be inspected. This is a gradient convolution operation. ( () is for channel stacking operations.

[0038] It should be noted that a gradient enhancement representation mechanism is introduced to achieve dual-modal fusion of brightness and structure. This overcomes the limitation of traditional CNNs that rely solely on raw grayscale pixels for feature mapping. By explicitly introducing spatial gradient components at the data-aware front end, a dual-channel input representation with geometric prior is constructed. The Sobel operator is used to capture pixel-level intensity abrupt changes and extract edge energy distribution, achieving second-order saliency enhancement of mineral spot features and boundaries at the physical level.

[0039] Meanwhile, in this embodiment, through data processing by the gradient enhancement representation module, a leap from single brightness representation to a dual-modal representation of "brightness-structure" is achieved. By introducing gradient features, the image possesses enhanced edge perception capabilities from the very beginning of entering the backbone network, providing high signal-to-noise ratio geometric prior information for the subsequent SEEM module to perform accurate semantic alignment and background noise suppression.

[0040] Step S20: Input the dual-channel composite tensor into the pre-trained multimodal contrast network model and output the ore sorting results; In some embodiments, the multimodal contrastive network model includes a backbone network, a feature pyramid, a semantic embedding module, a contrastive learning module, a spatial embedding module, and a classification head. The backbone network is a ResNet34 backbone network, comprising convolutional blocks and multiple residual blocks for layer-by-layer feature extraction. The output of the backbone network is connected to both the feature pyramid and the semantic embedding module. Features extracted from each layer by the backbone network are fed into the feature pyramid, which uses a top-down path to upsample and fuse the deep semantic information contained in higher layers with the high-resolution features of lower layers pixel by pixel. Through this stage, the model generates feature maps with rich semantics and accurate spatial information. Simultaneously, the semantic enhancement module dynamically weights and filters the original convolutional features. The semantic enhancement module filters noise interference through a cross-attention mechanism, initially enhancing the semantic features related to mineral textures and generating multiple sets of high-quality features.

[0041] Furthermore, at the output of the feature pyramid, the feature flow is divided into two parallel branches, which are input to the contrastive learning module and the spatial embedding module, respectively. The contrastive learning module compresses the feature map input from the feature pyramid into a 256-dimensional vector using global adaptive average pooling, and then maps it to a low-dimensional contrastive space via a projection head. The class labels are used as supervision signals to calculate the contrastive loss (InfoNCE Loss). The spatial embedding module calculates the feature loss with the output features of the semantic embedding module, incorporating optimized edge details into the deep semantic features to recover boundary details lost during downsampling, before feeding them into the classification head for classification.

[0042] It should be noted that a feature pyramid structure is introduced on top of the backbone network. Through top-down paths and bypass connections, deep semantic information and shallow spatial features are fused at multiple scales. This allows the model to capture both the microscopic textures reflecting mineral patches and global spatial features when processing feature maps. This design compensates for the loss of detailed features caused by downsampling in the backbone network, significantly enhancing the model's robustness in feature extraction from ore particles of different sizes. Contrastive learning is used to supervise feature alignment and discriminative reshaping, directly using class labels to guide the optimization of feature space distribution. By connecting a contrastive learning module to the output of the feature pyramid, the model uses InfoNCE loss to force the feature distance of similar ores to be closer and push away dissimilar samples, making similar features more compact and different class boundaries clearer, suppressing blurry and noisy features, and achieving discriminative reshaping of the feature space.

[0043] A feature enhancement mechanism is constructed throughout the entire feature extraction process using semantic embedding and spatial embedding modules. The semantic embedding module utilizes deep semantic information to guide shallow edge information, thereby reducing redundant noise and highlighting correct edge features. The spatial embedding module incorporates optimized edge details into deep semantic features to recover boundary details lost during downsampling.

[0044] In some embodiments, combined with Figure 3 The data processing of the multimodal contrastive network model includes the following steps S210 to S240.

[0045] Step S210: Extract residual features from the dual-channel composite tensor to generate multiple initial feature maps at different stages, and input the multiple initial feature maps into a preset semantic embedding module in the model to generate semantically enhanced features; In this step, when the dual-channel composite tensor is input into the multimodal contrastive network model, firstly, four stages of residual convolution processing are sequentially performed on the dual-channel composite tensor. Each stage is processed through multiple residual blocks, and each stage outputs an initial feature map with a different scale. The data processing of the semantic embedding module includes: using the initial feature maps output from the corresponding stages in the low and deep layers as low-level edge features and deep semantic features, respectively; performing channel alignment on the low-level edge features and deep semantic features through convolution operations; upsampling the channel-aligned deep semantic features to make their spatial resolution consistent with the low-level edge features; processing the upsampled deep semantic features through an activation function to generate a global attention weight map; and performing a matrix dot product operation between the global attention weight map and the low-level edge features to generate semantically enhanced features.

[0046] In one example, combined Figure 2 It should be understood that this step corresponds to the data processing of the backbone network and semantic embedding module. For the residual blocks of the backbone network, each residual block contains two consecutive 3×3 convolution operations and one shortcut connection. The network is divided into four main stages, namely residual block 2 to residual block 5 (Conv2_x to Conv5_x), with 3, 4, 6, and 3 residual blocks in each stage, respectively. This design effectively avoids the gradient vanishing problem in deep network training through residual learning, ensuring that feature information can be transmitted deep within the network, thereby significantly improving the training efficiency and classification performance of the model for ore images.

[0047] Specifically, the input two-channel composite tensor (size 224×224×2) first passes through the Conv1 layer (convolutional block 1). In this layer, the input data undergoes a 7×7 convolution operation, increasing the number of output channels to 64 while maintaining the original image spatial resolution of 224×224. After the convolution operation, batch normalization is used to maintain the consistency of the feature distribution, and a nonlinear transformation is introduced using the ReLU activation function. The corresponding expression for this process is:

[0048] Subsequently, the features enter various stages consisting of residual blocks. Each residual block contains two convolutional operations: the first convolutional layer is expressed as:

[0049] The second convolutional layer further extracts high-level spatial features, which, after batch normalization, are directly passed to the residual connection part.

[0050] The core idea of ​​residual blocks is to introduce skip connections to integrate input features. With the output of the second convolution layer The gradients are added together to mitigate the vanishing gradient problem in deep networks. The specific calculation is as follows:

[0051] This connection allows the network to more easily learn the residual differences between the input and output. If the input and output sizes do not match, the skip connection aligns the input channels using a 1×1 convolution, ensuring lossless feature fusion.

[0052] Between each stage, the backbone network can downsample using convolutional operations with a stride of 2, gradually reducing the spatial resolution of the feature maps and expanding the receptive field. The output sizes for each stage can be: 112×112×256 for Conv2_x, 56×56×512 for Conv3_x, 28×28×1024 for Conv4_x, and 14×14×2048 for Conv5_x. These multi-scale features are then fed into a Feature Pyramid Network (FPN) to achieve deep fusion of spatial information and semantic features. Simultaneously, the features from Conv1, Conv2_x, Conv3_x, and Conv5_x are also passed to the semantic enhancement module for semantic enhancement processing.

[0053] In one example, combined Figure 4The Semantic Embedding Module (SEEM), serving as a feature refinement component at the model's front end, is deployed after the initial convolutional layers and residual stages of the backbone network. Its core design goal is to leverage the high-level semantic information of deep networks as guidance to adaptively filter redundant noise in shallow features, thereby extracting clean edge information. This module receives low-level edge features from the backbone network (…). ) and semantic features from deep layers ( First, the module aligns the two feature channels through convolution and uses bilinear interpolation to upscale the low-resolution semantic features to a spatial resolution consistent with the edge features. Then, a sigmoid activation function is used to non-linearly process the semantic features, generating a global attention weight map. This weight map is then multiplied by the edge features using a matrix dot product, leveraging semantic information to enhance the feature response intensity of the target region while effectively suppressing redundant noise from the non-target background, thereby improving the discriminative power of the features. The calculation process is as follows:

[0054] In the formula, ( () represents the convolution operation. ( () is the Sigmoid activation function. ( ) represents an upsampling operation.

[0055] Step S220: Perform point-by-point convolution processing on multiple initial feature maps to obtain intermediate feature maps with a uniform number of channels. Upsample the intermediate feature maps to match the corresponding bypass features, then perform pixel-by-pixel addition and fusion, and generate a fused feature map after spatial convolution processing. It should be noted that this step corresponds to the data processing of the feature pyramid network, used to balance the deep, high-dimensional semantic information extracted by the backbone network with the shallow, high-resolution spatial features. When the feature pyramid network receives output feature maps (Conv2_x to Conv5_x) from different stages of the backbone network, it performs deep integration through a top-down path and bypass connections. This structure ensures that each layer of feature maps has strong expressive power when the model processes ore targets at different scales.

[0056] In one example, the outputs from each stage of the backbone network (with dimensions respectively) , , , Channel compression is performed through a convolutional layer (Conv2d, 1x1, s1) to unify the number of channels in each layer to 256, facilitating subsequent feature fusion. The expression is as follows:

[0057] Subsequently, the feature pyramid network employs a top-down upsampling mechanism to achieve multi-scale information interaction. Deep features (such as...) After upsampling, bilinear interpolation is performed to improve its spatial resolution to match that of the previous layer (e.g., ...). The features are consistent with the corresponding bypass features and are then added and fused pixel-by-pixel. This fusion process ensures that high-level semantics can be effectively injected into the low-level path, enhancing the discriminative power of the low-level features. Its expression is:

[0058] The fused feature map then goes through a A convolutional layer (Conv2d, 3x3, s1) eliminates the aliasing effect during upsampling and further refines the fused spatial features. After this processing, the FPN outputs four sets of multi-scale feature maps with a unified number of 256 channels, and their spatial resolutions are as follows: , , as well as .

[0059] In the final feature transition, the highest resolution output layer of the feature pyramid network (i.e., The feature maps are simultaneously fed into the contrastive learning module. In this module, the model performs supervised feature alignment, which enhances the model's accuracy in distinguishing similar ores by constraining the high-quality representations fused from the feature pyramid network. Simultaneously, the output features from each level of the feature pyramid network are further passed to the spatial embedding module for spatial awareness enhancement, providing accurate input support for the final classification task.

[0060] Step S230: Input the fused feature map and the semantic enhancement feature into the preset spatial embedding module in the model, and output the target feature map; In this step, the spatial embedding module is integrated into the terminal representation layer of the feature fusion network. It is used to deeply fuse enhanced edge features with multi-scale semantic features. Its design goal is to repair the boundary details lost during multiple downsampling processes of semantic features through physical compensation of spatial information, thereby obtaining accurate edge feature representations. Specifically, this module receives semantically enhanced features refined by the semantic enhancement module (…). ) and the corresponding fusion feature map ( The module takes two sets of pixel-level feature maps as input. First, it aligns the two feature maps by channel and spatial scale. Then, it performs pixel-level addition on the two sets of pixel-level feature maps of the same size, directly compensating for the boundaries of the semantic features using high-resolution edge information. Finally, it performs convolution and smoothing on the fused features through convolutional layers to eliminate numerical abrupt changes caused by feature superposition. The specific algorithm formula is expressed as follows:

[0061] In the formula, For the target feature map, ( () represents the convolution operation. For semantic enhancement features, To fuse feature maps, ( ) represents an upsampling operation.

[0062] Step S240: After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results of the predicted probability distribution of each category are obtained.

[0063] In this step, it's important to note that the classification head, as the final output of the model, is deployed after the feature extraction and enhancement module. Its main function is to map the high-dimensional features obtained through multi-scale fusion to the target category space, outputting the probability prediction results for tungsten ore types. This module receives the final feature map output from the Spatial Embedding Module (SPFM). First, global average pooling is used to aggregate and compress the row space dimension of the feature map. This operation compresses the three-dimensional feature tensor into a one-dimensional global feature vector by calculating the average value of the feature map for each channel. This design effectively reduces the number of model parameters and improves the network's robustness to target translation and deformation. Subsequently, the extracted global feature vector is input to the fully connected layer. The fully connected layer maps the feature vector to a predefined number of categories through linear transformation and normalizes it using the Softmax function, generating the predicted probability distribution for each category, where the expression is:

[0064] In the formula, Represents the classification weight matrix. This is a bias term.

[0065] In the above method steps, by constructing a dual-channel composite tensor containing original grayscale information and gradient magnitude information, the feature representation of edge texture in ore images is improved, laying the foundation for feature extraction of small mineral spots. Multi-stage residual feature extraction generates initial feature maps with different spatial resolutions, covering multi-dimensional visual information from macroscopic ore particle geometric contours to microscopic mineral spot details. The semantic embedding module uses deep semantic information to guide the selection of shallow edge features through cross-level feature interaction, preserving the boundary information of fine and weak mineral spots that are easily weakened by the convolution downsampling process. After unifying the number of channels of each initial feature map through pointwise convolution, the effective integration of features at different scales is achieved through top-down upsampling and pixel-by-pixel addition mechanisms, adapting to the changes in image resolution and ore particle proportion caused by differences in ore fragmentation particle size. After spatial convolution, the fused features are further deeply integrated with semantic enhancement features to compensate for the boundary details lost during feature extraction. The method improves the model's adaptability to the recognition of tungsten ore particles of different scales and improves the recognition accuracy of difficult-to-classify samples such as fine and weak mineral spots.

[0066] In some embodiments, a training method for a multimodal contrastive network model is provided. This method involves collecting ore samples using an industrial detector, selecting images with various mineral features, dividing the dataset proportionally (e.g., 8:1:1) into a training set, a validation set, and a test set, and then feeding the image dataset into the multimodal contrastive network model for training. The image input size can be set to 224×224, the batch size to 16, and the number of epochs to 300. After training, the model weights are updated to obtain the multimodal contrastive network model.

[0067] Specifically, during the training process, to achieve synergistic optimization of ore category recognition and feature discrimination capabilities, this embodiment adopts a training strategy jointly supervised by contrastive learning loss and classification loss. The data processing of the contrastive learning module includes: receiving a high-dimensional feature map from the dual-channel composite tensor fusion feature map; performing spatial tensor conversion and spatial dimension compression operations on the obtained high-dimensional feature map to obtain a global feature vector; inputting the obtained dual-channel composite tensor global feature vector into a projection head composed of a multilayer sensing mechanism for nonlinear transformation to obtain a projected feature vector; normalizing the obtained dual-channel composite tensor projected feature vector; and calculating the first loss value (contrastive learning loss) from the feature vectors of positive samples with the same category label as the current sample after normalization. The functional expression of the first loss value is:

[0068] In the formula, This is the normalized projected feature vector of the current sample; For the current batch of samples Feature vectors of positive samples with the same category label; For the current batch of samples A set of negative samples with different category labels; This refers to temperature hyperparameters.

[0069] It should be noted that the contrastive learning module calculates supervised contrastive loss based on sample labels, which is used to constrain the distribution of the feature space, making ore samples of the same category more clustered in the feature space, and further separating samples of different categories.

[0070] Furthermore, the class probability distribution output by the classification head is first compared with the true labels to calculate a second loss value (cross-entropy loss), which measures the difference between the model's prediction and the true class. The calculation formula is as follows:

[0071] Where N is the number of samples in the current batch. Let be the predicted probability of the i-th sample in the true class; The total loss value is determined based on the first loss value and the second loss value, and the parameters of the multimodal contrastive network model are updated according to the total loss value. The functional expression of the total loss value is:

[0072] in, For classification cross-entropy loss, To monitor and compare learning loss, The loss weights are used to balance the contributions of classification and feature alignment tasks to model training. During network training, the total loss function simultaneously updates the parameters of the backbone network, feature pyramid network, SEEM module, SPEM module, projection head, and classification head through backpropagation. Specifically, cross-entropy loss improves the accuracy of ore category prediction, while contrastive learning loss enhances the discriminative power of the feature space, thereby improving the model's ability to identify difficult-to-classify samples such as fine-spotted ore patches and weak ore patches.

[0073] In some embodiments, please refer to Figure 7 This embodiment provides a comparative experiment between the above-mentioned multi-scale ore sorting method based on the fusion of multimodal gradient representation and contrastive learning and existing methods. Specifically: 1. Introduction to the dataset All samples in the dataset used in the experiment were 16-bit grayscale ore images acquired by industrial X-ray ore sorting equipment. The dataset contains 10,884 ore images, which can well reflect the surface texture, morphology of ore patches, and grayscale distribution characteristics of ore in real industrial scenarios. Compared with ordinary 8-bit images, 16-bit grayscale images can retain richer density information and fine-grained grayscale variations, providing higher information fidelity for the identification of ore patch features.

[0074] Based on the size, texture distribution, and grayscale characteristics of mineral spots on the ore surface, the dataset was divided into seven categories. These categories are not completely independent discrete categories, but rather exhibit a clear continuity and order relationship. Specifically, the main differences between categories are gradual variations in spot size and texture intensity; adjacent categories often only show minor variations in fine-grained features. This can easily lead to category confusion, increasing the difficulty of model recognition. This continuous distribution characteristic better reflects actual ore sorting scenarios and also places higher demands on the model's fine-grained feature extraction capabilities.

[0075] like Figure 7 As shown, the seven types of ore, from left to right, are: pure black ore, ore with obvious large spots, ore with obvious small spots, ore with fine spots, ore with weak spots, waste rock with high grayscale, and waste rock with low grayscale. Among them, the "fine spots" and "weak spots" categories are characterized by small spot size, blurred edges, and weak grayscale differences, making them the most challenging categories in this dataset.

[0076] 2. Implementation details The networks used were implemented using PyTorch 2.8 and CUDA 12.8 frameworks. MGC-Net (the tungsten ore multi-scale sorting method corresponding to this application) and various comparative models were trained and tested on an NVIDIA GeForce RTX 4060 GPU platform equipped with 8GB of video memory.

[0077] To ensure fairness in the experiment, all models adopted the same data partitioning and training strategy. The batch size during training was uniformly set to 16, and the initial learning rate was set to [value missing]. The total training iterations were 300 epochs. Data augmentation operations, including rotation, translation, and horizontal flipping, were also employed.

[0078] All input data consisted of 16-bit grayscale mineral images. To accommodate the input characteristics of different network structures, the model input layers were uniformly adapted: for the Vision Transformer (ViT) model, the convolutional projection layer in the Patch Embedding module was reconstructed, and the number of input channels was adjusted from 3 to 1 to accommodate single-channel grayscale input; for CNN-type models, the first convolutional structure was uniformly modified to enable it to directly process single-channel 16-bit grayscale images.

[0079] 3. Evaluation Indicators The following metrics were selected as evaluation indicators: number of parameters, Top-1 accuracy, Neighbor Tolerant Accuracy, F1 score, Throughput / FPS, and Real-Time Efficiency.

[0080] The number of parameters is used to measure the size and complexity of a model, measured in millions (M). A smaller number of parameters usually means lower storage overhead and deployment costs, making the model more suitable for industrial edge devices or real-time sorting scenarios.

[0081] Top-1 accuracy measures the overall classification accuracy of a model. It represents the proportion of samples whose predicted class matches the true class exactly, and is one of the most commonly used evaluation metrics for classification tasks. This metric directly reflects the model's overall recognition ability.

[0082] The F1 score takes into account both the model's precision and recall, and can more comprehensively reflect the model's classification performance under imbalanced class conditions.

[0083] Throughput (FPS) measures the inference speed of a model, representing the number of images the model can process per second. This metric reflects the model's real-time processing capability in a sorting system; a higher FPS indicates stronger model inference efficiency.

[0084] Because the ore categories in this experiment exhibit clear continuity and order, adjacent categories often only differ in the size of mineral patches or the intensity of texture. Therefore, relying solely on strict Top-1 accuracy is insufficient to fully reflect the model's actual sorting performance. To address this, this paper introduces the Neighbor Tolerant Accuracy metric to measure the model's tolerance for "misclassification of adjacent categories." A prediction is considered acceptable in ore sorting scenarios when the difference between the predicted and true categories does not exceed 1. Its definition is as follows:

[0085] in, Represents the total number of samples. Indicates the true label, Indicates the predicted label, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. Compared to traditional accuracy, NTA is better suited to the engineering requirements of actual ore sorting tasks and more reasonably evaluates the model's ability to identify fine-grained continuous categories.

[0086] Furthermore, practical industrial sorting requires models to possess not only high accuracy but also high inference efficiency. To comprehensively measure the model's ability to balance "classification performance" and "real-time performance," this paper further designs a combined accuracy-speed index (Real-Time Efficiency). This index introduces throughput information based on NTA and uses a logarithmic function to reduce the dominant influence of excessively high FPS on the results. Its definition is as follows:

[0087] NTA measures the model's fault-tolerant classification ability, while FPS measures the model's inference speed. By introducing a logarithmic function, the problem of insufficient accuracy being masked by the speed advantage of high-FPS models can be effectively avoided, thus providing a more objective evaluation of the model's overall performance in industrial real-time ore sorting scenarios. A higher RTE value indicates a better balance between classification accuracy and real-time inference ability.

[0088] 5. Experimental Results Table 1: Experimental results of each method

[0089] The proposed MGC-Net was compared with commonly used models to verify its effectiveness and overall performance in fine-grained ore classification tasks. The comparison models included MobileNetV3-Large, DenseNet-121, ResNet-34, ConvNeXt-Tiny, and ViT-Base / 16. Experimental results are shown in Table 1. The proposed MGC-Net achieved state-of-the-art performance on several key evaluation metrics. Specifically, the model ranked first in Top-1 Accuracy, NTA, F1-score, and RTE. MGC-Net's Top-1 Accuracy reached 83.71%, improving upon DenseNet-121, ConvNeXt-Tiny, and ViT-Base / 16 by 0.75%, 1.03%, and 0.94%, respectively. Simultaneously, MGC-Net's F1-score reached 0.8364, also outperforming all comparison models. This indicates that the proposed network can more effectively extract fine-grained texture information and weak mineral patch features from ore images, and has a stronger ability to distinguish between complex ore categories.

[0090] In terms of Neighborhood Tolerance Accuracy (NTA), MGC-Net achieved 97.11%, second only to DenseNet's 97.77%. Because the ore categories in this study exhibit clear continuity and order, with only gradual differences in ore patch size and grayscale texture between adjacent categories, NTA more realistically reflects the model's actual performance in industrial ore sorting. Experimental results demonstrate that MGC-Net not only improves strict classification accuracy but also effectively reduces the "large-span misclassification" phenomenon, concentrating erroneous predictions within the range of adjacent categories.

[0091] Furthermore, in terms of the combined accuracy-speed ratio (RTE), MGC-Net achieved the highest score of 2.7465, outperforming ResNet-34's 2.7288 and MobileNetV3-Large's 2.6852. This result indicates that while some models have higher FPS, their overall industrial application value is limited due to insufficient classification accuracy; while MGC-Net maintains high inference efficiency while ensuring high fault tolerance in classification, thus exhibiting superior overall performance in industrial online ore sorting tasks.

[0092] Please see Figure 8 , Figure 8 This embodiment illustrates a multi-scale ore sorting system that integrates multimodal gradient representation and comparative learning. In this embodiment, the multi-scale ore sorting system integrating multimodal gradient representation and comparative learning is used to perform the above-mentioned... Figure 1 The steps in the corresponding embodiments. Please refer to the details. Figure 1 as well as Figure 1 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 8 The ore multi-scale sorting system 200, which integrates multimodal gradient representation and contrastive learning, includes: The acquisition module 210 is configured to acquire an image of the ore to be detected, perform convolution operations and gradient magnitude calculations on the ore image respectively, and generate a dual-channel composite tensor. The detection module 220 is configured to input the dual-channel composite tensor into a pre-trained multimodal contrast network model and output the ore sorting results; The data processing steps for the multimodal contrastive network model include: Residual features are extracted from the dual-channel composite tensor to generate multiple initial feature maps at different stages. These initial feature maps are then input into a pre-defined semantic embedding module in the model to generate semantically enhanced features. Multiple initial feature maps are convolved pointwise to obtain intermediate feature maps with a uniform number of channels. The intermediate feature maps are upsampled to match the corresponding bypass features and then fused pixel by pixel. After spatial convolution, a fused feature map is generated. The fused feature map and semantically enhanced features are input into the pre-defined spatial embedding module in the model, and the target feature map is output. After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results with predicted probability distributions for each category are obtained.

[0093] The sorting system provided in this embodiment improves the feature representation of edge texture in ore images by constructing a dual-channel composite tensor containing original grayscale information and gradient magnitude information, laying the foundation for feature extraction of small mineral spots. Multi-stage residual feature extraction generates initial feature maps with different spatial resolutions, covering multi-dimensional visual information from macroscopic ore particle geometry to microscopic mineral spot details. The semantic embedding module guides the selection of shallow edge features through cross-level feature interaction, using deep semantic information to preserve the boundary information of fine and weak mineral spots that are easily weakened by convolutional downsampling. After unifying the number of channels in each initial feature map through pointwise convolution, effective integration of features at different scales is achieved through top-down upsampling and pixel-by-pixel addition, adapting to changes in image resolution and ore particle proportion caused by differences in ore fragmentation particle size. The fused features are further deeply integrated with semantic enhancement features after spatial convolution, compensating for boundary details lost during feature extraction. This method improves the model's adaptability to the recognition of tungsten ore particles at different scales and enhances the recognition accuracy of difficult-to-classify samples such as fine and weak mineral spots.

[0094] It should be understood that the modules in the system provided in this embodiment are used to execute... Figure 1 and Figure 3 The steps in the corresponding embodiments, and for Figure 1 and Figure 3 The steps in the corresponding embodiments have been explained in detail in the above embodiments. Please refer to them for details. Figure 1 and Figure 3 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0095] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes at least one processor 901 and a memory 902. Optionally, the electronic device 90 further includes a communication component 903. The processor 901, memory 902, and communication component 903 are connected via a bus 904. In the specific implementation process, at least one processor 901 executes computer execution instructions stored in memory 902, causing at least one processor 901 to execute the above-mentioned multi-scale ore sorting method that integrates multimodal gradient representation and comparative learning.

[0096] The specific implementation process of processor 901 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0097] In the above embodiments, it should be understood that the processor 901 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0098] The memory 902 can be an internal storage unit of the electronic device 90, such as a hard drive or memory in a server. The memory 902 can also be an external storage terminal device of the electronic device 90, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the electronic device 90.

[0099] Furthermore, the memory 902 may include both internal storage units of the electronic device 90 and external storage terminal devices. The memory 902 is used to store computer programs and other programs and data required by the turntable terminal device. The memory 902 can also be used to temporarily store data that has been output or will be output.

[0100] Bus 904 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0101] Those skilled in the art will understand that Figure 9 This is merely an example of electronic device 90 and does not constitute a limitation on electronic device 90. It may include more or fewer components than shown, or combine certain components, or different components. For example, a turntable terminal device may also include input / output terminal devices, network access terminal devices, etc.

[0102] In one embodiment, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the ore multi-scale sorting method integrating multimodal gradient representation and contrastive learning as described in the above embodiments.

[0103] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0104] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0105] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0108] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects and not to describe a particular order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, it may include a series of steps or units, or optionally, steps or units not listed, or other steps or units inherent to these processes, methods, products, or devices.

[0109] The accompanying drawings show only the portions relevant to this application, not all of them. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations may be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0110] The terms “component,” “module,” “system,” “unit,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, and / or distributed between two or more computers. Furthermore, these units can be executed from various computer-readable media on which various data structures are stored. Units can communicate, for example, via signals having one or more data packets (e.g., data from a second unit interacting with another unit between a local system, a distributed system, and / or a network; for example, the Internet interacting with other systems via signals).

[0111] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0112] Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily indicate the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A multi-scale ore sorting method integrating multimodal gradient representation and contrastive learning, characterized in that, include: Acquire an image of the ore to be detected, perform convolution operations and gradient magnitude calculations on the ore image respectively, and generate a dual-channel composite tensor; The dual-channel composite tensor is input into a pre-trained multimodal comparison network model to output ore sorting results; The data processing procedure for the multimodal contrastive network model includes: Residual feature extraction is performed on the dual-channel composite tensor to generate multiple initial feature maps at different stages, and the multiple initial feature maps are input into a preset semantic embedding module in the model to generate semantically enhanced features. This includes: performing four stages of residual convolution processing on the dual-channel composite tensor in sequence, with each stage being processed by multiple residual blocks in sequence, and each stage outputting the initial feature maps at different scales. The data processing of the semantic embedding module includes: using the initial feature maps output from the corresponding low-level and deep-level stages as low-level edge features and deep-level semantic features, respectively; performing channel alignment on the low-level edge features and the deep-level semantic features through convolution operations; performing upsampling on the channel-aligned deep-level semantic features to make their spatial resolution consistent with that of the low-level edge features; processing the upsampled deep-level semantic features through an activation function to generate a global attention weight map; and performing a matrix dot product operation between the global attention weight map and the low-level edge features to generate semantically enhanced features. Multiple initial feature maps are subjected to pointwise convolution to obtain intermediate feature maps with a uniform number of channels. The intermediate feature maps are upsampled to match the corresponding bypass features and then fused pixel by pixel. After spatial convolution, a fused feature map is generated. The fused feature map and the semantically enhanced feature are input into a preset spatial embedding module in the model, and a target feature map is output. This includes receiving a high-dimensional feature map from the fused feature map, performing a spatial tensor-to-continuous vector representation conversion and spatial dimension compression on the high-dimensional feature map to obtain a global feature vector; inputting the global feature vector into a projection head composed of a multilayer perceptron for nonlinear transformation to obtain a projected feature vector; normalizing the projected feature vector, and calculating a first loss value from the positive sample feature vectors with the same class label as the current sample after normalization, wherein the functional expression of the first loss value is: In the formula, This is the normalized projected feature vector of the current sample; For the current batch of samples Feature vectors of positive samples with the same category label; For the current batch of samples A set of negative samples with different category labels; This refers to temperature hyperparameters. After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results with predicted probability distributions for each category are obtained.

2. The multi-scale ore sorting method integrating multimodal gradient representation and contrastive learning according to claim 1, characterized in that, The step of acquiring the image of the ore to be detected, performing convolution operations and gradient magnitude calculations on the ore image to generate a dual-channel composite tensor includes: The ore image is subjected to gradient convolution operations in the horizontal and vertical directions to obtain a horizontal gradient response map and a vertical gradient response map. Based on the horizontal gradient response map and the vertical gradient response map, the fused gradient magnitude is calculated, and the gradient magnitude is normalized with the ore image and a channel stacking operation is performed to generate the dual-channel composite tensor.

3. The multi-scale ore sorting method integrating multimodal gradient representation and comparative learning according to claim 2, characterized in that, The steps of calculating the fused gradient magnitude based on the horizontal gradient response map and the vertical gradient response map, and of uniformly normalizing the gradient magnitude with the ore image and performing a channel stacking operation to generate the dual-channel composite tensor include: The expression for calculating the horizontal gradient response plot is: The formula for calculating the vertical gradient response map is: The expression for calculating the gradient magnitude is: The expression for calculating the two-channel composite tensor is: In the formula, The image shows the ore to be inspected. This is a gradient convolution operation. ( () is for channel stacking operations.

4. The multi-scale ore sorting method integrating multimodal gradient representation and contrastive learning according to claim 1, characterized in that, In the step of inputting the fused feature map and the semantically enhanced feature into a preset spatial embedding module in the model and outputting the target feature map, the data processing process of the spatial embedding module includes: Alignment operations are performed on the semantic enhancement features and the fused feature map for both channel and spatial scale. The aligned fused feature map and the semantic enhancement features are then added pixel-by-pixel. The added features are then convolved and smoothed to generate the target feature map. The corresponding algorithm expression is as follows: In the formula, For the target feature map, ( () represents the convolution operation. For semantic enhancement features, To fuse feature maps, ( ) represents an upsampling operation.

5. The multi-scale ore sorting method integrating multimodal gradient representation and comparative learning according to claim 1, characterized in that, The training process of the multimodal contrastive network model includes: Based on the predicted probability distribution of the model output and the true labels, the second loss value is calculated, where the functional expression of the second loss value is: Where N is the number of samples in the current batch. Let be the predicted probability of the i-th sample in the true class; The total loss value is determined based on the first loss value and the second loss value, and the parameters of the multimodal contrastive network model are updated according to the total loss value.

6. A multi-scale ore sorting system integrating multimodal gradient representation and contrastive learning, characterized in that, The system is applied to the multi-scale ore sorting method as described in any one of claims 1-5, comprising: The acquisition module is configured to acquire an image of the ore to be detected, perform convolution operations and gradient magnitude calculations on the ore image, and generate a dual-channel composite tensor. The detection module is configured to input the dual-channel composite tensor into a pre-trained multimodal contrastive network model and output the ore sorting results; The data processing procedure for the multimodal contrastive network model includes: Residual features are extracted from the dual-channel composite tensor to generate multiple initial feature maps at different stages. The multiple initial feature maps are then input into a preset semantic embedding module in the model to generate semantically enhanced features. Multiple initial feature maps are subjected to pointwise convolution to obtain intermediate feature maps with a uniform number of channels. The intermediate feature maps are upsampled to match the corresponding bypass features and then fused pixel by pixel. After spatial convolution, a fused feature map is generated. The fused feature map and the semantically enhanced feature are input into a preset spatial embedding module in the model, and the target feature map is output. After performing global average pooling on the target feature map, it is mapped to the target category space. After normalization, the ore sorting results with predicted probability distributions for each category are obtained.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the ore multi-scale sorting method that integrates multimodal gradient representation and contrastive learning as described in any one of claims 1-5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the ore multi-scale sorting method that integrates multimodal gradient representation and comparative learning as described in any one of claims 1-5.

Images