Ore image small sample target detection method and system

Abstract

The present application provides a kind of ore image small sample target detection method and system, belong to target detection technical field, comprising: based on ore image sample set obtains ore mark database;Based on sample enhancement module, double attention region generation network and pre-set light weight multi-relation detection head, construct target detection framework;Obtain pre-set standard data set, utilize pre-set standard data set to carry out basic training to target detection framework, utilize ore mark database to carry out fine-tuning training to target detection framework, obtain ore small sample target detection model;The ore image to be detected is input to ore small sample target detection model, and the ore target detection result is obtained.The present application is aimed at the condition that ore sample quantity is few, adopts based on sample enhancement module, double attention region generation network and light weight multi-relation detection head to construct target detection framework, effectively solve network overfitting problem in ore image granularity analysis task under the condition of small amount of samples, significantly improve the extraction precision of ore granularity information.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and system for small-sample target detection in mineral images. Background Technology

[0002] In ore testing, ore particle size information, such as area, perimeter, and particle size, is an important data indicator reflecting the crushing effect. Particle size analysis of the crushed product can ensure the required particle size distribution.

[0003] Large block detection is a task in ore granulation analysis, referring to the identification of oversized ore within an ore belt. Typically, object detection networks are used to obtain the coordinates of the ore, and then bounding boxes are used to define the bounding rectangular region of each ore. Finally, a threshold is checked based on the obtained region to determine if any large blocks exist on the conveyor belt. Semantic segmentation is not suitable for this type of task compared to object detection. The problem is that when segmenting stacked ore, each overlapping ore cannot be separated because they are assigned to the same pixel class. This leads to two or more ore areas being treated as a single ore when calculating the area of ​​each individual ore in the image. In recent years, significant progress has been made in object detection due to the development of convolutional neural networks. Traditional object detectors require a large amount of bounding box labels for training, making the acquisition of such high-quality ore label data costly and time-consuming. When labeled data becomes scarce, convolutional neural networks suffer from severe overfitting and fail to generalize. Therefore, it is difficult to apply object detectors to detect new objects that are not present in the dataset. At the same time, ensuring the accuracy and real-time performance of ore particle size detection is extremely difficult given the limited labeling data and common issues such as occlusion, adhesion, and changes in ore appearance.

[0004] Traditional object detection methods require a large amount of labeled data as prior knowledge to train models with high generalization performance before they can be applied to new data. However, these labeled data conditions are difficult to meet in special and complex scenarios. Some few-shot learning-based methods have offered profound insights into the data scarcity problem in practical engineering problems. These methods either use different learning strategies, such as metric learning and meta-learning, or perform data feature-level processing to address the data scarcity problem. For example, Dong et al. proposed a metric learning-based method for road damage detection. In the aerospace field, Pierre et al. proposed a new object detection data augmentation method, and a new support cropping strategy improved the application of few-shot object detection in aerial imagery. Zhou et al. proposed a multi-learning model using a Siamese convolutional neural network to improve the accuracy of intelligent anomaly detection in industrial cyber-physical systems. However, the above methods are not suitable for the field of ore detection.

[0005] Therefore, it is necessary to propose a target detection method for small sample sizes in ore images to solve problems such as overfitting of the network model due to insufficient sample size. Summary of the Invention

[0006] This invention provides a method and system for small-sample target detection in ore images, which solves the defects in the prior art, such as network overfitting and slow detection speed, when detecting ore particle size information when the number of ore image samples is too small.

[0007] In a first aspect, the present invention provides a method for small-sample target detection in ore images, comprising:

[0008] Collect a set of ore sample images, and obtain an ore annotation database based on the set of ore image samples;

[0009] A target detection framework is constructed based on a sample augmentation module, a dual-attention region generation network, and a pre-defined lightweight multi-relation detection head.

[0010] Obtain a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample target detection model for ore.

[0011] The image of the ore to be detected is input into the small sample target detection model of the ore to obtain the target detection result of the ore.

[0012] According to the present invention, a method for small-sample target detection of ore images includes the following steps: acquiring a set of ore sample images and obtaining an ore annotation database based on the set of ore image samples.

[0013] Collect multiple ore sample images of different sizes and locations, and cut the multiple ore sample images into multiple preset small-sized images;

[0014] The multiple preset small-sized images are divided into a support set and a query set with several category labels.

[0015] According to the present invention, a method for small-sample target detection in mineral images is provided, wherein the target detection framework is constructed based on a sample enhancement module, a dual-attention region generation network, and a preset lightweight multi-relation detection head, comprising:

[0016] A learnable linear matrix is ​​used to transform and reconstruct the ore sample input support feature map to obtain an ore feature map with enhanced edge information.

[0017] The ore feature map with enhanced edge information is input into the channel attention mechanism module and the spatial attention mechanism module in the dual attention region generation network to obtain the channel attention feature map and the spatial attention feature map. The channel attention feature map and the spatial attention feature map are then fused to obtain the final feature map.

[0018] The target detection framework is obtained by using the preset lightweight multi-relationship detection head to perform similarity calculation and classification prediction on the final feature map.

[0019] According to the present invention, a method for small-sample target detection in ore images includes a step of transforming and reconstructing the input support feature map of the ore sample using a learnable linear matrix to obtain an ore feature map with enhanced edge information, comprising:

[0020] Determine the length, width, and number of channels of the ore sample input support feature map;

[0021] Based on the position of the same pixel in each channel and the set of all pixels in the ore sample input support feature map, the set of all pixel positions in the ore sample input support feature map is obtained;

[0022] A fully connected layer is formed over the set of all pixel locations to obtain the learnable linear matrix I. e :

[0023] I e =FC(T) PE (Y)),

[0024] Where Y is the feature map, Y∈R H×W×C H, W, and C are the length, width, and number of channels of the feature map, respectively, and T... PE Y represents the set of all pixels in each channel, Ω∈HW represents the set of all pixels, and Y represents the set of all pixels. i This indicates the position of the same pixel in each channel of the feature map; FC represents a fully connected layer.

[0025] The activation value of each feature map location is obtained by processing the learnable linear matrix with a normalized exponential function. Then, each activation value is multiplied by the set of all pixel locations to obtain the weighted feature map R for each pixel in each channel.

[0026]

[0027] Where β refers to the normalization exponent softmax. It is the product symbol;

[0028] The weighted feature map of each pixel in each channel is processed by a parameterized linear correction unit, multiplied by a preset activation coefficient, and then superimposed with the ore sample input support feature map to obtain the ore feature map Y with enhanced edge information. E :

[0029] Y E =Y + μ·LeakyReLU(R)

[0030] Among them, the mineral image enhancement effect is best when the activation coefficient μ is 0.5. LeakyReLU is a parameterized linear correction unit.

[0031] According to the present invention, a method for small-sample target detection of ore images includes inputting the ore feature map with enhanced edge information into the channel attention mechanism module and the spatial attention mechanism module of the dual attention region generation network to obtain a channel attention feature map and a spatial attention feature map, and fusing the channel attention feature map and the spatial attention feature map to obtain a final feature map, comprising:

[0032] The supporting features in the ore feature map of the enhanced edge information are determined based on the average pooling kernel size and the number of channels, and the query features in the ore feature map of the enhanced edge information are determined based on the length, the width, and the number of channels.

[0033] Based on the dimension of the average pooling kernel size, the support features are used as convolution kernels to perform convolution calculations on the deep cross-correlation features of the query features. The similarity between the support features and the query features is traversed to obtain the spatial attention feature map A. s :

[0034]

[0035] Among them, A s This represents the generated spatial attention feature map, where c, h, and w are the length, width, and number of channels of the feature map, respectively, and m, n ∈ {1, ..., P}, with supporting features X ∈ t. C×P×P C is the number of feature channels, P is the kernel size after average pooling, and the query feature is Y∈t. C×H×W H and W are the length and width of the feature map, and t is the vector space;

[0036] The support features are compressed through one-dimensional convolution channels and then reconstructed and normalized sequentially to obtain the support feature matrix. Similarly, the query features are compressed through one-dimensional convolution channels and then reconstructed to obtain the query feature matrix. The channel attention feature map A is obtained by multiplying the support feature matrix and the query feature matrix and then performing an XOR operation with the query features. c :

[0037] A c =F SG [τ1(Conv s (Y)×F SM (τ2(Conv s (X))]⊙Y

[0038] Among them, A c This represents the generated channel attention feature map, where τ1 and τ2 are tensor reshape operations, and F... SG Represents sigmoid, F SM Represents softmax, ⊙ represents multiplication concatenation, Conv s It is a convolution operation;

[0039] The channel attention feature map and the spatial attention feature map are fused and calculated to obtain the final feature map.

[0040] According to the present invention, a method for small-sample target detection in ore images, wherein the target detection framework is obtained by using the preset multi-relation detection head to perform similarity calculation and classification prediction on the final feature map, includes:

[0041] The channel attention feature map and the spatial attention feature map are superimposed and connected using a fully connected layer to obtain the global relation head GR(X, Y):

[0042] GR(X,Y)=FC(Cat(X,Y))

[0043] Where Cat represents the superposition of two feature maps, FC represents a fully connected layer, X represents a support feature, and Y represents a query feature;

[0044] Perform a pre-defined depthwise convolution operation on the channel attention feature map and the spatial attention feature map to obtain the local relation header LR(X, Y):

[0045] LR(X,Y)=Cross(X,Y)

[0046] Where Cross(X, Y) represents the depthwise convolution operation;

[0047] Determine the query embedding vector of the query feature, obtain the average query embedding vector of all pixels from the query embedding, determine the key embedding vector of the supporting feature, obtain the average key embedding vector of the pixels from the key embedding, and map the transpose of the difference between the query embedding vector and the average query embedding vector of all pixels to the difference between the key embedding vector and the average key embedding vector of the pixels to obtain the fragmented relation header PR(X, Y):

[0048] PR(X,Y)=φ((Q-μ)Q ) T (K-μ K ))

[0049] Where φ is the mapping operation, Q represents the query embedding of query feature Y, K represents the key embedding of supporting feature X, and μ Q and μ K These represent the average values ​​of all pixels in the query embedding and key embedding, respectively.

[0050] The confidence Γ of the target detection framework is obtained by weighted summation of the first weight adjustment coefficient, the global relation header, the second weight adjustment coefficient, the local relation header, the third weight adjustment coefficient, and the scattered relation header.

[0051] Γ=σ1GR(X,Y)+σ2LR(X,Y)+σ3PR(X,Y)

[0052] Where σ1, σ2, and σ3 are the weight adjustment coefficients for each relation head.

[0053] According to the present invention, a method for small-sample target detection in ore images includes obtaining a preset standard dataset, performing basic training on the target detection framework using the preset standard dataset, and fine-tuning the target detection framework using the ore annotation database to obtain a small-sample target detection model for ore images.

[0054] Extract a predetermined number of category images from the predetermined standard dataset to perform basic training on the object detection framework;

[0055] A preset number of samples are selected from the ore labeling database. Based on a preset intersection-union ratio (IU) threshold, the accuracy of the model performance is verified using the preset number of samples. The inference memory usage, inference speed, and model size of the model performance are also verified. The optimal value of the model performance is determined, and the ore small sample target detection model is obtained.

[0056] According to the present invention, a method for small-sample target detection in ore images includes inputting the ore image to be detected into the small-sample target detection model to obtain the ore target detection result, comprising:

[0057] The coordinates of the ore image to be detected are output by the ore small sample target detection model, and the bounding box of the ore image to be detected is determined based on the coordinates;

[0058] The area of ​​the ore boundary rectangle is calculated based on the bounding box.

[0059] Secondly, the present invention also provides a small-sample target detection system for ore images, comprising:

[0060] The acquisition module is used to acquire a set of ore sample images and obtain an ore annotation database based on the set of ore image samples;

[0061] The building module is used to construct an object detection framework based on the sample augmentation module, the dual attention region generation network, and the preset lightweight multi-relation detection head;

[0062] The training module is used to acquire a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample target detection model for ore.

[0063] The detection module is used to input the image of the ore to be detected into the small sample target detection model of the ore to obtain the target detection result of the ore.

[0064] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the small sample target detection method for ore images as described above.

[0065] The present invention provides a small sample target detection method and system for ore images. For situations where the number of ore samples is small, a target detection framework is constructed based on a sample enhancement module, a dual attention region generation network, and a multi-relation detection head. This effectively solves the network overfitting problem in the ore image particle size analysis task under the condition of a small number of samples, and significantly improves the extraction accuracy of ore particle size information. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0067] Figure 1 This is a flowchart illustrating the small sample target detection method for ore images provided by the present invention;

[0068] Figure 2 This is a diagram showing the overall structure of the real-time small-sample target detection framework for ore images provided by this invention.

[0069] Figure 3 This is a structural diagram of the sample enhancement module provided by the present invention;

[0070] Figure 4 This is a comparison image of the features before and after the sample enhancement module provided by this invention.

[0071] Figure 5 This is a diagram of the dual attention region generation network structure provided by the present invention;

[0072] Figure 6 This is a comparison chart of the detection results of the ore image detection method provided by this invention with other small sample target detection methods;

[0073] Figure 7 This is a schematic diagram of the structure of the small sample target detection system for ore images provided by the present invention;

[0074] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0076] Figure 1 This is a flowchart illustrating the small-sample target detection method for ore images provided in this embodiment of the invention, as shown below. Figure 1 As shown, it includes:

[0077] Step 100: Collect a set of ore sample images, and obtain an ore annotation database based on the set of ore image samples;

[0078] Step 200: Construct an object detection framework based on the sample augmentation module, dual attention region generation network, and preset lightweight multi-relation detection head;

[0079] Step 300: Obtain a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample target detection model for ore.

[0080] Step 400: Input the image of the ore to be detected into the small sample target detection model of the ore to obtain the target detection result of the ore.

[0081] This invention proposes a real-time few-shot object detection method for ore images (A Real-time Few-Shot Object Detector for Ore Images, OreFSDet). It employs a Shot Enhancement Block (SEB) to enhance the semantic information of the support set and develops a Dual Attention-Region Proposal Network (DA-RPN) to fully utilize support features to guide the generation of accurate candidate proposals. Furthermore, a lightweight multi-relation detection head is designed to effectively evaluate the similarity between the query set and the support set, thereby significantly improving the network model's guidance performance and addressing the network overfitting problem in granular analysis tasks of ore images under limited sample conditions.

[0082] Specifically, such as Figure 2 As shown, firstly, images of ore at different sizes and locations are acquired using an image acquisition system to create an ore annotation database, i.e., ground truth (GT). Then, an object detection framework is constructed, and basic training is performed using Microsoft COCO2017. Fine-tuning training is then performed using the ore annotation database to obtain a small-sample ore object detection model. Finally, the ore images to be detected are input into the model to obtain the ore object detection results.

[0083] This invention addresses the issue of scarce ore samples by constructing a target detection framework based on a sample enhancement module, a dual attention region generation network, and a multi-relation detection head. This effectively solves the network overfitting problem in ore image particle size analysis under limited sample conditions and significantly improves the extraction accuracy of ore particle size information.

[0084] Based on the above embodiments, step 100 includes:

[0085] Collect multiple ore sample images of different sizes and locations, and cut the multiple ore sample images into multiple preset small-sized images;

[0086] The multiple preset small-sized images are divided into a support set and a query set with several category labels.

[0087] Specifically, by changing the location of the ore, such as its density and thickness, ore images of different scales were collected. Then, the ore images were preprocessed to cut large ore images into smaller ones, creating an labeled database of 5,120 ore images.

[0088] It's important to note that in the few-shot task setting, the training set consists of multiple datasets, each including a support set and a query set. The test set also consists of similar two parts. During training, the specific data in the support set and query set do not overlap, but they belong to the same categories. For each training subtask, N categories are randomly sampled, and K+1 labeled samples are sampled from each category. These K samples constitute the support set, and the remaining sample from each category constitutes the query set, creating multiple (N-way K-shot) few-shot tasks.

[0089] The database is processed to generate support sets and query sets with several categories labeled under different shots for the training set; and to generate support set feature information containing the corresponding shot for the validation set, which is used as input to the support images in the network during validation.

[0090] Based on the above embodiments, step 200 includes:

[0091] A learnable linear matrix is ​​used to transform and reconstruct the ore sample input support feature map to obtain an ore feature map with enhanced edge information.

[0092] The ore feature map with enhanced edge information is input into the channel attention mechanism module and the spatial attention mechanism module in the dual attention region generation network to obtain the channel attention feature map and the spatial attention feature map. The channel attention feature map and the spatial attention feature map are then fused to obtain the final feature map.

[0093] The target detection framework is obtained by using the preset lightweight multi-relationship detection head to perform similarity calculation and classification prediction on the final feature map.

[0094] The step of transforming and reconstructing the ore sample input support feature map using a learnable linear matrix to obtain an ore feature map with enhanced edge information includes:

[0095] Determine the length, width, and number of channels of the ore sample input support feature map;

[0096] Based on the position of the same pixel in each channel and the set of all pixels in the ore sample input support feature map, the set of all pixel positions in the ore sample input support feature map is obtained;

[0097] A fully connected layer is applied to the set of all pixel locations to obtain the learnable linear matrix.

[0098] The activation value at each feature map location is obtained by processing the learnable linear matrix with a normalized exponential function, and then...

[0099] The activation value of each feature map location is multiplied by the set of all pixel locations to obtain the weighted feature map of each pixel in each channel;

[0100] The weighted feature map of each pixel in each channel is processed by a parameterized linear correction unit, multiplied by a preset activation coefficient, and then superimposed with the ore sample input support feature map to obtain the ore feature map with enhanced edge information.

[0101] The process of inputting the enhanced edge information ore feature map into the channel attention mechanism module and the spatial attention mechanism module of the dual attention region generation network to obtain a channel attention feature map and a spatial attention feature map, and fusing the channel attention feature map and the spatial attention feature map to obtain the final feature map, includes:

[0102] The supporting features in the ore feature map of the enhanced edge information are determined based on the average pooling kernel size and the number of channels, and the query features in the ore feature map of the enhanced edge information are determined based on the length, the width, and the number of channels.

[0103] Based on the dimension of the average pooling kernel size, the support features are used as convolution kernels to perform convolution calculation on the deep cross-correlation features of the query features, and the similarity between the support features and the query features is traversed to obtain the spatial attention feature map;

[0104] The support features are compressed through one-dimensional convolution channels and then reconstructed and normalized to obtain the support feature matrix. The query features are compressed through one-dimensional convolution channels and then reconstructed to obtain the query feature matrix. The channel attention feature map is obtained by multiplying the support feature matrix and the query feature matrix and then performing an XOR operation with the query features.

[0105] The channel attention feature map and the spatial attention feature map are fused and calculated to obtain the final feature map.

[0106] The step of using the preset multi-relationship detection head to perform similarity calculation and classification prediction on the final feature map to obtain the target detection framework includes:

[0107] The channel attention feature map and the spatial attention feature map are superimposed and connected using a fully connected layer to obtain the global relation header;

[0108] Perform a pre-defined depth convolution operation on the channel attention feature map and the spatial attention feature map to obtain the local relation header;

[0109] Determine the query embedding vector of the query feature, obtain the average query embedding vector of all pixels from the query embedding, determine the key embedding vector of the supporting feature, obtain the average key embedding vector of the pixels from the key embedding, and map the transpose of the difference between the query embedding vector and the average query embedding vector of all pixels to the difference between the key embedding vector and the average key embedding vector of the pixels to obtain the fragmented relation header;

[0110] The confidence level of the target detection framework is obtained by weighted summation of the first weight adjustment coefficient, the global relation header, the second weight adjustment coefficient, the local relation header, the third weight adjustment coefficient, and the scattered relation header.

[0111] Specifically, in the design of the sample enhancement module, the input support feature map is first transformed and reconstructed using a linear matrix, so that the support feature information is concentrated at the edge of the ore and irrelevant background information is suppressed.

[0112] like Figure 3 As shown, for feature Y∈R H×W×C (H, W, and C are the length, width, and number of channels of the feature map, respectively), to obtain the linear learnable matrix. Using the learnable linear matrix I... e ∈R HW×1 The calculation process is as follows:

[0113] I e =FC(T) PE (Y)),

[0114] Where Y is the feature map, Y∈R H×W×C H, W, and C are the length, width, and number of channels of the feature map, respectively, and T... PE Y represents the set of all pixels in each channel, Ω∈HW represents the set of all pixels, and Y represents the set of all pixels. i This indicates the position of the same pixel in each channel of the feature map, and FC represents a fully connected layer.

[0115]

[0116] Where β refers to the normalization exponent softmax. It is a product sign, and the activation value at each location in the feature map is obtained through softmax (stored in I). e (Middle). These activation values ​​will be used as Y. i The weighting coefficients, and then with Y i Multiplying these results in a feature map with adjusted weights for each pixel in each channel. R represents the attention score for each pixel in feature map Y. This attention mask is superimposed on feature map Y, preserving relevant features and weakening irrelevant features, resulting in a mineral feature map Y with enhanced edge information. E :

[0117] Y E =Y + μ·LeakyReLU(R)

[0118] The best image enhancement effect is achieved when the activation coefficient μ is 0.5. LeakyReLU is a parameterized linear correction unit, such as... Figure 4 As shown, the left column is the input image, the middle column is before SEB processing, and the right column is after SEB processing. It can be seen that after SEB processing, the supporting feature information is concentrated at the edge of the ore, and irrelevant background information is suppressed.

[0119] Table 1 shows the impact of the Sample Augmentation Module (SEB) on model performance. As can be seen from Table 1, four attention mechanisms were used as modules to enhance the semantic information of the images. The concept of SEB differs from Spatial Attention (SGE) and External Attention (EANet), the latter being lightweight but with insufficient performance. Furthermore, unlike CoTAttention, which uses input context information to guide the learning of the dynamic attention matrix, SEB is lightweight enough not to significantly increase the model's memory footprint or speed burden. The results show that the embodiments of this invention effectively enhance the semantic information of the mineral images and outperform other modules in terms of accuracy, especially...

[0120] Table 1

[0121]

[0122] The structure of dual-attention RPN is as follows Figure 5 As shown, it includes a channel attention mechanism module and a spatial attention mechanism module. The channel attention mechanism module and the spatial attention mechanism module receive support feature X and query feature Y respectively, generate two different attention maps, namely spatial attention feature map and channel attention feature map, merge the two feature maps together to generate the final feature map, and then deliver them to RPN for classification and regression respectively.

[0123] In the spatial attention module, the correlation between the query and supporting feature maps is evaluated in a deep manner, and candidate boxes are constructed using similarity. Supporting features are described as X∈t. C×P×P (Kernel size after average pooling P = 1), query feature description is Y∈t C×H×W (H, W, and C are the length, width, and number of channels of the feature map), specifying the similarity as follows:

[0124]

[0125] Among them, A sThis represents the generated spatial attention feature map, where c, h, and w are the length, width, and number of channels of the feature map, respectively, and m, n ∈ {1, ..., P}, with supporting features X ∈ t. C×P×P C is the number of feature channels, P is the kernel size after average pooling, and the query feature is Y∈t. C×H×W H and W are the length and width of the feature maps, and t is the vector space. Each support feature map of X is used as a convolution kernel to perform convolution operations on the query feature map corresponding to the deep cross-correlation of Y.

[0126] In the channel attention module, X and Y are first processed using a 1×1 convolution. After processing, the channels of X are fully compressed, while the channel dimension of Y remains at a relatively high level (C / 2). Therefore, normalized softmax is used to enhance the information of the support features. Then, the support features and query features are multiplied by a matrix, followed by a 1×1 convolution, layerNorm, and sigmoid. LayerNorm increases the channel dimension from C / 2 to C, and the final sigmoid is used to keep all parameters between 0 and 1, as follows:

[0127] A c =F SG [τ1(COnv s (Y)×F SM (τ2(Conv s (X))]⊙Y

[0128] Among them, A c This represents the generated channel attention feature map, where τ1 and τ2 are tensor reshape operations, and F... SG Represents sigmoid, F SM Represents softmax, ⊙ represents multiplication concatenation, Conv s It is a convolution operation.

[0129] Table 2 shows the impact of dual-attention RPN on model performance. By comparing the four different candidate box generation methods for the ore images in Table 2, it can be found that: AttentionRPN uses only a simple spatial attention method to reweight query image features using class prototypes, which can generate class-specific proposals and improve accuracy. FGN encodes the support set and uses it to guide the generation of candidates from the query image. DAnA uses a different method to generate candidate proposals. It calculates the correlation between query and support image features through matrix multiplication, and then sends the fused feature map to the RPN. Table 3 shows the impact of different numbers of channels in the RPN detection head on model performance.

[0130] Table 2

[0131]

[0132] Table 3

[0133]

[0134] To better establish the relationship between support and query, the concepts of channel and spatial attention are introduced. Support semantic information is used to weight the query feature map, resulting in feature maps with both spatial and channel attention. Finally, the two feature maps are fused and sent to the RPN. The DA-RPN outperforms other modules in accuracy, especially... The results show that DA-RPN makes better use of the semantic information of the supporting images.

[0135] Furthermore, a lightweight multi-relation detection head is employed to obtain features that preserve accuracy. Table 4 compares the performance of the lightweight multi-relation detection head with other detection heads in few-shot object detection methods on the ore dataset. Meta-R-CNN performs better in terms of model size and inference speed, but its accuracy is lower. By relying on a large number of convolutional layers and full connection layers, FGN and AttentionRPN achieve good accuracy results, but at the cost of additional speed and memory burden. Therefore, based on AttentionRPN, we designed a lightweight multi-relation detection head to ensure inference speed, accuracy, and model size.

[0136] Table 4

[0137]

[0138] The detection head proposed in this embodiment consists of four modules. First, a 1×1 convolution is designed as a weight-shared convolutional layer to achieve cross-channel and information integration, reducing the number of channels in the output tensor from 2048 to 1024. Then, the inputs for the following three parallel prediction branches are generated. The global relation head obtains global information through global average pooling. To reduce the weight of the detection head, the two original fully connected layers are removed. The local relation head discovers the correlation between support and query suggestions along the channel level. The original 1×1×2048 shared convolutional layer is removed, and each input support feature map is directly used as a convolution kernel to perform channel-level convolution on the corresponding 7×7 query feature map. In the scattered relation head, a novel method is used to evaluate the similarity between query set features and support set features. This is achieved through matrix W. q and W k We obtain the query embedding Q of query feature Y and the key embedding K of supporting feature X, respectively. The correspondence between X and Y can be calculated as follows, μ Q and μ K This query retrieves the average value of all pixels for both the query embedding and the key embedding. The relation headers are calculated as follows:

[0139] Global relation header:

[0140] GR(X,Y)=FC(Cat(X,Y))

[0141] Local relation header:

[0142] LR(X,Y)=Cross(X,Y)

[0143] Scattered relationships:

[0144] PR(X,Y)=φ((Q-μ) Q ) T (K-μ K ))

[0145] Where Cat represents the superposition of two feature maps, FC represents a fully connected layer, Cross(X, Y) is a depthwise convolution operation, φ is a mapping operation, Q represents the query embedding of query feature Y, K represents the key embedding of supporting feature X, and μ Q and μ K These represent the average values ​​of all pixels in the query embedding and key embedding, respectively. The operation is similar to the formula for calculating the spatial attention feature map, except that the size of the convolution kernel is adjusted to 7×7.

[0146] Finally, the confidence scores Γ calculated under the three relationships are summed together for judgment:

[0147] Γ=σ1GR(X,Y)+σ2LR(X,Y)+σ3PR(X,Y)

[0148] Where σ1, σ2, and σ3 are the weight adjustment coefficients for each relation head.

[0149] Table 5 compares the experimental results of different relation head combinations in the lightweight multi-relation detection head (1 is the local relation head, 2 is the scattered relation head, and 3 is the global relation head). By merging all relation heads, a complete multi-relation detection head was achieved and its performance was optimized. This shows that the three relation heads complement each other and can better distinguish between targets and non-matching items.

[0150] Table 5

[0151]

[0152] Based on the above embodiments, step 300 includes:

[0153] Extract a predetermined number of category images from the predetermined standard dataset to perform basic training on the object detection framework;

[0154] A preset number of samples are selected from the ore labeling database. Based on a preset intersection-union ratio (IU) threshold, the accuracy of the model performance is verified using the preset number of samples. The inference memory usage, inference speed, and model size of the model performance are also verified. The optimal value of the model performance is determined, and the ore small sample target detection model is obtained.

[0155] Specifically, this embodiment of the invention uses the Microsoft COCO 2017 dataset for basic training and a mineral image dataset for fine-tuning training. For fair comparison, the method uses the common settings of previous few-shot object detection methods on the MS COCO dataset: 60 categories for basic training, with no limit on the number of samples allowed per category.

[0156] The difference lies in the training process itself. Fine-tuning is performed on the ore dataset, with a limitation on the number of samples (shots) in the training set, typically not exceeding 30. Then, testing is conducted on the validation set (1060 images) to measure the detector's performance with a smaller sample size. Six metrics are used to validate the effectiveness of the proposed detection framework, namely AP. box , Inference memory usage, inference speed, and model size. Accuracy is measured by the first three metrics, representing precision at different IoU (Intersection over Union) thresholds (from 0.5 to 0.95, with a step size of 0.05), where the computational cost of the model is reflected in the inference speed.

[0157] Based on the above embodiments, step 400 includes:

[0158] The coordinates of the ore image to be detected are output by the ore small sample target detection model, and the bounding box of the ore image to be detected is determined based on the coordinates;

[0159] The area of ​​the ore boundary rectangle is calculated based on the bounding box.

[0160] Specifically, in this embodiment of the invention, the coordinates of the ore are obtained through a trained small-sample ore target detection model, and the area of ​​the ore's bounding rectangle is calculated using bounding boxes. Extensive experimental results demonstrate the effectiveness of the method; on a new ore image dataset, the model achieves a real-time detection speed of 30 frames per second. OreFSDet not only rivals traditional object detection methods but also achieves high performance in AP... box / It also outperforms the best algorithm for Few-Shot Object Detection (FSOD) in terms of metrics. Furthermore, among FSOD methods, OreFSDet performs best in terms of inference time, model size, and inference memory.

[0161] Figure 6 To compare the detection results of the ore image detection method of this invention with other few-shot object detection methods, Table 6 shows the experimental results of the few-shot object detection method on the ore dataset under different shots, and Table 7 shows the experimental results of the traditional object detection method and the few-shot object detection method on the ore dataset. The traditional object detection method uses 4060 images for training and 1060 images for validation. Under the few-shot task setting, 60 categories from the Microsoft COCO 2017 dataset are used for basic training. The difference is that 20 new classes are replaced with a new category, ore. Then, fine-tuning training and validation of the detector are performed under different shots. During validation, in order to compare performance with the traditional object detection method, the validation set is set to the same 1060 images.

[0162] Table 6

[0163]

[0164] Table 7

[0165]

[0166]

[0167] It can be seen that, when the number of samples is small, this invention can detect complete and independent ores more accurately and in real time compared with traditional target detection methods.

[0168] The small sample target detection system for ore images provided by the present invention is described below. The small sample target detection system for ore images described below can be referred to in correspondence with the small sample target detection method for ore images described above.

[0169] Figure 7 This is a schematic diagram of the structure of the small sample target detection system for ore images provided in an embodiment of the present invention, as shown below. Figure 7 As shown, it includes: a data acquisition module 71, a construction module 72, a training module 73, and a detection module 74, wherein:

[0170] The acquisition module 71 is used to acquire a set of ore sample images and obtain an ore annotation database based on the ore image sample set; the construction module 72 is used to construct a target detection framework based on a sample augmentation module, a dual attention region generation network, and a preset lightweight multi-relation detection head; the training module 73 is used to acquire a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample ore target detection model; the detection module 74 is used to input the ore image to be detected into the small sample ore target detection model to obtain the ore target detection result.

[0171] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a small-sample target detection method for ore images. The method includes: acquiring a set of ore sample images; obtaining an ore annotation database based on the ore image sample set; constructing a target detection framework based on a sample augmentation module, a dual-attention region generation network, and a preset lightweight multi-relation detection head; acquiring a preset standard dataset; performing basic training on the target detection framework using the preset standard dataset; fine-tuning the target detection framework using the ore annotation database to obtain a small-sample target detection model for ore; and inputting the ore image to be detected into the small-sample target detection model to obtain the ore target detection result.

[0172] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for small-sample target detection in ore images, characterized in that, include: Collect a set of ore sample images, and obtain an ore annotation database based on the set of ore image samples; A target detection framework is constructed based on a sample augmentation module, a dual-attention region generation network, and a pre-defined lightweight multi-relation detection head. Obtain a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample target detection model for ore. The image of the ore to be detected is input into the small sample target detection model of the ore to obtain the target detection result of the ore; The target detection framework constructed based on the sample augmentation module, dual attention region generation network, and preset lightweight multi-relation detection head includes: A learnable linear matrix is ​​used to transform and reconstruct the ore sample input support feature map to obtain an ore feature map with enhanced edge information. The ore feature map with enhanced edge information is input into the channel attention mechanism module and the spatial attention mechanism module in the dual attention region generation network to obtain the channel attention feature map and the spatial attention feature map. The channel attention feature map and the spatial attention feature map are then fused to obtain the final feature map. The target detection framework is obtained by using the preset lightweight multi-relationship detection head to perform similarity calculation and classification prediction on the final feature map; The process of transforming and reconstructing the ore sample input support feature map using a learnable linear matrix to obtain an ore feature map with enhanced edge information includes: Determine the length, width, and number of channels of the ore sample input support feature map; Based on the position of the same pixel in each channel and the set of all pixels in the ore sample input support feature map, the set of all pixel positions in the ore sample input support feature map is obtained; A fully connected layer is applied to the set of all pixel locations to obtain the learnable linear matrix. : in, It is a feature map. , H, W and C These are the length, width, and number of channels of the feature map, respectively. Represents the set of all pixels in each channel. Represents the set of all pixels. This indicates the position of the same pixel in each channel of the feature map. Indicates a fully connected layer; The activation value of each feature map location is obtained by processing the learnable linear matrix with a normalized exponential function. Then, each activation value is multiplied by the set of all pixel locations to obtain the weighted feature map of each pixel in each channel. : in, Refers to the normalization index , It is the product symbol; The weighted feature map of each pixel in each channel is processed by a parameterized linear correction unit, multiplied by a preset activation coefficient, and then superimposed with the ore sample input support feature map to obtain the ore feature map with enhanced edge information. : Where, when the activation coefficient The image enhancement effect is best when the value is 0.

5. It is a parameterized linear correction unit.

2. The method for small-sample target detection in ore images according to claim 1, characterized in that, The collected ore sample image set, based on which an ore annotation database is obtained, includes: Collect multiple ore sample images of different sizes and locations, and cut the multiple ore sample images into multiple preset small-sized images; The multiple preset small-sized images are divided into a support set and a query set with several category labels.

3. The method for small-sample target detection in ore images according to claim 1, characterized in that, The process involves inputting the enhanced edge information ore feature map into the channel attention mechanism module and the spatial attention mechanism module of the dual attention region generation network, respectively, to obtain a channel attention feature map and a spatial attention feature map. The channel attention feature map and the spatial attention feature map are then fused to obtain the final feature map, including: The supporting features in the ore feature map of the enhanced edge information are determined based on the average pooling kernel size and the number of channels, and the query features in the ore feature map of the enhanced edge information are determined based on the length, the width, and the number of channels. Based on the dimension of the average pooling kernel size, the support features are used as convolution kernels to perform convolution calculations on the deep cross-correlation features of the query features. By traversing the similarity between the support features and the query features, the spatial attention feature map is obtained. : in, This represents the generated spatial attention feature map, where c, h, and w are the length, width, and number of channels of the feature map, respectively. Supporting features are , The number of feature channels, The kernel size after average pooling is the query feature. , H, W These are the length and width of the feature map. t It is a vector space; The support features are compressed through one-dimensional convolution channels and then reconstructed and normalized sequentially to obtain the support feature matrix. Similarly, the query features are compressed through one-dimensional convolution channels and then reconstructed to obtain the query feature matrix. The channel attention feature map is obtained by multiplying the support feature matrix and the query feature matrix, and then performing an XOR operation with the query features. : in, This represents the generated channel attention feature map. and It is a tensor operate, represent , express , Indicates multiplication concatenation. It is a convolution operation; The channel attention feature map and the spatial attention feature map are fused and calculated to obtain the final feature map.

4. The method for small-sample target detection in ore images according to claim 3, characterized in that, The step of using the preset lightweight multi-relation detection head to perform similarity calculation and classification prediction on the final feature map to obtain the target detection framework includes: The channel attention feature map and the spatial attention feature map are superimposed and connected using a fully connected layer to obtain the global relation header. : in, This indicates that two feature maps are superimposed. Indicates a fully connected layer. Indicates supporting features, Indicates query characteristics; Perform a pre-defined depthwise convolution operation on the channel attention feature map and the spatial attention feature map to obtain the local relation head. : in, Represents depthwise convolution operations; Determine the query embedding vector of the query feature, and obtain the average query embedding vector of all pixels from the query embedding. Determine the key embedding vector of the supporting feature, and obtain the average key embedding vector of the pixels from the key embedding. Map the transpose of the difference between the query embedding vector and the average query embedding vector of all pixels to the difference between the key embedding vector and the average key embedding vector of the pixels to obtain the fragmented relation header. : in, It is a mapping operation. Indicate query characteristics Y Query embedding, Indicates support features Key embedding, and These represent the average values ​​of all pixels in the query embedding and key embedding, respectively. The confidence level of the target detection framework is obtained by weighted summation of the first weight adjustment coefficient, the global relation header, the second weight adjustment coefficient, the local relation header, the third weight adjustment coefficient, and the scattered relation header. : in, ， ， These are the weight adjustment coefficients for each relation head.

5. The method for small-sample target detection in ore images according to claim 1, characterized in that, The process of obtaining a preset standard dataset, performing basic training on the target detection framework using the preset standard dataset, and fine-tuning the target detection framework using the ore annotation database to obtain a small-sample ore target detection model includes: Extract a predetermined number of category images from the predetermined standard dataset to perform basic training on the object detection framework; A preset number of samples are selected from the ore labeling database. Based on a preset intersection-union ratio (IU) threshold, the accuracy of the model performance is verified using the preset number of samples. The inference memory usage, inference speed, and model size of the model performance are also verified. The optimal value of the model performance is determined, and the ore small sample target detection model is obtained.

6. The method for small-sample target detection in ore images according to claim 1, characterized in that, The step of inputting the image of the ore to be detected into the small sample target detection model to obtain the ore target detection result includes: The coordinates of the ore image to be detected are output by the ore small sample target detection model, and the bounding box of the ore image to be detected is determined based on the coordinates; The area of ​​the ore boundary rectangle is calculated based on the bounding box.

7. A small-sample target detection system for ore images, based on the small-sample target detection method for ore images according to any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire a set of ore sample images and obtain an ore annotation database based on the set of ore image samples; The building module is used to construct an object detection framework based on the sample augmentation module, the dual attention region generation network, and the preset lightweight multi-relation detection head; The training module is used to acquire a preset standard dataset, perform basic training on the target detection framework using the preset standard dataset, and fine-tune the target detection framework using the ore annotation database to obtain a small sample target detection model for ore. The detection module is used to input the image of the ore to be detected into the small sample target detection model of the ore to obtain the target detection result of the ore.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the small sample target detection method for ore images as described in any one of claims 1 to 6.

Images