Scrap image fine-grained classification method combined with progressive network of heterogeneous local graph

Abstract

The application provides a scrap steel image fine-grained classification method combining a progressive network of heterogeneous local graphs, which is used for grade division of scrap steels, and the progressive network is a convolutional neural network, and scrap steel images collected by a monitoring camera are used as a data set for training of the convolutional neural network; the method first integrates attention information from different training iteration stages and network feature extraction stages to locate a target region containing key information from an original image under the condition of only using an image category label, then samples a salient local part in the target image, learns key local detail information, and finally constructs a complete heterogeneous local graph to learn semantic relationships between different local regions, so that the discriminability of features is enhanced; the application can combine global and local information, and further learn the relevance of the local part, so that the accuracy of scrap steel image classification is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a fine-grained classification method for scrap steel images that combines a progressive network of heterogeneous local graphs. Background Technology

[0002] Scrap steel image classification falls under the category of fine-grained image classification within computer vision and pattern recognition. Due to the significant price differences between different grades of scrap steel, its quality directly impacts the production costs and product quality of steel enterprises. Therefore, classifying scrap steel into grades is an important and highly challenging research topic. Unlike ordinary image classification, scrap steel image classification aims to further refine the subclassification of broad categories of scrap steel images, thereby achieving grade classification for different types of scrap steel. Because different categories of scrap steel have similar appearances, the differences between classes are more subtle. Furthermore, different images may exhibit significant variance within the same category due to interference from factors such as the placement of the scrap steel, lighting, and background. These issues make fine-grained scrap steel image classification even more challenging. Recent studies have shown that the key to solving the fine-grained classification problem lies in learning more discriminative representations and locating salient parts of the target features, and that there are certain connections between these local parts. Y.Ding used sparse attention to sample salient parts of the target. WangZ designed a graph propagation subnetwork to find salient local regions and used GCN to learn the internal semantic correlations between feature vectors of these regions. S. Wang used GNNs to model the semantic relationships between salient regions of subclasses to learn important attributes. However, these studies essentially only considered learning independently on a single image, ignoring the clustering characteristics of the feature space as a whole. Due to the characteristics of scrap steel image datasets, the spatial distribution of data is often more discrete, making it impossible to learn feature representations with sufficient discriminative power. Summary of the Invention

[0003] This invention proposes a fine-grained classification method for scrap steel images that combines a progressive network with heterogeneous local graphs. This method can combine global and local information and further learn the correlation of local information to improve the accuracy of scrap steel image classification.

[0004] The present invention adopts the following technical solution.

[0005] A fine-grained classification method for scrap steel images, combining a progressive network with a heterogeneous local map, is proposed to classify scrap steel into different grades. The progressive network is a convolutional neural network, using scrap steel images captured by surveillance cameras as the training dataset for the convolutional neural network. The method first integrates attention information from different training iterations and network feature extraction stages using only image category labels to locate target regions containing key information in the original image. Then, salient local components are sampled in the target image to learn key local detail information. Finally, a complete heterogeneous local map is constructed to learn the semantic relationships between different local regions, thereby enhancing the discriminative power of the features.

[0006] The method includes the following steps;

[0007] Step S1: Input a mini-batch of images into a convolutional neural network to extract the feature maps F∈R. H×W×C Where H, W, and C represent the height, width, and number of channels of the feature map, respectively. Channel attention is used to aggregate the feature map F, reducing the number of channels to a single channel to generate the corresponding attention map A∈R. H×W :

[0008]

[0009] The above formula This involves bitwise multiplication, followed by a simple normalization. Proceed with subsequent operations;

[0010] Step S2: Establish an attention accumulation map M to store the high-response regions of the attention maps A in each training iteration stage. First, initialize M using the attention map A1 generated in the first iteration stage. Here, the initialized attention accumulation map is denoted as M1. When training enters the second stage, update M1 using the attention map A2 generated in the current stage.

[0011] Formula 2: M2 = max(M1, A2);

[0012] Here, `max()` represents the operation of comparing the values ​​of each element of the two input items and taking the maximum value. Similarly, when the training iteration enters the k-th stage, the cumulative attention map M is updated in the same way:

[0013] M K =max(M K-1 A K Formula 3;

[0014] Find the mean of the updated attention cumulative graph M. The points at each location in M ​​are divided using a threshold, resulting in a binary image.

[0015] Step S3: Based on the binary image, crop out the largest 1-connected region from the image as the target image, adjust its size, and then input it into the convolutional neural network for feature extraction;

[0016] Step S4: Based on the sliding window concept in object detection, multiple predefined anchors of different scales and proportions are used to sample the salient regions of the target based on the feature map of the target image as part images. The size is adjusted and then input into the convolutional neural network for feature extraction.

[0017] Step S5: Perform global average pooling on the feature map of the component image to obtain the local component feature vector. in Let F represent the p-th local feature of the i-th image. Construct a complete heterogeneous local graph G = (F, E) using all local features as nodes. Let F be the local region feature and E be the edge set. This graph has two different types of edges: positive edges where the local region corresponding to a node comes from the same class of images, and negative edges where two nodes come from different classes of images. The weights of the edges between graph nodes are calculated using a self-attention operation.

[0018]

[0019] Where W∈d h ×d f It is a transformation matrix. Here, the original node feature representation is first linearly transformed. || represents the concat of the feature representations of two nodes in the channel dimension. g is a fully connected layer that calculates the edge weights between all pairs of nodes.

[0020] Step S6: To enable comparison of edge weights between different nodes, the calculated adjacency matrix E is fed into a softmax layer for normalization.

[0021]

[0022] This represents the association weight coefficient between the p-th local region of the i-th image and the q-th local region of the j-th image in a mini-batch. The node representation is based on this association weight coefficient. Perform a linear combination to update the original node features:

[0023]

[0024] Where σ represents the ELU nonlinear activation function;

[0025] Step S7: Optimize the entire network parameters by minimizing the following objective function:

[0026] L total =L cls +α1*L ar +α2*L distill Formula 7;

[0027] Where α1 and α2 are hyperparameters, set to 0.01 and 1 respectively; thus,

[0028] (1) L cls The classification loss includes the cross-entropy loss for classifying the original image, the target image, and the part image, where y is the true label of the input image, and C represents a fully connected layer and a softmax layer.

[0029] (2) L ar For attention regularization loss, the weights of positive and negative edges in the heterogeneous local graph are constrained to guide the graph to correctly learn the interaction relationships between feature vectors of different nodes. bce (,) represents the binary cross-entropy loss, and τ represents the true value. If If the edge is directly opposite, then τ equals 1; otherwise, τ equals 0.

[0030] (3) L distill For distillation loss, β is a hyperparameter set to 0.7. To enable the network to directly learn the optimization information generated by the constructed heterogeneous local graph, L... distill Self-distillation is performed between the feature extraction network and the heterogeneous local map, allowing the feature extraction network to directly generate optimized feature representations. The new node feature representations generated by the heterogeneous local map are only used to calculate the classification loss during the training iteration phase. The final image classification result is...

[0031] In step S3, the target image is obtained using a multi-layer ensemble method. The corresponding binary mask images are calculated using the features output by the last convolutional block Conv_final and the previous convolutional block Conv_before. and The location of the target connected region is determined jointly by these two factors.

[0032] In step S4, when sampling regions of the target image, eight anchors of three different sizes are set: {[4×4, 3×5], [6×6, 5×7], [8×8, 6×10, 7×9, 7×10]}. The mean value of the corresponding region of each window on the attention map of the target image features is calculated. (H W and W W We use the height and width of the anchor as scores to find valuable local regions. In order to ensure that the sampled regions contain different information as much as possible and to avoid information redundancy caused by the overlap between windows, we finally use Non-Maximum Suppression (NMS) to select a fixed number of windows as the sampling results.

[0033] The method performs the following steps when outputting the predicted image category label;

[0034] Step A1: Input a mini-batch of multiple scrap steel images x1,...,x n ;

[0035] Step A2: Use a convolutional neural network (CNN) to extract features from each image;

[0036] Step A3: Aggregate the channels of each feature map to generate the corresponding attention map A;

[0037] Step A4: Integrate the attention maps from different iteration stages of the image by taking the maximum value to generate the attention cumulative map M;

[0038] Step A5: Calculate the binary mask image using the attention accumulation map of features from different network layers. Integrate the positioning results of the target area;

[0039] Step A6: Crop the target image o1,...,o n The image size was adjusted and then input into the network;

[0040] Step A7: Extract features from each target image;

[0041] Step A8: Use anchors of different scales and sizes to sample salient regions in the attention map of the target image;

[0042] Step A9: Crop the sampled area into a component image. The image size was adjusted before being input into the network;

[0043] Step A10: Extract features from the images of each component;

[0044] Step A11: Construct a heterogeneous local graph G, update the features of the component images, and calculate the attention regularization loss L. ar ;

[0045] Step A12: Calculate the distillation loss L distill The updated feature optimization information is fed back to the input;

[0046] Step A13: Calculate the total loss L total =L cls +α1*L ar +α2*L distill Backpropagation optimizes network parameters;

[0047] Step A14: Repeat the above steps until the required number of iterations is met;

[0048] Step A15: Output the predicted image category label

[0049] The scrap steel images were acquired using on-site surveillance cameras, with anti-shake technology applied to the cameras to prevent vibrations from the scrap steel yard after vehicles unload heavy loads from interfering with image acquisition. Three cameras were used to capture images from different angles to reduce dataset bias caused by occlusion due to different placement of scrap steel.

[0050] The scrap steel includes self-produced scrap steel, processed scrap steel, and used scrap steel. Self-produced scrap steel refers to scrap steel generated during the processing of steel products. Processed scrap steel refers to scrap steel generated by downstream manufacturers during the steel processing process. Used scrap steel mainly includes depreciated scrap steel and waste scrap steel. Depreciated scrap steel refers to scrap steel from end-use automobiles, machinery, construction, etc. that have reached their scrap age, while waste scrap steel refers to useless furniture from daily life.

[0051] Compared to existing technologies, this invention combines global and local information and further learns the correlation of local data to improve the accuracy of scrap steel image classification, resulting in the following beneficial effects:

[0052] 1) A multi-branch network for fine-grained classification of scrap steel images was established, which can learn features of different granularities from coarse to fine, and effectively combine global and local information of scrap steel images;

[0053] 2) By utilizing the relationships between local components in a scrap steel image, a heterogeneous local graph is constructed, which can fully learn the semantic relationships between local components. Furthermore, based on a contrastive learning approach, the relationships between positive edges of the same category and negative edges of different categories are constrained accordingly, thereby enabling the learned features to have more significant information about the category. Attached Figure Description

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0055] Appendix Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0056] As shown in the figure, a fine-grained classification method for scrap steel images, combining a progressive network with a heterogeneous local map, is used to distinguish the grades of scrap steel. The progressive network is a convolutional neural network, using scrap steel images captured by surveillance cameras as the training dataset for the convolutional neural network. The method first integrates attention information from different training iteration stages and network feature extraction stages using only image category labels to locate target regions containing key information in the original image. Then, it samples salient local components in the target image and learns key local detail information. Finally, it constructs a complete heterogeneous local map and learns the semantic relationships between different local regions to enhance the discriminative power of the features.

[0057] The method includes the following steps;

[0058] Step S1: Input a mini-batch of images into a convolutional neural network to extract the feature maps F∈R. H×W×C Where H, W, and C represent the height, width, and number of channels of the feature map, respectively. Channel attention is used to aggregate the feature map F, reducing the number of channels to a single channel to generate the corresponding attention map A∈R. H×W :

[0059]

[0060] The above formula This involves bitwise multiplication, followed by a simple normalization. Proceed with subsequent operations;

[0061] Step S2: Establish an attention accumulation map M to store the high-response regions of the attention maps A in each training iteration stage. First, initialize M using the attention map A1 generated in the first iteration stage. Here, the initialized attention accumulation map is denoted as M1. When training enters the second stage, update M1 using the attention map A2 generated in the current stage.

[0062] Formula 2: M2 = max(M1, A2);

[0063] Here, `max()` represents the operation of comparing the values ​​of each element of the two input items and taking the maximum value. Similarly, when the training iteration enters the k-th stage, the cumulative attention map M is updated in the same way:

[0064] M K =max(M K-1 A K Formula 3;

[0065] Find the mean of the updated attention cumulative graph M. The points at each location in M ​​are divided using a threshold, resulting in a binary image.

[0066] Step S3: Based on the binary image, crop out the largest 1-connected region from the image as the target image, adjust its size, and then input it into the convolutional neural network for feature extraction;

[0067] Step S4: Based on the sliding window concept in object detection, multiple predefined anchors of different scales and proportions are used to sample the salient regions of the target based on the feature map of the target image as part images. The size is adjusted and then input into the convolutional neural network for feature extraction.

[0068] Step S5: Perform global average pooling on the feature map of the component image to obtain the local component feature vector. in Let F represent the p-th local feature of the i-th image. Construct a complete heterogeneous local graph G = (F, E) using all local features as nodes. Let F be the local region feature and E be the edge set. This graph has two different types of edges: positive edges where the local region corresponding to a node comes from the same class of images, and negative edges where two nodes come from different classes of images. The weights of the edges between graph nodes are calculated using a self-attention operation.

[0069]

[0070] Where W∈d h ×d f It is a transformation matrix. Here, the original node feature representation is first linearly transformed. || represents the concat of the feature representations of two nodes in the channel dimension. g is a fully connected layer that calculates the edge weights between all pairs of nodes.

[0071] Step S6: To enable comparison of edge weights between different nodes, the calculated adjacency matrix E is fed into a softmax layer for normalization.

[0072]

[0073] This represents the association weight coefficient between the p-th local region of the i-th image and the q-th local region of the j-th image in a mini-batch. The node representation is based on this association weight coefficient. Perform a linear combination to update the original node features:

[0074]

[0075] Where σ represents the ELU nonlinear activation function;

[0076] Step S7: Optimize the entire network parameters by minimizing the following objective function:

[0077] L total =L cls +α1*L ar +α2*L distill Formula 7;

[0078] Where α1 and α2 are hyperparameters, set to 0.01 and 1 respectively; thus,

[0079] (1)

[0080] L cls The classification loss includes the cross-entropy loss for classifying the original image, the target image, and the part image, where y is the true label of the input image, and C represents a fully connected layer and a softmax layer.

[0081] (2)

[0082] L ar For attention regularization loss, the weights of positive and negative edges in the heterogeneous local graph are constrained to guide the graph to correctly learn the interaction relationships between feature vectors of different nodes. bce (,) represents the binary cross-entropy loss, and τ represents the true value. If If the edge is directly opposite, then τ equals 1; otherwise, τ equals 0.

[0083] (3)

[0084] L distill For distillation loss, β is a hyperparameter set to 0.7. To enable the network to directly learn the optimization information generated by the constructed heterogeneous local graph, L... distill Self-distillation is performed between the feature extraction network and the heterogeneous local map, allowing the feature extraction network to directly generate optimized feature representations. The new node feature representations generated by the heterogeneous local map are only used to calculate the classification loss during the training iteration phase. The final image classification result is...

[0085] In step S3, the target image is obtained using a multi-layer ensemble method. The corresponding binary mask images are calculated using the features output by the last convolutional block Conv_final and the previous convolutional block Conv_before. and The location of the target connected region is determined jointly by these two factors.

[0086] In step S4, when sampling regions of the target image, eight anchors of three different sizes are set: {[4×4, 3×5], [6×6, 5×7], [8×8, 6×10, 7×9, 7×10]}. The mean value of the corresponding region of each window on the attention map of the target image features is calculated. (H W and W W We use the height and width of the anchor as scores to find valuable local regions. In order to ensure that the sampled regions contain different information as much as possible and to avoid information redundancy caused by the overlap between windows, we finally use Non-Maximum Suppression (NMS) to select a fixed number of windows as the sampling results.

[0087] The method performs the following steps when outputting the predicted image category label;

[0088] Step A1: Input a mini-batch of multiple scrap steel images x1,...,x n ;

[0089] Step A2: Use a convolutional neural network (CNN) to extract features from each image;

[0090] Step A3: Aggregate the channels of each feature map to generate the corresponding attention map A;

[0091] Step A4: Integrate the attention maps from different iteration stages of the image by taking the maximum value to generate the attention cumulative map M;

[0092] Step A5: Calculate the binary mask image using the attention accumulation map of features from different network layers. Integrate the positioning results of the target area;

[0093] Step A6: Crop the target image o1,...,o n The image size was adjusted and then input into the network;

[0094] Step A7: Extract features from each target image;

[0095] Step A8: Use anchors of different scales and sizes to sample salient regions in the attention map of the target image;

[0096] Step A9: Crop the sampled area into a component image. The image size was adjusted before being input into the network;

[0097] Step A10: Extract features from the images of each component;

[0098] Step A11: Construct a heterogeneous local graph G, update the features of the component images, and calculate the attention regularization loss L.ar ;

[0099] Step A12: Calculate the distillation loss L distill The updated feature optimization information is fed back to the input;

[0100] Step A13: Calculate the total loss L total =L cls +α1*L ar +α2*L distill Backpropagation optimizes network parameters;

[0101] Step A14: Repeat the above steps until the required number of iterations is met;

[0102] Step A15: Output the predicted image category label

[0103] The scrap steel images were acquired using on-site surveillance cameras, with anti-shake technology applied to the cameras to prevent vibrations from the scrap steel yard after vehicles unload heavy loads from interfering with image acquisition. Three cameras were used to capture images from different angles to reduce dataset bias caused by occlusion due to different placement of scrap steel.

[0104] The scrap steel includes self-produced scrap steel, processed scrap steel, and used scrap steel. Self-produced scrap steel refers to scrap steel generated during the processing of steel products. Processed scrap steel refers to scrap steel generated by downstream manufacturers during the steel processing process. Used scrap steel mainly includes depreciated scrap steel and waste scrap steel. Depreciated scrap steel refers to scrap steel from end-use automobiles, machinery, construction, etc. that have reached their scrap age, while waste scrap steel refers to useless furniture from daily life.

[0105] Example:

[0106] In this example, during iterative network training, the original image size is adjusted to 448×448 before being input into the network. The size of the cropped target image is adjusted to the same size as the original image, while all local part images are adjusted to 224×224. When sampling regions of the target image, 14 different local part images are sampled, with 4 for the lowest-level anchor, 6 for the second-level, and 4 for the highest-level. The hyperparameters λ1 and λ2 of the objective function are set to 0.01 and 1, respectively, and the balance factor for distillation loss is 0.7. SGD is used as the optimizer, with the optimizer momentum set to 0.9, the learning rate initialized to 0.001 and multiplied by 0.1 every 60 training iterations, for a total of 200 iterations. The weight decay is set to 0.0001, and the minimum batch size is 6.

[0107] In this example, some images in the dataset contain impurities mixed in with the scrap steel.

[0108] In this example, the dataset contains 690 images with 7 different levels of labels. The images at each level of label are basically balanced across categories. 483 images are randomly selected for network training, and 207 images are used to validate the training results.

[0109] In this example, scrap steel is classified into seven different grades according to different grade labels in the dataset.

Claims

1. A fine-grained classification method for scrap steel images combining a progressive network with heterogeneous local graphs, used to classify scrap steel by grade, characterized in that: The progressive network is a convolutional neural network, using scrap steel images captured by surveillance cameras as the training dataset for the convolutional neural network. The method first integrates attention information from different training iteration stages and network feature extraction stages using only image category labels to locate the target region containing key information in the original image. Then, it samples salient local parts in the target image, learns key local detail information, and finally constructs a complete heterogeneous local map to learn the semantic relationships between different local regions, thereby enhancing the discriminative power of the features. The method performs the following steps when outputting the predicted image category label; Step A1: Input a mini-batch of multiple scrap steel images. ; Step A2: Use a convolutional neural network (CNN) to extract features from each image; Step A3: Aggregate the channels of each feature map to generate the corresponding attention map A; Step A4: Integrate the attention maps from different iteration stages of the image by taking the maximum value to generate the attention cumulative map M; Step A5: Calculate the binary mask image using the attention accumulation map of features from different network layers. Integrate the positioning results of the target area; Step A6: Crop the target image The image size was adjusted and then input into the network; Step A7: Extract features from each target image; Step A8: Use anchors of different scales and sizes to sample salient regions in the attention map of the target image; Step A9: Crop the sampled area into a component image. The image size was adjusted before being input into the network; Step A10: Extract features from the images of each component; Step A11: Construct heterogeneous local graphs Update the features of the component image and calculate the attention regularization loss. ; Step A12: Calculate distillation loss The updated feature optimization information is fed back to the input; Step A13: Calculate the total loss Backpropagation optimizes network parameters; For classification loss; , For hyperparameters; Step A14: Repeat the above steps until the required number of iterations is met; Step A15: Output the predicted image category label ; The first part represents the local features of the image. indivual.

2. The fine-grained classification method for scrap steel images using a progressive network combining heterogeneous local graphs as described in claim 1, characterized in that: The method includes the following steps; Step S1: Input a mini-batch of images into a convolutional neural network to extract the feature maps of the images. ,in , and These represent the height, width, and number of channels of the feature map, respectively. Channel attention is used to process the feature map. Aggregation is performed to reduce the number of channels to a single channel, generating the corresponding attention map. : Formula 1; The above formula This involves bitwise multiplication, followed by a simple normalization. Proceed with subsequent operations; Step S2: Construct an attention accumulation map M to store the high-response regions of the attention map A for each training iteration. First, use the attention map generated in the first iteration. We initialize M, and here we denote the initialized attention accumulation graph as... When training enters the second phase, the attention map generated in the current phase is used. right Update: Formula 2; Here, `max()` represents the operation of comparing the values ​​of each element of the two input items and taking the maximum value. Similarly, when the training iteration enters the k-th stage, the cumulative attention map M is updated in the same way: Formula 3; Find the mean of the updated attention cumulative graph M. The points at each location in M ​​are divided using a threshold, resulting in a binary image. ; Step S3: Based on the binary image, crop out the largest 1-connected region from the image as the target image, adjust its size, and then input it into the convolutional neural network for feature extraction; Step S4: Based on the sliding window concept in object detection, multiple predefined anchors of different scales and proportions are used to sample the salient regions of the target based on the feature map of the target image as part images. The size is adjusted and then input into the convolutional neural network for feature extraction. Step S5: Perform global average pooling on the feature map of the component image to obtain the local component feature vector. ,in Indicates the first The first picture Each local feature is used as a node to construct a complete heterogeneous local graph. Let the local region features F be the nodes of the graph, and E be the set of edges. There are two different types of edges in this graph: one is a positive edge where the local region corresponding to a node comes from the same class of image, and the other is a negative edge where two nodes come from different classes of image. The weights of the edges between graph nodes are calculated through a self-attention operation. Formula 4; in It is a transformation matrix. Here, the original node feature representation is first linearly transformed. || represents the concat of the feature representations of two nodes in the channel dimension. g is a fully connected layer that calculates the edge weights between all pairs of nodes. Step S6: To allow for comparison of edge weights between different nodes, the calculated adjacency matrix is... The data is fed into a softmax layer for normalization. Formula 5; This represents the first element in a mini-batch. The first image The local region and the first The first image The association weight coefficient of each local region is used to represent the nodes. Perform a linear combination to update the original node features: Formula Six; in Represents the ELU nonlinear activation function; Step S7: Optimize the entire network parameters by minimizing the following objective function: Formula 7; in and As hyperparameters, set them to 0.01 and 1 respectively; (The result is...) （1） Formula 8; The classification loss includes the cross-entropy loss of the original image, the target image, and the part image, where y is the true label of the input image, and C represents a fully connected layer and a softmax layer; （2） Formula Nine; To address the attention regularization loss, the weights of positive and negative edges in the heterogeneous local graph are constrained, guiding the graph to correctly learn the interaction relationships between feature vectors of different nodes. For binary cross-entropy loss, Represents the truth value, if If it is the opposite side, then It equals 1, otherwise Equal to 0; (3) Formula 10; Distillation loss, of which The hyperparameter was set to 0.7 to allow the network to directly learn the optimization information generated by the constructed heterogeneous local graph. Self-distillation is performed between the feature extraction network and the heterogeneous local map, allowing the feature extraction network to directly generate optimized feature representations. The new node feature representations generated by the heterogeneous local map are only used to calculate the classification loss during the training iteration phase. The final image classification result is... .

3. The fine-grained classification method for scrap steel images using a progressive network combining heterogeneous local graphs as described in claim 2, characterized in that: In step S3, the target image is obtained using a multi-layer ensemble method. The corresponding binary mask images are calculated using the features output by the last convolutional block Conv_final and the previous convolutional block Conv_before. and The location of the target connected region is determined jointly by the two. .

4. The fine-grained classification method for scrap steel images using a progressive network combining heterogeneous local graphs as described in claim 3, characterized in that: In step S4, when sampling regions of the target image, eight anchors of three different sizes are set: {[4 × 4, 3 × 5], [6 × 6, 5 × 7], [8 × 8, 6 × 10, 7 × 9, 7 × 10]}; the mean value of the corresponding region of each window on the attention map of the target image features is calculated. , and The height and width of the anchor are used as scores to find valuable local regions. In order to ensure that the sampled regions contain different information as much as possible and to avoid information redundancy caused by window overlap, NMS (non-maximum suppression) is finally used to select a fixed number of windows as the sampling results.

5. The fine-grained classification method for scrap steel images using a progressive network combining heterogeneous local graphs as described in claim 1, characterized in that: The scrap steel images were acquired using on-site surveillance cameras, with anti-shake technology applied to the cameras to prevent vibrations from the scrap steel yard after vehicles unload heavy loads from interfering with image acquisition. Three cameras were used to capture images from different angles to reduce dataset bias caused by occlusion due to different placement of scrap steel.

6. The fine-grained classification method for scrap steel images using a progressive network combining heterogeneous local graphs as described in claim 1, characterized in that: The scrap steel includes self-produced scrap steel, processed scrap steel, and used scrap steel. Self-produced scrap steel refers to scrap steel generated during the processing of steel products. Processed scrap steel refers to scrap steel generated by downstream manufacturers during the steel processing process. Used scrap steel mainly includes depreciated scrap steel and waste scrap steel. Depreciated scrap steel refers to scrap steel from end-use automobiles, machinery, construction, etc. that have reached their scrap age, while waste scrap steel refers to useless furniture from daily life.

Images