A method for grading unbalanced scrap steel samples based on tree-structured hierarchical network

Abstract

The present invention discloses a method for grading unbalanced scrap steel samples based on a tree-type hierarchical progressive network, and relates to the field of metal processing technology. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical progressive network can solve the serious long-tail effect existing in existing scrap steel grading data sets, thereby avoiding severely biased prediction results and improving the unbiasedness of the overall prediction of the model. By integrating various representative features stored in the dynamic memory module, the feature representation of the current target is enhanced, further improving the prediction accuracy of the overall model. By displaying and mining significant differences and subtle differences between different categories through a tree-type classification network, the overall generalization ability of the model for tail predicates is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of metal processing, and in particular to a method for grading unbalanced scrap steel samples based on a tree-type hierarchical network. Background Art

[0002] The steel industry is a pillar of national economic development, radiating to a range of downstream sectors, such as real estate, automobiles, and shipbuilding. The classification, grading, and recycling of scrap steel are crucial for cost reduction across the entire steel industry, playing a crucial role in promoting the industry's overall low-carbon development and improving steel recycling. With the advancement of deep learning technology, its integration with various industries has become increasingly close, including the steel industry. Traditional scrap steel inspection and grading relies primarily on manual visual inspection, caliper measurement, and joint judgment by supervisors at production plants and technical centers. These grading procedures are cumbersome and time-consuming, and are subject to human subjective perception and accumulated experience. This results in significant labor and material resources being consumed during actual grading operations, and leads to numerous quality disputes.

[0003] Robust and generalizable deep learning models rely on large-scale datasets with sufficient and relatively balanced sample distribution. However, in the real-world scrap steel grading task, due to the complex detection scenarios and the wide variety of scrap steel types, scrap steel classification datasets constructed by existing methods often suffer from a severe long-tail distribution effect. Specifically, only a small number of classes (the head classes) have sufficient and diverse training samples, while the remaining large number of classes (the tail classes) often have very sparse training samples. This extremely unbalanced sample data distribution causes the model to focus solely on fitting the data for the head classes during training, while neglecting to learn the tail classes. This leads to significant bias in prediction results, with only a small number of the head classes being accurately predicted, while a large number of the tail classes are misclassified or ignored. This bias in prediction reduces scrap steel grading accuracy and can lead to significant economic and profit losses for steel companies due to misclassification. To address this issue, we propose a method for grading unbalanced scrap steel samples based on a tree-structured hierarchical progressive network. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In view of the shortcomings of the existing technology, the present invention provides a method for grading unbalanced scrap steel samples based on a tree-type hierarchical network, which solves the problems mentioned in the above background technology.

[0006] (2) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for grading unbalanced scrap steel samples based on a tree-type hierarchical network, the method comprising the following steps:

[0008] S1: Construct a correspondence table between scrap steel types and discrimination levels, and build a conventional scrap steel classification model based on deep learning.

[0009] S2: Iteratively train the scrap steel classification model and build and update the corresponding scrap steel category tree accordingly.

[0010] S3: Build and train a deep learning encoder based on a dynamic memory module to extract and enhance the differentiated features of scrap steel samples.

[0011] S4: Build and train a deep learning decoder based on a tree-structured hierarchical network to enhance the overall model's unbiased prediction capability for scrap steel samples, thereby improving the model's overall precision and accuracy in scrap steel grading.

[0012] S5: Obtain a trained deep learning model based on dynamic memory modules and tree-structured hierarchical networks, and use it in actual scrap steel grading operations.

[0013] Preferably, the S1 specifically includes the following steps:

[0014] S11: Define the set C of all scrap steel categories that may be included in this scenario all ;

[0015] C all ={c1, c2, ..., c m}

[0016] S12: Define all scrap steel grading category sets Z according to actual production needs;

[0017] Za={z1,z2,...,z A}

[0018] S13: Build the corresponding scrap steel classification dataset;

[0019] S14: Build a conventional scrap steel classification model Model_N based on deep learning.

[0020] Preferably, said S2 comprises the following steps:

[0021] S21: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized (1) .

[0022] S22: Use all training set image samples to train scrap steel classification model Model_N (1) , using Model_N (1) Predict all test set images and get a single category c m The detailed prediction result vector of

[0023]

[0024] in represent The test sample is predicted to be probability.

[0025] S23: For each category c m Q (1) The prediction result vector of

[0026] Preferably, the step S2 further comprises the following steps:

[0027] S24: Label the scrap steel targets of each training set i , according to the parent class mapping function of the first round, adjust it to the annotation

[0028] S25: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized. (2) .

[0029] Preferably, the step S3 specifically includes the following steps:

[0030] S31: Constructing a visual feature extraction model Encoder based on the target detection module Faster R_CNN raw .

[0031] S32: Constructing an auxiliary decoder based on a multi-layer perceptron raw .

[0032] S33: Construct a dynamic memory feature library Memory containing all M types of scrap steel categories;

[0033]

[0034] And correspondingly maintain an M-dimensional memory coefficient matrix COFF;

[0035] COFF=[coff1,coff2,…,coff M ].

[0036] Preferably, the step S3 further comprises the following steps:

[0037] S34: During the training of the model, the original visual features corresponding to each candidate target in the input image I are

[0038] S35: Based on original visual features The corresponding category label l n , get the prediction result vector No. 1 n Dimension value Based on this, the dynamic memory feature library Memory and the corresponding memory coefficient matrix COFF are updated. n term, as shown in the following formula:

[0039]

[0040] S36: For each candidate target in Figure I, combined with the original visual features and the updated corresponding dynamic memory features

[0041] Preferably, the S4 specifically includes the following steps:

[0042] S41: Constructing a parent class decoder based on a multi-layer perceptron f , and according to the scrap steel classification tree CTree obtained in step S25 (K) All M F parent class nodes, and construct the same number of sub-class decoders based on multi-layer perceptrons.

[0043] S42: For each candidate target in Figure 1, first obtain its corresponding category label, and then use the parent class mapping function in step S25 to Get category l n The only corresponding parent class annotation Assume that the parent class label is located in the scrap steel classification tree CTree (K) Middle nodes.

[0044] S43: Based on the enhanced visual feature v in step S36 n , pass it through the parent class decoder Decoder f , get the category prediction of its parent node Then the cross entropy loss function is used for parameter optimization, and its formula is as follows:

[0045]

[0046] S44: Based on the enhanced visual feature v in step S36 n , pass it through the Subclass decoder Get its subclass node category prediction Then the cross entropy loss function is used for parameter optimization, and its formula is as follows:

[0047]

[0048] S45: The calculation formula of the overall loss function is as follows:

[0049]

[0050] Preferably, S5 specifically includes the following steps:

[0051] S51: Fully train the model to obtain a robust visual feature extraction model Encoder raw .

[0052] S52: Given a test image I * , using the visual feature extraction model Encoder raw Get its corresponding feature v * , and accordingly pass the parent class decoder Decoder f , get its parent class prediction

[0053] Then the feature v * Input to (P f ) * The corresponding m * Subclass decoder , obtain the final scrap steel category forecast

[0054] S53: Predict p based on the scrap steel category obtained * , combined with the unique mapping function f from scrap steel category to grading category in step S12 c→z , obtain and output the corresponding scrap steel grading category Z * =f c→z (P * ).

[0055] (3) Beneficial effects

[0056] The present invention provides a method for grading unbalanced scrap steel samples based on a tree-type hierarchical network. It has the following beneficial effects:

[0057] (1) This unbalanced scrap steel sample grading method based on a tree-structured hierarchical network can solve the serious long-tail effect existing in the existing scrap steel grading data set, thereby avoiding seriously biased prediction results and improving the unbiasedness of the overall prediction of the model.

[0058] (2) The unbalanced scrap steel sample grading method based on the tree-structured hierarchical network enhances the feature representation of the current target by integrating various representative features stored in the dynamic memory module, thereby further improving the prediction accuracy of the overall model.

[0059] (3) This method for grading unbalanced scrap steel samples based on a tree-type hierarchical network uses a tree-type classification network to display and mine significant differences and subtle differences between different categories, thereby improving the overall generalization ability of the model for tail predicates. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a structural schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] See also Figure 1 The present invention provides a technical solution: a method for grading unbalanced scrap steel samples based on a tree-type hierarchical network, comprising the following steps:

[0063] S1: Construct a correspondence table between scrap steel types and discrimination levels, and build a conventional scrap steel classification model based on deep learning.

[0064] S2: Iteratively train the scrap steel classification model and build and update the corresponding scrap steel category tree accordingly.

[0065] S3: Build and train a deep learning encoder based on a dynamic memory module to extract and enhance the differentiated features of scrap steel samples.

[0066] S4: Build and train a deep learning decoder based on a tree-structured hierarchical network to enhance the overall model's unbiased prediction capability for scrap steel samples, thereby improving the model's overall precision and accuracy in scrap steel grading.

[0067] S5: Obtain a trained deep learning model based on dynamic memory modules and tree-structured hierarchical networks, and use it in actual scrap steel grading operations.

[0068] In this embodiment, S1 specifically includes the following steps:

[0069] S11: Define the set C of all scrap steel categories that may be included in this scenario all ;

[0070] C all ={c1, c2, ..., c m}

[0071] where c mRepresents the mth category, and M represents the total number of all candidate scrap steel categories.

[0072] S12: Based on actual production needs, define a set Z of all scrap steel grading categories.

[0073] Za={z1,z2,...,z A}

[0074] where z a Indicates the a-th category grading level, A indicates the total number of all grading categories. And according to the set of all scrap steel categories C all , define scrap steel category c m To classification category z a The only mapping function f C →z.

[0075] S13: Build the corresponding scrap steel classification dataset.

[0076] S14: Build a conventional scrap steel classification model Model_N based on deep learning. This model can detect all scrap steel targets in the input image l and give its category prediction set.

[0077]

[0078] in Represents the predicted category of the nth scrap steel target, and satisfies N represents the total number of scrap steel targets detected in the figure.

[0079] In this embodiment, step S2 specifically includes the following steps:

[0080] S21: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized (1) , the classification space of the model is the set C of all scrap steel categories all .

[0081] S22: Use all training set image samples to train scrap steel classification model Model_N (1) After it converges, use Model_N (1) Predict all test set images and get a single category c m The detailed prediction result vector of

[0082]

[0083] in represent The test sample is predicted to be The probability, obviously

[0084] ∑Qm (1) =1.

[0085] S23: For each category c m The detailed prediction result vector Qm (1) , get its maximum value

[0086]

[0087] Corresponding category And the category As the parent node, c m As a child node, forming an edge of the tree Traverse the prediction result vectors of all categories to obtain the first round of scrap steel classification tree CTree (1) , and establish the first round of parent class mapping function It can target each type of input Uniquely query the corresponding parent category

[0088] S24: Label the scrap steel targets of each training set i , according to the parent class mapping function of the first round, adjust it to the annotation

[0089] S25: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized. (2) The classification space of the model is the set of all parent nodes of the scrap steel classification tree obtained in S23. Then the process from S22 to S24 is repeated until the scrap steel classification tree CTree obtained for the Kth time is obtained. (K) , and the scrap steel classification tree CTree obtained in the K-1th time (K-1) When the parent nodes are exactly the same (i.e. the parent node set is the same, the child node set is the same, and the edge set from the parent node to the child node is the same), exit the loop process. Save the parent class mapping function at this time

[0090] In this embodiment, S3 specifically includes the following steps:

[0091] S31: Constructing a visual feature extraction model Encoder based on the target detection module Faster R_CNN raw , the model can give all the scrap steel target candidate frames according to the input image I, and extract the original visual features corresponding to the nth scrap steel candidate target

[0092] S32: Constructing an auxiliary decoder based on a multi-layer perceptron raw , the decoder can input a single candidate target raw visual feature Give its category prediction And generate the corresponding M-dimensional prediction result vector Obviously

[0093] S33: Construct a dynamic memory feature library Memory containing all M types of scrap steel categories;

[0094]

[0095] And correspondingly maintain an M-dimensional memory coefficient matrix COFF;

[0096] COFF=[coff1,coff2,…,coff M ]

[0097] At the beginning of training, all items in the dynamic memory feature library Memory and the corresponding memory coefficient matrix COFF are set to 0.

[0098] S34: During the training of the model, the original visual features corresponding to each candidate target in the input image I are Input it to the auxiliary decoder Decoder raw Get the corresponding prediction result vector

[0099] S35: Based on original visual features The corresponding category label l n , get the prediction result vector No. 1 n Dimension value Based on this, the dynamic memory feature library Memory and the corresponding memory coefficient matrix COFF are updated. n term, as shown in the following formula:

[0100]

[0101] S36: For each candidate target in Figure I, combined with the original visual features and the updated corresponding dynamic memory features Through the attention mechanism, the corresponding enhanced visual feature v is obtained n , as shown in the following formula:

[0102]

[0103] Here, tanh(·) represents the hyperbolic tangent function, MLP(·) represents the multilayer perceptron, and represents the Hadamard product.

[0104] In this embodiment, step S4 specifically includes the following steps:

[0105] S41: Constructing a parent class decoder based on a multi-layer perceptron f , and according to the scrap steel classification tree CTree obtained in step S25 (K) All M F parent class nodes, construct the same number of sub-class decoders based on multi-layer perceptrons

[0106] S42: For each candidate target in Figure 1, first obtain c Take the corresponding category label l n , then according to the parent class mapping function in step S25 Get category l n The only corresponding parent class annotation Assume that the parent class label is located in the scrap steel classification tree CTree (K) Middle nodes.

[0107] S43: Based on the enhanced visual feature v in step S36 n , pass it through the parent class decoder Decoder f , get the category prediction of its parent node Then the cross entropy loss function is used for parameter optimization, and its formula is as follows:

[0108]

[0109] S44: Based on the enhanced visual feature v in step S36 n , pass it through the Subclass decoder Get its subclass node category prediction Then the cross entropy loss function is used for parameter optimization, and its formula is as follows:

[0110]

[0111] S45: Therefore, when training the overall model, the calculation formula of its overall loss function is as follows:

[0112]

[0113] Where B represents the total number of candidate targets to be trained.

[0114] In this embodiment, S5 specifically includes the following steps:

[0115] S51: Fully train the model to obtain a robust visual feature extraction model Encoder raw , parent class decoder Decoder f and a series of subclass decoders

[0116] S52: Given a test image I * , using the visual feature extraction model Encoder raw Get its corresponding feature v * , and accordingly pass the parent class decoder Decoder f , get its parent class prediction

[0117] Then the feature v * Input to (P f ) * The corresponding m * Subclass decoder , obtain the final scrap steel category forecast

[0118] S53: Predict p based on the scrap steel category obtained * , combined with the unique mapping function f from scrap steel category to grading category in step S12 c→z , obtain and output the corresponding scrap steel grading category Z * =f c→z (P * ).

[0119] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for grading unbalanced scrap steel samples based on a tree-type hierarchical network, characterized by: The method comprises the following steps: S1: Construct a correspondence table between scrap steel types and discrimination grades, and build a conventional scrap steel classification model based on deep learning; S2: Iteratively train the scrap steel classification model and build and update the corresponding scrap steel category tree accordingly; S3: Build and train a deep learning encoder based on a dynamic memory module to extract and enhance the differentiated features of scrap steel samples; S31: Constructing a visual feature extraction model Encoder based on the target detection module Faster R_CNN raw ; S32: Constructing an auxiliary decoder based on a multi-layer perceptron raw ; S33: Construct a dynamic memory feature library Memory containing all M types of scrap steel categories; And correspondingly maintain an M-dimensional memory coefficient matrix COFF; COFF=[coff1,coff2,…,coff M ]; represents the dynamic memory feature vector of the mth type of scrap steel; coff M It represents the memory coefficient of the mth type of scrap steel; S34: During the training of the model, the original visual features corresponding to each candidate target in the input image I are recorded as Input it to the auxiliary decoder Decoder raw Get the corresponding prediction result vector S35: Based on original visual features The corresponding category label l n , get the prediction result vector No. 1 n Dimension value Based on this, the dynamic memory feature library Memory and the corresponding memory coefficient matrix COFF are updated. n term, as shown in the following formula: S36: For each candidate target in Figure I, combined with the original visual features and the updated corresponding dynamic memory features S4: Build and train a deep learning decoder based on a tree-structured hierarchical network to enhance the overall model's unbiased prediction capabilities for scrap steel samples, thereby improving the model's overall precision and accuracy in scrap steel grading. S5: Obtain a trained deep learning model based on dynamic memory modules and tree-structured hierarchical networks, and use it in actual scrap steel grading operations.

2. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical network according to claim 1, characterized in that: The S1 specifically includes the following steps: S11: Define the set C of all scrap steel categories that may be included in this scenario all ; C all ={c1,c2,...,c m } S12: Define all scrap steel grading category sets Z according to actual production needs; Za={z1,z2,...,z A } S13: Build the corresponding scrap steel classification dataset; S14: Build a conventional scrap steel classification model Model_N based on deep learning; where z a Indicates the a-th category grading level, A indicates the total number of all grading categories, and according to the set of all scrap steel categories C all , define scrap steel category c m To classification category z a The only mapping function f C →z.

3. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical network according to claim 1, characterized in that: The S2 comprises the following steps: S21: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized (1) ; S22: Use all training set image samples to train scrap steel classification model Model_N (1) , using Model_N (1) Predict all test set images and get a single category c m The detailed prediction result vector of in represent The test sample is predicted to be probability; S23: Get its maximum value 4. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical network according to claim 3, characterized in that: The S2 further comprises the following steps: S24: Label the scrap steel targets of each training set i , according to the parent class mapping function of the first round, the scrap steel target label l of each training set is obtained i The parent class mapping function S25: Copy the conventional scrap steel classification model Model_N to obtain a new model Model_N whose parameters have not been trained and optimized. (2) .

5. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical network according to claim 1, characterized in that: The S4 specifically includes the following steps: S41: Constructing a parent class decoder based on a multi-layer perceptron f , and according to the scrap steel classification tree CTree obtained in step S25 (K) All M F parent class nodes, construct the same number of sub-class decoders based on multi-layer perceptrons; S42: For each candidate target in Figure 1, first obtain its corresponding category label, and then use the parent class mapping function in step S25 to Get category l n The only corresponding parent class annotation Assume that the parent class label is located in the scrap steel classification tree CTree (K) Middle nodes; S43: Based on the enhanced visual feature v in step S36 n , pass it through the parent class decoder Decoder f , get the category prediction of its parent node Then the cross entropy loss function is used for parameter optimization, and its formula is as follows: S44: Based on the enhanced visual feature v in step S36 n , pass it through the Subclass decoder Get its subclass node category prediction Then the cross entropy loss function is used for parameter optimization, and its formula is as follows: S45: The calculation formula of the overall loss function is as follows:

6. The method for grading unbalanced scrap steel samples based on a tree-type hierarchical network according to claim 4, characterized in that: S5 specifically includes the following steps: S51: Fully train the model to obtain a robust visual feature extraction model Encoder raw ; S52: Given a test image I * , using the visual feature extraction model Encoder raw Get its corresponding feature v * , and accordingly pass the parent class decoder Decoder f , get its parent class prediction Then the feature v * Input to (P f ) * The corresponding m * Subclass decoder , obtain the final scrap steel category forecast S53: Predict p based on the scrap steel category obtained * , combined with the unique mapping function f from scrap steel category to grading category in step S12 c→z , obtain and output the corresponding scrap steel grading category Z * =f c→z (P * ).

Images