A method for classifying aluminum profiles

Abstract

The present invention discloses a method for classifying aluminum profiles, including: Step 1: Construct a sample library and a feature library for the cross-sections of aluminum profiles; Step 2: Based on the QATM algorithm, extract the positions where the cross-sections of aluminum profiles appear from the input image to generate candidate boxes, use a pre-trained model to extract features from the input image, and obtain feature maps of different layers through each convolutional network of the pre-trained model; Step 3: Scale the feature maps obtained in Step 2 to a specified size, and then take the mean of all the feature maps to obtain a mixed template feature map. Compared with the prior art, this method for classifying aluminum profiles does not need to be frequently retrained as the types of targets change, can flexibly and efficiently adapt to the rapidly changing requirements in aluminum profile factories, and at the same time meet the requirements of industrial automated production.

Description

Technical Field

[0001] The present invention relates to the field of profile classification, and in particular to a method for classifying aluminum profiles. Background Art

[0002] With the continuous improvement of the demand for automation and intelligence in the industrial field, the application of computer vision technology in factories has become increasingly widespread. In actual application scenarios, if the background environment is relatively simple and the tasks are relatively straightforward, traditional image processing methods can usually be used to complete the tasks. However, when the background is complex and the task requirements are diverse, deep learning technology is required to provide higher robustness and accuracy.

[0003] Traditional deep learning algorithms usually need to go through a training process to achieve the application effect. If the characteristics such as the types and shapes of the targets change frequently in actual applications, the algorithms also need to be trained frequently accordingly to adapt to the new requirements.

[0004] Taking an aluminum profile factory as an example, in its production process, post-treatment operations such as spraying need to be carried out on the formed aluminum profiles. In order to achieve automated production, the factory needs to first identify the types of aluminum profiles and call the corresponding spraying schemes according to the identification results. However, the types of aluminum profiles are updated extremely fast, and different types of aluminum profiles need to be processed almost every day. If traditional deep learning classification algorithms are used, frequent model training and updating for new types of aluminum profiles are required, which obviously cannot meet the requirements of production efficiency. Summary of the Invention

[0005] Aiming at the problems existing in the above-mentioned prior art, the present invention provides a method for classifying aluminum profiles, which does not need to be frequently retrained as the types of targets change, so as to more efficiently and flexibly adapt to the rapidly changing requirements in aluminum profile factories, and at the same time meet the requirements of industrial automated production.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A method for classifying aluminum profiles, comprising:

[0008] Step 1: Construct a sample library and a feature library of aluminum profile cross-sections;

[0009] Step 2: Based on the QATM algorithm, extract the positions where the aluminum profile cross-sections appear from the input image to generate candidate boxes, use a pre-trained model to extract features from the input image, and pass through each convolutional network layer of the pre-trained model to obtain feature maps of different layers ;

[0010] Step 3: Scale the feature maps obtained in Step 2 to a specified size, and then for all the feature maps Take the average value to obtain the mixed template feature map ;

[0011] Step 4: Normalize the feature map and the template feature map respectively to obtain the normalized input feature map and the template feature map . Set the shape of the input feature map to [x, c, a, b], and set the shape of the template feature map to [x, c, d, e] to obtain the similarity distance [x, a, b, d, e], where x is the batch dimension, c is the number of channels, a and b are the height and width of the input feature map , and d and e are the height and width of the template feature map ;

[0012] The similarity distance [x, a, b, d, e] represents the similarity between each pixel point of the input feature map and the template feature map ;

[0013] Step 5: Calculate the normalized confidence of the similarity distance D in Step 4, and then obtain the highest confidence of each point in the feature map relative to the template feature map by taking the maximum value of the confidence by row;

[0014] Step 6: Generate the candidate box position confidence map;

[0015] Step 7: Perform non-maximum suppression on the candidate boxes;

[0016] Step 8: Input the candidate boxes into the image retrieval algorithm and match them with the constructed feature library to determine the category of the aluminum profile.

[0017] Preferably, Step 1 includes:

[0018] Step (1): Collect cross-sectional images of all types of the aluminum profiles and store them to construct the sample library;

[0019] Step (2): According to Step (1), extract the features of all the cross-sectional images of the aluminum profiles in the sample library through the image retrieval algorithm, denoted as ;

[0020] Step (3): Cluster the features belonging to the same aluminum profile category according to the X, so as to obtain K cluster centers for each aluminum profile category;

[0021] Step (4): After the clustering is completed, the K cluster centers will represent the features of the aluminum profile category, and store the features of the aluminum profile category corresponding to the cluster centers to construct a feature library for representing each aluminum profile category;

[0022] The image retrieval algorithm is the CLIP-ReID model, the clustering is the K-Means clustering algorithm, and n is the number of samples in the sample library.

[0023] Preferably, the input image is input into the pre-trained model, and through the convolutional networks of each layer of the pre-trained model, the feature maps of different layers are obtained , and the feature maps between different layers are scaled to the same size, and the feature maps of different scales are concatenated along the channel dimension to form a multi-scale feature map :

[0024]

[0025] In the step 4, the similarity distance [x, a, b, d, e] is calculated by the formula:

[0026]

[0027] In the step 5, the similarity distance is reshaped to obtain , let subtract the maximum value of the row and column respectively to obtain and , and the formula for calculating the confidence is:

[0028]

[0029] The confidence represents the confidence of the feature map at position i and the template feature map at position j;

[0030] By taking the maximum value of each row of the confidence matrix , the highest confidence of each point in the feature map relative to the template feature map is obtained:

[0031]

[0032] The is the feature map of the first layer, and the α is a hyperparameter, which is the temperature parameter of softmax;

[0033] The pre-trained model is VGG19, and the specified sizes of the template feature map include (128, 128), (128, 256) and (256, 128).

[0034] Preferably, in the step 6, a sliding window with a specified size slides on the to calculate the mean value of the confidence within the sliding window, and the confidence that the center point of the candidate box of this size falls on each pixel position is obtained;

[0035] In the step 7, when performing non-maximum suppression on the candidate boxes of the same size, sort them according to the confidence of each candidate box, select the candidate box with the highest confidence as the benchmark, and calculate the intersection over union (IoU) of the remaining candidate boxes and the benchmark candidate box. For two candidate boxes A and B, the formula for calculating their IoU is as follows:

[0036]

[0037] The candidate box A is the remaining candidate box, and the candidate box B is the benchmark candidate box;

[0038] The IoU includes the intersection and the union. The intersection refers to the overlapping area of the remaining candidate box and the benchmark candidate box, and the union refers to the total area of the remaining candidate box and the benchmark candidate box.

[0039] Preferably, if the IoU of the remaining candidate box and the benchmark candidate box is greater than the threshold, the remaining candidate box is deleted from the candidate box set;

[0040] If the IoU of the remaining candidate box and the benchmark candidate box is less than the threshold, the remaining candidate box with an IoU less than the threshold is retained until all the remaining candidate boxes are processed;

[0041] The threshold is 0.01 - 0.3.

[0042] Preferably, the candidate boxes are input into the image retrieval algorithm one by one to extract the corresponding features, and the extracted features are matched with each feature in the pre-constructed feature library. The similarity degree is measured by calculating the distance between the features, and the calculation result is converted into the confidence;

[0043] Perform non-maximum suppression on the candidate bounding boxes based on the confidence levels to eliminate redundant overlapping regions, and further remove the candidate bounding boxes with confidence levels lower than the threshold that are determined to be the background. Finally, determine the position where the aluminum profile cross-section is located.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This classification method does not need to be frequently retrained as the types of targets change, and can more efficiently and flexibly adapt to the rapidly changing requirements in aluminum profile factories, while meeting the requirements of industrial automated production. Brief Description of the Drawings

[0046] Figure 1 is a schematic flowchart of a method for classifying aluminum profiles according to the present application.

[0047] Figure 2 is a response diagram of a method for classifying aluminum profiles according to the present application under sliding windows of different sizes. Detailed Description of the Preferred Embodiments

[0048] The following will be further described in detail in conjunction with the attached Figure 1-2 drawings to make the technical solutions of the present invention easier to understand and master.

[0049] In this embodiment, it should be understood that the orientation or positional relationships indicated by terms such as "middle", "upper", "lower", "top", "right side", "end", "front", "back", "middle", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0050] In addition, in this specific embodiment, if not otherwise specified, the connection or fixing methods between components can be bolt fixing, nail pin fixing, or pin shaft connection, etc., which are commonly used in the prior art. Therefore, they will not be elaborated in this embodiment.

[0051] The following will be further described in detail in conjunction with the attached Figure 1-2 drawings to further illustrate the present application.

[0052] As Figure 1-2 shown, a method for classifying aluminum profiles includes:

[0053] Step 1: Construct a sample library and a feature library for aluminum profile cross-sections. Specifically:

[0054] Step (1): Collect images of aluminum profile cross-sections of all types and store them to construct a sample library;

[0055] Step (2): According to step (1), extract the features of all aluminum profile cross-section images in the sample library through an image retrieval algorithm, denoted as , where n is the number of samples in the sample library;

[0056] Step (3): Cluster the features belonging to the same aluminum profile category according to X respectively, so as to obtain K cluster centers for each aluminum profile category ;

[0057] Step (4): After clustering, the K cluster centers will represent the features of the aluminum profile category. Store the features of the aluminum profile category corresponding to the cluster centers to construct a feature library for representing each aluminum profile category.

[0058] In this embodiment, the image retrieval algorithm is the CLIP-ReID model, and the clustering is the K-Means clustering algorithm.

[0059] Step 2: Based on the QATM (Quality-Aware Template Matching) algorithm, extract the positions where the aluminum profile cross-section appears from the input image to generate candidate boxes. Specifically, use a pre-trained model to extract features from the input image I First, input the input image I into the pre-trained model, and obtain feature maps of different layers through the convolutional networks of each layer of the pre-trained model , that is, obtain feature maps of different layers , where represents the feature map of the I-th layer;

[0060] Since the feature maps between different layers are often different, it is necessary to first scale the feature maps between different layers to the same size, and splice the feature maps of different scales along the channel dimension to form a multi-scale feature map :

[0061]

[0062] The splicing operation is performed along the channel dimension, that is, the feature maps of each layer are merged by channels to form a more abundant multi-scale feature map , and this multi-scale feature map contains spatial and semantic information of different levels.

[0063] Step 3: Extract features from all template images in the same way as in step 2, scale the feature maps obtained in step 2 to a specified size (different sizes and aspect ratios can better respond to target objects of different sizes), and then for all feature maps Take the mean value to obtain the mixed template feature map . In particular, if the template feature map remains unchanged, the template feature map can be saved and does not need to be regenerated repeatedly each time.

[0064] Step 4: Normalize the feature map and the template feature map respectively to obtain the normalized input feature map and the template feature map . Set the shape of the input feature map to [x, c, a, b], and set the shape of the template feature map to [x, c, d, e], where x is the batch dimension, c is the number of channels, a and b are the spatial dimensions (height and width) of the input feature map . Set the shape of the template feature map to [x, c, d, e]. x and c have the same meanings as above, that is, x is the batch dimension, c is the number of channels, and d and e are the spatial dimensions (height and width) of the template feature map . Then the calculation formula for the similarity distance [x, a, b, d, e] is as follows:

[0065]

[0066] The similarity distance [x, a, b, d, e] represents the similarity between each pixel point of the input feature map and the template feature map ;

[0067] Step 5: Calculate the normalized confidence of the similarity distance D in Step 4. Reshape the similarity distance to obtain . Let subtract the maximum value of the row and column respectively to obtain and . The formula for calculating the confidence is as follows:

[0068]

[0069] Where α is a hyperparameter, which is the temperature parameter of softmax. When α approaches 0, the output probability will tend to be uniform, such as 60% and 40%; if α approaches infinity, the output probability will strengthen the maximum response and suppress other responses, and the output may be 99% and 1%. Therefore, in this embodiment, α = 25; confidence represents the feature map at position i and the template feature map at position j.

[0070] Subsequently, by taking the maximum value of each row of the confidence matrix the feature map is obtained, and the highest confidence of each point in the feature map relative to the template feature map

[0071]

[0072] Step 6: Generate the candidate box position confidence map; interpolate the result obtained in Step 5 to the size of the input image I Then, use a sliding window of a specified size to slide on and calculate the mean value of the confidence within the sliding window to obtain the confidence that the center point of the candidate box of this size falls on each pixel position. In particular, in this step, sliding windows of multiple sizes will be specified to generate multiple candidate box position confidence maps to adapt to aluminum profiles of different sizes. The sizes of the sliding windows include (128, 256), (256, 128), and (256, 256).

[0073] Step 7: Perform non-maximum suppression on the candidate boxes; when performing non-maximum suppression (Non-Maximum Suppression, abbreviated as NMS) on candidate boxes of the same size, sort according to the confidence of each candidate box, select the candidate box with the highest confidence as the reference, and calculate the intersection over union (Intersection over Union, abbreviated as IoU) of the remaining candidate boxes and the reference candidate box. For two candidate boxes A and B (candidate box A is the remaining candidate box, and candidate box B is the reference candidate box), the calculation formula for its intersection over union is as follows:

[0074]

[0075] The intersection over union includes the intersection and the union. The intersection refers to the overlapping area of the two candidate boxes, and the union refers to the total area of the two candidate boxes.

[0076] If the intersection over union (IoU) between the remaining candidate bounding boxes and the reference candidate bounding box is greater than the threshold, the remaining candidate bounding boxes are deleted from the set of candidate bounding boxes; if the IoU between the remaining candidate bounding boxes and the reference candidate bounding box is less than the threshold, the remaining candidate bounding boxes with an IoU less than the threshold are retained until all the remaining candidate bounding boxes are processed.

[0077] For candidate bounding boxes of different sizes, the IoU comparison is not performed. Therefore, the above NMS operation is executed separately for candidate bounding boxes of each size without interacting with candidate bounding boxes of other sizes. Finally, the set of candidate bounding boxes after non-maximum suppression is output, where each box has a relatively high confidence value and a small overlapping area with other candidate bounding boxes, ensuring the accuracy of the localization result. At the same time, since there is no interaction between candidate bounding boxes of different sizes, multiple candidate bounding boxes of the same object are generated, improving the output quality of the candidate bounding boxes.

[0078] In this embodiment, the pre-trained model is VGG19, and the output feature maps of the second and eighth convolutional layers are selected for concatenation in the channel dimension to generate a multi-scale feature map ; the specified sizes of the template feature map include (128, 128), (128, 256), and (256, 128); the threshold is 0.01 - 0.3, and the specific threshold can be selected as 0.05. In other embodiments, it can be flexibly adjusted according to actual needs.

[0079] Step 8: Input the candidate bounding boxes into the image retrieval algorithm and match them with the constructed feature library to determine the category of the aluminum profile. Specifically:

[0080] Input the candidate bounding boxes into the image retrieval algorithm one by one to extract the corresponding features, and match the extracted features with each feature in the pre-constructed feature library. Calculate the distance between the features to measure the similarity degree, and convert the calculation result into a confidence level;

[0081] Perform non-maximum suppression on the candidate bounding boxes based on the confidence level to eliminate redundant overlapping regions, and further remove the candidate bounding boxes with a confidence value lower than the threshold determined as the background. Finally, determine the position where the aluminum profile cross-section is located.

[0082] In this embodiment, the image retrieval algorithm is the same as in Step 1, and the CLIP-ReID model trained on some known aluminum profile categories is adopted.

[0083] As Figure 2 shown, the QATM algorithm of the present invention can provide higher-quality candidate bounding boxes at more scales and more ratios for the subsequent image retrieval algorithm to determine the category compared with the original algorithm.

[0084] The present invention makes full use of the existing feature library and image retrieval mechanism, and can efficiently complete the classification task of new categories, significantly improving its adaptability and flexibility in scenarios where categories are frequently updated. Especially in the industrial production environment where the types of aluminum profiles change frequently, this method can quickly adapt to new classification requirements without frequent training of the model or readjustment of parameters, reducing the consumption of computing resources and significantly improving the efficiency and practicality of the industrial automation process.

[0085] Through the above classification method, users do not need to retrain frequently as the target types change, and can more efficiently and flexibly adapt to the rapidly changing requirements in the aluminum profile factory, while meeting the requirements of industrial automation production.

[0086] The technical effects of the present invention are mainly reflected in the following aspects:

[0087] This classification method does not need to retrain frequently as the target types change, and can more efficiently and flexibly adapt to the rapidly changing requirements in the aluminum profile factory, while meeting the requirements of industrial automation production.

[0088] Of course, the above are only typical examples of the present invention. In addition, the present invention can also have many other specific implementation manners. Any technical solutions formed by equivalent replacement or equivalent transformation fall within the scope of protection required by the present invention.

Claims

1. A method for classifying aluminum profiles, characterized in that, Including: Step 1: Construct a sample library and a feature library for the aluminum profile cross-section; Step 2: Based on the QATM algorithm, extract the positions where the aluminum profile cross-sections appear from the input image to generate candidate boxes, and use a pre-trained model to perform feature extraction on the input image. Through the convolutional networks of each layer of the pre-trained model, feature maps of different layers are obtained ; Step 3: Scale the feature map obtained in the above Step 2 to a specified size, and then calculate the mean value of all the feature maps to obtain a mixed template feature map ; Step 4: Normalize the feature map and the template feature map respectively to obtain the normalized input feature map and the template feature map . Set the shape of the input feature map to [x, c, a, b], and set the shape of the template feature map to [x, c, d, e], to obtain the similarity distance [x, a, b, d, e], where x is the batch dimension, c is the number of channels, a and b are the height and width of the input feature map , and d and e are the height and width of the template feature map ; The similarity distance [x, a, b, d, e] represents the input feature map and the template feature map The similarity between each pixel point; Step 5: Calculate the normalized confidence of the similarity distance D in Step 4, and then obtain the feature map by taking the maximum value of the confidence by row for each point in the template feature map Step 6: Generate the confidence map for the candidate box positions; Step 7: Perform non-maximum suppression on the candidate boxes; Step 8: Input the candidate boxes into the image retrieval algorithm and match them with the constructed feature library to determine the category of the aluminum profile.

2. The aluminum profile classification method according to claim 1, characterized in that The said Step 1 includes: Step (1): Collect the cross-section images of all types of the aluminum profiles and store them to construct the sample library; Step (2): According to the said step (1), extract the features of all the aluminum profile cross-section images in the sample library through an image retrieval algorithm, expressed as ; Step (3): Cluster the features belonging to the same aluminum profile category according to the said X respectively, so as to obtain K clustering centers for each aluminum profile category; Step (4): After the clustering is completed, the K clustering centers will represent the features of the aluminum profile category. Store the features of the aluminum profile category corresponding to the clustering centers to construct the feature library for representing each aluminum profile category; The said image retrieval algorithm is the CLIP-ReID model, the said clustering is the K-Means clustering algorithm, and n is the number of samples in the sample library.

3. The aluminum profile classification method according to claim 1, wherein, Input the input image into the pre-trained model, and through the convolutional networks of each layer of the pre-trained model, obtain the feature maps of different layers , and the feature maps between different layers are scaled to the same size, and the feature maps of different scales are concatenated along the channel dimension to form a multi-scale feature map : ， In the said step 4, the similarity distance [x, a, b, d, e] is calculated by the following formula: ， In the step 5, the similarity distance is reshaped to obtain , and let subtract the maximum values of the rows and columns respectively to obtain and , and the formula for calculating the confidence is: ， The confidence level indicates the confidence level of the feature map at position i and the template feature map at position j; By taking the maximum value of each row of the confidence matrix the feature map is obtained, and the highest confidence of each point in the feature map relative to the template feature map is obtained: ， The is the feature map of the first layer, and α is a hyperparameter, which is the temperature parameter of softmax; The pre-trained model is VGG19, and the specified sizes of the template feature maps include (128, 128), (128, 256), and (256, 128).

4. The aluminum profile classification method according to claim 3, wherein In step 6, a sliding window of a specified size is used to slide on the , and the mean value of the confidence within the sliding window is calculated to obtain the confidence of the center point of the candidate box of this size falling on each pixel position. In the said Step 7, when performing non-maximum suppression on the candidate boxes of the same size, sort them according to the confidence of each candidate box, select the candidate box with the highest confidence as the benchmark, calculate the intersection over union (IoU) of the remaining candidate boxes and the benchmark candidate box. For two candidate boxes A and B, the formula for calculating their intersection over union is as follows: ， The said candidate box A is the remaining candidate box, and the said candidate box B is the benchmark candidate box; The said intersection over union includes the intersection and the union. The intersection refers to the overlapping area of the remaining candidate box and the benchmark candidate box, while the union refers to the total area of the remaining candidate box and the benchmark candidate box.

5. The aluminum profile classification method according to claim 4, wherein If the intersection over union of the remaining candidate box and the benchmark candidate box is greater than the threshold, delete the remaining candidate box from the candidate box set; If the intersection over union of the remaining candidate box and the benchmark candidate box is less than the threshold, retain the remaining candidate boxes with the intersection over union less than the threshold until all the remaining candidate boxes are processed; The said threshold is 0.01 - 0.

3.

6. The aluminum profile classification method according to claim 5, wherein Input the candidate boxes into the image retrieval algorithm one by one to extract the corresponding features, match the extracted features with each feature in the pre-constructed feature library, measure the similarity by calculating the distance between the features, and convert the calculation result into the confidence; Perform non-maximum suppression on the candidate boxes based on the confidence to eliminate redundant overlapping areas, and further remove the candidate boxes with the confidence value lower than the threshold determined as the background, and finally determine the position where the aluminum profile cross-section is located.

Images