Aluminum profile classification method

Abstract

The invention discloses an aluminum profile classification method. The method comprises the steps of 1, constructing a sample library and a feature library of aluminum profile sections; 2, based on a QATM algorithm, extracting the position where the section of the aluminum profile appears from the input image, and generating a candidate box; and 3, inputting the candidate box into an image retrieval algorithm, and matching the candidate box with the constructed feature library so as to determine the category of the aluminum profile. Compared with the prior art, the aluminum profile classification method has the advantages that frequent retraining along with the change of the target type is not needed, the rapid change requirement in an aluminum profile factory is flexibly and efficiently met, and meanwhile the requirement of industrial automatic production is met.

Description

Technical Field

[0001] The invention relates to the field of profile classification, and in particular to a method for classifying aluminum profiles. Background Art

[0002] As the demand for automation and intelligence in the industrial field continues to increase, the application of computer vision technology in factories has become increasingly widespread. In actual application scenarios, if the background environment is relatively simple and the task is relatively simple, traditional image processing methods can usually be used to complete the task. However, when the background is complex and the task requirements are diverse, deep learning technology is needed to provide higher robustness and accuracy.

[0003] Traditional deep learning algorithms usually need to go through a training process to achieve application results. If the type, shape and other characteristics of the target change frequently in actual applications, the algorithm also needs to be trained frequently to adapt to new needs.

[0004] Take the aluminum profile factory as an example. During the production process, the formed aluminum profiles need to be sprayed and other post-processing operations. In order to achieve automated production, the factory needs to first identify the type of aluminum profile and call the corresponding spraying scheme based on the identification result. However, the types of aluminum profiles are updated very quickly, and different types of aluminum profiles need to be processed almost every day. If the traditional deep learning classification algorithm is used, it is necessary to frequently train and update the model for new types of aluminum profiles, which is obviously difficult to meet the needs of production efficiency. Summary of the invention

[0005] In response to the problems existing in the above-mentioned prior art, the present invention provides an aluminum profile classification method, which does not need to be frequently retrained as the target type changes, so as to adapt to the rapidly changing needs in aluminum profile factories more efficiently and flexibly, while meeting the requirements of industrial automated production.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: A method for classifying aluminum profiles, comprising: Step 1: Build a sample library and feature library of aluminum profile sections; Step 2: Use the pre-trained model to perform feature extraction on the input image, extract the location where the aluminum profile cross section appears from the input image, and generate a candidate frame; Step 3: The feature map obtained in step 2 Scale to the specified size, and then perform Take the average value to get the mixed template feature map ; Step 4: For the feature map With the template feature map Normalize them separately to get the normalized input feature map With template feature map , for the input feature map The shape is set to [x, c, a, b], for the template feature map The shape of is set to [x, c, d, e] to get the similarity distance [x, a, b, d, e], where x is the batch dimension, c is the number of channels, and a and b are the input feature maps The height and width of the template are d and e. The height and width of Step 5: Calculate the normalized confidence of the similarity distance D in step 4, and then obtain the feature map by taking the maximum value of the confidence by row. Each point in the template feature map The highest confidence level; Step 6: Generate the candidate box position confidence map; Step 7: Perform non-maximum suppression on the candidate box; Step 8: Input the candidate frame into the image retrieval algorithm and match it with the constructed feature library to determine the category of the aluminum profile.

[0007] Preferably, the step 1 comprises: Step (1): collecting cross-sectional images of all types of aluminum profiles and storing them to construct the sample library; Step (2): According to step (1), the features of all the aluminum profile cross-section images in the sample library are extracted by an image retrieval algorithm, which is expressed as ; Step (3): clustering the features belonging to the same aluminum profile category according to X, thereby obtaining K cluster centers of each aluminum profile category; Step (4): After clustering is completed, the K cluster centers will represent the features of the aluminum profile category, and the features of the aluminum profile category corresponding to the cluster centers will be stored to construct the feature library for representing each aluminum profile category; The image retrieval algorithm is the CLIP-ReID model, the clustering is the K-Means clustering algorithm, and n is the number of samples in the sample library.

[0008] Preferably, the input image is input into the pre-trained model, and the feature maps of different layers are obtained through the convolutional networks of the pre-trained model. , the feature maps between different layers The feature maps of different scales are scaled to the same size Splicing along the channel dimension to form a multi-scale feature map :

[0009] In step 4, the similarity distance The calculation formula for [x, a, b, d, e] is:

[0010] The similarity distance [x, a, b, d, e] represents the input feature map With the template feature map The similarity between each pixel; In step 5, the similarity distance Reshape ,make Subtract the maximum value of the row and column respectively to get and , forming the confidence level calculated The formula is: , The confidence Represents the feature map At position i and the template feature map Confidence at position j; By the confidence matrix Take the maximum value by row and get the feature map Each point in the template feature map The highest confidence level is:

[0011] The α is a hyperparameter, which is the temperature parameter of softmax; The pre-trained model is VGG19, and the template feature map Specified sizes include (128,128), (128,256), and (256,128).

[0012] Preferably, in step 6, a sliding window of a specified size is used in the Slide upward, calculate the mean of the confidence within the sliding window, and obtain the confidence that the center point of the candidate box of this size falls on each pixel position; In step 7, when the non-maximum suppression is performed on the candidate boxes of the same size, the confidence of each candidate box is sorted, the candidate box with the highest confidence is selected as the benchmark, and the intersection-and-union ratio of the remaining candidate boxes and the benchmark candidate box is calculated. For two candidate boxes A and B, the calculation formula of the intersection-and-union ratio is as follows:

[0013] The candidate frame A is a remaining candidate frame, and the candidate frame B is a reference candidate frame; The intersection-over-union ratio includes an intersection and a union, the intersection refers to an overlapping area of ​​the remaining candidate box and the reference candidate box, and the union refers to a total area of ​​the remaining candidate box and the reference candidate box.

[0014] Preferably, if the intersection-over-union ratio of the remaining candidate box and the reference candidate box is greater than a threshold, the remaining candidate box is deleted from the candidate box set; The intersection-and-union ratio of the remaining candidate frames and the reference candidate frames is smaller than the threshold, and the remaining candidate frames whose intersection-and-union ratio is smaller than the threshold are retained until the remaining candidate frames are processed; The threshold value is 0.01-0.3.

[0015] Preferably, the candidate frames are input into the image retrieval algorithm one by one to extract corresponding features, and the extracted features are matched with each feature in the pre-built feature library, the distance between the features is calculated to measure the similarity, and the calculation result is converted into the confidence level; The non-maximum suppression is performed on the candidate frames based on the confidence level to eliminate redundant overlapping areas, and the candidate frames whose confidence levels are lower than the threshold and are judged as backgrounds are further removed, and finally the position of the aluminum profile cross section is determined.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This classification method does not need to be frequently retrained as the target types change, so it can adapt to the rapidly changing needs in aluminum profile factories more efficiently and flexibly, while meeting the requirements of industrial automated production. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of an aluminum profile classification method of the present application.

[0018] Figure 2 This is a response diagram of an aluminum profile classification method of the present application under sliding windows of different sizes. DETAILED DESCRIPTION

[0019] The following is combined with Figure 1-2, the specific implementation methods of the present invention are further described in detail to make the technical solution of the present invention easier to understand and grasp.

[0020] In this embodiment, it should be understood that the terms "middle", "upper", "lower", "top", "right side", "end", "front", "back", "middle", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0021] In addition, in this specific embodiment, if the connection or fixing method between components is not specifically described, the connection or fixing method can be through bolt fixing or pin fixing, or pin shaft connection and the like commonly used in the prior art, and therefore, it is not described in detail in this embodiment.

[0022] The following is combined with Figure 1-2 This application is described in further detail.

[0023] like Figure 1-2 As shown, a method for classifying aluminum profiles includes: Step 1: Build a sample library and feature library of aluminum profile sections. Specifically: Step (1): Collect all kinds of aluminum profile cross-section images and store them to build a sample library; Step (2): Based on step (1), the features of all aluminum profile cross-section images in the sample library are extracted through the image retrieval algorithm, which is expressed as , where n is the number of samples in the sample library; Step (3): Cluster the features belonging to the same aluminum profile category according to X, so as to obtain K cluster centers for each aluminum profile category. ; Step (4): After clustering is completed, K cluster centers will represent the characteristics of the aluminum profile category. The characteristics of the aluminum profile category corresponding to the cluster center will be stored to construct a feature library for representing each aluminum profile category.

[0024] In this embodiment, the image retrieval algorithm is the CLIP-ReID model, and the clustering is the K-Means clustering algorithm.

[0025] Step 2: Based on the QATM (Quality-Aware Template Matching) algorithm, extract the location of the aluminum profile section from the input image and generate a candidate frame. Specifically, use the pre-trained model to match the input image. I Perform feature extraction. First, input image IInput into the pre-trained model, pass through each layer of the convolutional network of the pre-trained model, and obtain the feature maps of different layers , that is, obtaining feature maps of different layers ,in Represents the feature map of layer I; Since the feature maps between different layers are often different, it is necessary to first Scaling to the same size, feature maps of different scales Splicing along the channel dimension to form a multi-scale feature map :

[0026] The concatenation operation is performed along the channel dimension, that is, the feature map of each layer Merge by channel to form a richer multi-scale feature map , the multi-scale feature map Contains spatial and semantic information at different levels.

[0027] Step 3: Extract features from all template images in the same way as in step 2, and extract the feature maps obtained in step 2. Scale to a specified size (different sizes and aspect ratios can better respond to target objects of different sizes), and then Take the average value to get the mixed template feature map In particular, if the template feature map No change has occurred, and the template feature map can be Save it and don't need to generate it again every time.

[0028] Step 4: Feature map With template feature map Normalize them separately to get the normalized input feature map With template feature map , for the input feature map The shape of is set to [x, c, a, b], for the template feature map The shape is set to [x, c, d, e], where x is the batch dimension, c is the number of channels, and a and b are input feature maps. The spatial dimensions (height and width) of the template feature map The shape of is set to [x, c, d, e], where x and c have the same meaning as above, that is, x is the batch dimension, c is the number of channels, and d and e are template feature maps. The spatial dimensions (height and width) of The calculation formula for [x, a, b, d, e] is:

[0029] Similarity distance [x, a, b, d, e] represents the input feature map With template feature map The similarity between each pixel; Step 5: Calculate the confidence of the normalized similarity distance D in step 4 and convert the similarity distance Reshape ,make Subtract the maximum value of the row and column respectively to get and , forming the calculated confidence The formula is:

[0030] α is a hyperparameter, which is the temperature parameter of softmax. When α is close to 0, the probability of output will be close to uniform, such as 60% and 40%; if α is close to infinity, the probability of output will strengthen the maximum response and suppress other responses, and the output may be 99% and 1%, so in this embodiment, α=25; confidence Representation feature map At position i and the template feature map The confidence at position j.

[0031] Then, the confidence matrix Take the maximum value by row and get the feature map Each point in the template feature map The highest confidence level is:

[0032] Step 6: Generate a candidate box position confidence map; Interpolate to input image I The size of the candidate box is then specified, and a sliding window of the specified size is used to slide on conf, and the mean of the built-in confidence of the sliding window is calculated to obtain the confidence that the center point of the candidate box of this size falls on each pixel position. In particular, in this step, sliding windows of multiple sizes are specified to generate multiple candidate box position confidence maps to accommodate aluminum profiles of different sizes. The sliding window sizes include (128, 256), (256, 128) and (256, 256).

[0033] Step 7: Perform non-maximum suppression on the candidate boxes; when performing non-maximum suppression (NMS) on candidate boxes of the same size, sort them according to the confidence of each candidate box, select the candidate box with the highest confidence as the benchmark, and calculate the intersection over union (IoU) of the remaining candidate boxes and the benchmark candidate box. For two candidate boxes A and B (candidate box A is the remaining candidate box, and candidate box B is the benchmark candidate box), the calculation formula for the intersection over union is as follows:

[0034] The intersection-over-union ratio includes intersection and union. The intersection refers to the overlapping area of ​​two candidate boxes, while the union refers to the total area of ​​the two candidate boxes.

[0035] If the intersection-over-union ratio of the remaining candidate boxes to the benchmark candidate boxes is greater than the threshold, the remaining candidate boxes are deleted from the candidate box set; if the intersection-over-union ratio of the remaining candidate boxes to the benchmark candidate boxes is less than the threshold, the remaining candidate boxes with the intersection-over-union ratio less than the threshold are retained until the remaining candidate boxes are processed.

[0036] For candidate boxes of different sizes, no comparison of intersection over union is performed, so each size of the candidate box will perform the above NMS operation separately without interacting with candidate boxes of other sizes. Finally, a set of candidate boxes after non-maximum suppression is output, in which each box has a high confidence value and a small overlap area with other candidate boxes, ensuring the accuracy of the positioning results. At the same time, because candidate boxes of different sizes do not interact with each other, the same target will generate candidate boxes of multiple sizes, which improves the output quality of the candidate boxes.

[0037] In this embodiment, the pre-trained model is VGG19, and the output feature maps of the 2nd and 8th convolution layers are selected for channel dimension splicing to generate a multi-scale feature map. ; Template feature map The specified sizes include (128, 128), (128, 256) and (256, 128); the threshold is 0.01-0.3, and the specific threshold can be selected as 0.05. In other embodiments, it can be flexibly adjusted according to actual needs.

[0038] Step 8: Input the candidate box into the image retrieval algorithm and match it with the constructed feature library to determine the category of the aluminum profile. Specifically: Input candidate boxes into the image retrieval algorithm one by one to extract corresponding features, and match the extracted features with each feature in the pre-built feature library. Calculate the distance between features to measure the similarity, and convert the calculation results into confidence. Based on the confidence level, non-maximum suppression is performed on the candidate boxes to eliminate redundant overlapping areas, and the candidate boxes whose confidence values ​​are lower than the threshold and are judged as background are further removed, and finally the location of the aluminum profile section is determined.

[0039] In this embodiment, the image retrieval algorithm is consistent with step 1, and a CLIP-ReID model trained on some known aluminum profile categories is used.

[0040] like Figure 2 As shown, compared with the original algorithm, the QATM algorithm of the present invention can provide better candidate boxes at more scales and more proportions for subsequent image retrieval algorithms to make category determinations.

[0041] The present invention makes full use of the existing feature library and image retrieval mechanism, can efficiently complete the classification task of the newly added category, and significantly improves its adaptability and flexibility in the scenario of frequent category updates. Especially in the industrial production environment where the types of aluminum profiles change frequently, this method can quickly adapt to new classification requirements without frequent model training or re-adjustment of parameters, reducing the consumption of computing resources, and significantly improving the efficiency and practicality of industrial automation processes.

[0042] Through the above classification method, users do not need to retrain frequently as the target types change, so as to adapt to the rapidly changing needs in aluminum profile factories more efficiently and flexibly, while meeting the requirements of industrial automated production.

[0043] The technical effects of the present invention are mainly reflected in the following aspects: This classification method does not need to be frequently retrained as the target types change, so it can adapt to the rapidly changing needs in aluminum profile factories more efficiently and flexibly, while meeting the requirements of industrial automated production.

[0044] Of course, the above are only typical examples of the present invention. In addition, the present invention may also have many other specific implementations. All technical solutions formed by equivalent replacement or equivalent transformation fall within the scope of protection required by the present invention.

Claims

1. A method for classifying aluminum profiles, characterized in that: include: Step 1: Build a sample library and feature library of aluminum profile sections; Step 2: Use the pre-trained model to perform feature extraction on the input image, extract the location where the aluminum profile cross section appears from the input image, and generate a candidate frame; Step 3: The feature map obtained in step 2 Scale to the specified size, and then perform Take the average value to get the mixed template feature map ; Step 4: For the feature map With the template feature map Normalize them separately to get the normalized input feature map With template feature map , for the input feature map The shape is set to [x, c, a, b], for the template feature map The shape of is set to [x, c, d, e] to get the similarity distance [x, a, b, d, e], where x is the batch dimension, c is the number of channels, and a and b are the input feature maps The height and width of the template are d and e. The height and width of Step 5: Calculate the normalized confidence of the similarity distance D in step 4, and then obtain the feature map by taking the maximum value of the confidence by row. Each point in the template feature map The highest confidence level; Step 6: Generate the candidate box position confidence map; Step 7: Perform non-maximum suppression on the candidate box; Step 8: Input the candidate frame into the image retrieval algorithm and match it with the constructed feature library to determine the category of the aluminum profile.

2. The aluminum profile classification method according to claim 1, characterized in that: The step 1 comprises: Step (1): collecting cross-sectional images of all types of aluminum profiles and storing them to construct the sample library; Step (2): According to step (1), the features of all the aluminum profile cross-section images in the sample library are extracted by an image retrieval algorithm, which is expressed as ; Step (3): clustering the features belonging to the same aluminum profile category according to X, thereby obtaining K cluster centers of each aluminum profile category; Step (4): After clustering is completed, the K cluster centers will represent the features of the aluminum profile category, and the features of the aluminum profile category corresponding to the cluster centers will be stored to construct the feature library for representing each aluminum profile category; The image retrieval algorithm is the CLIP-ReID model, the clustering is the K-Means clustering algorithm, and n is the number of samples in the sample library.

3. The aluminum profile classification method according to claim 1, characterized in that: The input image is input into the pre-trained model, and the feature maps of different layers are obtained through the convolutional networks of each layer of the pre-trained model. , the feature maps between different layers The feature maps of different scales are scaled to the same size Splicing along the channel dimension to form a multi-scale feature map : ， In step 4, the similarity distance The calculation formula for [x, a, b, d, e] is: ， The similarity distance [x, a, b, d, e] represents the input feature map With the template feature map The similarity between each pixel; In step 5, the similarity distance Reshape ,make Subtract the maximum value of the row and column respectively to get and , forming the confidence level calculated The formula is: ， The confidence Represents the feature map At position i and the template feature map Confidence at position j; By the confidence matrix Take the maximum value by row and get the feature map Each point in the template feature map The highest confidence level is: ， Said is the feature map of the Ith layer, and α is a hyperparameter, which is the temperature parameter of softmax; The pre-trained model is VGG19, and the template feature map Specified sizes include (128,128), (128,256), and (256,128).

4. The aluminum profile classification method according to claim 3, characterized in that: In step 6, a sliding window of a specified size is used to Slide upward, calculate the mean of the confidence within the sliding window, and obtain the confidence that the center point of the candidate box of this size falls on each pixel position; In step 7, when the non-maximum suppression is performed on the candidate boxes of the same size, the confidence of each candidate box is sorted, the candidate box with the highest confidence is selected as the benchmark, and the intersection-and-union ratio of the remaining candidate boxes and the benchmark candidate box is calculated. For two candidate boxes A and B, the calculation formula of the intersection-and-union ratio is as follows: ， The candidate frame A is a remaining candidate frame, and the candidate frame B is a reference candidate frame; The intersection-over-union ratio includes an intersection and a union, the intersection refers to an overlapping area of ​​the remaining candidate box and the reference candidate box, and the union refers to a total area of ​​the remaining candidate box and the reference candidate box.

5. The aluminum profile classification method according to claim 4, characterized in that: If the intersection-over-union ratio of the remaining candidate frame and the reference candidate frame is greater than a threshold, the remaining candidate frame is deleted from the candidate frame set; The intersection-and-union ratio of the remaining candidate frames and the reference candidate frames is smaller than the threshold, and the remaining candidate frames whose intersection-and-union ratio is smaller than the threshold are retained until the remaining candidate frames are processed; The threshold value is 0.01-0.

3.

6. The aluminum profile classification method according to claim 5, characterized in that: Inputting the candidate frames into the image retrieval algorithm one by one to extract corresponding features, and matching the extracted features with each feature in the pre-built feature library, calculating the distance between the features to measure the similarity, and converting the calculation result into the confidence level; The non-maximum suppression is performed on the candidate frames based on the confidence level to eliminate redundant overlapping areas, and the candidate frames whose confidence levels are lower than the threshold and are judged as backgrounds are further removed, and finally the position of the aluminum profile cross section is determined.

Images