Visual identification method, device, equipment and storage medium based on steel surface defect classification

Abstract

The present invention provides a visual recognition method, device, equipment, and storage medium based on steel surface defect classification, relating to the field of defect recognition in industrial scenarios. The method includes the following steps: extracting steel surface feature information, screening out a number of important feature data, sorting them according to a predetermined strategy, and dividing them into a training set and a test set; constructing a two-dimensional feature learning network, inputting the training set into it and mapping it to a two-dimensional feature plane, and obtaining the area corresponding to each steel surface defect category; fixing the network model parameters after multiple rounds of training; inputting the test set into the two-dimensional feature learning network with fixed network model parameters, obtaining the mapped two-dimensional feature plane, and according to the mapped two-dimensional feature coordinates, defining the situation based on the intersection elliptical surface coverage strategy, and determining the steel surface defect category to which it belongs. This method can intuitively observe the differences between training set samples under different categories and define the area to which each category sample belongs.

Description

Technical Field

[0001] The present invention relates to the field of defect recognition in industrial scenarios, and in particular to a visual recognition method, device, equipment and storage medium based on steel surface defect classification. Background Art

[0002] Currently, defect classification through visual recognition has become mainstream in the field of steel processing automation. For example, Chinese invention patent publication number CN117557520A proposes a lightweight defect recognition algorithm for hot-rolled steel strip surface defects; another example is Chinese invention patent publication number CN118898582A, which proposes an improved YOLOX steel surface defect detection method.

[0003] However, in real-world scenarios, the sample under test may contain multiple defect types simultaneously, or even new types that have not been accurately defined. In this case, evaluation metrics such as precision and recall alone cannot accurately describe the differences between the sample under test and known defect types. For steel surface defect detection tasks, the sample under test may belong to one of the known defect categories, multiple categories simultaneously, or even an unknown defect category. Existing technologies cannot effectively address these issues. Summary of the Invention

[0004] Purpose of the invention: To propose a visual identification method, device, equipment and storage medium based on steel surface defect classification to solve the above-mentioned problems existing in the prior art.

[0005] In a first aspect of the present invention, a visual identification method based on steel surface defect classification is proposed, comprising the following steps:

[0006] Extract the surface feature information of steel and select a predetermined number of important feature data;

[0007] Using the Gini coefficient as an evaluation index for the important feature data, the important feature data are sorted from the center to the periphery into a square of size N×N according to the evaluation index, and training set data and test set data are divided from the sorted important feature data according to a predetermined ratio;

[0008] A two-dimensional feature learning network is constructed and inputted with training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtains each steel surface defect category and the region to which it belongs; the test set data is inputted into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and the situation attribution is defined based on the mapped two-dimensional feature coordinates to determine the steel surface defect category to which it belongs; after multiple rounds of training and performance evaluation, a final steel surface defect classification and recognition model is obtained;

[0009] The steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0010] In a further embodiment of the first aspect, the surface feature information of the steel material is extracted from the open source dataset UCI SteelPlates Faults Data Set. After extracting the surface feature information of the steel material of the first sample quantity from the open source dataset, the second sample quantity ( ), so that they can be arranged into an N×N square pattern later, where N is a positive integer, preferably N≥5.

[0011] In a further embodiment of the first aspect, if N is an odd number, the first important feature coordinate is one, and its position is ;

[0012] The second important feature coordinates are expanded outwards around the first important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position range of

[0013] The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple positions, and their horizontal and vertical coordinates are both located at within the position range.

[0014] In a further embodiment of the first aspect, if N is an even number, there are multiple first important feature coordinates, and their horizontal and vertical coordinates are both located at within the position range of

[0015] The second important feature coordinates are expanded outwards around the first important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position range of

[0016] The third important feature coordinates continue to expand outward around the second important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position range.

[0017] In a further embodiment of the first aspect, the two-dimensional feature learning network includes a weighted convolutional layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer;

[0018] The training set data that meets the predetermined square size passes through the convolution layer, batch normalization layer, ReLU layer, pooling layer, residual network layer, adaptive average pooling layer, and fully connected layer in sequence, and is reduced to a two-dimensional feature plane through t-SNE. The sample data in the two-dimensional feature plane is classified based on the intersection elliptical surface covering strategy to determine the category of steel surface defects.

[0019] In a further embodiment of the first aspect, the process of defining the situation attribution of sample data in a two-dimensional feature plane based on the union and intersection ellipse surface covering strategy specifically includes:

[0020] Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set data categories, and mark the real training set category and noise category to which the sample data belongs;

[0021] Calculate the elliptical area bounded by each true training set category and noise category:

[0022]

[0023]

[0024]

[0025] Where, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse respectively;

[0026] The sample data in the two-dimensional feature plane is classified into three categories according to the different UMI areas defined:

[0027]

[0028] Where m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part.

[0029] In a further embodiment of the first aspect, the steel surface defect categories include the following three:

[0030] The test sample falls into the UMI area under Case 1 , when falling into When there are elliptical areas, it is determined to be the category of the i-th training set;

[0031] The test sample falls into the UMI area under Case 2 , when falling into When the elliptical surface area covered by the noise category area represented by the sample may contain common characteristics of multiple known categories. When the following set is met:

[0032]

[0033] Samples to be tested It is determined to be the category of the i-th training set, because The formula of case 2 is satisfied, so i here is not unique. At this time, the sample contains common characteristics of multiple known categories.

[0034] The test samples that fall into the UMI area under situation 3 are judged as unknown classes. When the unknown class forms a certain scale and is concentrated in one or several areas within the two-dimensional feature plane, it is defined as a new class.

[0035] In a further embodiment of the first aspect, based on the mapped two-dimensional feature plane, the following classification indicators are used to perform performance evaluation on the two-dimensional feature learning network:

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] Wherein, TP represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the known class as the known class; TN represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the unknown class as the unknown class; FP represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the unknown class as the known class; FN represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the known class as the unknown class; Indicates the accuracy of known class recognition; Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

[0043] A second aspect of the present invention provides a device for classifying and identifying surface defects of steel materials, the device comprising:

[0044] A data extraction module is used to extract surface feature information of steel materials and select a predetermined number of important feature data;

[0045] a data processing module, using the Gini coefficient as an evaluation index for the important feature data, sorting the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and dividing the sorted important feature data into training set data and test set data according to a predetermined ratio;

[0046] A model building module is used to build a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain the category of each steel surface defect and the area to which it belongs; the test set data is input into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and the mapped two-dimensional feature coordinates are used to define the situation and determine the steel surface defect category to which it belongs; after multiple rounds of training and testing, the final steel surface defect classification and recognition model is obtained;

[0047] The result output module is used to import the steel surface image captured by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0048] According to a third aspect of the present invention, an electronic device is provided, comprising a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the visual recognition method based on steel surface defect classification as disclosed in the first aspect is implemented.

[0049] In a fourth aspect of the present invention, a computer-readable storage medium is proposed, which stores at least one executable instruction. When the executable instruction is run on an electronic device, the electronic device executes the visual identification method based on steel surface defect classification as disclosed in the first aspect.

[0050] Beneficial effects: The present invention maps both the training set and the test set data onto a visual two-dimensional plane, which not only allows intuitive observation of the differences between training set samples under different categories and defines the region to which each category of samples belongs, but also allows judgment through outliers on whether certain samples have characteristics of multiple categories at the same time; in addition, it is possible to identify new categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0052] Figure 1 Schematic diagram of the structure of a two-dimensional feature learning network in an embodiment of the present invention.

[0053] Figure 2 Schematic diagram of mapping training set data to a two-dimensional feature plane in an embodiment of the present invention.

[0054] Figure 3 Schematic diagram of mapping training set and test set data to a two-dimensional feature plane in an embodiment of the present invention.

[0055] Figure 4 Schematic diagram of the structure of the device for classifying and identifying steel surface defects in an embodiment of the present invention.

[0056] Figure 5 Schematic diagram of the structure of the data processing module in an embodiment of the present invention.

[0057] Figure 6 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention.

[0059] Traditional multi-classification algorithms focus on improving model efficiency or accuracy, but in real scenarios, the samples to be tested may contain multiple defect types at the same time, or they may be new types that have not been accurately defined before.

[0060] To address the challenges of traditional classification algorithms, this invention maps both the training and test data sets onto a visual two-dimensional plane. This not only allows for intuitive observation of differences between trained defect categories and the regions to which samples from each category belong, but also allows for the use of outliers to determine the similarity between these samples and other known categories. In the test data set, some defect samples automatically cluster in multiple areas of the two-dimensional plane, suggesting that these samples may represent genuine new categories.

[0061] This embodiment discloses a visual identification method based on steel surface defect classification. The method may include steps 1 to 3, the specific details of which are as follows:

[0062] Step 1: Manually extract the surface feature information of steel and select Important feature data (N is a positive integer, preferably N ≥ 5). Sort all the important feature data that have been screened out, and divide the training set data and test set data from the sorted important feature data.

[0063] Step 2: Construct a 2D feature learning network and input the sorted training data. After multiple rounds of training, a set of network model parameters are fixed. The feature layer of the 2D feature learning network is extracted, and the training data is mapped to a 2D feature plane using a dimensionality reduction algorithm. Based on the 2D feature coordinates of the mapped training data, a strategy of intersecting elliptical surfaces is used to classify the surface defects and determine the corresponding steel surface defect category.

[0064] Step 3: Input the test set data into the two-dimensional feature learning network with fixed network model parameters, and use the dimensionality reduction algorithm again to obtain the mapped two-dimensional feature plane. According to the two-dimensional feature coordinates of the mapped test set data, determine the category of steel surface defects.

[0065] Step 4: After training and testing in steps 2 and 3, a final steel surface defect classification and recognition model is obtained; the steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and the recognition and classification results are output.

[0066] As a preferred embodiment, in step 1, the open source dataset UCI Steel Plates FaultsData Set is used to extract the defect types of steel surface feature information. The dataset contains 7 types of steel defects, namely decoration, Z scratches, K scratches, stains, dirt, bumps, and other defects, which are defined as categories 1 to 7 respectively. The feature data is independent features of 28 dimensions. The Gini coefficient is used as an evaluation indicator for important feature data, and the evaluation indicator is used to diffuse and sort from the center of a square of size N×N to the surrounding areas according to a predetermined sorting strategy. Table 1 shows the results of sorting independent feature information of N=5, a total of 25 dimensions. The numbers in brackets in Table 2 indicate the feature importance ranking.

[0067] Table 1: Ranking results of important feature data of size 5×5

[0068] Edge X Indicator (22) Edge Y indicator (14) Square Indicator (10) Luminosity Index (15) Y circumference (24) Edge indicators (18) Logarithmic X Index (6) Conveyor length (2) Steel plate thickness (7) Empty index (20) Minimum brightness (12) Total brightness (4) Regional logarithm (1) External X-index (5) Y_min(13) Maximum brightness (19) X minimum value (8) Pixel area (3) X maximum value (9) Logarithm of area (21) Logarithmic Y-Index (23) Y maximum value (16) Directional indicators (11) X circumference (17) Is it A300 steel? (25)

[0069] Furthermore, if N is an odd number, taking N=5 as an example, the first important feature coordinate position "1" is , the distribution of the second important feature coordinate position "2-9" expands outward, and the horizontal and vertical coordinates are both located at In the position range, the distribution of the third important feature coordinate position "10-25" continues to expand outward, and the horizontal and vertical coordinates are both located at , and so on.

[0070] Furthermore, if N is an even number, taking N=6 as an example, the horizontal and vertical coordinates of the first important feature coordinate position "1-4" are both located at In the position range, the distribution of the second important feature coordinate position "5-16" expands outward, and the horizontal and vertical coordinates are both located at In the position range, the distribution of the third important feature coordinate position "17-36" continues to expand outward, and the horizontal and vertical coordinates are both located at , and so on.

[0071] Furthermore, if N is an odd number, for example, N=7, the ranking method for the same important features is: non-vertex features are more important than vertex features. If N is an odd number, the importance rankings between 2-5 can be swapped. Similarly, the importance rankings between 6-9, 10-21, 22-25, 26-45, and 46-49 can also be swapped. Table 2 shows the permutation results for a 9×9 size.

[0072] Table 2: Ranking results of important feature data of size 9×9

[0073] 46 38 30 26 31 39 47 42 22 14 10 15 23 44 34 18 6 2 7 20 36 28 12 4 1 5 13 29 35 19 8 3 9 21 37 43 24 16 11 17 25 45 48 40 32 27 33 41 49

[0074] In addition, if N is an even number, taking N=6 as an example, the sorting method when facing the same important features is: the positions ranked between "1-4" in importance can be interchanged. Similarly, the positions within the positions ranked between "5-12", "13-16", "17-32", and "33-36" in importance can also be interchanged. Table 3 shows the arrangement results when the square size is 6×6.

[0075] Table 3: Ranking results of important feature data of size 6×6

[0076] 33 25 17 18 26 34 29 13 5 6 14 31 21 9 1 2 11 23 22 10 3 4 12 24 30 15 7 8 16 32 35 27 19 20 28 36

[0077] As a preferred embodiment, see Figure 1 As shown in Figure 2, the two-dimensional feature learning network in step 2 includes a weighted convolution layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer.

[0078] The entire dataset was first divided into three parts: training set, test set 1, and test set 2. The training set consisted of 80% random samples from category 1 decorative defects, category 2 Z scratch defects, and category 6 bump defects. Test set 1 consisted of the remaining 20% ​​of samples from category 1 decorative defects, category 2 Z scratch defects, and category 6 bump defects. Test set 2 consisted of category 3 (K scratches) and category 4 (stains). The training set data, sorted according to Table 1, was sequentially processed through convolutional layers, batch normalization layers, ReLU layers, pooling layers, residual network layers, adaptive average pooling layers, and fully connected layers. Dimensionality was reduced to a two-dimensional feature plane using t-SNE. The sample data within the two-dimensional feature plane was then classified using a union-intersection elliptical surface covering strategy to determine the class of the test data.

[0079] Through experiments, the accuracy of the training set is 93.8% and the accuracy of the test set 1 is 88.7% under the combination of the designed two-dimensional feature learning network and input features. Figure 2 In the figure, the data within the ellipse are considered to be the true type 1 decorative defects, type 2 Z scratch defects, type 6 bump defects and outliers, respectively.

[0080] As a preferred embodiment, the situation attribution is defined based on the intersection ellipse coverage strategy, and the specific process is as follows:

[0081] Step 2.1: Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set, and mark the categories and corresponding labels of these samples. The noise class is marked as "-1";

[0082] Step 2.2: Calculate the elliptical area bounded by each true class and the "-1" class:

[0083]

[0084]

[0085]

[0086] Where, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse respectively;

[0087] Step 2.3: Use the intersection ellipse coverage strategy to define the situation of the sample data in the two-dimensional plane:

[0088]

[0089] Where m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part.

[0090] As a preferred embodiment, in step 3, the test sample falling into the UMI region under case 1 is determined to belong to the category of the corresponding training set; the test sample falling into the UMI region under case 2 is determined to have characteristics of multiple known categories; and the test sample falling into the UMI region under case 3 is determined to belong to a new category.

[0091] To optimize the above technical solution, the following classification indicators can be used for the two-dimensional feature plane:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] Where TP is the number of samples that the model correctly predicts as known classes; TN is the number of samples that the model correctly predicts as unknown classes; FP is the number of samples that the model incorrectly predicts as known classes; FN is the number of samples that the model incorrectly predicts as known classes. represents the accuracy of known class recognition, Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

[0099] The results show that the accuracy of test set 1 is 91.4%, and the accuracy of test set 2 is 82.4%, indicating that the algorithm can correctly distinguish most samples in test set 1 and test set 2, corresponding to Figure 3 The case 1 is framed by a rectangular box. It can also be seen from the figure that there are indeed some outliers whose Euclidean distances from the known two or three categories are quite similar, such as Figure 3 Case 2, outlined by a rectangular frame, has attributes that simultaneously meet the characteristics of categories 1, 2, and 6. Regarding the specific problem, the test sample has decoration, Z scratches, and bump defects. Figure 3 The results also show that most of the samples of category 3 and category 4 in the test set are not within the four elliptical surfaces in the figure, corresponding to Figure 3 Case 3 is outlined in a rectangular box. In this case, the rectangular boxes are considered to have formed a certain scale in this embodiment and are concentrated in certain areas within the two-dimensional feature plane. Each of these categories can be considered a new category. As shown in the figure, category 4 in the true category is well clustered together, successfully identifying it as a new category. However, category 3 is dispersed and clustered in multiple areas of the two-dimensional feature plane. Under this method, the three areas of category 3 that form a certain scale can be considered as three new categories.

[0100] The present application also provides a steel surface defect classification and identification device 400, such as Figure 4 The device is composed of a data extraction module 401, a data processing module 402, a model building module 403, and a result output module 404.

[0101] The data extraction module 401 is used to extract the surface feature information of the steel material and select a predetermined number of important feature data;

[0102] The data processing module 402 uses the Gini coefficient as an evaluation index for the important feature data, and sorts the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and divides the sorted important feature data into training set data and test set data according to a predetermined ratio;

[0103] The model building module 403 is used to build a two-dimensional feature learning network and input the training set data. The two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and then obtains the category of each steel surface defect and the area to which it belongs. The test set data is input into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane. Based on the mapped two-dimensional feature coordinates, the situation is defined and the steel surface defect category is determined. After multiple rounds of training and testing, the final steel surface defect classification and recognition model is obtained.

[0104] The result output module 403 is used to import the steel surface image captured by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0105] When the steel surface defect classification and identification device 400 is run, the visual identification method based on steel surface defect classification disclosed in the above embodiment can be automatically executed, which will not be described in detail here.

[0106] As a preferred embodiment, Figure 5 As shown, the data processing module 402 includes at least one sorting unit 402a and at least one data partitioning unit 402b.

[0107] The sorting unit 402a uses the Gini coefficient as an evaluation index for important feature data and performs a spread sorting from the center of an N×N square to the surrounding areas according to the evaluation index and a predetermined sorting strategy. The data partitioning unit 402b can automatically partition the training set data into the test set data according to the sorting results.

[0108] As a preferred embodiment, a feasible sorting method of the above sorting unit 402a when performing data sorting is disclosed:

[0109] First determine whether N is an odd number or an even number. If N is an odd number, the first important feature coordinate is one, and its position is ;

[0110] The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position range of

[0111] The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple ones, and their horizontal and vertical coordinates are both located at within the position range of

[0112] And so on.

[0113] If N is an even number, the first important feature coordinates are multiple, and their horizontal and vertical coordinates are both located within the position range of

[0114] The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position range of

[0115] The third important feature coordinate continues to expand outward around the second important feature coordinate into multiple ones, and their horizontal and vertical coordinates are both located at within the position range of

[0116] And so on.

[0117] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to perform the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding, and will not be repeated here.

[0118] The embodiment of the present application provides an electronic device 500, see Figure 6 The electronic device 500 includes a memory 501 and a processor 502. The memory 501 stores a computer program that can be executed on the processor. When the processor 502 executes the program, some or all of the steps in the above method are implemented. The memory 501 may encapsulate at least one computer-readable storage medium 503, which stores a computer program. When the computer program is executed by the processor 502, some or all of the steps in the above method are implemented. The computer-readable storage medium 503 can be transient or non-transient.

[0119] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between the various embodiments, and their similarities or similarities can be referenced to each other. The descriptions of the above device, storage medium, computer program, and computer program product embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the description of the method embodiments of this application for understanding.

[0120] It should be noted that in the embodiments of the present application, if the above-mentioned method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk. In this way, the embodiments of the present application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0121] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A visual identification method based on steel surface defect classification, characterized in that: The steps include: Extract the surface feature information of steel and select a predetermined number of important feature data; Using the Gini coefficient as an evaluation index for the important feature data, the important feature data are sorted from the center to the periphery into a square of size N×N according to the evaluation index, and training set data and test set data are divided from the sorted important feature data according to a predetermined ratio; If N is an odd number, the first important feature coordinate is one, and its position is ; The second important feature coordinates are expanded outwards around the first important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position range of The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple positions, and their horizontal and vertical coordinates are both located at within the position range of If N is an even number, the first important feature coordinates are multiple, and their horizontal and vertical coordinates are both located within the position interval; The second important feature coordinates are expanded outwards around the first important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position interval; The third important feature coordinates continue to expand outward around the second important feature coordinates into multiple ones, whose horizontal and vertical coordinates are both located at within the position range of A two-dimensional feature learning network is constructed and training set data is input; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtains each steel surface defect category and the area to which it belongs; the test set data is input into a two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane. Based on the mapped two-dimensional feature coordinates, the situation is defined based on the intersection elliptical surface coverage strategy to determine the steel surface defect category. According to the different UMI areas defined, it is divided into three situations: Where m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part; The test sample falls into the UMI area under Case 1 , when it falls into any elliptical area When , it is determined to be the category of the i-th training set; The test sample falls into the UMI area under Case 2 , when it falls into any elliptical area , and the following set is satisfied: Samples to be tested It is determined to be the category of the i-th training set; The test sample falls into the UMI area under case 3 It is judged as an unknown class; After multiple rounds of training and performance evaluation, the final steel surface defect classification and recognition model was obtained; The steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

2. The visual identification method based on steel surface defect classification according to claim 1 is characterized in that: The two-dimensional feature learning network includes a weighted convolution layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer; The training set data that meets the predetermined square size passes through the convolution layer, batch normalization layer, ReLU layer, pooling layer, residual network layer, adaptive average pooling layer, and fully connected layer in sequence, and is reduced to a two-dimensional feature plane through t-SNE. The sample data in the two-dimensional feature plane is classified based on the intersection elliptical surface covering strategy to determine the category of steel surface defects.

3. The visual identification method based on steel surface defect classification according to claim 2 is characterized in that: The ellipse surface coverage strategy based on union and intersection defines the situation attribution of sample data in the two-dimensional feature plane, specifically including: Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set data categories, and mark the real training set category and noise category to which the sample data belongs; Calculate the elliptical area bounded by each true training set category and noise category: Where, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse, respectively.

4. The visual identification method based on steel surface defect classification according to claim 3 is characterized in that: Also includes: Based on the mapped two-dimensional feature plane, the following classification indicators are used to evaluate the performance of the two-dimensional feature learning network: Wherein, TP represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the known class as the known class; TN represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the unknown class as the unknown class; FP represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the unknown class as the known class; FN represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the known class as the unknown class; Indicates the accuracy of known class recognition; Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

5. A device for classifying and identifying steel surface defects, which executes the visual identification method based on steel surface defect classification according to any one of claims 1 to 4, characterized in that: include: A data extraction module is used to extract surface feature information of steel materials and select a predetermined number of important feature data; a data processing module, using the Gini coefficient as an evaluation index for the important feature data, sorting the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and dividing the sorted important feature data into training set data and test set data according to a predetermined ratio; A model building module is used to build a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain the category of each steel surface defect and the area to which it belongs; the test set data is input into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and the mapped two-dimensional feature coordinates are used to define the situation and determine the steel surface defect category to which it belongs; After multiple rounds of training and testing, the final steel surface defect classification and recognition model was obtained; The result output module is used to import the steel surface image captured by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

6. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the visual recognition method based on steel surface defect classification according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the visual recognition method based on steel surface defect classification according to any one of claims 1 to 4.

Images