Visual recognition method, device and equipment based on steel surface defect classification and storage medium

Abstract

The invention provides a visual recognition method, device and equipment based on steel surface defect classification and a storage medium, and relates to the field of defect recognition in industrial scenes. The method comprises the following steps: extracting steel surface feature information, screening out a plurality of important feature data, sorting the important feature data according to a predetermined strategy, and dividing a training set and a test set; constructing a two-dimensional feature learning network, inputting the training set into the two-dimensional feature learning network, mapping the training set to a two-dimensional feature plane, and obtaining an area corresponding to each steel surface defect category; network model parameters are fixed through multiple rounds of training; and inputting the test set into a two-dimensional feature learning network of fixed network model parameters to obtain a mapped two-dimensional feature plane, and according to a mapped two-dimensional feature coordinate, carrying out condition affiliation definition on the mapped two-dimensional feature plane based on a cross ellipsoid coverage strategy, and judging a steel surface defect category to which the mapped two-dimensional feature plane belongs. According to the method, the difference between training set samples under different categories can be visually observed, and the area to which each category of samples belongs is defined.

Description

Technical Field

[0001] The present invention relates to the field of defect recognition in industrial scenarios, and in particular to a visual recognition method, device, equipment and storage medium based on steel surface defect classification. Background Art

[0002] At present, in the field of steel processing automation, defect classification through visual recognition has become the mainstream. For example, the Chinese invention patent with publication number CN117557520A proposed a lightweight defect recognition algorithm for hot-rolled steel strip surface defects; another example is the Chinese invention patent with publication number CN118898582A that proposed an improved YOLOX steel surface defect detection method.

[0003] However, in real scenarios, the samples to be tested may contain multiple defect types at the same time, or they may be new types that have not been accurately defined before. In this case, only evaluation indicators such as accuracy and recall cannot describe the differences between the samples to be tested and the known types. For the steel surface defect task, the sample to be tested may belong to one of the known defect categories, or it may belong to multiple known defect categories at the same time, or it may belong to an unknown defect category. The above problems cannot be effectively solved by existing technologies. Summary of the invention

[0004] Purpose of the invention: To propose a visual identification method, device, equipment and storage medium based on steel surface defect classification to solve the above-mentioned problems existing in the prior art.

[0005] In a first aspect of the present invention, a visual recognition method based on steel surface defect classification is proposed, and the steps are as follows:

[0006] Extract the surface characteristic information of steel materials and select a predetermined number of important characteristic data;

[0007] Using the Gini coefficient as an evaluation index of the important feature data, the important feature data are diffused and sorted from the center to the periphery into a square of size N×N according to the evaluation index, and training set data and test set data are divided from the sorted important feature data according to a predetermined ratio;

[0008] Construct a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain each type of steel surface defect category and the area to which it belongs; input the test set data into a two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and define the situation according to the mapped two-dimensional feature coordinates to determine the steel surface defect category to which it belongs; after multiple rounds of training and performance evaluation, the final steel surface defect classification and recognition model is obtained;

[0009] The steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and the recognition and classification results are output.

[0010] In a further embodiment of the first aspect, the surface feature information of the steel material is extracted from the open source dataset UCI SteelPlates Faults Data Set. After extracting the surface feature information of the steel material of the first sample quantity from the open source dataset, a second sample quantity ( ), so as to facilitate the subsequent arrangement into an N×N square pattern, where N is a positive integer, preferably N≥5.

[0011] In a further embodiment of the first aspect, if N is an odd number, the first important feature coordinate is one, and its position is ;

[0012] The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval;

[0013] The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple positions, and their horizontal and vertical coordinates are both located at within the position range.

[0014] In a further embodiment of the first aspect, if N is an even number, there are multiple first important feature coordinates, and their horizontal and vertical coordinates are both located at within the position interval;

[0015] The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval;

[0016] The third important feature coordinates continue to expand outward around the second important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position range.

[0017] In a further embodiment of the first aspect, the two-dimensional feature learning network includes a weighted convolutional layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer;

[0018] The training set data that meets the predetermined square size passes through the convolution layer, batch normalization layer, ReLU layer, pooling layer, residual network layer, adaptive average pooling layer, and fully connected layer in turn, and is reduced to a two-dimensional feature plane through t-SNE. The sample data in the two-dimensional feature plane is classified based on the intersection elliptical surface covering strategy to identify the surface defect category of the steel.

[0019] In a further embodiment of the first aspect, the process of defining the situation attribution of sample data in the two-dimensional feature plane based on the union and intersection ellipse surface coverage strategy specifically includes:

[0020] Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set data category, and mark the real training set category and noise category to which the sample data belongs;

[0021] Calculate the elliptical area bounded by each true training set category and noise category:

[0022]

[0023]

[0024]

[0025] In the formula, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse respectively;

[0026] The sample data in the two-dimensional feature plane is classified into three situations according to the different UMI areas defined:

[0027]

[0028] In the formula, m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part.

[0029] In a further embodiment of the first aspect, the steel surface defect categories include the following three types:

[0030] The sample to be tested falls into the UMI area under case 1 , when it falls into When there are elliptical areas, it is judged to be the category of the i-th training set;

[0031] The sample to be tested falls into the UMI area under case 2 , when falling into When the elliptical area covered by the noise category area represented by When the following set is met:

[0032]

[0033] Samples to be tested It is judged to be the category of the i-th training set, because The formula of case 2 is satisfied, so i here is not unique. At this time, the sample contains common characteristics of multiple known categories.

[0034] The samples to be tested that fall into the UMI area under situation 3 are judged as unknown classes. When the unknown classes form a certain scale and are concentrated in one or several areas in the two-dimensional feature plane, they are defined as new classes.

[0035] In a further embodiment of the first aspect, based on the mapped two-dimensional feature plane, the following classification indicators are used to complete the performance evaluation of the two-dimensional feature learning network:

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] In the formula, TP represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the known class as the known class; TN represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the unknown class as the unknown class; FP represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the unknown class as the known class; FN represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the known class as the unknown class; Indicates the accuracy of known class recognition; Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

[0043] In a second aspect of the present invention, a device for classifying and identifying surface defects of steel is provided, the device comprising:

[0044] A data extraction module is used to extract the surface characteristic information of steel materials and select a predetermined number of important characteristic data;

[0045] A data processing module, using the Gini coefficient as an evaluation index of the important feature data, and sorting the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and dividing the training set data and the test set data from the sorted important feature data according to a predetermined ratio;

[0046] A model building module is used to build a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain each type of steel surface defect category and the area to which it belongs; the test set data is input into a two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and according to the mapped two-dimensional feature coordinates, the situation is defined and the steel surface defect category is determined; after multiple rounds of training and testing, the final steel surface defect classification and recognition model is obtained;

[0047] The result output module is used to import the steel surface image collected by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0048] According to a third aspect of the present invention, an electronic device is provided, comprising a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the visual recognition method based on steel surface defect classification as disclosed in the first aspect is implemented.

[0049] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, in which at least one executable instruction is stored. When the executable instruction is executed on an electronic device, the electronic device executes the visual recognition method based on steel surface defect classification as disclosed in the first aspect.

[0050] Beneficial effects: The present invention maps both the training set and the test set data into a visual two-dimensional plane, which can not only intuitively observe the differences between training set samples under different categories and define the area to which each category of samples belongs, but also determine whether some samples have the characteristics of multiple categories at the same time through outliers; in addition, it is also possible to identify new categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0052] Figure 1 Schematic diagram of the structure of a two-dimensional feature learning network in an embodiment of the present invention.

[0053] Figure 2 It is a schematic diagram of mapping training set data to a two-dimensional feature plane in an embodiment of the present invention.

[0054] Figure 3 Schematic diagram of mapping training set and test set data to a two-dimensional feature plane in an embodiment of the present invention.

[0055] Figure 4 Schematic diagram of the structure of the device for classifying and identifying steel surface defects in an embodiment of the present invention.

[0056] Figure 5 Schematic diagram of the structure of the data processing module in the embodiment of the present invention.

[0057] Figure 6 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described.

[0059] Traditional multi-classification algorithms focus on improving model efficiency or accuracy, but in real scenarios, the samples to be tested may contain multiple defect types at the same time, or may be new types that have not been accurately defined before.

[0060] In order to solve the problems of traditional classification algorithms, the present invention maps both the training set and the test set data into a visualized two-dimensional plane, which can not only visually observe the differences between the trained defect categories and define the area to which each category sample belongs, but also judge the similarity between these samples and other known categories through outliers. In the test set data, some defect samples automatically cluster in other areas of the two-dimensional plane, and these samples may be real new categories.

[0061] This embodiment discloses a visual recognition method based on steel surface defect classification, which may include steps 1 to 3, and the specific details are as follows:

[0062] Step 1: Manually extract the surface feature information of steel and select Important feature data (here N is a positive integer, preferably N ≥ 5). Sort all the selected important feature data, and divide the training set data and the test set data from the sorted important feature data.

[0063] Step 2: Construct a two-dimensional feature learning network, input the sorted training set data, and after multiple rounds of training, fix a set of network model parameters. Extract the feature layer of the two-dimensional feature learning network, and map the training set data to the two-dimensional feature plane through the dimensionality reduction algorithm. According to the two-dimensional feature coordinates of the mapped training set, the situation is defined based on the intersection elliptical surface coverage strategy to determine the surface defect category of the steel.

[0064] Step 3: Input the test set data into the two-dimensional feature learning network with fixed network model parameters, and obtain the mapped two-dimensional feature plane through the dimensionality reduction algorithm again. According to the two-dimensional feature coordinates of the mapped test set data, determine the category of steel surface defects.

[0065] Step 4: After training and testing in steps 2 and 3, a final steel surface defect classification and recognition model is obtained; the steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0066] As a preferred embodiment, in step 1, the open source dataset UCI Steel Plates FaultsData Set is used to extract the defect types of steel surface feature information. The dataset contains 7 types of steel defects, namely decoration, Z scratches, K scratches, stains, dirtiness, bumps, and other defects, which are defined as categories 1 to 7 respectively. The feature data is independent features of 28 dimensions. The Gini coefficient is used as an evaluation index for important feature data, and the evaluation index is used to diffuse and sort from the center of a square of size N×N to the surrounding area according to a predetermined sorting strategy. Table 1 shows the results of sorting independent feature information of N=5, a total of 25 dimensions. The numbers in brackets in Table 2 indicate the feature importance ranking.

[0067] Table 1: Ranking results of important feature data of size 5×5

[0068] Edge X Indicator (22) Edge Y indicator (14) Square Indicator (10) Luminosity Index (15) Y circumference (24) Edge indicators (18) Logarithmic X Index (6) Conveyor Length(2) Steel plate thickness (7) Empty index (20) Minimum brightness (12) Total brightness (4) Area logarithm (1) Outside X-Indicator (5) Y_min(13) Maximum brightness (19) X minimum value (8) Pixel area (3) X Max(9) Logarithm of area (21) Logarithmic Y Indicator (23) Y maximum value (16) Directional indicators (11) X circumference (17) Is it A300 steel? (25)

[0069] Furthermore, if N is an odd number, taking N=5 as an example, the first important feature coordinate position "1" is , the distribution of the second most important feature coordinate position "2-9" expands outward, and the horizontal and vertical coordinates are both located at In the position range, the distribution of the third important feature coordinate position "10-25" continues to expand outward, and the horizontal and vertical coordinates are both located at , and so on.

[0070] Furthermore, if N is an even number, taking N=6 as an example, the horizontal and vertical coordinates of the first important feature coordinate position "1-4" are both located at In the position interval, the distribution of the second important feature coordinate position "5-16" expands outward, and the horizontal and vertical coordinates are both located at In the position range, the distribution of the third important feature coordinate position "17-36" continues to expand outward, and the horizontal and vertical coordinates are both located at , and so on.

[0071] In addition, if N is an odd number, taking N=7 as an example, the sorting method when facing the same important features is: the importance of non-vertex position features is higher than that of vertex position features. If N is an odd number, the positions of importance ranking between "2-5" can be interchanged. Similarly, the positions of importance ranking between "6-9", "10-21", "22-25", "26-45", and "46-49" can also be interchanged. Table 2 shows the arrangement results when the size is 9×9.

[0072] Table 2: Ranking results of important feature data of size 9×9

[0073] 46 38 30 26 31 39 47 42 22 14 10 15 23 44 34 18 6 2 7 20 36 28 12 4 1 5 13 29 35 19 8 3 9 21 37 43 24 16 11 17 25 45 48 40 32 27 33 41 49

[0074] In addition, if N is an even number, taking N=6 as an example, the sorting method when facing the same important features is: the positions ranked between "1-4" in importance can be interchanged. Similarly, the positions ranked between "5-12", "13-16", "17-32", and "33-36" in importance can also be interchanged. Table 3 shows the arrangement results when the square size is 6×6.

[0075] Table 3: Ranking results of important feature data of size 6×6

[0076] 33 25 17 18 26 34 29 13 5 6 14 31 21 9 1 2 11 23 22 10 3 4 12 24 30 15 7 8 16 32 35 27 19 20 28 36

[0077] As a preferred embodiment, see Figure 1 As shown in the figure, the two-dimensional feature learning network in step 2 includes a weighted convolution layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer.

[0078] First, the entire data set is divided into three parts: training set, test set 1 and test set 2. The training set is 80% of the random samples of the first category of decorative defects, the second category of Z scratch defects and the sixth category of bump defects. Test set 1 is the remaining 20% ​​of the samples of the first category of decorative defects, the second category of Z scratch defects and the sixth category of bump defects. Test set 2 is category 3 (K scratches) and category 4 (stains). The training set data sorted in Table 1 is sequentially passed through the convolution layer, batch normalization layer, ReLU layer, pooling layer, residual network layer, adaptive average pooling layer, and fully connected layer. After t-SNE dimension reduction to a two-dimensional feature plane, the sample data in the two-dimensional feature plane is defined based on the union and intersection ellipse surface covering strategy to determine the category of the test data.

[0079] Through experiments, with the combination of the designed two-dimensional feature learning network and input features, the accuracy of the training set is 93.8%, and the accuracy of the test set 1 is 88.7%. Figure 2 The data within the ellipse are considered to be the true type 1 decorative defects, type 2 Z scratch defects, type 6 bump defects and outliers, respectively.

[0080] As a preferred embodiment, the situation attribution is defined based on the intersection ellipse coverage strategy, and the specific process is as follows:

[0081] Step 2.1: Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set categories, and mark the categories to which these samples belong and the corresponding labels. The noise class is marked as the "-1" class;

[0082] Step 2.2: Calculate the elliptical area bounded by each true class and the "-1" class:

[0083]

[0084]

[0085]

[0086] In the formula, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse respectively;

[0087] Step 2.3: Use the intersecting ellipse coverage strategy to define the data of the samples in the two-dimensional plane:

[0088]

[0089] In the formula, m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part.

[0090] As a preferred embodiment, in step 3, the test samples falling into the UMI area under situation 1 are determined to be in the category of the corresponding training set; the test samples falling into the UMI area under situation 2 are determined to have characteristics of multiple known categories at the same time; and the test samples falling into the UMI area under situation 3 are determined to be new categories.

[0091] To optimize the above technical solution, the two-dimensional feature plane can adopt the following classification indicators:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] In the formula, TP represents the number of samples that the model correctly predicts as known classes; TN represents the number of samples that the model correctly predicts as unknown classes; FP represents the number of samples that the model incorrectly predicts as known classes; FN represents the number of samples that the model incorrectly predicts as known classes. represents the accuracy of known class recognition, Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

[0099] The results show that the accuracy of test set 1 is 91.4%, and the accuracy of test set 2 is 82.4%, indicating that the algorithm can correctly distinguish most samples in test set 1 and test set 2, corresponding to Figure 3 The case 1 is framed by a rectangular box. It can also be seen from the figure that there are indeed some outliers whose Euclidean distances from the known two or three categories are comparable, such as Figure 3 Case 2, which is framed by a rectangular box, has attributes that simultaneously satisfy the characteristics of categories 1, 2, and 6. Regarding the specific problem, the test sample has decoration, Z scratch, and bump defects. Figure 3 The results also show that most samples of category 3 and category 4 in the test set are not within the four ellipses in the figure, corresponding to Figure 3 In this case, the categories framed by the rectangular frame are considered to form a certain scale in this embodiment, and are respectively concentrated in several areas in the two-dimensional feature plane, and can be respectively regarded as a new category. From the figure, it can be seen that category 4 in the real category is well clustered together, and one new category is successfully identified, but category 3 is scattered and gathered in multiple areas of the two-dimensional feature plane. Under this method, the three categories 3 that form a certain scale can be respectively regarded as three new categories.

[0100] The present application also provides a steel surface defect classification and identification device 400, such as Figure 4 The device is composed of a data extraction module 401, a data processing module 402, a model building module 403, and a result output module 404.

[0101] The data extraction module 401 is used to extract the surface characteristic information of the steel material and select a predetermined number of important characteristic data therefrom;

[0102] The data processing module 402 uses the Gini coefficient as an evaluation index for the important feature data, and sorts the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and divides the sorted important feature data into training set data and test set data according to a predetermined ratio;

[0103] The model building module 403 is used to build a two-dimensional feature learning network and input the training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain each type of steel surface defect category and the area to which it belongs; the test set data is input into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and the situation is defined according to the mapped two-dimensional feature coordinates to determine the steel surface defect category to which it belongs; after multiple rounds of training and testing, the final steel surface defect classification and recognition model is obtained;

[0104] The result output module 403 is used to import the steel surface image collected by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

[0105] When the steel surface defect classification and identification device 400 is run, the visual identification method based on steel surface defect classification disclosed in the above embodiment can be automatically executed, which will not be described in detail here.

[0106] As a preferred embodiment, Figure 5 As shown, the data processing module 402 includes at least one sorting unit 402a and at least one data partitioning unit 402b.

[0107] The sorting unit 402a uses the Gini coefficient as an evaluation index for important feature data, and performs a diffusion sorting from the center of a square of size N×N to the periphery according to the evaluation index and a predetermined sorting strategy. The data division unit 402b can automatically divide the training set data and the test set data according to the sorting result.

[0108] As a preferred embodiment, a feasible sorting method when the above sorting unit 402a performs data sorting is disclosed:

[0109] First, determine whether N is an odd number or an even number. If N is an odd number, the first important feature coordinate is one, and its position is ;

[0110] The second important feature coordinates expand outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval;

[0111] The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple positions, and their horizontal and vertical coordinates are both located at within the position interval;

[0112] And so on.

[0113] If N is an even number, there are multiple first important feature coordinates, and their horizontal and vertical coordinates are both located within the position interval;

[0114] The second important feature coordinates expand outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval;

[0115] The third important feature coordinates continue to expand outward around the second important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval;

[0116] And so on.

[0117] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding, and no further description is given here.

[0118] The present application embodiment provides an electronic device 500, Figure 6 , the electronic device 500 includes a memory 501 and a processor 502. The memory 501 stores a computer program that can be run on the processor, and the processor 502 implements some or all of the steps in the above method when executing the program. The memory 501 can encapsulate at least one computer-readable storage medium 503, and the computer-readable storage medium 503 stores a computer program, and the computer program implements some or all of the steps in the above method when executed by the processor 502. The computer-readable storage medium 503 can be transient or non-transient.

[0119] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0120] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0121] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A visual recognition method based on steel surface defect classification, characterized in that: The steps include: Extract the surface characteristic information of steel materials and select a predetermined number of important characteristic data; Using the Gini coefficient as an evaluation index of the important feature data, the important feature data are diffused and sorted from the center to the periphery into a square of size N×N according to the evaluation index, and training set data and test set data are divided from the sorted important feature data according to a predetermined ratio; Construct a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain each type of steel surface defect category and the area to which it belongs; input the test set data into a two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and define the situation according to the mapped two-dimensional feature coordinates to determine the steel surface defect category to which it belongs; after multiple rounds of training and performance evaluation, the final steel surface defect classification and recognition model is obtained; The steel surface image captured by the industrial camera is imported into the final steel surface defect classification and recognition model to complete the automatic recognition and classification of steel surface defects, and the recognition and classification results are output.

2. The visual identification method based on steel surface defect classification according to claim 1 is characterized in that: If N is an odd number, the first important feature coordinate is one, and its position is ; The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval; The third important feature coordinate position continues to expand outward around the second important feature coordinate into multiple positions, and their horizontal and vertical coordinates are both located at within the position range.

3. The visual identification method based on steel surface defect classification according to claim 1 is characterized in that: If N is an even number, there are multiple first important feature coordinates, and their horizontal and vertical coordinates are both located within the position interval; The second important feature coordinates are expanded outward around the first important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position interval; The third important feature coordinates continue to expand outward around the second important feature coordinates into multiple ones, and their horizontal and vertical coordinates are both located at within the position range.

4. The visual identification method based on steel surface defect classification according to claim 1 is characterized in that: The two-dimensional feature learning network includes a weighted convolution layer, a batch normalization layer, a ReLU layer, a pooling layer, a residual network layer, an adaptive average pooling layer, and a fully connected layer; The training set data that meets the predetermined square size passes through the convolution layer, batch normalization layer, ReLU layer, pooling layer, residual network layer, adaptive average pooling layer, and fully connected layer in turn, and is reduced to a two-dimensional feature plane through t-SNE. The sample data in the two-dimensional feature plane is classified based on the intersection elliptical surface covering strategy to identify the surface defect category of the steel.

5. The visual identification method based on steel surface defect classification according to claim 4 is characterized in that: The ellipse surface coverage strategy based on union and intersection defines the situation attribution of sample data in the two-dimensional feature plane, specifically including: Use the DBSCAN clustering algorithm to obtain the same number of categories as the training set data category, and mark the real training set category and noise category to which the sample data belongs; Calculate the elliptical area bounded by each true training set category and noise category: ； ； ； In the formula, is the center of the k-th sample ellipse; is the qth sample of the kth class; is the total number of samples in the kth class; and are the major and minor axes of the ellipse respectively; The sample data in the two-dimensional feature plane is classified into three situations according to the different UMI areas defined: ； In the formula, m is the total number of categories in the training set; Represents the elliptical area covered by all true training set categories; Represents the elliptical area covered by all noise categories; Represents the entire 2D feature plane exclusion area part.

6. The visual identification method based on steel surface defect classification according to claim 5 is characterized in that: The types of steel surface defects include: The sample to be tested falls into the UMI area under case 1 , when it falls into any elliptical area When , it is judged to be the category of the i-th training set; The sample to be tested falls into the UMI area under case 2 , when it falls into any elliptical area , and the following set is satisfied: ； Samples to be tested It is judged to be the category of the i-th training set; The samples to be tested fall into the UMI area under case 3 It is classified as unknown.

7. The visual identification method based on steel surface defect classification according to claim 5 is characterized in that: Also includes: Based on the mapped two-dimensional feature plane, the following classification indicators are used to evaluate the performance of the two-dimensional feature learning network: ； ； ； ； ； ； In the formula, TP represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the known class as the known class; TN represents the number of samples that the final steel surface defect classification and recognition model correctly predicts the unknown class as the unknown class; FP represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the unknown class as the known class; FN represents the number of samples that the final steel surface defect classification and recognition model incorrectly predicts the known class as the unknown class; Indicates the accuracy of known class recognition; Indicates the accuracy of unknown class recognition; represents the set of real known classes, Represents the set of true unknown classes.

8. A device for classifying and identifying surface defects of steel, characterized in that: include: A data extraction module is used to extract the surface characteristic information of steel materials and select a predetermined number of important characteristic data; A data processing module, using the Gini coefficient as an evaluation index of the important feature data, and sorting the important feature data from the center to the periphery into a square of size N×N according to the evaluation index, and dividing the training set data and the test set data from the sorted important feature data according to a predetermined ratio; A model building module is used to build a two-dimensional feature learning network and input training set data; the two-dimensional feature learning network maps the training set data to a two-dimensional feature plane to obtain the mapped two-dimensional feature coordinates, and obtain each type of steel surface defect category and the area to which it belongs; the test set data is input into the two-dimensional feature learning network with fixed network model parameters to obtain the mapped two-dimensional feature plane, and according to the mapped two-dimensional feature coordinates, the situation is defined and the steel surface defect category is determined; After multiple rounds of training and testing, the final steel surface defect classification and recognition model was obtained; The result output module is used to import the steel surface image collected by the industrial camera into the final steel surface defect classification and recognition model, complete the automatic recognition and classification of steel surface defects, and output the recognition and classification results.

9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the visual recognition method based on steel surface defect classification as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the visual recognition method based on steel surface defect classification according to any one of claims 1 to 7.

Images