A method for sorting marine module steel structure excess material

Abstract

The application discloses a kind of marine module steel structure scrap classification methods, by irregular steel structure scrap marking category, and each class is coded, and the image containing the whole outer contour of steel structure scrap is obtained by shooting to preprocess scrap image, the probability vector of the category to which the preprocessed steel structure scrap image belongs is obtained by the neural network model established, and the category to which the steel structure scrap belongs is judged according to the probability vector of the category.The method of the application classifies steel structure scrap image by CCD vision technology combined with convolutional neural network Resnet algorithm, and classifies scrap according to the classification result obtained by the algorithm, and is short in time, efficient and low in cost.

Description

Technical Field

[0001] This invention relates to the field of image classification algorithms, and specifically to a classification method for managing surplus steel structure materials based on image recognition technology. Background Technology

[0002] During the construction of marine modules, various shapes and sizes of steel structure scraps are typically generated. These scraps need to be classified and managed to increase the material utilization rate of the steel structures. Currently, the classification process mostly relies on manual processing and recording, which has a low degree of automation and is prone to errors. Secondly, the steel structure scraps in marine engineering are characterized by their large volume and heavy weight, which also increases the difficulty of manual sorting. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a highly efficient and low-cost method for classifying surplus materials in marine modular steel structures.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] The present invention provides a method for classifying surplus steel materials in marine modular structures, comprising the following steps:

[0006] Step 1: Based on the shape and size of the steel structure scrap, classify the steel structure scrap into 6 categories. The specific method is as follows: Calculate the area of ​​the largest surface of each piece of steel structure scrap, and classify the steel structure scrap into categories with an area S ≥ 3m based on the area size S and shape. 2 Rectangular steel structure scraps, 1.5m 2 ≤S<3m 2 Rectangular steel structure scrap, 0.5m 2 ≤S<1.5m 2 Rectangular steel structure scrap, S≥3m 2 Irregular steel structure scraps, 1.5m 2 ≤S<3m 2 Irregular steel structure scraps, 0.5m 2 ≤S<1.5m 2 The irregular steel structure scraps were collected, and the images were labeled with categories and each category was coded.

[0007] Step 2: Use an industrial camera to photograph the outer contour of each piece of leftover steel structure material in the factory to obtain an image containing the entire outer contour of the leftover steel structure material. The image should be taken against a white background and saved in JPEG format.

[0008] Step 3: Preprocess all steel structure scrap images to diversify the steel scrap images and obtain the RGB values ​​of the preprocessed images;

[0009] Step 4: Create a dataset of images of preprocessed steel structure scraps. The dataset contains images of preprocessed steel structure scraps and their corresponding category labels.

[0010] Step 5: Establish a neural network model. Take a portion of the pre-processed steel structure scrap images as the input set, process the RGB of the images, and output the probability vector set of the image's category.

[0011] Step Six: Train the neural network model established in Step Five. Take another portion of the preprocessed image as input and input it into the neural network model of Step Five to obtain the probability vector of the image's category. By comparing the consistency between the probability vector and the encoding in Step One, calculate the accuracy of the calculation result of the neural network model to determine the training degree of the model. If it meets the requirements, continue to the next step. If it does not reach the set accuracy, return to Step Five to adjust the weight matrix of the neural network model. Repeat Step Five and Step Six until the accuracy of the calculation result meets the requirements.

[0012] Step 7: The conveyor belt starts, the industrial camera CCD starts, and sorting begins. The industrial camera takes pictures of the steel structure scraps at set time intervals so that each steel structure scrap can be captured with clear outline information. The industrial camera CCD then uploads the captured images of the steel structure scraps to the industrial control system.

[0013] Step 8: The industrial control system preprocesses the collected images of steel structure scrap materials using the method described in Step 3;

[0014] Step 9: Input the preprocessed steel structure scrap image obtained in Step 8 into the model trained in Step 5. The output set is the probability vector of the category to which the preprocessed steel structure scrap image belongs. Based on the probability vector of the category, the category to which the steel structure scrap belongs is determined.

[0015] The beneficial effects of this invention are as follows:

[0016] This invention provides a method for classifying steel structure scrap materials. It uses CCD vision technology combined with the ResNet convolutional neural network algorithm to classify steel structure scrap material images and achieves the classification of scrap materials based on the classification results obtained by the algorithm. The method is time-saving, efficient, and low-cost. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for classifying steel structure scraps according to an embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1The present invention provides a method for classifying surplus steel materials in marine modular structures, comprising the following steps:

[0020] Step 1: Based on the shape and size of the steel structure scrap, classify the steel structure scrap into 6 categories. The specific method is as follows: Calculate the area of ​​the largest surface of each piece of steel structure scrap, and classify the steel structure scrap into categories with an area S ≥ 3m based on the area size S and shape. 2 Rectangular steel structure scraps, 1.5m 2 ≤S<3m 2 Rectangular steel structure scrap, 0.5m 2 ≤S<1.5m 2 Rectangular steel structure scrap, S≥3m 2 Irregular steel structure scraps, 1.5m 2 ≤S<3m 2 Irregular steel structure scraps, 0.5m 2 ≤S<1.5m 2 The collected images are categorized and coded. For example, if the steel structure scrap belongs to category 1, it is coded as [1,0,0,0,0,0]; if it belongs to category 2, the coded code is [0,1,0,0,0,0], and so on.

[0021] Step 2: Use an industrial camera to photograph the outer contour of each piece of leftover steel structure material from the factory to obtain an image containing the entire outer contour of the leftover steel structure material. The image should be taken against a white background and saved in JPEG format.

[0022] Step 3: Preprocess all steel structure scrap images to diversify the images and obtain the RGB values ​​of the preprocessed images. The specific method is as follows:

[0023] Step 1: Use photo processing tools to crop each image of the steel structure scrap material, keeping only the scrap material portion, and resize each image of the steel structure scrap material to a size of 224×224 pixels;

[0024] The second step, to increase data diversity and prevent model overfitting, involves transforming the image's coordinates to obtain a transformed image. The specific method is as follows: rotate the image by +10° from (a, b) to (a1, b1), then flip it horizontally and vertically to transform the coordinates from (a1, b1) to (a2, b2). The formula is as follows:

[0025] Rotate the image by +10° to transform the coordinates from (a, b) to (a1, b1), using the following formula:

[0026] The coordinates are transformed from (a1, b1) to (a2, b2), and the formula is: [a2, b2] = [224-1-b1, 224-1-a1];

[0027] Third step: Import the image processed in step two into image processing software. The RGB value of each pixel in the image is R. α ,

[0028] G α B α Where α represents the image number of the steel structure scrap, α = 1, 2, ..., n. The image is normalized in the software by dividing the RGB values ​​of each pixel by 255, thus normalizing the RGB value range of each pixel from 0-255 to 0-1. The specific formula is as follows:

[0029] r α =R α / 255

[0030] g α =G α / 255

[0031] b α =B α / 255

[0032] Where, r α g α b α This represents the R, G, and B values ​​of each pixel after normalization, resulting in the preprocessed image and the image's RGB values.

[0033] Step 4: Create a dataset of images of preprocessed steel structure scraps. The dataset contains images of preprocessed steel structure scraps and their corresponding category labels.

[0034] Step 5: Establish a neural network model. Input a portion of the pre-processed steel structure scrap images as the input set, process the RGB values ​​of the images, and output a probability vector set for the image's category. The specific process is as follows:

[0035] The first step is to process each preprocessed image using a 3x3 convolution kernel. The convolution kernel moves across the preprocessed image, performing a dot product operation on the RGB channels to obtain a new feature map. The specific process can be as follows:

[0036] The size of the preprocessed input image is 224×224×3 (width×height×number of channels). 64 convolutional kernels of size 3×3×3 (width×height×number of channels) are used. The convolutional kernels slide one pixel at a time on the preprocessed image, starting from the top left corner (stride is 1). This will result in a new feature map with a size of (224-3+1)×(224-3+1)×64, or 222×222×64.

[0037] The second step is to use a 2×2 max pooling window with a stride of 2, slide it across the feature map, select a 2×2 region each time, and then take the maximum value in this region as a pixel value in the new feature map.

[0038] For a feature map of size 222×222×64 obtained after a convolutional layer, after 2×2 max pooling, the size of the feature map becomes (222÷2)×(222÷2)×64, that is, 111×111×64.

[0039] The third step involves a series of convolution and pooling operations to obtain a 111×111×64 feature map, which is then flattened into a one-dimensional vector X with a length of 111×111×64=788544.

[0040] The fourth step is to connect a one-dimensional vector X of length 788544 to 6 neurons, where 6 represents the classification category.

[0041] The formula is as follows:

[0042]

[0043] y is the initial output vector regarding the degree of correlation between image categories. j1 represents the j1st element of the vector, indicating the relevance of the image to the j1st category. W is the weight matrix. i1j1 The element in the i1th column and j1st row of the matrix represents the weight assigned to the i1th element of vector X in the j1st type of the image. i1 represents the index of the i1th element of the input vector X, and j1 represents the category label of the output vector.

[0044] Step 5: Convert the vector y obtained in step 4 into a probability vector y'. The specific formula is as follows:

[0045] y′=[y′1,y′2,,,y′ j1 ,,,y′6]

[0046]

[0047] Where e represents the natural index, y' j1Let y' be the j1-th element in the image category probability vector related to the image category. j1 Let represent the probability that the remaining material belongs to the i-th class, where y'1 + y'2 + ... + y'6 = 1.

[0048] Step Six: Train the neural network model established in Step Five. Input the preprocessed portion of the image (the part not used in Step Five) into the neural network model from Step Five to obtain the probability vector of the image's category. Calculate the accuracy of the neural model's calculation results by comparing the consistency between the probability vector and the encoding from Step One. Determine the model's training level. If it meets the requirements, continue to the next step. If the set accuracy is not reached, return to Step Five to adjust the weight matrix of the neural network model. Repeat Steps Five and Six until the calculation accuracy meets the requirements. The specific process for adjusting the neural network model can be as follows:

[0049] Step 1: Divide the dataset created in Step 4 into a training set, a validation set, and a test set. The dataset should be divided according to a ratio of 70% for training, 20% for validation, and 10% for testing.

[0050] The second step is to determine the number of training rounds to be 100 and set the batch size to 32. Train the model on the training set, adjusting the element values ​​of the weight matrix W based on the results during training to ensure that the neural network output is consistent with the encoding result from step one.

[0051] The third step, after training and validation, involves testing. The preprocessed image is input into the model obtained in step four, which outputs a probability vector y'. The term with the highest probability in y' is taken as y''. i , where i represents the i-th category;

[0052] Step 4: Calculate the accuracy of the test. If the output probability vector corresponds to the encoding result in step 1, the model prediction is correct. The model correctly predicts T samples and incorrectly predicts F samples. The accuracy P = T / (T+F). When the accuracy reaches 99.9%, the model training is complete.

[0053] Step 7: The conveyor belt starts, the industrial camera CCD starts, and sorting begins. The industrial camera takes pictures of the steel structure scraps at set time intervals so that each steel structure scrap can be captured with clear outline information. The industrial camera CCD then uploads the captured images of the steel structure scraps to the industrial control system.

[0054] Step 8: The industrial control system uses the method in Step 3 to preprocess the collected images of steel structure scrap.

[0055] Step 9: Input the preprocessed steel structure scrap image obtained in Step 8 into the model trained in Step 5. The output set is the probability vector y' = [y'1, y'2, ..., y'6] of the category to which the preprocessed steel structure scrap image belongs. Determine the category of the steel structure scrap based on the probability vector of the category. y' represents the output probability vector, and y'1, y'2, ..., y'6 represent the category.

[0056] Where: if max[y'1,y'2,,,y'6]≤0.6, then it is determined that the steel structure scrap does not belong to the six specified scrap types;

[0057] If y'1 > 0.6, the output type is Category 1;

[0058] If y'2 > 0.6, the output type is Category 2;

[0059] If y'3 > 0.6, the output type is Category 3;

[0060] If y'4 > 0.6, the output type is Category 4;

[0061] If y'5 > 0.6, the output type is category five;

[0062] If y'6 > 0.6, the output type is Category 6.

Claims

1. A method of classifying offal from a steel structure of a marine module, characterized in that It comprises the following steps: Step one, according to the shape and size of the steel structure scrap, the steel structure scrap is divided into 6 categories, the specific method is as follows: calculate the area of the largest face of each steel structure scrap, according to the area size S and shape, the steel structure scrap is divided into S≥3m 2 Rectangular steel structure scrap, 1.5m 2 ≤S<3m 2 Rectangular steel structure scrap, 0.5m 2 ≤S<1.5m 2 Rectangular steel structure scrap, S≥3m 2 Irregular steel structure scrap, 1.5m 2 ≤S<3m 2 Irregular steel structure scrap, 0.5m 2 ≤S<1.5m 2 Irregular steel structure scrap, mark the collected images respectively, and encode each category; Step two, using industrial camera to take pictures of the outer contour of each piece of factory steel structure scrap, obtaining images containing the entire outer contour of the steel structure scrap, using white background, and finally saving the images in JPEG format; Step three, preprocessing all steel structure scrap images to diversify the steel scrap images and obtain the RGB values of the preprocessed images; Step four, establishing a data set for the preprocessed steel structure scrap images, which contains the preprocessed steel structure scrap images and corresponding class labels; Step five, establishing a neural network model, inputting a part of the preprocessed steel structure scrap images as input set, processing the RGB of the picture, and outputting the probability vector set of the image's category; Step six, training the neural network model established in step five, inputting another part of the preprocessed images as input into the neural network model in step five, obtaining the probability vector of the image's category; by comparing the consistency of the probability vector and the encoding in step one, calculating the accuracy of the neural model's calculation result, judging the model training degree, if it meets the requirements, continue to the next step, if it does not reach the set accuracy, return to step five to adjust the weight matrix of the neural network model, repeat steps five and six until the calculation result accuracy meets the requirements; Step seven, start the conveyor belt, start the industrial camera CCD, start the classification, the industrial camera takes pictures of the steel structure scrap at a set time interval so that each steel structure scrap can be individually photographed to obtain clear contour information, and then the industrial camera CCD uploads the photographed steel structure scrap images to the industrial control system; Step eight, the industrial control system preprocesses the collected steel structure scrap images using the method in step three; Step nine, input the preprocessed steel structure scrap images obtained in step eight into the model trained in step five as input set, output the probability vector of the category of the preprocessed steel structure scrap image, and determine the category of the steel structure scrap according to the probability vector of the category.

2. The method of claim 1, wherein: The method for determining the category of the steel structure scrap in step nine is: If max[y'1, y'2, …, y'6] ≤ 0.6, it is determined that the steel structure scrap does not belong to the specified six types of scrap; y' represents the output probability vector, y'1, y'2, …, y'6 represent the category; If y'1 > 0.6, the output type is category one; If y'2 > 0.6, the output type is category two; If y'3 > 0.6, the output type is category three; If y'4 > 0.6, the output type is category four; If y'5 > 0.6, the output type is category five; If y'6 > 0.6, the output type is category six.

Images