Intelligent color recognition device and method used in waste glass sorting process

Abstract

In order to solve the problem of intelligent color recognition in the waste glass sorting process, a convolutional neural network color recognition algorithm based on deep learning and embedded with spatial information is invented in a waste glass image collecting and sorting system, intelligent color recognition in the waste glass sorting process is achieved, and the intelligent color recognition efficiency is improved. And an intelligent color identification technology is provided for quickly and accurately sorting out waste glass with different colors during waste glass sorting.

Description

Technical Field

[0001] The present invention relates to an intelligent color recognition control system used in the waste glass sorting process, which mainly performs intelligent recognition and screening according to the color and particle size, and belongs to the glass screening technology in the fields of waste glass crushing and remanufacturing. Background Art

[0002] Austrian Pentax AG has developed a cullet color sorting system based on the CLARITY optical sorting machine. While removing metals and stones mixed in cullet, it also uses a camera system to identify the color spectrum to complete the specific color sorting of glass, but the camera performance is not high and the color sorting accuracy is not high. Japan's Sumitomo Metal Industries, Ltd. has developed equipment that can distinguish between green, brown, and white glass bottles and glass fragments, but the sorting accuracy of cullet is poor and the sorting speed is slow. Later, Japan's Kurashiki Bobo also developed equipment for glass bottle color sorting, which has improved the recognition accuracy to a certain extent and can distinguish between white, green, and blue glass bottles, but the color sorting effect of cullet is still poor.

[0004] Many companies in my country have begun to independently develop broken glass color sorters and have made some progress, but there are not many studies on broken glass color sorting equipment and its intelligent color recognition technology. In 2013, Zhong Chunhong and others designed a colored and colorless glass sorting device. According to the different characteristics of the reflected light due to the different colors of the glass, the standard color plate was compared to distinguish between colored glass and colorless glass. The sorting accuracy of the device is not high. It can only distinguish between colored and colorless, but cannot distinguish the specific color of the glass. In 2019, Li Danyang designed a waste glass recycling machine, which mainly designed a breaking and sorting mechanism to quickly break the glass bottles, and then sort the bottom of the bottle and the fragments, and did not sort the color.

[0005] Image recognition is to distinguish objects with known rules. The traditional image recognition process is divided into four steps: image acquisition, image preprocessing, feature extraction, and image recognition. In recent years, deep learning has developed rapidly, and the accuracy of image target recognition and classification in machine vision has been continuously improved. Now deep learning algorithms have been widely used in target detection. In 1994, Liao et al. used machine vision to study the classification of yellow and white corn based on color features, and extracted features in HIS and RGB color spaces respectively. The results showed that the accuracy of feature extraction in HIS space is higher than that in RGB space.

[0006] In 2016, Zhang Ziran and others used a spectrophotometer to extract CIE chromaticity values ​​and other parameters, and used the SVM model to classify the color-coated reinforced glass data, sorting out glass with different color differences with an accuracy rate of more than 90%. However, they only proposed an algorithm, and did not implement the relevant system.

[0007] The neural network based on deep learning automatically learns and extracts features from the input images. There is no need for manual feature extraction of the input images before sending them to the classifier for training, which reduces the error of manual operation. Therefore, it can well meet the high precision and high reliability of target detection. The deep learning algorithm for color recognition is still in its infancy.

[0008] At present, the color sorter for broken glass on the market can only sort one color. This design extracts the HIS feature space of waste glass according to the physical properties of waste glass, combines the convolutional neural network algorithm based on deep learning, and uses the SVM model to design an intelligent recognition technology for waste glass color recognition, realizing the intelligent color recognition and sorting in the waste glass sorting process, which has wide application value in improving the efficiency and sorting accuracy of the waste glass sorting system, thereby reducing costs. Summary of the invention

[0009] The present invention is to solve the problem of intelligent color recognition in the waste glass sorting process, and the intelligent color recognition technology in the waste glass sorting process is designed to quickly and accurately sort out waste glass of different colors during waste glass sorting. The technical solution adopted by the present invention is:

[0010] As a first aspect, the present invention provides an intelligent color recognition method for waste glass sorting, the steps of the method are as follows:

[0011] Step 1: collect images, and use the YOLOv3 network to detect, capture and save the target area of ​​the original image and segment the original image;

[0012] Step 2, analyzing the RGB three-color values ​​of the segmented image information to obtain the color histogram of each RGB component and form an RGB color space;

[0013] Step 3, convert the RGB color space into the HSV color space, and extract the H, S, and V three-channel color torches respectively;

[0014] Step 4, importing the RGB color space information and the HSV color space information into the feature extraction subnetwork of the SVM model to extract the combined color features of the collected image;

[0015] Step 5: Import the combined color features of the collected image into the classification subnetwork of the SVM model to obtain the color recognition and classification results of the collected image.

[0016] In combination with the above scheme, in any one of the schemes, preferably, the feature extraction subnetwork includes two feature extraction models based on convolutional neural networks, namely, an RGB feature extraction subnetwork and an HSV feature extraction subnetwork. The RGB feature extraction subnetwork and the HSV feature extraction subnetwork learn and extract the color features of the original image layer by layer. The feature extraction subnetwork includes five convolutional layers {C1, C2, C3, C4, C5}. The first convolutional layer includes 192 11×11×3 three-dimensional convolution kernels with a convolution step of 4; the second layer includes 256 5×5×192 convolution kernels; the third, fourth, and fifth layers include 256 3×3×256 convolution kernels;

[0017] Among them, the RGB feature extraction subnetwork has 4 convolutional layers, which uses the method of cascading low-dimensional features and high-dimensional features as the final classification features to extract and combine the color features of the input data;

[0018] For the output feature map of the first convolutional layer, it is first passed through a 2*2 maximum pooling layer, and then the pooling layer result is divided into five parts. For the output feature maps of the second and third convolutional layers, the output feature map is divided into five parts. The maximum pooling layer result and the division result are respectively passed through the average pooling layer corresponding to their sizes to obtain 6 feature maps of 1*1*channels size. These 6 feature maps are cascaded to obtain the final feature representation of each layer; the feature map is composed of five parts from left to right and from top to bottom, namely part1, part2, part3, part4, and part5. In the first four parts, each part completely covers the entire feature map without any overlap. In the middle of the feature map, the fifth part of the preset size is taken, and used together with the first four parts as the feature extraction area.

[0019] In combination with the above scheme, in any one of the schemes, preferably, the classification subnetwork includes three fully connected layers {fc6, fc7, fc8}, and the number of neural units in the three fully connected layers is 1024, 256 and 8 respectively; the first two fully connected layers contain 4096 neurons, and the last layer contains 8 neurons, which is consistent with the number of categories; wherein the outputs of the last three layers are used as color features for color recognition, the segmented image is divided into 4 sub-regions, and then the original image and the sub-regions are scaled to a fixed scale, and for each input image, each layer of the convolutional neural network can generate a response map.

[0020] In combination with the above schemes, in any of the schemes, preferably, the SVM model adds an aggregation layer to downsample the output results of the first, second and fifth layers, uses the maximum value or average value of the features in a certain area of ​​the image to reduce the size of the features, and reduces the length and width of the output feature map to half of the original.

[0021] In combination with the above scheme, in any one of the schemes, preferably, the convolution layer obtains a feature map of the image by performing a convolution operation on each sub-region of the input image through a linear filter on the image, and adding an offset and a nonlinear function. In the convolution operation, each convolution template can obtain a feature map h k , the feature map is composed of weights W k and b k It is calculated on the input of the previous layer, and the tanh function nonlinear transformation is added to the result of convolution. The calculation method is:

[0022]

[0023] The weight W can be expressed as a 4-dimensional tensor, where each dimension of the tensor is the length and width of the feature map of the input image and the output feature map, and the offset b represents the offset of each element in the output feature map.

[0024] In combination with the above schemes, in any of the schemes, preferably, the feature extraction subnetwork is a forward feedback network, which is trained using a standard backward propagation algorithm, and the weights are updated using a stochastic gradient descent method, and the formula is as follows:

[0025]

[0026] Among them, η represents the learning step size, and C represents the loss function;

[0027] The output of the last convolutional layer is stretched into a feature vector and then used as the input of the fully connected layer in the classification subnetwork of the SVM model.

[0028] In combination with the above schemes, in any of the schemes, preferably, each image is divided into two layers in the manner of a spatial pyramid, and then the color features of all sub-regions and the entire image are spliced ​​together, and the spliced ​​color features are expressed as the color information of the entire image.

[0029] In combination with the above solutions, in any one of the solutions, preferably, the training of the SVM model is:

[0030] Extract the image color feature vector of the collected image, and then obtain the regional data set that can be used for training and testing through manual color annotation;

[0031] Optimize the penalty factor and function of the SVM model through algorithms;

[0032] Run the SVM multi-classification model training to determine whether its model parameters have reached the optimal solution. If so, output the optimal SVM model. Otherwise, continue to optimize the penalty factor and function of the SVM model through the algorithm.

[0033] As a second aspect, an intelligent color recognition device for waste glass sorting, the device comprising a vibrating feeder, a conveyor belt, an intelligent color recognition device, a storage channel for waste glass of different colors, a sorting nozzle, and a controller, wherein the intelligent color recognition device comprises a light source, an image acquisition device, and an image analysis device;

[0034] According to the material conveying direction, they are arranged in sequence as follows: a vibrating feeder, a light source, an image acquisition device, and a sorting nozzle. The light source and the image acquisition device are distributed on the conveyor belt, and the sorting nozzle and the storage channels for waste glass of different colors are distributed on opposite sides of the conveyor belt;

[0035] A signal control connection is established between the controller and the vibration feeder, the conveyor belt, the intelligent color recognition device, and the sorting nozzle, and a data transmission connection is established between the image acquisition device and the image analysis device.

[0036] As a third aspect, a data processing device includes a central processing unit, a readable memory, and a data bus, characterized in that the central processing unit is data-connected to the readable memory via the data bus, and the central processing unit controls the readable memory according to operating instructions, thereby executing any one of the above methods.

[0037] Beneficial effects of the present invention

[0038] The present invention makes comprehensive use of the advantages of RGB and HSV color spaces and uses the features of the two color spaces to recognize the color of glass. Before formally using the RGB features of the input image and the HSV features to recognize the color of glass, the feature extraction processes of the two different color spaces of the image are pre-trained using the same training data set to ensure that the two feature extraction subnetworks have certain and directional feature learning capabilities, reduce the blindness of color model training, and accelerate the convergence speed of the network.

[0039] A deep learning-based convolutional neural network color recognition algorithm was invented that embeds spatial information. The spatial information can describe the structural information of the glass and improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a structural schematic diagram of the device of the present invention;

[0042] Figure 2 It is the image color feature extraction flow chart of the present invention;

[0043] Figure 3 It is the optimal SVM model acquisition flow chart of the present invention;

[0044] Figure 4 It is a schematic diagram of a waste glass color recognition framework based on a deep neural network of the present invention;

[0045] Figure 5 It is a schematic diagram of the HSV color space model of the present invention;

[0046] Figure 6 It is a schematic diagram of the classification sub-network structure of the present invention;

[0047] Figure 7 It is a schematic diagram of the overall framework of the method of the present invention;

[0048] Figure 8 It is a schematic diagram of the structure of the deep convolutional network of the present invention;

[0049] Fig. 9 It is a schematic diagram of the operation of the convolutional layer of the present invention;

[0050] Fig.10 is a schematic diagram of the operation of the polymeric layer of the present invention;

[0051] Fig.11 It is a schematic diagram of the method for embedding spatial information of the present invention.

[0052] The attached drawings are marked with: 1-vibrating feeder, 2-conveyor belt, 3-intelligent color recognition device, 4-storage channel for waste glass of different colors, 5-sorting nozzle. DETAILED DESCRIPTION

[0053] The above contents of the present invention are further described in detail below through examples, but this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples, and all technologies implemented based on the above contents of the present invention belong to the scope of the present invention.

[0054] The present invention is to solve the problem of intelligent color recognition in the waste glass sorting process, and the intelligent color recognition technology in the waste glass sorting process is designed to quickly and accurately sort out waste glass of different colors during waste glass sorting. In order to solve the above technical problems, the structural schematic diagram of the waste glass sorting device adopted by the present invention is shown in Figure 1As shown. The device includes a vibrating feeder 1, a conveyor belt 2, an intelligent color recognition device 3, a storage channel for waste glass of different colors 4, a sorting nozzle 5, and a controller, wherein the intelligent color recognition device includes a light source, an image acquisition device, and an image analysis device. According to the material conveying direction, they are arranged in sequence: a vibrating feeder 1, a light source, an image acquisition device, and a sorting nozzle 5. The light source and the image acquisition device are distributed on the conveyor belt, and the sorting nozzle and the storage channel for waste glass of different colors are distributed on opposite sides of the conveyor belt. A signal control connection is established between the controller and the vibrating feeder, the conveyor belt, the intelligent color recognition device, and the sorting nozzle, and the image acquisition device is connected to the image analysis device for data transmission. According to Figure 1 Schematic diagram, waste glass fed by a vibrating feeder 1 to a conveyor belt 2 is identified by an intelligent color recognition device 3. According to different colors of waste glass, a controller controls different sorting nozzles 5 to blow waste glass of different colors into a storage channel 4 of corresponding colors for storage.

[0055] For the waste glass image collected by the image acquisition device 4, after image segmentation, the RGB three-color values ​​of the segmented single glass image are analyzed by software to obtain the color histogram of each RGB component, and the RGB color space is converted to the HSV color space, and the H, S, and V three-channel color moments are extracted respectively, and then the eigenvalues ​​are combined to obtain the HSV space eigenvector value of the collected image (see the process of obtaining the HSV space eigenvector value of the image). Figure 2 ), after obtaining the HSV spatial feature vector value of the waste glass image, the neural network intelligent algorithm is used to perform SVM (Support Vector Machine, SVM) training.

[0056] The technical requirements that the image acquisition device 4 required in the present invention needs to meet are: a resolution of 1920*1080 pixels or above, automatic shooting under the control of the controller 1, continuous image acquisition under the control of the controller 1, and the ability to adapt to more complex environments.

[0057] The image acquisition in the present invention is performed by an image acquisition device, which acquires the image of the waste glass under the illumination of a circular light source, and performs image processing and recognition through an image analysis device.

[0058] The waste glass image collected by the image acquisition unit in the present invention needs to be processed by image interpolation algorithm. The image acquisition device 4 in the present invention adopts a CMOS photosensitive device, and its photosensitive substrate is covered with a color filter array. The color filter array samples only one of the three color components R, G, and B at each pixel, cross-samples the values ​​of the B and G color components in even rows and cross-samples the values ​​of the G and R color components in odd rows. In this way, different color components are sampled between adjacent pixels. Figure 6Here R, G, and B represent the color components sampled at this point.

[0059] According to the above technical scheme, the color recognition algorithm technology involved in the present invention is a spatial neural network algorithm based on deep learning to perform color recognition of waste glass. The deep learning method processes the features of the input data, and adaptively filters, extracts and combines the low-dimensional features of the input data at multiple levels to obtain high-dimensional abstract features for representing the input data, thereby obtaining classification or recognition results.

[0060] The principle of the algorithm is: by constructing two feature extraction models RGB Net and HSVNet based on convolutional neural networks to learn and extract the color features of waste glass layer by layer, and then constructing a classification network structure model, the input image is classified according to the color features obtained in the previous process, and the color classification labels of waste glass in natural scenes are obtained.

[0061] The overall framework of the waste glass color recognition process based on deep learning is shown in Figure 4 First, the YOLOv3 network is used to detect, capture and save the target area of ​​the original scene graph, and then the input data that can be used for the color recognition model is obtained through the manual color labeling process; after the manual color labeling step, a regional data set that can be used for training and testing is obtained; the training data set is passed through the two feature extraction sub-networks of MultiColor-Net, RGB Net and HSV Net, to learn the feature representation for classification; after combining the RGB features and HSV features corresponding to each input image, they are sent to the classification sub-network for the final color recognition step to obtain the color classification result.

[0062] The present invention first needs to preprocess the input image data. After using YOLOv3 to detect the target area, only image data without any corresponding labels is obtained. It is also necessary to manually annotate the obtained sample data with corresponding color information to obtain complete input data.

[0063] The image obtained by the image acquisition device of the present invention is RGB image data, and is converted into an HSV color space model. Compared with the RGB color space that describes colors in three color components, the HSV color space model divides colors into three dimensions: hue, saturation, and brightness. That is, it describes colors in a way that is easier for humans to imagine, such as color, depth, light and dark, and is closer to the way humans perceive colors. The model of the HSV color space is as follows Figure 5As shown in the figure, it consists of three coordinate axes: hue H, saturation S, and lightness V. The vertical direction is the V direction, and the range is [0,1]. The left boundary 0 represents black, and the process of changing from 0 to 1 corresponds to the process of the image becoming brighter and brighter. When the value reaches 1 at the right boundary, the image appears white as a whole; the rotation angle H determines the hue, and the range is [0°, 360°]; the radial dimension S represents the color saturation, and the range is [0,1]. The larger the value, the higher the color purity.

[0064] The present invention makes comprehensive use of the advantages of RGB and HSV color spaces and uses the features of the two color spaces to recognize the color of glass. Before formally using the RGB features of the input image and the HSV features to recognize the color of glass, the feature extraction processes of the two different color spaces of the image are pre-trained using the same training data set to ensure that the two feature extraction subnetworks have certain and directional feature learning capabilities, reduce the blindness of color model training, and accelerate the convergence speed of the network.

[0065] According to the above technical solution, the structure of the glass color recognition neural network model in the end-to-end real-time monitoring scenario used by the present invention is Figure 6 As shown in the figure. The input data are the RGB image of glass in natural scenes and the corresponding HSV image, and the output data are the model prediction results of the color of waste glass in the figure. The model mainly includes two parts: feature extraction subnetwork and classification subnetwork. The specific design of each part is as follows:

[0066] ①Feature extraction sub-network

[0067] The RGB feature extraction subnetwork (RGB Net) mainly has 4 convolutional layers, and the size, number, moving step and other parameters of the convolution kernel used in each convolutional layer. As the more convolutional layers the image passes through, the higher the feature dimension extracted and the more abstract it is, the more information contained in the low-dimensional feature map (Feature Map) will be lost. Therefore, the model uses the method of cascading low-dimensional features that retain more original information and more refined high-dimensional features as the final classification features to extract and combine the color features of the input data to ensure the integrity and comprehensiveness of the feature information.

[0068] In the feature extraction subnetwork, for the output feature map of the first convolutional layer, the method of first passing it through a 2*2 maximum pooling layer and then dividing the pooling layer result into five parts was adopted; for the output feature maps of the second and third convolutional layers, two processing methods were adopted: dividing the feature map into five parts and directly passing the feature map through a 2*2 maximum pooling layer. The maximum pooling layer result and the division result are respectively passed through the average pooling layer corresponding to their sizes, and 6 feature maps of 1*1*channels size will be obtained. These 6 feature maps are cascaded to obtain the final feature representation of each layer. The feature map is divided into five parts from left to right and from top to bottom: part1, part2, part3, part4, and part5. In the first four parts, each part can actually completely cover the entire feature map without any overlap. However, due to the characteristics of the convolution calculation method, the pixels at the edge cannot be utilized to the maximum extent. For the task of glass color recognition, the central area of ​​the feature map is different from the edge pixels that may belong to the background. Instead, it represents the glass color and has a very important influence on the judgment of the glass color. Therefore, a fifth part of a certain size is taken in the middle of the feature map and used together with the first four parts as the feature extraction area.

[0069] ②Classification sub-network

[0070] The fully connected layer can map the distributed feature representation learned by convolution and other operations back to the original sample label space. This model will use three fully connected layers as the color classification subnetwork to perform the final color classification work. The number of neural units in the three fully connected layers is 1024, 256, and 8 respectively.

[0071] like Figure 7 As shown, the present invention uses a convolutional neural network to generate color features corresponding to each image, and then uses SVM as a classifier to predict the color of a given glass image. In the process of learning features, the glass image is divided into 4 sub-areas, and then the original image and the sub-areas are scaled to a fixed scale. In the training process, given the training image and the corresponding annotation, the parameters in the network are continuously updated in an iterative manner through backward transfer. After the network training is completed, all parameters and the structure of the convolutional neural network are saved for generating features. For each input image, each layer of the convolutional neural network can generate a response map.

[0072] According to the above technical solution, the present invention uses the output of the last three layers of the network as features for color recognition. In order to add spatial information, each image is divided into two layers in the form of a spatial pyramid, and then the features of all sub-regions and the entire image are spliced ​​together, and the spliced ​​features are expressed as the color information of the entire image.

[0073] like Figure 7 As shown, according to the overall framework of the convolutional neural network in the present invention, firstly, given training data and corresponding annotation information, the parameters of the network are trained, and then the trained network is used to output features for each image. Finally, a SVM classifier is trained to predict the given glass features to obtain the final color judgment. Spatial information can describe the structural information of the glass and improve the accuracy of recognition.

[0074] like Figure 8 As shown in FIG. 1 , the present invention adopts a deep convolutional network structure to learn features. In the deep convolutional network structure, the output of each layer can be used as the learned feature. The entire network includes five convolutional layers {C1, C2, C3, C4, C5} and three fully connected layers {fc6, fc7, fc8} (see FIG. 1 ). Figure 7 ). The first convolutional layer contains 192 11×11×3 convolution kernels with a convolution step of 4. The second layer contains 256 5×5×192 convolution kernels. The third, fourth, and fifth layers contain 256 3×3×256 convolution kernels. The first two fully connected layers contain 4096 neurons, and the last layer contains 8 neurons, which is the same number as the number of categories. At the same time, in order to reduce the complexity of the calculation, an aggregation layer is added to downsample the output results of the first, second, and fifth layers, reducing the length and width of the output to half of the original. By adding a nonlinear transformation layer of ReLU, the network can fit any function. At the same time, the local normalization layer can normalize the output amplitude and improve the results of network training. The following introduces the various layers in the deep convolutional network.

[0075] According to the above technical solution, the function of the convolution layer in the present invention is to obtain a feature map ( Fig. 9 shows a schematic diagram of the convolution operation). In other words, the feature map of the image is obtained by convolution operation of the linear filter on the image, plus the offset and nonlinear function. In the convolution operation, each convolution template can obtain a feature map. Assume that the feature map of a given layer is h k , and this feature map is composed of weights W k and b k It is calculated on the input of the previous layer. A nonlinear transformation is added to the result of convolution, usually using the tanh function, and the calculation method is:

[0076]

[0077] In order to obtain richer data expressions, more convolution kernel functions can be selected to obtain more feature maps. (k) ,k=0,...,K}. The weight W can be expressed as a 4-dimensional tensor, where each dimension of the tensor is the length and width of the feature map of the input image and the feature map of the output. The offset b represents the offset of each element in the output feature map.

[0078] According to the above technical solution, the purpose of the ReLU layer (abbreviation of Rectified Linear Units) of the present invention is to perform nonlinear transformation on the previous output. ReLU is an approximate description of the function f(x)=log(1+exp(x)). There are other functions that can perform nonlinear transformation on the input, including tanh(x), |tanh(x)| and Sigmoid(x) functions. Compared with these nonlinear functions, ReLU can increase the speed several times during training, and can also reduce the problem of gradient disappearance or explosion.

[0079] In order to enhance the offset invariance of the input features, the aggregation layer uses the maximum or average value of the features in a certain area of ​​the image to reduce the size of the features, ensuring that the same features can be calculated for images with smaller offsets. The operation of the aggregation layer is similar to the operation of the convolution layer, which operates on the sub-areas of the image separately to obtain the average or maximum value ( Fig.10 (a) is a schematic diagram of the operation of the aggregation layer. Aggregation only changes the length and width of the feature map, but does not change the number of feature maps. Fig.10 What is shown is the aggregation of the input feature map in 2×2 regions, and then the length and width of the output feature map are half of the input.

[0080] Since the fully connected layer contains most of the parameters of the entire network, it is easily affected by overfitting during training. In order to reduce overfitting and greatly improve the time efficiency of training, the Dropout method is used. During the training process, Dropout assigns each neuron a probability value of switching, that is, at each iteration, half of the neurons are turned off and no output is made during the forward and backward passes. The biggest contribution of the Dropout method is that although 2 neurons are generated during training, n There are a total of 2 possible network structures, but during testing, only one network structure is needed to output the probability value of each class. Assuming that each neuron has a probability of 0.5 being closed during training, and each neuron is in an open state during testing, the output of this layer will be multiplied by 0.5 as the final output.

[0081] The deep convolutional network used in the present invention is a typical forward feedback network, which can be trained using a standard backward propagation algorithm. The weights can be updated using the stochastic gradient descent method, as shown in the following formula:

[0082]

[0083] Among them, η represents the learning step size and C represents the loss function.

[0084] The input of the network is an RGB image of 227×227×3. The output feature sizes of each convolutional layer are 55×55×96, 27×27×256, 13×13×384, 13×13×384, and 6×6×256, respectively. It is worth noting that the max aggregation after some convolutional layers can reduce the feature size of the convolutional layer output, thereby reducing the feature calculation time. The output of the last convolutional layer is stretched into a feature vector and then used as the input of the subsequent fully connected layer. After the convolutional neural network training is completed, the method of the present invention selects the output of the last three layers of the entire network as features. The reason why the outputs of other layers are not selected is that the previous four layers are not distinguishable enough, and the feature dimensions of these layers are large, which requires more resources when training the SVM classifier.

[0085] The present invention embeds spatial information in order to add spatial information to image features to improve classification accuracy. Fig.11 Shown is a method for embedding spatial information in deep learning. Fig.11 (a) shows that the features of each sub-region are extracted through the spatial pyramid model, and then the spatial information is added to the description of the image by stitching all the sub-region features together. Fig.11 (b) indicates that the four divided sub-regions are first scaled to the size of the full image, and then merged into an image matrix with a dimension size of 15, and then the features of the merged image matrix are extracted through the network. Fig.11 The advantage of (b) is that it can include the features of spatial information with only one calculation. Fig.11(a), the whole image and each sub-region need to be calculated, and a total of 5 network feature extraction processes are required. It should be noted here that due to the different dimensions of the input data, there are slight differences in the deep network structures used in the two spatial information embedding methods. The difference between the two networks is that the sizes of the first-layer convolution kernels are 11×11×3 and 11×11×15, respectively. Unlike the features based on the bag-of-words model, the method of the present invention obtains the features of each region by learning a good convolutional neural network. The bag-of-words model feature is a feature in the form of a histogram that encodes the original feature through a dictionary. The color feature of the present invention is a response map obtained by the output of the middle layer of the convolutional neural network as a feature. In order to be able to use local features to describe the image, the local features need to be converted into a feature vector through an aggregation function. In the process of aggregation, some local features are ignored and do not contribute to the final feature vector. The features learned by the method of the present invention take into account each pixel, so that the color is more descriptive and better distinguishability is obtained.

[0086] It should be understood that the above embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In addition, it should be understood that after reading the content of the present invention, those skilled in the art can make various changes and modifications to the present invention, and these equivalent forms also fall within the scope limited by the appended claims of the present application.

Claims

1. An intelligent color recognition method for waste glass sorting, characterized in that: The steps of the method are as follows: Step 1: collect images, and use the YOLOv3 network to detect, capture and save the target area of ​​the original image and segment the original image; Step 2, performing RGB tricolor value analysis on the segmented image information to obtain a color histogram of each RGB component to form an RGB color space; Step 3, convert the RGB color space into the HSV color space, and extract the H, S, and V three-channel color torches respectively; Step 4, importing the RGB color space information and the HSV color space information into the feature extraction subnetwork of the SVM model to extract the combined color features of the collected image; Step 5: Import the combined color features of the collected image into the classification subnetwork of the SVM model to obtain the color recognition and classification results of the collected image.

2. The intelligent color recognition method for waste glass sorting according to claim 1 is characterized in that: The feature extraction subnetwork includes two feature extraction models based on convolutional neural networks, namely, an RGB feature extraction subnetwork and an HSV feature extraction subnetwork. The RGB feature extraction subnetwork and the HSV feature extraction subnetwork learn and extract the color features of the original image layer by layer. The feature extraction subnetwork includes five convolutional layers {C1, C2, C3, C4, C5}. The first convolutional layer includes 192 11×11×3 convolution kernels with a convolution step of 4; the second layer includes 256 5×5×192 convolution kernels; the third, fourth, and fifth layers include 256 3×3×256 convolution kernels; Among them, the RGB feature extraction subnetwork has 4 convolutional layers, which uses the method of cascading low-dimensional features and high-dimensional features as the final classification features to extract and combine the color features of the input data; For the output feature map of the first convolutional layer, it is first passed through a 2*2 maximum pooling layer, and then the pooling layer result is divided into five parts. For the output feature maps of the second and third convolutional layers, the output feature map is divided into five parts. The maximum pooling layer result and the division result are respectively passed through the average pooling layer corresponding to their sizes to obtain 6 feature maps of 1*1*channels size. These 6 feature maps are cascaded to obtain the final feature representation of each layer; the feature map is composed of five parts from left to right and from top to bottom, namely part1, part2, part3, part4, and part5. In the first four parts, each part completely covers the entire feature map without any overlap. In the middle of the feature map, the fifth part of the preset size is taken, and used together with the first four parts as the feature extraction area.

3. The intelligent color recognition method for waste glass sorting according to claim 2 is characterized in that: The classification subnetwork includes three fully connected layers {fc6, fc7, fc8}, the number of neural units in the three fully connected layers are 1024, 256 and 8 respectively; the first two fully connected layers contain 4096 neurons, and the last layer contains 8 neurons, which is consistent with the number of categories; the outputs of the last three layers are used as color features for color recognition, the segmented image is divided into 4 sub-regions, and then the original image and the sub-regions are scaled to a fixed scale. For each input image, each layer of the convolutional neural network can generate a response map.

4. The intelligent color recognition method for waste glass sorting according to claim 3 is characterized in that: The SVM model adds an aggregation layer to downsample the output results of the first, second and fifth layers, and uses the maximum or average value of the features in a certain area of ​​the image to reduce the size of the features, reducing the length and width of the output feature map to half of the original.

5. The intelligent color recognition method for waste glass sorting according to claim 4 is characterized in that: The convolution layer obtains the feature map of the image by convolution operation on each sub-region of the input image through a linear filter, and adds an offset and a nonlinear function. In the convolution operation, each convolution template can obtain a feature map h k , the feature map is composed of weights W k and b k It is calculated on the input of the previous layer, and the tanh function nonlinear transformation is added to the result of convolution. The calculation method is: The weight W can be expressed as a 4-dimensional tensor, where each dimension of the tensor is the length and width of the feature map of the input image and the output feature map, and the offset b represents the offset of each element in the output feature map.

6. The intelligent color recognition method for waste glass sorting according to claim 5 is characterized in that: The feature extraction subnetwork is a forward feedback network, which is trained using the standard backward propagation algorithm. The weights are updated by the stochastic gradient descent method. The formula is as follows: Among them, η represents the learning step size, and C represents the loss function; The output of the last convolutional layer is stretched into a feature vector and then used as the input of the fully connected layer in the classification subnetwork of the SVM model.

7. The intelligent color recognition method for waste glass sorting according to claim 6 is characterized in that: Each image is divided into two layers in the form of a spatial pyramid, and then the color features of all sub-regions and the entire image are spliced ​​together, and the spliced ​​color features are used as the color information of the entire image.

8. An intelligent color recognition method for waste glass sorting according to any one of claims 1 to 7, characterized in that: The training of the SVM model is: Extract the image color feature vector of the collected image, and then obtain the regional data set that can be used for training and testing through manual color annotation; Optimize the penalty factor and function of the SVM model through algorithms; Run the SVM multi-classification model training to determine whether its model parameters have reached the optimal solution. If so, output the optimal SVM model. Otherwise, continue to optimize the penalty factor and function of the SVM model through the algorithm.

9. An intelligent color recognition device for waste glass sorting, characterized in that: The device includes a vibrating feeder, a conveyor belt, an intelligent color recognition device, storage channels for waste glass of different colors, a sorting nozzle, and a controller, wherein the intelligent color recognition device includes a light source, an image acquisition device, and an image analysis device; According to the material conveying direction, they are arranged in sequence as follows: a vibrating feeder, a light source, an image acquisition device, and a sorting nozzle. The light source and the image acquisition device are distributed on the conveyor belt, and the sorting nozzle and the storage channels for waste glass of different colors are distributed on opposite sides of the conveyor belt; A signal control connection is established between the controller and the vibration feeder, the conveyor belt, the intelligent color recognition device, and the sorting nozzle, and a data transmission connection is established between the image acquisition device and the image analysis device.

10. A data processing device, comprising a central processing unit, a readable memory, and a data bus, characterized in that: The central processing unit is data-connected to the readable memory via a data bus, and the central processing unit controls the readable memory according to operation instructions, thereby executing the method described in any one of claims 1 to 8.

Images