A Neural Network-Based Image Stain Analysis Method for Sorting Machines

Abstract

This invention discloses a neural network-based image contamination analysis method for sorting machines, belonging to the field of sorting equipment technology. The method is as follows: A background removal algorithm is used on the original image to preserve the integrity of the material, retaining any contaminants that cannot be removed; images without contaminants are classified according to material characteristics and assigned different labels, while images with contaminants are grouped into one category and also assigned labels, thus constructing a dataset; a neural network with feature reuse and a large receptive field is designed and used to train the dataset; the model is deployed to the edge computing platform of the sorting machine, where the sorting machine provides images, and the processed image information is returned to the control board; after receiving the information from the edge computing platform, the control board summarizes the information and displays the material category and contamination level of the transmission channel on the client side. The method proposed in this invention can achieve accurate material classification and early warning of contamination levels in the transmission channel.

Description

Technical Field

[0001] This invention belongs to the field of sorting equipment technology, specifically relating to a method for analyzing image contamination in sorting machines based on neural networks. Background Technology

[0002] With the continuous development of deep learning technology, it has achieved great success in the field of image recognition. Using neural networks for image recognition is a highly efficient and accurate method, and many tasks that previously required human visual discrimination can now be replaced by machines equipped with image recognition capabilities. Sorting machines are equipped with fine-grained image recognition devices. Materials are transported to rollers via a feeding mechanism, and the rollers drive the material to rotate, capturing surface images of the materials through a camera. Neural networks then extract and identify features from the background-removed images to achieve material sorting. During machine operation, the conveyor channels are frequently used, leading to the accumulation of dirt. When this dirt accumulates to a certain level, subsequent background removal cannot completely remove the contaminants, and these residual contaminants affect the image data distribution, significantly impacting the subsequent prediction of material types using neural networks. Furthermore, without knowing the degree of contamination in the conveyor channels, frequent machine cleaning to achieve ideal sorting results results in significant time and resource consumption.

[0003] Sorting machines are currently mostly used for fine-grained classification of materials, where the differences between images of different categories of the same product are small. For example, the difference between an image of a fruit with a wormhole and one of a good fruit is just a small black dot. In the field of fine-grained image classification, neural networks such as ResNet, VGG, DenseNet, and EfficientNet have not achieved significant advantages. The main task of sorting machines is to distinguish the fine-grained nature of materials and complete the sorting. Due to the variable nature of customer site environments, edge computing devices are needed to replace computers for calculations in order to maintain equipment stability. However, the computational load of the aforementioned neural networks is too large to be deployed on mobile devices. Therefore, a lightweight new type of neural network is needed for application in the field of fine-grained image classification.

[0004] Current technologies employ traditional image recognition techniques to remove blemishes as thoroughly as possible. However, in practical applications, this technology lacks versatility and struggles to completely filter out blemishes. Summary of the Invention

[0005] (1) The technical problem to be solved by the present invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a neural network-based image contamination analysis method for sorting machines, thereby solving the following technical problems: the problem of reduced image recognition accuracy caused by contamination of the sorting machine's transport channel; the problem of fine-grained image classification of normal materials and materials affected by contaminants; and the problem of determining the degree of contamination of the transport channel and when it should be cleaned.

[0007] (2) The technical solution adopted in this invention

[0008] The method proposed in this invention can accurately classify materials and provide early warning of contamination levels in transport channels. The specific steps include the following:

[0009] Step S1: Preserve the integrity of the material by using a general background removal algorithm on the original image, and retain any blemishes that cannot be removed;

[0010] Step S2: Classify the unblemished images according to material characteristics and assign different labels, and group the blemished images into one category and assign labels to them, thus constructing the sample dataset required for training;

[0011] Step S3: Design a neural network with feature reuse and a large receptive field, and use the neural network to train the dataset in Step S2. Use the ONNX model to save the trained network weights.

[0012] Step S4: Deploy the model obtained in step S3 to the edge computing platform of the sorting machine. The operation of the sorting machine provides images to the edge computing platform, and the information after image processing is returned to the control board by the edge computing platform.

[0013] Step S5: After receiving the information returned by the edge computing platform, the control board summarizes the information and displays the material category and the degree of contamination of the transmission channel on the client. The summary results are used to control the mechanical device to sort materials, and the degree of contamination of the transmission channel is used to inform the client of the equipment's operating status.

[0014] Furthermore, step S2 includes the following steps:

[0015] S2.1: Assign labels to the collected material images according to their categories;

[0016] S2.2: Images with blemishes are uniformly labeled regardless of their material type.

[0017] Furthermore, step S3 includes the following steps:

[0018] S3.1: Design a lightweight neural network with depthwise separable convolution as the backbone and residual structures and large convolutional kernels in some positions on the backbone. The neural network includes an input layer, a hidden layer, an output layer and a softmax classification layer. The feature extraction layers before and after are connected by residual structures to realize the reuse of feature information. The receptive field is improved by a set of convolutional kernels with different sizes. These two structures can better fit fine-grained image datasets.

[0019] S3.2: Import the dataset into the neural network using the PyTorch framework for training. Use the SGD optimizer and cross-entropy loss function to update the neural network weights. Adjust the learning rate of SGD to make the neural network fit the dataset to the best.

[0020] S3.3: Save the neural network weights using the ONNX model after training is complete.

[0021] Furthermore, in the neural network, a 3*3 convolution is first used on the input image to increase the image dimension, and then a depthwise separable convolution is used to achieve preliminary image feature extraction. The intermediate feature extraction layer uses a set of convolution kernels of different sizes. The large convolution kernel captures high-resolution features, and the small convolution kernel captures low-resolution features. The feature map channels are divided in a ratio of 4:2:1:1. The small convolution kernel convolves more feature maps to reduce the amount of computation. After three sets of convolution kernels extract features, the high-dimensional information of the image is fused by the residual structure. The residual structure connects the previous and subsequent feature extraction layers, performing repeated fine processing on the image and superimposing information of different dimensions.

[0022] The neural network uses ReLU6 and BN layers throughout the process to enhance its nonlinear expressive power and constrain the distribution of image data.

[0023] Preferably, the learning rate of the SGD optimizer in the neural network is initially 0.01, and decreases by 10% every ten iterations. After calculating the loss L using the cross-entropy loss function, the learning rate determines the step size for weight updates, and the loss L determines the direction of weight updates; wherein,

[0024]

[0025] In the formula, y is the true label value of the image. This represents the probability value predicted for the image.

[0026] Furthermore, step S5 includes the following steps:

[0027] S5.1: The control panel summarizes the prediction results of different sides of the same object and finally determines the type of the object. If it is determined to have stains, it will not be sorted and will be returned to prevent accidental blowing. The count will also be performed.

[0028] S5.2: The sorting machine has a total of N wheels in one week. Each wheel will be photographed during operation. In the first week of operation, A images are judged to have stains. In the second week of operation, B images are judged to have stains. In the third week of operation, C images are judged to have stains. The degree of contamination of the transmission channel is displayed based on the statistics of the three weeks.

[0029]

[0030] S5.3: Display the dirty value on the client, set a pollution threshold alarm, and notify the user that the transmission channel needs to be cleaned when the threshold is exceeded.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] (1) The machine of the present invention can distinguish between material images with and without contaminants;

[0033] (2) The present invention uses a novel neural network to improve the accuracy of fine-grained image classification without affecting the inference speed;

[0034] (3) The present invention can statistically analyze pollutant images to determine the degree of contamination in the transmission channel.

[0035] Compared to previous methods that strive for complete background removal, this invention eliminates the need for overly complex background removal algorithms, saving program runtime. Using neural networks for image recognition offers higher accuracy and stronger generalization capabilities than traditional algorithms. Employing lightweight neural networks with feature reuse and a large receptive field, the machine can more quickly and accurately distinguish between images of materials containing and without contaminants. After obtaining the image results, the machine can provide feedback to the client, informing them of the equipment's operational status and the contamination level of the transmission channel, thus facilitating human-computer interaction. Attached Figure Description

[0036] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0037] Figure 1 This is a flowchart of the method according to an embodiment of the present invention;

[0038] Figure 2 This is a structural diagram of the neural network in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Example

[0040] Because contaminants attached to the conveyor channel vary widely in shape, size, orientation, and location, traditional background removal methods struggle to completely remove them while preserving the integrity of the material. Therefore, using traditional techniques results in low material identification accuracy during sorting machine operation and fails to accurately inform users of the contamination level in the conveyor channel for timely machine cleaning.

[0041] This embodiment proposes a neural network-based image blemish analysis method for sorting machines, combined with... Figures 1-2 As shown, it includes the following steps:

[0042] Step S1: Preserve the integrity of the material by using a general background removal algorithm on the original image, and retain any blemishes that cannot be removed;

[0043] Step S2: Classify the unblemished images according to material characteristics and assign different labels, and group the blemished images into one category and assign labels to them, thus constructing the sample dataset required for training;

[0044] Based on the above, in some specific embodiments, step S2 includes the following steps:

[0045] S2.1: Assign labels to the collected material images according to their categories;

[0046] S2.2: Images with blemishes are uniformly labeled regardless of their material type.

[0047] Step S3: Design a neural network with feature reuse and a large receptive field, and use the network to train the dataset in Step S2. Use the ONNX model to save the trained network weights.

[0048] Based on the above, in some specific embodiments, step S3 includes the following steps:

[0049] S3.1: Design a lightweight neural network with depthwise separable convolution as the backbone and residual structures and large convolutional kernels in some positions on the backbone. The neural network includes an input layer, a hidden layer, an output layer and a softmax classification layer. The feature extraction layers before and after are connected by residual structures to realize the reuse of feature information. The receptive field is improved by a set of convolutional kernels with different sizes. These two structures can better fit fine-grained image datasets.

[0050] S3.2: Import the dataset into the neural network using the PyTorch framework for training. Use the SGD optimizer and cross-entropy loss function to update the neural network weights. Adjust the learning rate of SGD to make the neural network fit the dataset to the best.

[0051] S3.3: Save the neural network weights using the ONNX model after training is complete.

[0052] Furthermore, in the neural network, a 3x3 convolution is first applied to the input image to enhance its dimensionality, followed by depthwise separable convolution to achieve preliminary image feature extraction. Throughout the neural network process, ReLU6 and BN layers are used to enhance the network's non-linear expressive power and constrain the image data distribution. The intermediate feature extraction layer employs a set of convolutional kernels of varying sizes. Large kernels capture high-resolution features, while small kernels capture low-resolution features, with feature map channels segmented in a 4:2:1:1 ratio. Small kernels convolve more feature maps to reduce computational complexity. After feature extraction through three sets of convolutional kernels, a residual structure fuses the high-dimensional information of the image. This residual structure connects the preceding and following feature extraction layers, performing repeated refinement of the image and superimposing information from different dimensions, ensuring high precision of image features and accuracy of neural network recognition.

[0053] The learning rate of the SGD optimizer is initially 0.01, and decreases by 10% every ten iterations. After the loss L is calculated using the cross-entropy loss function, the step size of the weight update is determined by the learning rate, and the direction of the weight update is determined by L.

[0054]

[0055] In the formula, y is the true label value of the image. This represents the probability value predicted for the image.

[0056] Step S4: Deploy the model obtained in step S3 to the edge computing platform of the sorting machine. The operation of the sorting machine provides images to the edge computing platform, and the information after image processing is returned to the control board by the edge computing platform.

[0057] Step S5: After receiving the information returned by the edge computing platform, the control board summarizes the information and displays the material category and the degree of contamination of the transmission channel on the client. The summary results are used to control the mechanical device to sort materials, and the degree of contamination of the transmission channel is used to inform the client of the equipment's operating status.

[0058] Based on the above, in some specific embodiments, step S5 includes the following steps:

[0059] S5.1: The control panel summarizes the prediction results of different sides of the same object and finally determines the type of the object. If it is determined to have stains, it will not be sorted and will be returned to prevent accidental blowing. The count will also be performed.

[0060] S5.2: The sorting machine has a total of N wheels per week. Each wheel will be photographed during operation. In the first week of operation, A images are judged to have stains. In the second week of operation, B images are judged to have stains. In the third week of operation, C images are judged to have stains. The degree of contamination of the transmission channel is displayed based on the statistical results of the three weeks.

[0061]

[0062] S5.3: Display the dirty value on the client, set a pollution threshold alarm, and notify the user that the transmission channel needs to be cleaned when the threshold is exceeded.

[0063] The features of the sorting machine image contamination analysis method in this embodiment include: the ability to classify contaminated images using neural networks; the proposal of a feature reuse neural network that can be deployed on mobile devices; and the ability to statistically calculate the degree of contamination in the transmission channel from contaminated images and display it on the client side.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for analyzing image contamination in a sorting machine based on a neural network, characterized in that, Includes the following steps: Step S1: Preserve the integrity of the material by using a general background removal algorithm on the original image, and retain any blemishes that cannot be removed; Step S2: Classify the unblemished images according to material characteristics and assign different labels, and group the blemished images into one category and assign labels to them, thus constructing the sample dataset required for training; Step S3: Design a neural network with feature reuse and a large receptive field, and use the neural network to train the dataset in Step S2. Use the ONNX model to save the trained network weights. Step S4: Deploy the model obtained in step S3 to the edge computing platform of the sorting machine. The operation of the sorting machine provides images to the edge computing platform, and the information after image processing is returned to the control board by the edge computing platform. Step S5: After receiving the information returned by the edge computing platform, the control board summarizes the information and displays the material category and the degree of contamination of the transmission channel on the client. The summary results are used to control the mechanical device to sort materials, and the degree of contamination of the transmission channel is used to inform the client of the equipment's operating status.

2. The method according to claim 1, characterized in that, Step S2 includes the following steps: S2.1: Assign labels to the collected material images according to their categories; S2.2: Images with blemishes are uniformly labeled regardless of their material type.

3. The method according to claim 1, characterized in that, Step S3 includes the following steps: S3.1: Design a lightweight neural network with depthwise separable convolution as the backbone, and residual structures and large convolutional kernels in some positions of the backbone. The neural network includes an input layer, a hidden layer, an output layer and a softmax classification layer. The feature extraction layers before and after are connected by residual structures to realize the reuse of feature information. The receptive field is increased by a set of convolutional kernels with different sizes. S3.2: Import the dataset into the neural network using the PyTorch framework for training. Use the SGD optimizer and cross-entropy loss function to update the neural network weights. Adjust the learning rate of the SGD optimizer to make the neural network fit the dataset to its best. S3.3: Save the neural network weights using the ONNX model after training is complete.

4. The method according to claim 3, characterized in that, In the neural network, a 3*3 convolution is first used on the input image to increase the image dimension, and then a depthwise separable convolution is used to achieve preliminary image feature extraction. The intermediate feature extraction layer uses a set of convolutional kernels of different sizes. Large convolutional kernels capture high-resolution features, while small convolutional kernels capture low-resolution features. The feature map channels are divided in a 4:2:1:1 ratio. Small convolutional kernels convolve the feature map to reduce computation. After three sets of convolutional kernels extract features, the high-dimensional information of the image is fused by the residual structure. The residual structure connects the previous and subsequent feature extraction layers, performing repeated refinement processing on the image and superimposing information from different dimensions.

5. The method according to claim 4, characterized in that, The neural network uses ReLU6 and BN layers throughout the process to enhance the network's nonlinear expressive power and limit the distribution of image data.

6. The method according to claim 5, characterized in that, The learning rate of the SGD optimizer in the neural network is initially 0.01, and decreases by 10% every ten iterations. After the loss L is calculated using the cross-entropy loss function, the step size of the weight update is determined by the learning rate, and the direction of the weight update is determined by the loss L.

7. The method according to claim 1, characterized in that, Step S5 includes the following steps: S5.1: The control panel summarizes the prediction results of different sides of the same object and finally determines the type of the object. If it is determined to have stains, it will not be sorted and will be returned to prevent accidental blowing. The count will also be performed. S5.2: The sorting machine has a total of N wheels per cycle. Each wheel is photographed during operation. In the first week, A images are identified as having stains; in the second week, B images are identified as having stains; and in the third week, C images are identified as having stains. The degree of contamination in the transport channel is displayed based on the statistical results of these three cycles. ； S5.3: Display the dirty value on the client, set a pollution threshold alarm, and notify the user that the transmission channel needs to be cleaned when the threshold is exceeded.

Images