Raw material sorting method and system based on image recognition technology

Abstract

The invention relates to the technical field of image recognition, and discloses a raw material sorting method and system based on an image recognition technology, and the raw material sorting method based on the image recognition technology comprises the following steps: S101, collecting waste image data in a raw material sorting process; s102, preprocessing the waste image data to obtain a first feature sequence; s103, constructing a material identification model according to the first feature sequence, and outputting a segmentation map with the same size as a sequence unit of the first feature sequence; and S104, calculating the proportion of the materials in the raw materials according to the category of the raw materials to which the pixels of the segmentation map belong, and generating response measures. According to the method, the material identification model is constructed based on the convolutional neural network and the improved gating circulation unit, so that the sorting efficiency and the automation level are improved, and errors caused by manual operation are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a raw material sorting method and system based on image recognition technology. Background Art

[0002] Raw material sorting plays a vital role in industries such as agriculture, mining and pharmaceuticals. Its purpose is to remove impurities and improve the purity and quality of products. As the market demand for high-quality products continues to increase, traditional manual sorting methods have gradually shown their limitations. Traditional sorting often relies on manual operation, which is inefficient and easily affected by human factors, resulting in low sorting accuracy. The identification between normal materials and impurities is often not accurate enough, resulting in economic losses.

[0003] Current sorting technology is mainly based on open-loop control systems, using static parameter settings to guide the sorting process. The drawback of this approach is that the operator's experience and judgment often determine the results of the sorting, and the debugging process is complex and time-consuming, which can easily lead to the mistaken rejection of normal materials. Due to the lack of real-time feedback and adaptive adjustment capabilities, traditional technologies often seem powerless when dealing with complex and changeable raw material characteristics. For example, in the case of large changes in particle size, color or shape, static parameter settings may not achieve the ideal sorting effect, further affecting production efficiency. In addition, existing sorting technologies often have problems with response delays and insufficient processing capacity when processing large batches of materials. This not only affects the continuity of the production process, but may also lead to increased costs and waste of resources.

[0004] Therefore, a raw material separation method is urgently needed to solve the above problems. Summary of the invention

[0005] The present invention provides a raw material sorting method and system based on image recognition technology, which solves the technical problems of low sorting efficiency, poor accuracy, poor adaptability and high misjudgment rate in related technologies.

[0006] The present invention provides a raw material sorting method and system based on image recognition technology, comprising the following steps:

[0007] Step S101, using a camera to collect image data of waste materials after raw material sorting within a first preset time period T at a preset time interval t, the waste image data including video data from the appearance of waste materials to the disappearance of waste materials in the camera;

[0008] Step S102, preprocessing the waste image data to obtain a first feature sequence, wherein a sequence unit of the first feature sequence is a preprocessed video frame image;

[0009] Step S103, constructing a material recognition model according to the first feature sequence, outputting a segmentation map with the same sequence unit size as the first feature sequence in the second preset time period Y, wherein each pixel of the segmentation map corresponds to the category of the raw material to which it belongs, and the raw materials are divided into materials and waste materials;

[0010] Step S104, calculating the proportion of materials in the raw materials within the second preset time period Y according to the raw material categories to which the pixels of the sequence units of the first characteristic sequence within the second preset time period Y belong, and generating response measures.

[0011] Further, the step of preprocessing the waste image data includes:

[0012] Step S201, performing frame extraction on waste material image data using a preset frame rate according to the material flow rate to obtain N video frames;

[0013] Step S202, by Perform noise removal on the video frame, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and G(x, y) represents the Gaussian function value at (x, y), that is, the weight of the point;

[0014] Step S203, by The video frame is enhanced, where H(u, v) represents the frequency response of the filter, D(u, v) represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter;

[0015] Step S204, adjusting the white balance of the video frame;

[0016] Step S205 , normalizing the video frame using the maximum-minimum normalization method to obtain a first feature sequence.

[0017] Furthermore, the material recognition model includes: a first hidden layer, an expansion layer, a second hidden layer and a first classifier;

[0018] The first hidden layer inputs the sequence unit of the first feature sequence and outputs a feature matrix;

[0019] The expansion layer is used to expand the feature matrix into the first eigenvector;

[0020] The second hidden layer includes N first units, the nth first unit inputs the nth sequence unit of the video frame sequence composed of the sequence units as the first feature vector, and outputs the second feature vector, wherein the video frame sequence includes N sequence units;

[0021] The second feature vector output by the Nth first unit is input into the first classifier, and the classification space of the first classifier represents the raw material category of each pixel of the sequence unit of the first feature sequence.

[0022] Furthermore, the first hidden layer is constructed based on convolutional neural network and SegNet.

[0023] Furthermore, the second hidden layer is constructed based on the improved gated recurrent unit. The specific improvements include: adding an adjustment coefficient in the update gate, adding a scaling factor in the reset gate, embedding a residual connection, and introducing a trend gate.

[0024] Furthermore, manual labeling is used as sample labels for training the material recognition model.

[0025] Furthermore, the N video frames of the first hidden layer are pre-trained before the material recognition model is trained. During the pre-training process, the N feature vectors expanded from the N video frames are input into the second classifier. The classification space of the second classifier represents whether the video frame pixels are wheat, and the classification labels of the second classifier are manually labeled.

[0026] Furthermore, K-fold cross validation is used to evaluate the accuracy of the material recognition model. The specific steps include:

[0027] Step S301, evenly divide the training data set into K subsets, each subset is equal in size;

[0028] Step S302, each time one subset is selected from the K subsets as a validation set, and the remaining K-1 subsets are used as training sets;

[0029] Step S303, repeat the above process K times, each time selecting a different subset as the validation set;

[0030] Step S304, compare the raw material recognition results of K times of verification with the manually labeled raw material categories, and calculate the mean square error between the two to evaluate the accuracy of the model.

[0031] Furthermore, the raw material category to which the pixels of the segmentation image belong is binarized, the normal material area is set to 1, and the waste area is set to 0 to form a binary image, and the proportion of the material in the waste is calculated by calculating the number of pixels in the material area and the entire image, and the proportion of the material is compared with a preset threshold. When the proportion of the material is large, the raw material sorting parameters are automatically adjusted to ensure that the proportion of the material in the waste is lower than the preset threshold, wherein the calculation formula of the material proportion is: Where R represents the proportion of materials, A m Indicates the number of material pixels, A s Indicates the total number of pixels.

[0032] The present invention provides a raw material sorting system based on image recognition technology, comprising:

[0033] A data acquisition module, which is used to collect waste material image data during the raw material sorting process;

[0034] A data preprocessing module, which is used to preprocess the waste image data to obtain a first feature sequence;

[0035] A material identification module, which is used to construct a material identification model according to the first feature sequence and output a segmentation map with the same size as the sequence unit of the first feature sequence;

[0036] The proportion calculation and response generation module is used to calculate the proportion of materials in the raw materials according to the raw material categories to which the pixels of the segmentation map belong, and generate response measures.

[0037] The beneficial effects of the present invention are as follows: by constructing a material recognition model based on a convolutional neural network and an improved gated recurrent unit, the present invention can efficiently and accurately classify waste materials at the pixel level, can detect the material ratio in the waste materials in real time, and when the ratio exceeds a preset threshold, the system can automatically adjust the sorting parameters. This mechanism reduces manual intervention, improves the efficiency and automation level of sorting, and reduces the errors caused by human operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the raw material sorting method based on image recognition technology of the present invention;

[0039] Figure 2 It is a flow chart of preprocessing waste map data of the present invention. DETAILED DESCRIPTION

[0040] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0041] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0042] like Figure 1-Figure 2 As shown, the raw material sorting method and system based on image recognition technology include the following steps:

[0043] Step S101, using a camera to collect image data of waste materials after raw material sorting within a first preset time period T at a preset time interval t, the waste image data including video data from the appearance of waste materials to the disappearance of waste materials in the camera;

[0044] Step S102, preprocessing the waste image data to obtain a first feature sequence, wherein a sequence unit of the first feature sequence is a preprocessed video frame image;

[0045] Step S103, constructing a material recognition model according to the first feature sequence, outputting a segmentation map with the same sequence unit size as the first feature sequence in the second preset time period Y, wherein each pixel of the segmentation map corresponds to the category of the raw material to which it belongs, and the raw materials are divided into materials and waste materials;

[0046] Step S104, calculating the proportion of materials in the raw materials within the second preset time period Y according to the raw material categories to which the pixels of the sequence units of the first characteristic sequence within the second preset time period Y belong, and generating response measures.

[0047] In one embodiment of the present invention, a USB area array camera is installed at a suitable position of the material drop channel to ensure that the field of view covers the entire channel and is parallel to the material flow direction, and the focal length and shooting angle of the camera are adjusted according to the size and shape of the material to ensure that the image is clear and can capture sufficient details. The use of a USB area array camera can achieve high-speed data transmission and real-time video acquisition and processing.

[0048] In one embodiment of the present invention, the step of preprocessing the waste image data includes:

[0049] Step S201, performing frame extraction on waste material image data using a preset frame rate according to the material flow rate to obtain N video frames;

[0050] Step S202, by Perform noise removal on the video frame, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and G(x, y) represents the Gaussian function value at (x, y), that is, the weight of the point;

[0051] Step S203, by The video frame is enhanced, where H(u, v) represents the frequency response of the filter, D(u, v) represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter;

[0052] Step S204, adjusting the white balance of the video frame to ensure the authenticity of the image color;

[0053] Step S205 , normalizing the video frame using the maximum-minimum normalization method to obtain a first feature sequence.

[0054] In one embodiment of the present invention, the material recognition model includes: a first hidden layer, an expansion layer, a second hidden layer and a first classifier;

[0055] The first hidden layer inputs the sequence unit of the first feature sequence and outputs a feature matrix;

[0056] The expansion layer is used to expand the feature matrix into the first eigenvector;

[0057] The second hidden layer includes N first units, the nth first unit inputs the nth sequence unit of the video frame sequence composed of the sequence units as the first feature vector, and outputs the second feature vector, wherein the video frame sequence includes N sequence units;

[0058] The second feature vector output by the Nth first unit is input into the first classifier, and the classification space of the first classifier represents the raw material category of each pixel of the sequence unit of the first feature sequence.

[0059] In one embodiment of the present invention, the first hidden layer is constructed based on a convolutional neural network and SegNet.

[0060] In one embodiment of the present invention, the second hidden layer is constructed based on the improved gated recurrent unit, and the specific improvements include: adding an adjustment coefficient in the update gate, adding a scaling factor in the reset gate, embedding a residual connection, and introducing a trend gate. In these improvements, the adjustment coefficient is used to flexibly adjust the influence of the hidden state at the previous moment, so that the model can more easily balance the proportion of historical information and current input, thereby improving the accuracy and adaptability of the prediction; adding a scaling factor in the reset gate to reduce the interference of short-term fluctuations on the model, helping the model to more accurately retain key long-term trend information, avoiding overreaction caused by short-term noise, and making the model more stable in dynamic changes; through residual connections, the original input information can be directly transmitted to subsequent levels to prevent the loss of key information when processing short-term fluctuations, and improve the model's expressiveness and anti-interference ability in complex time series; the trend gate specifically captures and retains the long-term trend information of the input data, so that the model has the ability to learn the data change pattern over a long period of time. Through this gated design, the model can effectively focus on long-term trends while processing short-term fluctuations, significantly improving the overall prediction effect.

[0061] Specifically, the calculation formula of the nth first unit of the second hidden layer includes:

[0062] The calculation formula of the update gate of the nth first unit in the second hidden layer is:

[0063]

[0064] Among them, z n represents the update gate of the nth first unit in the second hidden layer, x n represents the nth sequence unit of the second feature sequence currently input, h n-1 represents the feature vector output by the first unit of the n-1th order, and They represent the first weight parameter, the second weight parameter and the bias parameter corresponding to the update gate of the nth sequence unit of the second feature sequence, respectively. θ represents the adjustment coefficient, which is used to adjust the hidden state h at the previous moment. n-1 The weight in the update gate increases the sensitivity of the update gate to short-term changes;

[0065] The calculation formula of the reset gate of the nth first unit in the second hidden layer is:

[0066]

[0067] Among them, r n represents the reset gate of the nth first unit in the second hidden layer, and denote the first weight parameter, the second weight parameter and the bias parameter corresponding to the reset gate of the nth sequence unit of the second feature sequence respectively; δ denotes a scaling factor, which is used to adjust the response speed of the reset gate to the current input and hidden state, prevent overreaction to short-term noise, and retain long-term information;

[0068] The calculation formula of the trend gate of the nth first unit in the second hidden layer is:

[0069]

[0070] Among them, τ n represents the trend gate of the nth first unit of the second hidden layer, and Respectively represent the first weight parameter, the second weight parameter and the bias parameter corresponding to the trend gate of the nth sequence unit of the second feature sequence;

[0071] The calculation formula for the candidate hidden state of the nth first unit of the second hidden layer is:

[0072]

[0073] in, represents the candidate hidden state of the nth first unit of the second hidden layer, r n and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate hidden state of the nth sequence unit of the second feature sequence;

[0074] The calculation formula for updating the hidden state of the nth first unit in the second hidden layer is:

[0075]

[0076] Among them, h n represents the characteristic vector of the output of the nth first unit of the second hidden layer, ⊙ represents point-by-point multiplication, and γ represents the weight coefficient of the residual connection, which controls the direct impact of the current input on the hidden state and prevents excessive smoothing of information.

[0077] In one embodiment of the present invention, manual labeling is used as sample labels for training the material recognition model.

[0078] In one embodiment of the present invention, ImageNet is used to pre-train N video frames of the first hidden layer before training the material recognition model. During the pre-training process, N feature vectors expanded from the N video frames are input into the second classifier. The classification space of the second classifier represents whether the video frame pixels are wheat, and the classification labels of the second classifier are manually labeled.

[0079] In one embodiment of the present invention, a weighted cross entropy loss function is used as the loss function of the material recognition model, and the calculation formula of the weighted cross entropy loss function is:

[0080]

[0081] Among them, M represents the total number of training samples, y i represents the true label of the i-th sample, which can represent the raw material category of the sample, p(y i |x i ) indicates that the material recognition model predicts the label y for the i-th sample i The probability of It indicates that the weight for each label category can be set according to the frequency of samples appearing in the training set.

[0082] In one embodiment of the present invention, K-fold cross validation is used to evaluate the accuracy of the material recognition model, and the specific steps include:

[0083] Step S301, evenly divide the training data set into K subsets, each subset is equal in size;

[0084] Step S302, each time one subset is selected from the K subsets as a validation set, and the remaining K-1 subsets are used as training sets;

[0085] Step S303, repeat the above process K times, each time selecting a different subset as the validation set;

[0086] Step S304, compare the raw material recognition results of K times of verification with the manually labeled raw material categories, and calculate the mean square error between the two to evaluate the accuracy of the model.

[0087] In one embodiment of the present invention, the raw material category to which the pixels of the segmentation image belong is binarized, the normal material area is set to 1, and the waste area is set to 0 to form a binary image, and the proportion of the material in the waste is calculated by calculating the number of pixels in the material area and the entire image, and the proportion of the material is compared with a preset threshold. When the proportion of the material is large, the raw material sorting parameters are automatically adjusted to ensure that the proportion of the material in the waste is lower than the preset threshold, wherein the calculation formula of the material proportion is: Where R represents the proportion of materials, A m Indicates the number of material pixels, A s Indicates the total number of pixels.

[0088] In one embodiment of the present invention, a raw material sorting system based on image recognition technology is also provided, comprising:

[0089] A data acquisition module, which is used to collect waste material image data during the raw material sorting process;

[0090] A data preprocessing module, which is used to preprocess the waste image data to obtain a first feature sequence;

[0091] A material identification module, which is used to construct a material identification model according to the first feature sequence and output a segmentation map with the same size as the sequence unit of the first feature sequence;

[0092] The proportion calculation and response generation module is used to calculate the proportion of materials in the raw materials according to the raw material categories to which the pixels of the segmentation map belong, and generate response measures.

[0093] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A raw material sorting method and system based on image recognition technology, characterized in that: The following steps are involved: Step S101, using a camera to collect image data of waste materials after raw material sorting within a first preset time period T at a preset time interval t, the waste image data including video data from the appearance of waste materials to the disappearance of waste materials in the camera; Step S102, preprocessing the waste image data to obtain a first feature sequence, wherein a sequence unit of the first feature sequence is a preprocessed video frame image; Step S103, constructing a material recognition model according to the first feature sequence, outputting a segmentation map with the same sequence unit size as the first feature sequence in the second preset time period Y, wherein each pixel of the segmentation map corresponds to the category of the raw material to which it belongs, and the raw materials are divided into materials and waste materials; Step S104, calculating the proportion of materials in the raw materials within the second preset time period Y according to the raw material categories to which the pixels of the sequence units of the first characteristic sequence within the second preset time period Y belong, and generating response measures.

2. The raw material sorting method and system based on image recognition technology according to claim 1, characterized in that: The steps for preprocessing the waste image data include: Step S201, performing frame extraction on waste material image data using a preset frame rate according to the material flow rate to obtain N video frames; Step S202, by Perform noise removal on the video frame, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and G(x, y) represents the Gaussian function value at (x, y), that is, the weight of the point; Step S203, by The video frame is enhanced, where H(u, v) represents the frequency response of the filter, D(u, v) represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter; Step S204, adjusting the white balance of the video frame; Step S205 , normalizing the video frame using the maximum-minimum normalization method to obtain a first feature sequence.

3. The raw material sorting method and system based on image recognition technology according to claim 1, characterized in that: The material recognition model includes: a first hidden layer, an expansion layer, a second hidden layer and a first classifier; The first hidden layer inputs the sequence unit of the first feature sequence and outputs a feature matrix; The expansion layer is used to expand the feature matrix into the first eigenvector; The second hidden layer includes N first units, the nth first unit inputs the nth sequence unit of the video frame sequence composed of the sequence units as the first feature vector, and outputs the second feature vector, wherein the video frame sequence includes N sequence units; The second feature vector output by the Nth first unit is input into the first classifier, and the classification space of the first classifier represents the raw material category of each pixel of the sequence unit of the first feature sequence.

4. The method and system for raw material sorting based on image recognition technology according to claim 3, characterized in that: The first hidden layer is built based on convolutional neural network and SegNet.

5. The method and system for raw material sorting based on image recognition technology according to claim 3, characterized in that: The second hidden layer is built based on the improved gated recurrent unit. The specific improvements include: adding an adjustment coefficient in the update gate, adding a scaling factor in the reset gate, embedding a residual connection, and introducing a trend gate.

6. The method and system for raw material sorting based on image recognition technology according to claim 1, characterized in that: Manual labeling is used as sample labels for training material recognition models.

7. The method and system for raw material sorting based on image recognition technology according to claim 1, characterized in that: Before training the material recognition model, the N video frames of the first hidden layer are pre-trained. During the pre-training process, the N feature vectors expanded from the N video frames are input into the second classifier. The classification space of the second classifier represents whether the video frame pixels are wheat, and the classification labels of the second classifier are manually labeled.

8. The method and system for raw material sorting based on image recognition technology according to claim 1, characterized in that: K-fold cross validation is used to evaluate the accuracy of the material recognition model. The specific steps include: Step S301, evenly divide the training data set into K subsets, each subset is equal in size; Step S302, each time one subset is selected from the K subsets as a validation set, and the remaining K-1 subsets are used as training sets; Step S303, repeat the above process K times, each time selecting a different subset as the validation set; Step S304, compare the raw material recognition results of K times of verification with the manually labeled raw material categories, and calculate the mean square error between the two to evaluate the accuracy of the model.

9. The method and system for raw material sorting based on image recognition technology according to claim 1, characterized in that: The raw material category to which the pixels of the segmentation image belong is binarized, the normal material area is set to 1, and the waste area is set to 0 to form a binary image, and the proportion of the material in the waste is calculated by calculating the number of pixels in the material area and the entire image, and the proportion of the material is compared with the preset threshold. When the proportion of the material is large, the raw material sorting parameters are automatically adjusted to ensure that the proportion of the material in the waste is lower than the preset threshold, wherein the calculation formula of the material proportion is: Where R represents the proportion of materials, A m Indicates the number of material pixels, A s Indicates the total number of pixels.

10. The raw material sorting system based on image recognition technology is characterized by: The method for sorting raw materials based on image recognition technology as claimed in any one of claims 1 to 9 comprises: A data acquisition module, which is used to collect waste material image data during the raw material sorting process; A data preprocessing module, which is used to preprocess the waste image data to obtain a first feature sequence; A material identification module, which is used to construct a material identification model according to the first feature sequence and output a segmentation map with the same size as the sequence unit of the first feature sequence; The proportion calculation and response generation module is used to calculate the proportion of materials in the raw materials according to the raw material categories to which the pixels of the segmentation map belong, and generate response measures.

Images