Impurity real-time monitoring method and computer program product

Abstract

The invention discloses a real-time impurity detection method and a computer program product, and relates to the technical field of computer vision detection. The impurity real-time detection method comprises the following steps: acquiring a to-be-detected image in real time; inputting the to-be-detected image to a trained neural network model, and obtaining a first impurity detection result; inputting the to-be-detected image to a trained classification prediction model, and obtaining a second impurity detection result; and calculating the first impurity detection result and the second impurity detection result according to a preset weight ratio, and outputting the detection results. By adopting the technology provided by the invention, the accuracy of impurity detection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and in particular to a real-time impurity detection method and a computer program product. Background Art

[0002] The screening of raw materials for cash crops is crucial to ensure the quality and safety of agricultural products, especially for stem and leaf raw materials (such as tobacco). During the harvesting, primary roasting, re-roasting, fermentation and other processes, they will be mixed with various forms of impurities and foreign objects, such as hemp ropes, sawdust, foam, paper, feathers, plastic, rubber, metal, stones, or moldy tobacco leaves and stems.

[0003] Based on the above impurities, the current screening methods mainly include: (1) using screens and wind separators to remove larger impurities, such as hemp ropes, wood chips, stones, etc.; (2) using magnetic equipment to absorb and remove metal impurities; (3) using liquid flotation to remove light impurities, such as feathers, paper scraps, etc.; (4) using photoelectric color sorters to identify and remove moldy tobacco leaves and other impurities with abnormal colors based on color differences. Finally, manual re-inspection and selection are carried out to ensure that all impurities are removed, thereby improving the purity of the raw materials.

[0004] However, the manual re-inspection and selection rely on the workers' vision and experience, which is prone to fatigue and misjudgment. In addition, the selection is greatly affected by subjective factors, resulting in uneven quality of selected tobacco, making it difficult to ensure the consistency and accuracy required for production. Based on this, photoelectric imaging technology has gradually been used to take photos of tobacco in real time. By acquiring and interpreting physical optical information, it is possible to introduce higher-dimensional information on the basis of traditional geometric optical imaging.

[0005] However, this technology is currently mainly suitable for real-time identification and removal of impurities such as stems in tobacco, and is difficult to apply to large-scale data and diversified tasks. For the above-mentioned impurities with irregular shapes and similar characteristics to raw materials, its detection and removal accuracy is low, and the probability of missed detection and false detection is high. Summary of the invention

[0006] The present invention provides a real-time impurity detection method and a computer program product to solve the problem of high probability of missed detection and false detection of real-time identification and detection equipment in the prior art.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is to provide a real-time impurity detection method, which includes: acquiring the image to be inspected in real time; inputting the image to be inspected into a trained neural network model, and obtaining a first impurity detection result; inputting the image to be inspected into a trained classification prediction model, and obtaining a second impurity detection result; calculating the first impurity detection result and the second impurity detection result according to a preset weight ratio, and outputting the detection results.

[0008] In some embodiments, inputting the image to be inspected into a trained neural network model and obtaining a first impurity detection result includes: pre-acquiring a first image set x and a second image set x′, wherein the first image set x and the second image set x′ are different forms of the same raw material.

[0009] The neural network model is used to train the first image set x and the second image set x′ in sequence, and a first mapping value Φ(x) of the impurity feature in the first image set x and a second mapping value Φ(x′) of the impurity feature in the second image set x′ are calculated.

[0010] Calculate the joint probability distribution P of the first image set x and the second image set x′, which is expressed as:

[0011] P(z=c,z′=c′|x,x′)=Ф c (x)Ф c (x′)

[0012] Wherein, C is the total number of impurity classifications, z represents the classification result of the first image set x, and z′ represents the classification result of the second image set x′.

[0013] The maximum value of the common information I of the first image set x and the second image set x′ is calculated, which is expressed as:

[0014] MaxI(Ф(x),Ф(x′))

[0015] Perform marginalization processing on i first image sets x and i second image sets x′ to obtain:

[0016]

[0017] Wherein, T represents matrix transpose. Then the first image set x and the second image set x′ share information I, which can be expressed as:

[0018]

[0019] Maximize the common information I between the first image set x and the second image set x′.

[0020] By adopting the above technical solution, a mapping relationship is found through the above deep learning method to retain the common features of the first image set x and the second image set x′ as much as possible, so as to facilitate the subsequent distinction of the original products.

[0021] Furthermore, the neural network model includes an input layer, multiple convolutional layers, a pooling layer and a fully connected layer, the pooling layer is average pooling, and the fully connected layer is used to map the common information I to the image to be inspected, wherein the input and output dimensions of the fully connected layer are determined by the height and width of the image to be inspected.

[0022] Furthermore, the multiple convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the convolution kernel size of the first convolutional layer is configured as 4X4, the convolution kernel size of the second convolutional layer is configured as 4X4, and the convolution kernel size of the third convolutional layer is configured as 2X2.

[0023] In some embodiments, inputting the image to be tested into a trained classification prediction model and obtaining a second impurity detection result includes: pre-acquiring a fourth image set and extracting features of the fourth image set, wherein the feature is the sum of the grayscale values ​​of the image.

[0024] Define the number of K clusters, and repeatedly iterate the sum of squared errors within the cluster to train the classification prediction model; input the image to be tested into the trained classification prediction model, and extract the image features of the image to be tested; assign the image features to the nearest cluster center, and identify the impurity area of ​​the image to be tested. By adding the classification prediction model, the accuracy and robustness of the impurity classification can be predicted.

[0025] In some embodiments, inputting the image to be detected into the trained neural network model and obtaining the first impurity detection result includes: pre-acquiring a third image set Y, which is recorded as:

[0026] Y={y1,y2...y N}

[0027] The image y in the third image set Y N With label m n ∈{0, 1} k One-to-one correspondence, where k represents the total number of impurity classifications; minimize the objective function, recorded as:

[0028]

[0029] Among them, gw represents the classifier, f θ (y n ) represents the mapping operation of the third image set Y, and Loss represents a multi-class logical loss function, which is defined as:

[0030]

[0031] Where n represents the number of samples and k represents the total number of impurity categories.

[0032] In some embodiments, inputting the image to be inspected into the trained neural network model and obtaining the first impurity detection result further comprises: obtaining f θ (y n ); Learn the centroid matrix C∈R d×k And the classification results Z of N images n , recorded as:

[0033]

[0034] Among them, z n ∈{0, 1} k , classification results As a subsequent label, and update to m n ∈{0, 1} k ; Iterate the above process until the final solution is obtained.

[0035] Furthermore, the neural network model is an AlexNet architecture, and includes an input layer, multiple convolutional layers, multiple maximum pooling layers, multiple droupt layers, multiple fully connected layers and an output layer, wherein the maximum pooling layer is arranged after the first convolutional layer, the second convolutional layer and the fifth convolutional layer of the convolutional layer, and the droupt layer is arranged after the first fully connected layer and the second fully connected layer of the fully connected layer to prevent overfitting.

[0036] In some embodiments, the real-time acquisition of the image to be inspected on the production line includes: one or more cameras capturing the reflected image located on one side of the production line in real time.

[0037] The technical solution provided by the present invention has the following beneficial effects compared with the prior art:

[0038] The classification prediction is completed independently by the neural network model and the classification prediction model, and the prediction results of different models are weighted to form the final classification decision, which can effectively combine the two output results, thereby improving the accuracy of impurity detection. Among them, when the neural network model adopts the unsupervised clustering method, there is no need to manually label the data. The neural network is represented as an end-to-end process from input data to clustering, that is, from the input of the image to be inspected to the final clustering output of the impurity annotations and their types in the image, without additional intervention or adjustment. The above-mentioned objective function is the mutual information between the classifications of the first image set and the second image set, in order to optimize the model and improve the consistency and distinguishability of the clusters formed by different classification impurities, so as to detect and identify a variety of impurity types.

[0039] In addition, the neural network model is combined with the classification prediction model to identify impurities on the production line, such as hemp rope, wood chips, foam, paper, feathers, plastic, rubber, metal, stones, or moldy tobacco leaves and stems, etc. It can accurately detect and mark various impurities in real time, thereby improving the screening efficiency of the production line.

[0040] In some embodiments, the present application also provides a computer program product, including a computer program / instruction, which implements the above-mentioned real-time impurity detection method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work, among which:

[0042] Figure 1 This is a schematic diagram of a process of an embodiment of a real-time detection method for impurities provided by the present invention. Figure 1 ;

[0043] Figure 2 It is an image to be inspected in one embodiment of a real-time impurity detection method provided by the present invention;

[0044] Figure 3 This is a schematic diagram of a process of an embodiment of a real-time detection method for impurities provided by the present invention. Figure 2 ;

[0045] Figure 4 The neural network model structure of an embodiment of a real-time impurity detection method provided by the present invention is Figure 1 ;

[0046] Figure 5 It is a graph of various types of impurities provided by the present invention;

[0047] Figure 6 This is a schematic diagram of a process of an embodiment of a real-time detection method for impurities provided by the present invention. Figure 3 ;

[0048] Figure 7 This is a schematic diagram of a neural network structure of an embodiment of a real-time impurity detection method provided by the present invention. Figure 2 ;

[0049] Figure 8 This is a schematic diagram of a process of an embodiment of a real-time detection method for impurities provided by the present invention. Figure 4 ;

[0050] Fig. 9 A neural network structure of an embodiment of a real-time impurity detection method provided by the present invention is Figure 3 ;

[0051] Fig.10 A neural network structure of an embodiment of a real-time impurity detection method provided by the present invention is Figure 4 . DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] See also Figure 1 to Figure 2 As shown, Figure 1 A schematic diagram showing a process of an embodiment of a real-time detection method for impurities provided by the present application Figure 1 ; Figure 2 An image to be inspected according to an embodiment of a real-time impurity detection method provided by the present application is shown.

[0054] In some embodiments, the real-time detection method of impurities comprises:

[0055] Step S10, acquiring the image to be inspected in real time.

[0056] By installing a camera on the production line to shoot raw materials such as tobacco, etc. Figure 2 As shown, a three-phase camera can be fixed on the production line, and the working area on the production line is used as the viewing area for shooting, and the images taken by the cameras at various angles are combined as the image to be inspected. For example, the image to be inspected can also be pre-processed, such as adjusting the resolution of the image to be inspected and filtering.

[0057] Step S20, input the image to be detected into the trained neural network model, and obtain a first impurity detection result.

[0058] The neural network model can use unsupervised clustering methods to train the randomly initialized neural network into an end-to-end classification function. Figure 3 and Figure 4 As shown, Figure 3 A schematic diagram showing a process of an embodiment of a real-time detection method for impurities provided by the present application Figure 2 ; Figure 4 The neural network model structure of an embodiment of a real-time impurity detection method provided by the present application is shown. Figure 1 .

[0059] Step S200: pre-acquire a first image set x and a second image set x′.

[0060] The first image set x and the second image set x′ are different forms of the same raw material, that is, when the first image set x and the second image set x′ belong to the same raw product, it means that the two sets of image sets are similar to a certain extent, so that when the following deep learning model finds the mapping φ, it can maximize the capture of the commonalities of the raw materials of the first image set x and the second image set x′, such as the shape of the tobacco leaves and their spectral commonalities.

[0061] Exemplarily, the present application does not limit the number of the first image set x and the second image set x′, and the two sets of image sets can be uniformly preprocessed (such as normalizing pixels, etc.). When the first image set x and the second image set x′ are large enough, the robustness of the neural network model to raw materials such as tobacco leaves and tobacco shreds of different forms can be enhanced.

[0062] Step S210, using a neural network model to train the first image set x and the second image set x' in sequence, and calculating a first mapping value φ(x) of the impurity feature in the first image set x and a second mapping value φ(x') of the impurity feature in the second image set x'.

[0063] The clustering result output by clustering the first image set x and the second image set x′ is soft clustering, φ(x)∈[0,1]. When there are C impurity classifications, the distribution probability can be introduced to express the association between the elements (impurities) of each group of image sets, that is, the distribution probability of the first image set x is P(z|x); the distribution probability of the second image set x′ is P(z′|x′).

[0064] Step S220, calculate the joint probability distribution P of the first image set x and the second image set x′, denoted as:

[0065] P(z=c, z′=c′|x, x′)=Φ c (x)Φ c (x′)

[0066] Wherein, C is the total number of impurity classifications, z represents the classification results of the first image set x, and z′ represents the classification results of the second image set x′.

[0067] Step S230, calculate the maximum value of the common information I of the first image set x and the second image set x′, which is expressed as:

[0068] MaxI(Φ(x),Φ(x′))

[0069] Step S240, marginalization processing is performed on the i first image sets x and the second image sets x′ to obtain:

[0070]

[0071] Where T represents matrix transpose. Then the first image set x and the second image set x′ share information I, which can be expressed as:

[0072]

[0073] Maximize the common information I between the first image set x and the second image set x′.

[0074] In an embodiment of the present application, a loss function can be constructed by searching for the above-mentioned common information. After multiple iterations, the maximum common information can be obtained, and then a mapping relationship can be found to retain the common features of the first image set x and the second image set x′ as much as possible. The features may be impurity texture features, impurity shape features, etc., so that when the image to be inspected is input, the presence of impurities can be indicated by the above-mentioned common features, and then the impurities can be identified.

[0075] Combination Figures 5 and 6 As shown, Figure 5 Shows various types of impurity diagrams provided in this application; Figure 6 A schematic diagram showing a process of an embodiment of a real-time detection method for impurities provided by the present application Figure 3 .

[0076] Step S30, inputting the image to be detected into the trained classification prediction model, and obtaining a second impurity detection result.

[0077] For example, when industrially screening raw materials, if the growth cycle of the same variety of raw materials is similar or they come from the same origin, the impurities in the initial raw materials of the batch will also be similar. For example, tobacco leaves picked in the same tobacco growing area have similar types and shapes of impurities due to their common soil conditions, similar climate environment, etc. Figure 5 As shown, its visible light properties are similar and quite different from the raw materials.

[0078] Step S300, pre-acquire a fourth image set, and extract features of the fourth image set, wherein the features include the sum of grayscale values ​​of the images in the first image set.

[0079] By distinguishing impurities based on visible light properties, we can, to a certain extent, only consider the sensitive relationship between impurities and raw materials, that is, the sum of the grayscale values ​​of the image can be used as the main extracted feature.

[0080] Step S310, define K clusters, and repeatedly iterate the intra-cluster sum of squared errors to train the classification prediction model.

[0081] The K-means clustering method is introduced for iteration, namely: Among them, J represents the sum of squared errors within the cluster, K represents the number of clusters, n represents the total number of data points set for training, μ is the cluster center, and x j is an image feature. Exemplarily, K can be set to 2, and after training with the fourth image set samples of the above-mentioned estimated number n, a K-means-based classification prediction model can be obtained. Among them, the classification prediction model can also use other clustering algorithms, such as hierarchical clustering or Gaussian distribution to model the probability density function of data.

[0082] Step S320: input the image to be inspected into the trained classification prediction model, and extract image features of the image to be inspected.

[0083] Step S330, assign the image features to the nearest cluster center and identify the impurity region of the image to be inspected. And identify the impurity region based on the clustering result. For example, if a cluster represents impurity, the image region assigned to the cluster can be marked as an impurity region.

[0084] Step S40, calculating the first impurity detection result and the second impurity detection result according to the preset weight ratio, and outputting the detection results.

[0085] That is, Result = W1R1 + W2R2, where R1 represents the output result of the neural network model, and R2 represents the output result of the classification prediction model. W1 and W2 correspond to preset weights, where W1∈[0,1]; W2∈[0,1], W1+W2=1.

[0086] In an embodiment of the present application, classification prediction is completed independently by a neural network model and a classification prediction model, and the prediction results of different models are weighted to form a final classification decision, which can effectively combine the two output results, thereby improving the accuracy of impurity detection. Among them, when the neural network model adopts an unsupervised clustering method, there is no need to manually label the data. The neural network is represented as an end-to-end process from input data to clustering, that is, from inputting the image to be inspected to the final clustering output of the impurity annotations and their types in the image, without additional intervention or adjustment. The above-mentioned objective function is the mutual information between the classifications between the first image set and the second image set, in order to optimize the model and improve the consistency and distinguishability of the clusters formed by different classification impurities, so as to detect and identify a variety of impurity types.

[0087] Combination Figure 7 As shown, Figure 7 A schematic diagram of a neural network structure of an embodiment of a real-time impurity detection method provided by the present application is shown. Figure 2 .

[0088] In some embodiments, the neural network model includes an input layer, multiple convolutional layers, a pooling layer and a fully connected layer, the pooling layer is average pooling, and the fully connected layer is used to map the common information I to the image to be inspected, wherein the input and output dimensions of the fully connected layer are determined by the height and width of the image to be inspected.

[0089] Exemplarily, the input layer obtains the image to be inspected taken by the camera, which is usually an RGB three-channel image, and the multiple convolution layers include a first convolution layer, a second convolution layer, and a third convolution layer, wherein the convolution kernel size of the first convolution layer is configured as 4X4, the convolution kernel size of the second convolution layer is configured as 4X4, and the convolution kernel size of the third convolution layer is configured as 2X2. The above parameters can be adjusted and configured according to actual production needs.

[0090] See also Figures 8 to 9 As shown, Figure 8 A schematic diagram showing a process of an embodiment of a real-time detection method for impurities provided by the present application Figure 4 ; Fig. 9 The neural network structure of an embodiment of a real-time impurity detection method provided by the present application is shown. Figure 3 .

[0091] In some embodiments, the classification of impurities approaches infinity, and it is impossible to perform complete impurity classification and labeling. For example, the possible forms of stone impurities of the same type cannot be exhausted. Therefore, the neural network model can also use unsupervised feature learning methods to generate pseudo labels by classifying the visual features of the raw materials to determine the weights of the neural network, that is, converting the unsupervised problem into a supervised learning problem, thereby improving the accuracy of feature representation and reducing the errors that may occur in the clustering process.

[0092] Step S250: pre-acquire a third image set Y, denoted as: Y = {y1, y2...y N}. The third image set Y is preprocessed so that it can be converted into the same format according to a unified rule to become an image representation.

[0093] Step S260: the image yN in the third image set Y and the label m n ∈{0, 1} k One-to-one correspondence, where k represents the total number of impurity categories.

[0094] Among them, the label is a pseudo label, which is generated by the clustering algorithm and is a data point assigned based on the clustering result. For example, when a cluster subsequently clustered represents that its feature is a jagged edge, the label is based on the jagged feature.

[0095] Step S270, minimize the objective function, denoted as:

[0096]

[0097] Define a classifier consisting of three fully connected layers with parameter W to classify f θ y n ) to correctly label and classify. Among them, the parameter W and the parameter θ of the mapping relationship are jointly learned and optimized to achieve the goal of minimizing the objective formula.

[0098] Loss represents a multi-class logistic loss function, defined as:

[0099]

[0100] Where n represents the number of samples, and k represents the total number of impurity categories. Where l{} is an indicative function. Assume that there are N sample images in the third sample set, and each sample image has K possible categories. When the sample truly belongs to the corresponding category, y( j )=l, otherwise y( j )=0. Minimize the loss function, thereby improving the accuracy of neural network prediction by learning to adjust the parameter W and the parameter θ of the mapping relationship.

[0101] Step S280, obtaining f θ (y n ), and learn the centroid matrix C∈R d×k And the classification results Z of N images n , recorded as:

[0102]

[0103] Among them, z n ∈{0, 1} k , classification results As the subsequent label, and update to label m n ∈{0, 1} k . Take the output f of the convolutional layer θ (y n ) as input, set K independent cluster centers. For example, when the impurities are N categories, K is the raw material category plus the impurity category, that is, K = N + 1.

[0104] Step S290, iterate and alternate the above process until the final solution is obtained.

[0105] By continuously updating the classification labels through the above formula, the final solution can be obtained, and a trivial solution can be effectively avoided to ensure that the model learns meaningful features. Exemplarily, in order to prevent the occurrence of empty clusters (especially in the K-means algorithm used above), empty classifications are automatically assigned during optimization, that is, when a cluster becomes empty (that is, no data points are assigned to a certain cluster center), a non-empty cluster is randomly selected and a small random perturbation is performed on it to create a new center of the empty cluster, and the points belonging to the non-empty cluster are reallocated to the two result clusters.

[0106] In order to avoid overfitting of parameter θ when minimizing the objective function in step S270, for example, when one cluster is too large and other clusters are singletons, minimizing the objective function will hardly cause parameter θ to change. Images can be sampled based on the uniform distribution of classes or pseudo labels, that is, the input f in the above formula θ (y n )’s contribution to the loss function is weighted, and its value is the inverse of the assigned cluster size, thus preventing overly large clusters from affecting parameter training.

[0107] Combination Fig.10 As shown, Fig.10 The neural network structure of an embodiment of a real-time impurity detection method provided by the present application is shown. Figure 4 .

[0108] In some embodiments, the neural network model is an AlexNet architecture and includes an input layer, multiple convolutional layers, multiple maximum pooling layers, multiple droupt layers, multiple fully connected layers and an output layer, wherein a maximum pooling layer is set after the first convolutional layer, the second convolutional layer and the fifth convolutional layer of the convolutional layer, and a droupt layer is set after the first fully connected layer and the second fully connected layer of the fully connected layer to prevent overfitting.

[0109] AlexNet is a common deep convolutional neural network (CNN) architecture. The maximum pooling layer is set after the first convolutional layer, the second convolutional layer and the fifth convolutional layer, and the discarding rate of the Dropout layer can effectively extract features from the image, optimize the performance of the model, and improve its generalization ability.

[0110] In some embodiments, acquiring the image to be inspected on the production line in real time includes: one or more cameras capturing the reflected image on one side of the production line in real time.

[0111] In the embodiment of the present application, the camera is fixedly mounted above and below the production line or at other specific angles by a tripod or the like to capture the reflected image, wherein the reflected image is an image reflected from different angles by a reflector. For example, through the 90° reflection of the mirror, the camera can shoot in a vertical direction, thereby capturing the details of the tobacco shreds and tobacco leaves from different angles to improve the accuracy and comprehensiveness of the detection. In addition, the camera can also directly capture images of tobacco leaves and shreds.

[0112] In some embodiments, the present application also provides a computer program product, including a computer program / instruction, which implements the above-mentioned real-time impurity detection method when executed by a processor.

[0113] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Those skilled in the art may clearly understand that each embodiment may be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution may essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product may be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and may include a number of instructions for a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0114] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly used in other related technical fields, should be included in the protection scope of the present invention.

Claims

1. A real-time detection method for impurities, characterized in that: include: Acquire the image to be inspected in real time; Inputting the image to be inspected into the trained neural network model, and obtaining a first impurity detection result; Inputting the image to be inspected into the trained classification prediction model, and obtaining a second impurity detection result; The first impurity detection result and the second impurity detection result are calculated according to a preset weight ratio, and the detection results are output.

2. The method for real-time detection of impurities according to claim 1, characterized in that: The step of inputting the image to be inspected into the trained neural network model and obtaining a first impurity detection result comprises: Pre-acquire a first image set x and a second image set x′, wherein the first image set x and the second image set x′ are different forms of the same raw material; The neural network model is used to train the first image set x and the second image set x′ in sequence, and a first mapping value Φ(x) of an impurity feature in the first image set x and a second mapping value Φ(x′) of an impurity feature in the second image set x′ are calculated; Calculate the joint probability distribution P of the first image set x and the second image set x′, which is expressed as: P(z=c,z'=c'|x,x)=Φc(x)Φc(x') Wherein, C is the total number of impurity classifications, z represents the classification result of the first image set x, and z′ represents the classification result of the second image set x′; The maximum value of the common information I of the first image set x and the second image set x′ is calculated, which is expressed as: maxI(Ф(x),Ф(x′)) Perform marginalization processing on i first image sets x and i second image sets x′ to obtain: Where T represents matrix transpose; Then the first image set x and the second image set x′ share information I, which can be expressed as: Maximize the common information I between the first image set x and the second image set x′.

3. The real-time detection method for impurities according to claim 2, characterized in that: The neural network model includes an input layer, multiple convolutional layers, a pooling layer and a fully connected layer, wherein the pooling layer is average pooling, and the fully connected layer is used to map the common information I to the image to be inspected, wherein the input and output dimensions of the fully connected layer are determined by the height and width of the image to be inspected.

4. The real-time impurity detection method according to claim 3, characterized in that: The multiple convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the convolution kernel size of the first convolutional layer is configured as 4X4, the convolution kernel size of the second convolutional layer is configured as 4X4, and the convolution kernel size of the third convolutional layer is configured as 2X2.

5. The method for real-time detection of impurities according to claim 1, characterized in that: The step of inputting the image to be inspected into the trained classification prediction model and obtaining the second impurity detection result comprises: Pre-acquire a fourth image set, and extract features of the fourth image set, wherein the feature is a sum of grayscale values ​​of the images; Define the number of K clusters and repeatedly iterate the sum of squared errors within the cluster to train the classification prediction model; Inputting the image to be inspected into the trained classification prediction model, and extracting image features of the image to be inspected; The image features are assigned to the nearest cluster center, and the impurity area of ​​the image to be inspected is identified.

6. The real-time impurity detection method according to claim 1, characterized in that: The step of inputting the image to be inspected into the trained neural network model and obtaining a first impurity detection result comprises: Pre-acquire the third image set Y, denoted as: Y={y1y2…yN} The image y in the third image set Y N With label m n ∈{0, 1} k One-to-one correspondence, where k represents the total number of impurity classifications; Minimize the objective function, denoted as: Among them, gw represents the classifier, f θ (y n ) represents the mapping operation of the third image set Y, and Loss represents a multi-class logical loss function, which is defined as: Where n represents the number of samples and k represents the total number of impurity categories.

7. The real-time detection method for impurities according to claim 6, characterized in that: The step of inputting the image to be inspected into the trained neural network model and obtaining the first impurity detection result further comprises: Get f θ (y n ); Learn the centroid matrix C∈R d×k And the classification results Z of N images n , recorded as: Among them, z n ∈{0,1} k , classification results As a subsequent label, and update to m n ∈{0, 1} k ; Repeat the iterations until the final solution is obtained.

8. The method for real-time detection of impurities according to claim 6, characterized in that: The neural network model is an AlexNet architecture and includes an input layer, multiple convolutional layers, multiple maximum pooling layers, multiple droupt layers, multiple fully connected layers and an output layer, wherein the maximum pooling layer is arranged after the first convolutional layer, the second convolutional layer and the fifth convolutional layer of the convolutional layer, and the droupt layer is arranged after the first fully connected layer and the second fully connected layer of the fully connected layer to prevent overfitting.

9. The real-time impurity detection method according to claim 1, characterized in that: The real-time acquisition of the image to be inspected on the production line includes: one or more cameras capturing the reflected image on one side of the production line in real time.

10. A computer program product, characterized in that The method comprises a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method for real-time detection of impurities according to any one of claims 1 to 9 is implemented.

Images