A product classification method and device based on self-learning model

Abstract

The present invention discloses a product classification method and device based on a self-learning model, the method comprising: constructing a self-learning module, wherein a plurality of parallel neural network models are integrated in the self-learning module; training the self-learning module to obtain a self-learning module that has been preliminarily trained; monitoring defect classification instructions to obtain a product image to be defect-classified; activating a neural network model in the most recently trained self-learning module to a working state based on the product type and product image quality in the product image; dividing the product image into a plurality of sub-images of a specified size; inputting all sub-images into the most recently trained self-learning module to obtain a product defect classification result of the product image; updating the self-learning module based on the product image to be defect-classified to obtain the most recently trained self-learning module. The method makes the classification result accurate and can meet the needs of the industry.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a product classification and detection method and device based on a self-learning model. Background Art

[0002] How to quickly deploy and apply automated equipment to production lines has become one of the important factors in improving productivity. Based on this, many deep learning models are currently on the market. However, these learning models have relatively high requirements for personnel expertise, deployment time, and hardware configuration, which greatly prolongs the online cycle and increases costs.

[0003] Existing deep learning and machine learning technologies require obtaining a large number of defect images in advance, and then manually marking and classifying the images with these defect features, and then training them on a high-configuration industrial computer. This operation is time-consuming and labor-intensive.

[0004] In addition, as the accuracy requirements for detection and recognition in the current automation process become higher and higher, the target range is getting larger and the beat is getting faster and faster, high-resolution cameras can meet the shooting range and accuracy requirements. However, the higher the resolution of the camera, the larger the picture will be. Large pictures must be paired with high-performance industrial computers or even servers to run deep learning models. The present invention can solve the problem of heavy reliance on high-end industrial computers by setting the segmentation method and learning model for the picture with high precision. Summary of the invention

[0005] In view of this, the present invention provides a product classification method and device based on a self-learning model, which can solve the technical problems that current deep learning must rely on high-performance configuration for large image training, requires a lot of time to manually mark the locations of product defects, and deep learning relies on professional operations.

[0006] In order to solve the above technical problems, the present invention is implemented as follows.

[0007] A product classification method based on a self-learning model, comprising:

[0008] Step S1: construct a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, and when the self-learning module is in use, only one neural network model is in a working state; train the self-learning module to obtain a self-learning module that has been preliminarily trained; and make the self-learning module that has been trained the latest be the self-learning module that has been preliminarily trained;

[0009] Step S2: listen to the defect classification instruction and obtain the product image to be defect classified; activate a neural network model in the most recently trained self-learning module to a working state based on the product type and product image quality in the product image; select a segmentation template from a number of pre-set segmentation templates based on the position of the product in the product image, and segment the product image into multiple sub-images of specified sizes according to the array or irregular layout set by the segmentation template; input all the sub-images into the most recently trained self-learning module to obtain the product defect classification result of the product image; update the most recently trained self-learning module.

[0010] Preferably, the step S1: training the self-learning module to obtain a preliminarily trained self-learning module comprises:

[0011] Step S11: For each neural network model in the self-learning module, training is performed in the following manner:

[0012] Step S111: obtaining a number of product images;

[0013] Step S112: for each product image: score the entropy of the product image; put product images with scores greater than or equal to a score threshold into a first image set, and put product images with scores less than the score threshold into a second image set;

[0014] Step S113: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template;

[0015] Step S114: inputting the sub-images corresponding to the training samples into the neural network model, the neural network model is used to extract the feature differences between the defect-free product images and the defective images, classify the defects of the product images, and use the classification results as the output of the neural network model; the neural network model is trained based on the training samples to obtain a trained neural network model;

[0016] Step S12: Based on the fully trained neural network model, a preliminarily trained self-learning module is obtained.

[0017] Preferably, the step S2: updating the most recently trained self-learning module comprises:

[0018] Step S13: The neural network model in the working state in the self-learning module that has been trained the most recently is trained in the following manner:

[0019] Step S131: obtaining a first image set and a second image set; obtaining the product image to be defect-classified, and scoring the product image according to the entropy of the product image to be defect-classified; if the score is greater than or equal to a dynamic preset threshold, the product image to be defect-classified is stored in the first image set; otherwise, the product image to be defect-classified is stored in the second image set; manually annotating each product image in the second image set, adjusting the dynamic preset threshold, and storing the product images in the second image set that are higher than the dynamic preset threshold in the first image set;

[0020] Step S132: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template;

[0021] Step S133: inputting the sub-image corresponding to each training sample into the neural network model in working state, the neural network model in working state is used to extract the feature difference between the defect-free product image and the defective image, classify the defects of the product image, and use the classification result as the output of the neural network model; training the neural network model in working state based on each training sample to obtain a trained neural network model in working state;

[0022] Step S14: Based on the trained neural network model in working state, the most recently trained self-learning module is updated.

[0023] Preferably, the dynamic preset threshold is adjusted in the following manner:

[0024]

[0025]

[0026] Among them, k is the number of all product images in the first image set and the second image set, j is the contour points of all captured products in each product image in the second image set, PointGrayValue is the grayscale value of the contour points of the captured products, Ava is the intermediate variable to be calculated, y is the dynamic preset threshold, and i1 is a variable.

[0027] A product classification device based on a self-learning model, comprising:

[0028] Preliminary training module: configured to construct a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, wherein when the self-learning module is in use, only one neural network model is in a working state; the self-learning module is trained to obtain a self-learning module that has been preliminarily trained; and the self-learning module that has been trained the latest is the self-learning module that has been preliminarily trained;

[0029] Retraining module: configured to listen to defect classification instructions and obtain product images to be defect-classified; based on the product type and product image quality in the product image, activate a neural network model in the most recently trained self-learning module to a working state; based on the position of the product in the product image, select a segmentation template from a number of pre-set segmentation templates, and segment the product image into multiple sub-images of specified sizes according to the array or irregular layout set by the segmentation template; input all sub-images into the most recently trained self-learning module to obtain the product defect classification result of the product image; update the most recently trained self-learning module.

[0030] Beneficial effects:

[0031] (1) The present invention divides a large image into small images of custom sizes to replace the manual labeling process;

[0032] (2) The present invention can complete various industry classification requirements such as relatively complex defects and product front and back, presence or absence, etc. under low-configuration hardware conditions and without the need for personnel with professional backgrounds to operate.

[0033] (3) The self-learning machine learning model of the present invention can automatically learn to improve recognition accuracy and generalization ability, so that the classification results are accurate and can meet industry needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the product classification method based on the self-learning model of the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of a product classification device based on a self-learning model of the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0037] like Figure 1 As shown, the present invention proposes a product classification method based on a self-learning model, comprising:

[0038] Step S1: construct a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, and when the self-learning module is in use, only one neural network model is in a working state; train the self-learning module to obtain a self-learning module that has been preliminarily trained; and make the self-learning module that has been trained the latest be the self-learning module that has been preliminarily trained;

[0039] Step S2: listen to the defect classification instruction and obtain the product image to be defect classified; activate a neural network model in the most recently trained self-learning module to a working state based on the product type and product image quality in the product image; select a segmentation template from a number of pre-set segmentation templates based on the position of the product in the product image, and segment the product image into multiple sub-images of specified sizes according to the array or irregular layout set by the segmentation template; input all the sub-images into the most recently trained self-learning module to obtain the product defect classification result of the product image; update the most recently trained self-learning module.

[0040] The step S1: training the self-learning module to obtain a self-learning module that has been preliminarily trained, includes:

[0041] Step S11: For each neural network model in the self-learning module, training is performed in the following manner:

[0042] Step S111: obtaining a number of product images;

[0043] Step S112: for each product image: score the entropy of the product image; put product images with scores greater than or equal to a score threshold into a first image set, and put product images with scores less than the score threshold into a second image set;

[0044] Step S113: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template;

[0045] Step S114: inputting the sub-images corresponding to the training samples into the neural network model, the neural network model is used to extract the feature differences between the defect-free product images and the defective images, classify the defects of the product images, and use the classification results as the output of the neural network model; the neural network model is trained based on the training samples to obtain a trained neural network model;

[0046] Step S12: Based on the fully trained neural network model, a preliminarily trained self-learning module is obtained.

[0047] The step S2: updating the most recently trained self-learning module comprises:

[0048] Step S13: The neural network model in the working state in the self-learning module that has been trained the most recently is trained in the following manner:

[0049] Step S131: obtaining a first image set and a second image set; obtaining the product image to be defect-classified, and scoring the product image according to the entropy of the product image to be defect-classified; if the score is greater than or equal to a dynamic preset threshold, the product image to be defect-classified is stored in the first image set; otherwise, the product image to be defect-classified is stored in the second image set; manually annotating each product image in the second image set, adjusting the dynamic preset threshold, and storing the product images in the second image set that are higher than the dynamic preset threshold in the first image set;

[0050] Step S132: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template;

[0051] Step S133: inputting the sub-image corresponding to each training sample into the neural network model in working state, the neural network model in working state is used to extract the feature difference between the defect-free product image and the defective image, classify the defects of the product image, and use the classification result as the output of the neural network model; training the neural network model in working state based on each training sample to obtain a trained neural network model in working state;

[0052] Step S14: Based on the trained neural network model in working state, the most recently trained self-learning module is updated.

[0053] Furthermore, in the process of training the neural network model in the working state in the self-learning module, the weight corresponding to each sub-image in the product image is determined, and then the weight of the product image is determined. The weight P corresponding to the sub-image ir The calculation formula is:

[0054]

[0055] Wherein, 1≤i≤m, 1≤r≤n, m is the number of sub-images of the product image, and n represents the number of iterations. is the product area of ​​the sub-image obtained at the rth iteration.

[0056] The calculation formula of the weight P of the product image is:

[0057]

[0058] In the present invention, in order to solve the two major problems of time-consuming manual labeling and model recognition speed, when obtaining pictures of defective products, the inherent features on the product are matched through the template of the traditional algorithm, and the product size and location are accurately located to automatically divide a large picture into multiple small pictures and save them. The operation improves the speed of Deep Learning training and recognition speed, and at the same time, greatly reduces the requirements of Deep Learning on computer hardware computing power.

[0059] In the present invention, the neural network model is, for example, a convolutional neural network model, a machine learning model, etc.

[0060] The entropy of the product image is scored, and the scoring formula is:

[0061] H=Plog2(P)S

[0062] Among them, H is the score of the entropy of the product image, P is the weight of the product image, and S is the entropy of the product image.

[0063] The method for calculating the entropy of the product image is a conventional entropy calculation method in the art.

[0064] Furthermore, the dynamic preset threshold is adjusted in the following manner:

[0065]

[0066]

[0067] Among them, k is the number of all product images in the first image set and the second image set, j is the contour points of all captured products in each product image in the second image set, PointGrayValue is the grayscale value of the contour points of the captured products, Ava is the intermediate variable to be calculated, y is the dynamic preset threshold, and i1 is a variable.

[0068] In the present invention, in order to solve the problem that it takes a lot of manual time to mark defects, the previously segmented small images are directly used for Deep Learning to learn, and the feature differences between normal images and images with defects are automatically learned and extracted, and the images are identified and classified. The images above and below the set score threshold are output in two folders. In the early stage, the images with identification and classification results above the set threshold are manually corrected to the category of the wrong image. After the correction, the image is used as a training image for Deep Learning to learn. After learning, the set score threshold is lowered, and the images below the score threshold in the previous round are used as training images for Deep Learning to learn automatically. After repeated several times, a model with high recognition accuracy can be obtained; at this time, no manual intervention is required, and Deep Learning can perform the above inner loop operation by itself. Even during production, it will not affect the automatic learning operation of Deep Learning, thereby achieving the purpose of continuously Deep Learning self-optimizing the model and improving the generalization ability of the model.

[0069] The present invention performs image segmentation through feature positioning, and continuously and automatically optimizes and upgrades the model while ensuring rapid online launch, so as to continuously improve accuracy in all aspects without affecting production progress.

[0070] The present invention proposes a specific embodiment of a product classification method based on a self-learning model.

[0071] This implementation takes the identification of multiple products on a single disk in a complex environment as an example and is implemented in the following ways:

[0072] Environment: One PC, linear motor or other moving mechanism, one 20-megapixel black-and-white camera, and detection system.

[0073] 1. Preliminary preparation: fix the camera to the axis;

[0074] 2. Camera search;

[0075] Connect the camera to the PC through the specified connection method, click the search button of the camera module, configure the searched camera (photo taking method, exposure, ROI area, image saving path, whether it is a virtual camera, etc.), set the image display window, and open the camera.

[0076] 3. Create a configuration file;

[0077] Create corresponding folder types according to detection needs.

[0078] Direct the threshold comparison results to different folders.

[0079] 4. Positioning and segmentation;

[0080] Create positioning features based on a certain feature of the entire drawing.

[0081] Set the segmentation ratio and number for automatic segmentation.

[0082] 5. Create project processes;

[0083] Select TCP Client communication in the process module to communicate with the third-party TCP Server.

[0084] Select logical judgment IF and set the judgment condition.

[0085] Run the detection module, output the corresponding images to the specified folder, and feedback the results to the specified location.

[0086] Run the self-learning module to monitor whether learning is needed.

[0087] Output the learning results for next detection.

[0088] The present invention proposes a product classification device based on a self-learning model, comprising:

[0089] Preliminary training module: configured to construct a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, wherein when the self-learning module is in use, only one neural network model is in a working state; the self-learning module is trained to obtain a self-learning module that has been preliminarily trained; and the self-learning module that has been trained the latest is the self-learning module that has been preliminarily trained;

[0090] Retraining module: configured to listen to defect classification instructions and obtain product images to be defect-classified; based on the product type and product image quality in the product image, activate a neural network model in the most recently trained self-learning module to a working state; based on the position of the product in the product image, select a segmentation template from a number of pre-set segmentation templates, and segment the product image into multiple sub-images of specified sizes according to the array or irregular layout set by the segmentation template; input all sub-images into the most recently trained self-learning module to obtain the product defect classification result of the product image; update the most recently trained self-learning module.

[0091] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A product classification method based on a self-learning model, characterized in that: include: Step S1: constructing a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, and when the self-learning module is in use, only one neural network model is in a working state; training the self-learning module to obtain a self-learning module that has been preliminarily trained; Let the most recently trained self-learning module be the self-learning module that has been initially trained; Step S2: listen to the defect classification instruction and obtain the product image to be defect-classified; activate a neural network model in the most recently trained self-learning module to a working state based on the product type and product image quality in the product image; select a segmentation template from a plurality of pre-set segmentation templates based on the position of the product in the product image, and segment the product image into a plurality of sub-images of specified sizes according to the array or irregular layout set by the segmentation template; input all the sub-images into the most recently trained self-learning module to obtain the product defect classification result of the product image; Updating the self-learning module that has been trained the most recently; The step S2: updating the most recently trained self-learning module comprises: Step S13: The neural network model in the working state in the self-learning module that has been trained the most recently is trained in the following manner: Step S131: obtaining a first image set and a second image set; obtaining the product image to be defect-classified, and scoring the product image according to the entropy of the product image to be defect-classified; if the score is greater than or equal to a dynamic preset threshold, the product image to be defect-classified is stored in the first image set; otherwise, the product image to be defect-classified is stored in the second image set; manually annotating each product image in the second image set, adjusting the dynamic preset threshold, and storing the product images in the second image set that are higher than the dynamic preset threshold in the first image set; Step S132: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template; Step S133: inputting the sub-image corresponding to each training sample into the neural network model in working state, the neural network model in working state is used to extract the feature difference between the defect-free product image and the defective image, classify the defects of the product image, and use the classification result as the output of the neural network model; training the neural network model in working state based on each training sample to obtain a trained neural network model in working state; Step S14: Based on the trained neural network model in working state, the most recently trained self-learning module is updated.

2. The method according to claim 1, characterized in that The step S1: training the self-learning module to obtain a self-learning module that has been preliminarily trained, includes: Step S11: For each neural network model in the self-learning module, training is performed in the following manner: Step S111: obtaining a number of product images; Step S112: for each product image: score the entropy of the product image; put product images with scores greater than or equal to a score threshold into a first image set, and put product images with scores less than the score threshold into a second image set; Step S113: taking all product images in the first image set as training samples; for each training sample: selecting a segmentation template from a plurality of pre-set segmentation templates, and segmenting the product image corresponding to the training sample into a plurality of sub-images according to the array or irregular layout set by the segmentation template; Step S114: inputting the sub-images corresponding to the training samples into the neural network model, the neural network model is used to extract the feature differences between the defect-free product images and the defective images, classify the defects of the product images, and use the classification results as the output of the neural network model; the neural network model is trained based on the training samples to obtain a trained neural network model; Step S12: Based on the fully trained neural network model, a preliminarily trained self-learning module is obtained.

3. The method according to any one of claims 1 to 2, characterized in that: The parallel neural network model is a convolutional neural network model and a machine learning model.

4. The method according to claim 3, characterized in that The method of adjusting the dynamic preset threshold is: Among them, k is the number of all product images in the first image set and the second image set, j is the contour points of all captured products in each product image in the second image set, PointGrayValue is the grayscale value of the contour points of the captured products, Ava is the intermediate variable to be calculated, y is the dynamic preset threshold, and i1 is a variable.

5. A product classification device based on a self-learning model, used to execute the method according to any one of claims 1 to 4, characterized in that: include: Preliminary Training module: configured to construct a self-learning module, wherein the self-learning module integrates a plurality of parallel neural network models, wherein when the self-learning module is in use, only one neural network model is in a working state; the self-learning module is trained to obtain a self-learning module that has been preliminarily trained; Let the most recently trained self-learning module be the self-learning module that has been initially trained; A retraining module: configured to listen to defect classification instructions and obtain product images to be defect-classified; based on the product type and product image quality in the product image, a neural network model in the most recently trained self-learning module is activated to a working state; based on the position of the product in the product image, a segmentation template is selected from a plurality of pre-set segmentation templates, and the product image is segmented into a plurality of sub-images of specified sizes according to the array or irregular layout set by the segmentation template; all the sub-images are input into the most recently trained self-learning module to obtain a product defect classification result of the product image; Update the most recently trained self-learning module.

6. A computer-readable storage medium, characterized in that: The storage medium stores a plurality of instructions; the plurality of instructions are used for a processor to load and execute the method as described in any one of claims 1 to 4.

7. An electronic device, characterized in that: The electronic device comprises: A processor, which is used to execute multiple instructions; A memory for storing a plurality of instructions; The plurality of instructions are used to be stored in the memory and loaded and executed by the processor according to any one of claims 1 to 4.

Images