Toothbrush sorting optimization method based on image recognition and machine learning

Abstract

The invention relates to the technical field of image recognition, and discloses a toothbrush sorting optimization method based on image recognition and machine learning, which comprises the following steps of: classifying toothbrushes, and judging whether the toothbrushes belong to categories in a classifier, a first increment classifier and a second increment classifier; if the first increment classifier and the second increment classifier belong to the classifiers, the classifications in the first increment classifier and the second increment classifier are normally classified; and if the toothbrush does not belong to the classified toothbrush in the classifier, the first increment classifier and the second increment classifier, not classifying the toothbrush, calculating a new increment classifier, and adding the new increment classifier to a machine learning classifier. Even if an enterprise produces new types of toothbrushes, the toothbrushes can be learned and classified, it is guaranteed that the types of the toothbrushes are continuously updated in real time in the toothbrush sorting process, and the actual production requirements of the enterprise are met.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a toothbrush sorting optimization method based on image recognition and machine learning. Background Art

[0002] Image recognition refers to the technology of using computers to process, analyze and understand images in order to identify targets and objects of various different patterns. It is a practical application of deep learning algorithms. In recent years, with the continuous improvement of my country's industrial automation level, machine vision positioning and sorting technology has been increasingly used in various industrial product sorting production lines.

[0003] Invention publication number in China: CN 107804514 A toothbrush sorting method based on image recognition proposed by A includes the following steps: S1: putting the toothbrush on a climbing feeding conveyor and sending it into a horizontal conveyor belt; S2: separating the toothbrushes one by one at the initial position of the horizontal conveyor belt; S3: when the photoelectric sensor on the horizontal conveyor belt detects that a toothbrush has passed, it triggers the industrial camera to complete image acquisition by sending a signal to the industrial computer; S4: the industrial computer calculates the posture information of the toothbrush by performing image processing and analysis on the acquired image; S5: finally, the robot grabs the toothbrush according to the calculated posture information, corrects the posture of the toothbrush and then passes it to the next process to complete. This scheme can improve the automation degree of the toothbrush packaging machine and avoid the disadvantage of large workload due to the complex variety of toothbrushes. However, this scheme has the following problems: in the increasingly competitive market, the development of new toothbrush varieties is an inevitable trend. When the company produces new toothbrushes, the original machine recognition system cannot distinguish the new toothbrushes and cannot realize the sorting of the newly produced toothbrushes. Therefore, a toothbrush sorting optimization method based on image recognition and machine learning is proposed to solve the above problems. Summary of the invention

[0004] 1. Technical issues to be resolved The purpose of the present invention is to solve the problem that in the increasingly competitive market, the development of new toothbrush varieties is an inevitable trend. When an enterprise produces a new toothbrush, the original machine recognition system cannot distinguish the new toothbrush and cannot perform the sorting of the newly produced toothbrush. A toothbrush sorting optimization method based on image recognition and machine learning is proposed.

[0005] (II) Technical solution The technical solution of the present invention to solve the above technical problems is as follows: A toothbrush sorting optimization method based on image recognition and machine learning comprises the following steps: S1: First, collect the background image during the sorting process , the background image After preprocessing, the background image is obtained ' ; S2: Collection The toothbrush images are divided into a training set T and a test set M. The images in the training set T are randomly scaled and cropped to obtain an image set T of size m×n. ’ ; S3: The image set T ’ After image processing, the toothbrush area is obtained , calculate the image set T ’ The regional feature vector , the regional feature vector Added as a classifier to the machine learning classifier; S4: Randomly scale and crop the images in the test set M to obtain an image set M of size m×n ’ , and M ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. ; S5: Select the toothbrush images with abnormal areas ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added as the first incremental classifier to the machine learning classifier; S6: Collection The images of toothbrushes are used as the test set N. The images of the test set N are randomly scaled and cropped to obtain an image set N of size m×n. ’ , and N ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. ; S7: Selecting the toothbrush image with abnormal area ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added to the machine learning classifier as a second incremental classifier; S8: Classify the toothbrushes produced subsequently using the classifier, the first incremental classifier and the second incremental classifier to determine whether the image belongs to the classifier, the first incremental classifier and the second incremental classifier.

[0006] Based on the above technical solution, the present invention can also be improved as follows.

[0007] Preferably, the random scaling cropping includes: fixed size cropping, random size cropping, single-side cropping, minimum external matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling; The original image is converted into an image of size m×n by a random scaling and cropping method that combines two or more of fixed size cropping, random size cropping, single-side cropping, minimum circumscribed matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling.

[0008] Preferably, the regional feature vector of the sample in S3 includes: toothbrush area , the rectangularity R of the toothbrush area, the area A of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, and the grayscale mean G of the hole area.

[0009] Preferably, selecting the toothbrush image with abnormal area comprises the following steps: Get the regional feature vector of the toothbrush image to be sorted, and compare the regional feature vector of the toothbrush image with the regional feature vector , eigenvector and the eigenvector All feature vectors are compared to find the regional feature vector that is most similar to the regional feature vector of the toothbrush image. Then the toothbrush image with a feature overlap less than F is confirmed as a regional abnormality. F is set to 0.95. The formula for feature overlap is as follows: , , is the area of ​​the toothbrush region of the toothbrush image to be sorted, is the rectangularity of the toothbrush area of ​​the toothbrush image to be sorted, is the area of ​​the hole area enclosed by the toothbrush area of ​​the toothbrush image to be sorted and the right boundary of its minimum circumscribed rectangle, is the grayscale mean of the hole area of ​​the toothbrush image to be sorted, is the most similar toothbrush area, is the rectangularity of the most similar toothbrush area, is the area of ​​the hole enclosed by the most similar toothbrush area and the right boundary of its smallest circumscribed rectangle, is the grayscale mean of the most similar hole area, is the toothbrush area influencing factor, is the influencing factor of the rectangularity of the toothbrush area, is the area influence factor of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, is the influencing factor of the grayscale mean value in the hole area.

[0010] Preferably, the specific process of S8 includes: Classify the subsequently produced toothbrushes to determine whether they belong to the classifier, the first incremental classifier, and the second incremental classifier; If it belongs to a classifier, the categories it belongs to in the first incremental classifier and the second incremental classifier are classified normally; If it does not belong to the classifier, the toothbrush classified in the first incremental classifier and the second incremental classifier is not classified, and its image is named ,Will After image processing, the toothbrush area is obtained ,calculate The regional feature vector And added to the machine learning classifier as a new incremental classifier.

[0011] (III) Beneficial effects Compared with the prior art, the technical solution of this application has the following beneficial technical effects: 1. The present invention keeps adding classifiers so that the entire system is always in a learning process. Even if the company produces new categories of toothbrushes, the toothbrushes can be learned and classified, ensuring that the categories of toothbrushes are constantly updated in real time during the toothbrush sorting process to meet the actual production needs of the company.

[0012] 2. The present invention uses the characteristic overlap formula to calculate the toothbrush area The four parameters of the toothbrush area rectangularity R, the hole area area A enclosed by the toothbrush area and the right border of its smallest circumscribed rectangle, and the grayscale mean G of the hole area are integrated into one formula. Compared with a single parameter, it is more accurate to select the toothbrush image with abnormal area. DETAILED DESCRIPTION

[0013] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0014] A toothbrush sorting optimization method based on image recognition and machine learning, characterized in that it comprises the following steps: S1: First, collect the background image during the sorting process , the background image After preprocessing, the background image is obtained ', establish a background model, continuously collect several empty horizontal conveyor belt background images, calculate the average pixel value of each corresponding position, and synthesize a new image as the background image; S2: Collection The toothbrush images are divided into a training set T and a test set M. The images in the training set T are randomly scaled and cropped to obtain an image set T of size m×n. ’ ; S3: The image set T ’ After image processing, the toothbrush area is obtained , calculate the image set T ’ The regional feature vector , the regional feature vector Added as a classifier to the machine learning classifier; S4: Randomly scale and crop the images in the test set M to obtain an image set M of size m×n ’ , and M ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. , this operation can smooth the region contour and remove both small and large connected regions; S5: Select the toothbrush images with abnormal areas ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added as the first incremental classifier to the machine learning classifier; S6: Collection The images of toothbrushes are used as the test set N. The images of the test set N are randomly scaled and cropped to obtain an image set N of size m×n. ’ , and N ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. ; S7: Selecting the toothbrush image with abnormal area ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added to the machine learning classifier as a second incremental classifier; S8: Classify the toothbrushes produced subsequently using the classifier, the first incremental classifier and the second incremental classifier to determine whether the image belongs to the classifier, the first incremental classifier and the second incremental classifier.

[0015] It should be noted that the eigenvector , the eigenvector and the eigenvector The efficacy was consistent in the machine learning classifier during image recognition of subsequent toothbrushes.

[0016] The random scaling cropping includes: fixed size cropping, random size cropping, single-side cropping, minimum external matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling; The original image is converted into an image of size m×n by a random scaling and cropping method that combines two or more of fixed size cropping, random size cropping, single-side cropping, minimum circumscribed matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling.

[0017] The regional feature vectors of the samples in S3 include: toothbrush area , the rectangularity R of the toothbrush area, the area A of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, and the grayscale mean G of the hole area.

[0018] Toothbrush area It can be obtained directly by counting the number of white pixels in the binary image. In Python, the OpenCV library can be used to complete this task. The following is the Python code: import cv2 import numpy as np # Assume that img is a binary toothbrush image # Calculate the number of white pixels, that is, the area of ​​the toothbrush area area = np.sum(img == 255).

[0019] The rectangularity R of the toothbrush region is the ratio of the area of ​​the region to the area of ​​its minimum enclosing rectangle. First, we need to find the minimum enclosing rectangle of the region and then calculate its area. The following is the Python code: # Find contours contours, _= cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) # Assume the largest contour is the toothbrush contour max_contour = max(contours, key=cv2.contourArea) # Calculate the minimum enclosing rectangle x, y, w, h = cv2.boundingRect(max_contour) rect_area = w * h # Calculate rectangularity rectangularity = area / rect_area.

[0020] The area A of the hole region enclosed by the toothbrush region and the right boundary of its minimum circumscribed rectangle refers to the area between the toothbrush region and the right boundary of its minimum circumscribed rectangle.

[0021] First, create a black image of the same size as the original image.

[0022] Draw the minimum bounding rectangle of the toothbrush on the image (filled with white).

[0023] Perform a logical AND operation (AND) on the original binary image (the toothbrush is white) and this rectangular image. In the result, the part outside the toothbrush area (including the holes) will be black.

[0024] Calculate the area of ​​the hole region (i.e. the number of white pixels in the resulting image, but minus the area of ​​the toothbrush region).

[0025] The calculation method of the grayscale mean G in the hole area is as follows: Use the above method (or a more precise method) to locate the hole area.

[0026] Create a mask image of the same size as the original image, where the hole areas are white (or a non-zero value) and the rest are black.

[0027] Use the mask image to perform a mask operation on the original grayscale image to obtain an image that only contains the hole area.

[0028] Calculate the grayscale mean of the image.

[0029] In the case of a binary image, since the hole area is black (grayscale value is 0), the grayscale mean will be 0 (if the hole area is composed entirely of black pixels).

[0030] The step of selecting the toothbrush image with abnormal area comprises the following steps: Get the regional feature vector of the toothbrush image to be sorted, and compare the regional feature vector of the toothbrush image with the regional feature vector , eigenvector and the eigenvector All feature vectors are compared to find the regional feature vector that is most similar to the regional feature vector of the toothbrush image. Then the toothbrush image with a feature overlap less than F is confirmed as a regional abnormality. F is set to 0.95. The formula for feature overlap is as follows: , , is the area of ​​the toothbrush region of the toothbrush image to be sorted, is the rectangularity of the toothbrush area of ​​the toothbrush image to be sorted, is the area of ​​the hole area enclosed by the toothbrush area of ​​the toothbrush image to be sorted and the right boundary of its minimum circumscribed rectangle, is the grayscale mean of the hole area of ​​the toothbrush image to be sorted, is the most similar toothbrush area, is the rectangularity of the most similar toothbrush area, is the area of ​​the hole enclosed by the most similar toothbrush area and the right boundary of its smallest circumscribed rectangle, is the grayscale mean of the most similar hole area, is the toothbrush area influencing factor, is the influencing factor of the rectangularity of the toothbrush area, is the area influence factor of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, is the influencing factor of the grayscale mean value in the hole area.

[0031] The specific process of S8 includes: Classify the subsequently produced toothbrushes to determine whether they belong to the classifier, the first incremental classifier, and the second incremental classifier; If it belongs to a classifier, the categories it belongs to in the first incremental classifier and the second incremental classifier are classified normally; If it does not belong to the classifier, the toothbrush classified in the first incremental classifier and the second incremental classifier is not classified, and its image is named ,Will After image processing, the toothbrush area is obtained ,calculate The regional feature vector And added to the machine learning classifier as a new incremental classifier.

[0032] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0033] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A toothbrush sorting optimization method based on image recognition and machine learning, characterized in that: The following steps are involved: S1: First, collect the background image of the sorting process , the background image After preprocessing, the background image is obtained ' ; S2: Collection The toothbrush images are divided into a training set T and a test set M. The images in the training set T are randomly scaled and cropped to obtain an image set T of size m×n. ’ ; S3: The image set T ’ After image processing, the toothbrush area is obtained , calculate the image set T ’ The regional feature vector , the regional feature vector Added as a classifier to the machine learning classifier; S4: Randomly scale and crop the images in the test set M to obtain an image set M of size m×n ’ , and M ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. ; S5: Select the toothbrush images with abnormal areas ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added as the first incremental classifier to the machine learning classifier; S6: Collection The images of toothbrushes are used as the test set N. The images of the test set N are randomly scaled and cropped to obtain an image set N of size m×n. ’ , and N ’ Images and background images in ' Subtract and take the absolute value, then perform global fixed threshold processing to obtain a binary image, and finally perform morphological opening operation on the image to obtain the area of ​​the toothbrush in the image. ; S7: Selecting the toothbrush image with abnormal area ,Will After image processing, the toothbrush area is obtained , calculate the image set M ’ The regional feature vector , the regional feature vector Added to the machine learning classifier as a second incremental classifier; S8: Classify the toothbrushes produced subsequently using the classifier, the first incremental classifier and the second incremental classifier to determine whether the image belongs to the classifier, the first incremental classifier and the second incremental classifier.

2. A toothbrush sorting optimization method based on image recognition and machine learning according to claim 1, characterized in that: The random scaling cropping includes: fixed size cropping, random size cropping, single-side cropping, minimum external matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling; The original image is converted into an image of size m×n by a random scaling and cropping method that combines two or more of fixed size cropping, random size cropping, single-side cropping, minimum circumscribed matrix cropping, bilinear interpolation scaling and bicubic interpolation scaling.

3. The toothbrush sorting optimization method based on image recognition and machine learning according to claim 1, characterized in that: The regional feature vectors of the samples in S3 include: toothbrush area , the rectangularity R of the toothbrush area, the area A of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, and the grayscale mean G of the hole area.

4. The toothbrush sorting optimization method based on image recognition and machine learning according to claim 1, characterized in that: The step of selecting the toothbrush image with abnormal area comprises the following steps: Get the regional feature vector of the toothbrush image to be sorted, and compare the regional feature vector of the toothbrush image with the regional feature vector , eigenvector and the eigenvector All feature vectors are compared to find the regional feature vector that is most similar to the regional feature vector of the toothbrush image. Then the toothbrush image with a feature overlap less than F is confirmed as a regional abnormality. F is set to 0.

95. The formula for feature overlap is as follows: , , is the toothbrush area of ​​the toothbrush image to be sorted, is the rectangularity of the toothbrush area of ​​the toothbrush image to be sorted, is the area of ​​the hole area enclosed by the toothbrush area of ​​the toothbrush image to be sorted and the right boundary of its minimum circumscribed rectangle, is the grayscale mean of the hole area of ​​the toothbrush image to be sorted, is the most similar toothbrush area, is the rectangularity of the most similar toothbrush area, is the area of ​​the hole enclosed by the most similar toothbrush area and the right boundary of its smallest circumscribed rectangle, is the grayscale mean of the most similar hole area, is the influencing factor of the toothbrush area, is the influencing factor of the rectangularity of the toothbrush area, is the area influence factor of the hole area enclosed by the toothbrush area and the right boundary of its minimum circumscribed rectangle, is the grayscale mean influencing factor of the hole area.

5. The toothbrush sorting optimization method based on image recognition and machine learning according to claim 1, characterized in that: The specific process of S8 includes: Classify the subsequently produced toothbrushes to determine whether they belong to the classifier, the first incremental classifier, and the second incremental classifier; If it belongs to a classifier, the categories in the first incremental classifier and the second incremental classifier are classified normally; If it does not belong to the classifier, the toothbrush classified in the first incremental classifier and the second incremental classifier is not classified, and its image is named ,Will After image processing, the toothbrush area is obtained ,calculate The regional feature vector And added to the machine learning classifier as a new incremental classifier.