An intelligent automatic picking system based on the Internet of Things

Abstract

The present invention discloses an intelligent automatic picking system based on the Internet of Things, which belongs to the technical field of production management, and specifically comprises: storing defect sample images of products to be sorted, and marking corresponding defect types; generating a three-dimensional model of the product according to product parameters; measuring the actual distance between the product surface and the distance sensor when the product passes through the distance sensor by a distance sensor arranged above a conveyor belt, and calculating the theoretical distance between the model surface and the distance sensor according to the size of the product in the three-dimensional model, and marking the position where the difference between the actual distance and the theoretical distance is greater than a preset error as an abnormal position; collecting a regional image of the abnormal position, and pre-processing the regional image, constructing a defect recognition model according to the defect sample image, inputting the regional image into the defect recognition model, and outputting a corresponding defect type recognition result; the present invention optimizes the detection method for product defects and reduces the cost.

Description

Technical Field

[0001] The present invention relates to the technical field of production management, and in particular to an intelligent automatic picking system based on the Internet of Things. Background Art

[0002] With the rapid development of the Internet of Things and automation technology, intelligent automatic picking systems are playing an increasingly important role in modern production and logistics. Traditional picking systems usually rely on manual visual inspection to identify product defects, which is not only inefficient but also easily affected by operator fatigue and subjective judgment, resulting in unstable detection accuracy. In addition, manual inspection cannot meet the needs of high-speed production lines and becomes a bottleneck for improving production efficiency.

[0003] To solve the above problems, automatic inspection systems based on image processing have been proposed and applied to the identification of product defects. These systems capture product images through cameras and use computer vision algorithms to analyze images to identify defects. However, existing image processing systems have certain limitations when processing products in motion and with irregular surfaces, and real-time defect identification and detection requires a lot of computing power and cost. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent automatic picking system based on the Internet of Things to solve the following technical problems:

[0005] Existing image processing systems have certain limitations when processing products in motion or with irregular surfaces, and real-time defect identification and detection requires a lot of computing power and cost.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An intelligent automatic picking system based on the Internet of Things, comprising:

[0008] Image library, used to store defect sample images of products to be sorted, classify each defect sample image and mark the corresponding defect type;

[0009] Model building module, used to generate a three-dimensional model of the product according to the product design drawing and design size parameters;

[0010] an abnormality detection module, based on a distance sensor disposed above the conveyor belt, measuring the actual distance between the surface of the product and the distance sensor when the product passes through the distance sensor, and calculating the theoretical distance between the surface of the model and the distance sensor according to the size of the product in the three-dimensional model, and when a position where the difference between the actual distance and the theoretical distance is greater than a preset error is detected, marking the position as an abnormal position;

[0011] The defect recognition module is used to collect the regional image of the abnormal position and preprocess the regional image, build a defect recognition model based on the Transformer deep network according to the defect sample image, input the regional image into the defect recognition model, and output the corresponding defect type recognition result.

[0012] As a further solution of the present invention: the specific process of detecting the abnormal position in the abnormality detection module is:

[0013] A detection baseline is set in the detection screen of the distance sensor, the detection baseline is perpendicular to the moving direction of the conveyor belt, the conveying speed v of the conveyor belt is obtained, and the timing starts from the initial moment when the product is placed on the conveyor belt;

[0014] When the front end of the product contacts the detection baseline, the distance sensor starts to measure the actual distance l1 between the product surface and the sensor, and the measurement interval is t1, where t1 is a preset unit interval;

[0015] Taking the front end of the product as the base point, according to the conveying speed v and conveying time t of the conveyor belt, calculate the actual distance L1 of the product being transported, obtain the distance L2 between the distance sensor and the initial position of the product conveyance, and calculate the distance L'=L1-L2 between the measured product surface position and the front section of the product each time;

[0016] Obtain the theoretical height H1 of the position at a distance L' from the front end of the product from the three-dimensional model, obtain the height H2 of the conveyor belt, and the setting height H3 of the distance sensor, then the theoretical distance l2 between the L' position and the distance sensor = H3-H1-H2;

[0017] The difference between the actual distance l1 and the theoretical distance l2 is calculated. If the difference is greater than the preset error, the L' position is an abnormal position.

[0018] As a further solution of the present invention: the construction process of the defect recognition model is:

[0019] Obtain defect sample images with labeled types, preprocess the defect sample images, and convert the preprocessed defect sample images into image vectors. The image vectors and tokens used as prompts are encoded and sorted and then input into the Transformer deep network to extract the defect features of the defect sample images. The Transformer deep network is updated using the labeled type back propagation, the defect sample images and the model are repeatedly updated, and the iteration is repeated until the loss function of the model converges to generate a defect recognition model.

[0020] As a further solution of the present invention: the process of image preprocessing is:

[0021] The image is subjected to denoising, the edge detection is performed on the denoised image by using a Sobel edge detection operator, the image after edge detection is binarized by using a maximum inter-class variance method to obtain a binarized image, and the binarized image is subjected to denoising again.

[0022] As a further solution of the present invention: the process of the noise reduction processing is:

[0023] The random noise in the image is removed by Gaussian filtering, median filtering and wavelet transform, and the directional defect type is enhanced by directional sequential filtering method.

[0024] As a further solution of the present invention: the classification of the defect sample images is manually labeled.

[0025] As a further solution of the present invention: the regional image is centered on the abnormal position and has a preset length r as a radius to generate a circular regional image.

[0026] Beneficial effects of the present invention:

[0027] The present invention integrates three-dimensional model information and real-time distance measurement technology. It first identifies the position on the product surface where the size does not meet the product standard through distance measurement, thereby performing targeted image acquisition. The pre-trained model is then used to identify the defect type of the acquired image. It can accurately identify defects on the product surface without being restricted by the material and shape of the product, and does not require full sampling of the product surface, thereby improving the efficiency of image processing, having a wide range of applications, and significantly improving the detection speed, thereby meeting the needs of high-speed production lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further described below in conjunction with the accompanying drawings.

[0029] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] See also Figure 1 As shown, the present invention is an intelligent automatic picking system based on the Internet of Things, comprising:

[0032] Image library, used to store defect sample images of products to be sorted, classify each defect sample image and mark the corresponding defect type;

[0033] Model building module, used to generate a three-dimensional model of the product according to the product design drawing and design size parameters;

[0034] an abnormality detection module, based on a distance sensor disposed above the conveyor belt, measuring the actual distance between the surface of the product and the distance sensor when the product passes through the distance sensor, and calculating the theoretical distance between the surface of the model and the distance sensor according to the size of the product in the three-dimensional model, and when a position where the difference between the actual distance and the theoretical distance is greater than a preset error is detected, marking the position as an abnormal position;

[0035] The defect recognition module is used to collect the regional image of the abnormal position and preprocess the regional image, build a defect recognition model based on the Transformer deep network according to the defect sample image, input the regional image into the defect recognition model, and output the corresponding defect type recognition result.

[0036] The present invention integrates three-dimensional model information and real-time distance measurement technology. It first identifies the position on the product surface where the size does not meet the product standard through distance measurement, thereby performing targeted image acquisition. The pre-trained model is then used to identify the defect type of the acquired image. It can accurately identify defects on the product surface without being restricted by the material and shape of the product, and does not require full sampling of the product surface, thereby improving the efficiency of image processing, having a wide range of applications, and significantly improving the detection speed, thereby meeting the needs of high-speed production lines.

[0037] In another preferred embodiment of the present invention, the specific process of detecting the abnormal position in the abnormality detection module is:

[0038] A detection baseline is set in the detection screen of the distance sensor, the detection baseline is perpendicular to the moving direction of the conveyor belt, the conveying speed v of the conveyor belt is obtained, and the timing starts from the initial moment when the product is placed on the conveyor belt;

[0039] When the front end of the product contacts the detection baseline, the distance sensor starts to measure the actual distance l1 between the product surface and the sensor, and the measurement interval is t1, where t1 is a preset unit interval;

[0040] Taking the front end of the product as the base point, according to the conveying speed v and conveying time t of the conveyor belt, calculate the actual distance L1 of the product being transported, obtain the distance L2 between the distance sensor and the initial position of the product conveyance, and calculate the distance L'=L1-L2 between the measured product surface position and the front section of the product each time;

[0041] Obtain the theoretical height H1 of the position at a distance L' from the front end of the product from the three-dimensional model, obtain the height H2 of the conveyor belt, and the setting height H3 of the distance sensor, then the theoretical distance l2 between the L' position and the distance sensor = H3-H1-H2;

[0042] The difference between the actual distance l1 and the theoretical distance l2 is calculated. If the difference is greater than the preset error, the L' position is an abnormal position.

[0043] In another preferred embodiment of the present invention, the defect recognition model is constructed as follows:

[0044] Obtain defect sample images with labeled types, preprocess the defect sample images, and convert the preprocessed defect sample images into image vectors. The image vectors and tokens used as prompts are encoded and sorted and then input into the Transformer deep network to extract the defect features of the defect sample images. The Transformer deep network is updated using the labeled type back propagation, the defect sample images and the model are repeatedly updated, and the iteration is repeated until the loss function of the model converges to generate a defect recognition model.

[0045] The defect recognition model built using the Transformer deep network, through its unique self-attention mechanism and flexible architecture design, combined with precise image preprocessing methods, enables the system to accurately distinguish defects from normal surface changes even in high-noise environments, reducing false detection and missed detection rates.

[0046] In another preferred embodiment of the present invention, the image preprocessing process is:

[0047] The image is subjected to denoising, the edge detection is performed on the denoised image by using a Sobel edge detection operator, the image after edge detection is binarized by using a maximum inter-class variance method to obtain a binarized image, and the binarized image is subjected to denoising again.

[0048] In another preferred embodiment of the present invention, the process of the noise reduction processing is:

[0049] The random noise in the image is removed by Gaussian filtering, median filtering and wavelet transform, and the directional defect type is enhanced by directional sequential filtering method.

[0050] In the image preprocessing stage, commonly used filtering algorithms such as Gaussian filtering, median filtering and wavelet transform can effectively remove random noise in the image and reduce the interference of noise on subsequent defect detection.

[0051] For directional defect features, a directional sequential filtering method can be used. This method can not only remove noise, but also enhance the target information to be detected in the same direction, laying a good foundation for accurate defect segmentation.

[0052] In another preferred embodiment of the present invention, the classification of the defect sample images is performed by manual annotation.

[0053] In another preferred embodiment of the present invention, the regional image is centered on the abnormal position and has a preset length r as a radius to generate a circular regional image.

[0054] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. An intelligent automatic picking system based on the Internet of Things, It is characterized in that include: Image library, used to store defect sample images of products to be sorted, classify each defect sample image and mark the corresponding defect type; Model building module, used to generate a three-dimensional model of the product according to the product design drawing and design size parameters; an abnormality detection module, based on a distance sensor disposed above the conveyor belt, measuring the actual distance between the surface of the product and the distance sensor when the product passes through the distance sensor, and calculating the theoretical distance between the surface of the model and the distance sensor according to the size of the product in the three-dimensional model, and when a position where the difference between the actual distance and the theoretical distance is greater than a preset error is detected, marking the position as an abnormal position; A defect recognition module is used to collect the regional image of the abnormal position, pre-process the regional image, build a defect recognition model based on the Transformer deep network according to the defect sample image, input the regional image into the defect recognition model, and output the corresponding defect type recognition result; The specific process of detecting abnormal positions in the abnormality detection module is as follows: A detection baseline is set in the detection screen of the distance sensor, the detection baseline is perpendicular to the moving direction of the conveyor belt, the conveying speed v of the conveyor belt is obtained, and the timing starts from the initial moment when the product is placed on the conveyor belt; When the front end of the product contacts the detection baseline, the distance sensor starts to measure the actual distance l1 between the product surface and the sensor, and the measurement interval is t1, where t1 is a preset unit interval; Taking the front end of the product as the base point, according to the conveying speed v and conveying time t of the conveyor belt, calculate the actual distance L1 of the product being transported, obtain the distance L2 between the distance sensor and the initial position of the product conveyance, and calculate the distance L'=L1-L2 between the measured product surface position and the front section of the product each time; Obtain the theoretical height H1 of the position at a distance L' from the front end of the product from the three-dimensional model, obtain the height H2 of the conveyor belt, and the setting height H3 of the distance sensor, then the theoretical distance l2 between the L' position and the distance sensor = H3-H1-H2; The difference between the actual distance l1 and the theoretical distance l2 is calculated. If the difference is greater than the preset error, the L' position is an abnormal position.

2. According to the intelligent automatic picking system based on the Internet of Things according to claim 1, It is characterized in that The construction process of the defect recognition model is as follows: Obtain defect sample images with labeled types, preprocess the defect sample images, and convert the preprocessed defect sample images into image vectors. The image vectors and tokens used as prompts are encoded and sorted and then input into the Transformer deep network to extract the defect features of the defect sample images. The Transformer deep network is updated using the labeled type back propagation, the defect sample images and the model are repeatedly updated, and the iteration is repeated until the loss function of the model converges to generate a defect recognition model.

3. According to the intelligent automatic picking system based on the Internet of Things as described in claim 2, It is characterized in that The image preprocessing process is as follows: The image is subjected to denoising, the edge detection is performed on the denoised image by using a Sobel edge detection operator, the image after edge detection is binarized by using a maximum inter-class variance method to obtain a binarized image, and the binarized image is subjected to denoising again.

4. According to claim 3, an intelligent automatic picking system based on the Internet of Things, It is characterized in that The process of the noise reduction process is as follows: The random noise in the image is removed by Gaussian filtering, median filtering and wavelet transform, and the directional defect type is enhanced by directional sequential filtering method.

5. According to the intelligent automatic picking system based on the Internet of Things according to claim 1, It is characterized in that The classification of the defect sample images is manually marked.

6. According to claim 1, an intelligent automatic picking system based on the Internet of Things, It is characterized in that The regional image is centered at the abnormal position and has a preset length r as a radius to generate a circular regional image.

Images