A surface defect detection method for disposable tableware based on deep learning

Abstract

The present invention discloses a surface defect detection system for disposable tableware based on deep learning, comprising the following steps: Step S1: multiple processes acquire video streams in real time and, based on hardware trigger signals, acquire images of disposable tableware to be processed by each process; Step S2: each process performs data preprocessing operations on the images to be processed, including correcting cropping, adjusting the brightness and contrast of the images, and performing scaling; Step S3: the preprocessed images to be tested are sent to a defect detection module, where the types of defects detected include material defects and processing defects; Step S4: post-processing the detection results placed in a shared memory by the three processes, including reducing repeated push and storage, and merging the result encoding. The present invention relates to the field of target detection and recognition technology, and has the beneficial effect of using three processes for data reading, writing, and prediction, thereby ensuring the inference performance of the target detection model and the efficiency of computing chip utilization.

Description

Technical Field

[0001] The present invention relates to the field of target detection and recognition, and in particular to a surface defect detection system for disposable tableware based on deep learning. Background Art

[0002] Currently, the mainstream manufacturing production method is still characterized by assembly line production. However, due to the limitations of this standardized production, companies have difficulty accurately meeting consumers' growing personalized needs. With the booming development of new-generation information technology, represented by artificial intelligence, AI and other technologies are rapidly integrating into all areas of economic and social development. The integration effect is becoming increasingly apparent, significantly impacting production and life. Therefore, in the global manufacturing value distribution chain, AI can accelerate China's capture of high-value segments of the industry chain, such as technology research and development, product design, and high-value-added services, and help it climb to the high end of the industrial value chain.

[0003] In terms of production and manufacturing, artificial intelligence can penetrate into all aspects such as R&D design, production line equipment, production processes, and production decisions. Therefore, it can solve many pain points in the digital transformation and upgrading of the manufacturing industry. It can provide the manufacturing industry with solutions such as visual inspection, automated control, intelligent calibration, and root cause analysis, promote manufacturing equipment innovation, reduce the dependence of manufacturing automation on foreign technology and equipment, and ultimately bring huge value to the manufacturing industry's move towards intelligent manufacturing.

[0004] Product defect detection is a crucial component of industrial production. Since the dawn of the industrial age, manufacturers have been using various technologies to monitor process and product quality on assembly lines. Early product quality inspections relied primarily on manual labor. However, with the expansion of manufacturing and the advancement of industrial automation, relying on real-time manual monitoring of product quality issues on the production line has become increasingly difficult. Applying intelligent manufacturing methods to product defect detection can effectively address various issues inherent in manual inspections. These include the susceptibility of manual inspections to external environmental interference, the difficulty in ensuring accuracy and efficiency, and the susceptibility to subjective influences, which can lead to high rates of missed and missed inspections, low production capacity, and low yield rates. Furthermore, manual inspections are limited in time, have high initial personnel training costs, and are difficult to control. Furthermore, relying solely on visual inspection and empirical judgment makes it difficult to establish quantifiable quality standards. With the rapid advancement of computer technology, a variety of automated defect detection technologies based on digital image processing and computer vision have been applied across various manufacturing industries, such as ceramic tiles, electronic printing, and steel plates. These defect detection technologies have significantly driven the automation and intelligent development of manufacturing enterprises.

[0005] Amidst the dire global plastic pollution crisis, "plastic restriction" has become a global consensus, with countries implementing policies such as plastic restrictions and bans. Products made from natural plant materials, such as wood, are gaining widespread favor, with wooden tableware becoming the "best alternative" to plastic. Specifically, defect detection in the disposable tableware industry requires two key aspects of quality inspection: material defects and processing defects. First, inspect the wood material of the disposable tableware for defects such as decay and knots; second, inspect for processing defects caused by steam treatment and mold application, such as missing pieces, burrs, splits, and bends. Both types of defects must be detected and eliminated to prevent substandard products from reaching customers, impacting their dining experience and negatively impacting the company.

[0006] Currently, traditional quality inspection methods used by companies primarily rely on visual inspection. However, the high speed of the production line prevents real-time sorting during the production process, forcing post-process inspections to be performed. However, this method suffers from a low sampling rate, resulting in a high probability of missed inspections of substandard products. Therefore, finding a suitable, highly accurate, universal, and fast disposable tableware defect detection method for the disposable tableware industry is an urgent challenge. Summary of the Invention

[0007] The purpose of this invention is to solve the above problems and design a disposable tableware surface defect detection system based on deep learning.

[0008] The technical solution of the present invention to achieve the above-mentioned purpose is a disposable tableware surface defect detection system based on deep learning, comprising the following steps:

[0009] Step S1: multiple processes acquire video streams in real time, and acquire the disposable tableware images to be processed by each process according to the hardware trigger signal;

[0010] Step S2: Each process performs data preprocessing operations on the image to be processed, including correction and cropping, adjustment of image brightness and contrast, and scaling;

[0011] Step S3: sending the pre-processed image to be tested to the defect detection module, where the defect types detected include material defects and processing defects;

[0012] Step S4: Post-process the detection results put into the shared memory by the three processes, including reducing repeated push and storage, and merging the result encoding.

[0013] Preferably, in step S1, the disposable tableware to be inspected is placed in a stable light environment, and the ambient light source for each camera to capture images is configured according to the characteristics of the defects to be inspected;

[0014] Configuring different ambient light sources is to obtain better image information;

[0015] The camera takes the original image frame according to the trigger signal, and each process obtains the image to be processed.

[0016] Preferably, the correction cropping in step S2 is because the actual production line situation is more complicated, and the position of the tableware may be tilted due to external interference factors. It is necessary to first correct the position of the object to be measured in the image, and then crop it at a fixed position after the processing is completed, retaining the information of the object to be measured at the specified position and removing useless background information.

[0017] Preferably, the brightness and contrast of the image are adjusted in step S2 to make the defect features in the image to be tested clearer and more conducive to defect detection. Secondly, the image is scaled so that its size meets the input requirements of the detection model.

[0018] Preferably, the defect detection module in step S3 sends the image to be detected after data preprocessing to the defect detection model controlled by each process, and the output result of each process is the type and location information of the defect detected by the model.

[0019] Preferably, in step S3, before the defect detection model performs defect location information detection and type information identification, the method further includes:

[0020] Deep learning defect detection model training: Collect images of disposable tableware containing different types of defects and divide them into three categories according to the defect type. Perform the following operations on each data set: Annotate all data with rectangular box labels that represent the location of the defect; Divide the annotated images into training set, validation set, and test set according to material defects and processing defects; Use the divided training set and validation set to train the defect detection model respectively, input the test set into the model to obtain the type and location information of the defect, and perform model evaluation at the same time, and finally obtain a detection model that matches material defects and processing defects.

[0021] Material defect detection in step S3: Material defects include knot defects and decay defects. The main process is responsible for detecting material defects. It inputs the pre-processed image to the model responsible for material defect detection, obtains information such as defect type and confidence level, and performs post-processing. If the defect confidence score exceeds the set judgment threshold, the material defect detection module outputs a result indicating the presence of a material defect. Otherwise, the material defect detection module outputs a result indicating the absence of a material defect. Finally, this result is stored in a specified location in shared memory, and the result flag of the first process is set high, indicating that the first process has completed detection within the current cycle.

[0022] Processing defect detection in step S3: Processing defects include cracks, missing pieces, and oil stains. The two sub-processes initiated by the main process are responsible for detecting processing defects. The image to be tested, which has completed the above preprocessing, is input into the processing defect detection model, and information such as defect type and confidence level is obtained for post-processing. If at least one of the defect recognition scores output by the two sub-processes is greater than the set judgment threshold, the processing defect detection module outputs a result indicating the presence of a processing defect; otherwise, the processing defect detection module outputs a result indicating the absence of a processing defect. This result is then stored in shared memory, and the result flags of the second and third processes are set high, indicating that the second and third processes have completed detection within the current cycle.

[0023] Preferably, in step S4, repeated push and storage are reduced. First, the main process reads the three result flags in the shared memory. If all are high, it indicates that all three processes have been detected in this cycle. Secondly, the detection results of each process are read from the shared memory and statistically merged. The results of each process and the cycle of the camera distance kicking device are analyzed, and it is calculated whether there are repeated results for a certain tableware. If repeated results appear, no secondary storage is performed to reduce the pressure on the server and the industrial computer.

[0024] Preferably, the merge result encoding in step S4 refers to merging the output results of the main process and the two sub-processes, encoding the results and sending them to the industrial computer, which determines whether to discard the current disposable tableware based on the signal.

[0025] A deep learning-based surface defect detection system for disposable tableware, built using the technical solution of this invention, employs three processes for data reading, writing, and prediction, ensuring the inference performance of the target detection model and the efficient use of computing chips. The system employs intelligent visual inspection methods, tightly integrating quality inspection with the production process. This helps customers achieve automated production, standardized quality, and data-driven analysis. It eliminates challenges such as large fluctuations in manual sampling results, low detection accuracy, and the inability to control and quantify the inspection process. This ensures a high yield rate for disposable tableware and enhances the product's core competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic structural diagram of a disposable tableware surface defect detection system based on deep learning according to the present invention;

[0027] Figure 2 This is the production line distribution in an embodiment of a disposable tableware surface defect detection system based on deep learning described in the present invention. DETAILED DESCRIPTION

[0028] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1-2As shown, a disposable tableware surface defect detection system based on deep learning;

[0029] like Figure 1 As shown, the disposable tableware surface defect detection system based on deep learning technology of the present invention includes the following steps:

[0030] Step S1: After the system starts, the main process starts multiple sub-processes. Each sub-process is responsible for controlling a camera to obtain a real-time video stream and obtain the wooden spoon image to be processed by each process according to the hardware trigger signal. The main process monitors the running status of the two sub-processes at the same time.

[0031] It's important to note that disposable wooden spoons contain both material and processing defects. Material defects include decay and knots, while processing defects include missing flesh, cracks, and oil stains. The number and location of these defects in a single spoon are variable. Three processes control three cameras to capture separate streams, but the hardware trigger signal arrives simultaneously. The cameras controlled by each process capture frames and store them in their respective allocated memory.

[0032] In step S1, the disposable tableware to be inspected is placed in a stable light environment, and the lighting method used in each camera environment is determined according to the characteristics of the defect to be inspected.

[0033] It should be noted that due to the relatively open environment of the actual production line, a stable lighting environment was specially customized to capture images of the tableware set to prevent natural light from affecting the characteristics of the wooden spoon. For large cracks, bottom lighting was used to create a translucent effect, thereby making the large crack features more prominent. For small cracks, top lighting was used to make the texture and small crack features clearer. However, in practice, the brightness balance between the upper and lower lights and parameters such as camera gain must be accurately adjusted. The final solution adopted was to use a combination of bottom and top lighting in the camera environment controlled by the main process, while the camera environments controlled by the two sub-processes only used top lighting.

[0034] In step S1, the camera takes the original image frame according to the trigger signal, and each process obtains the image to be processed;

[0035] It should be noted that the three cameras are capturing images of different spoons at different positions on the production line. The results of the three cameras will be integrated and merged into the results of the spoon that has currently reached the scrapping position.

[0036] Step S2: Each process performs data preprocessing operations on the image to be processed, including correction and cropping, adjustment of image brightness and contrast, and scaling;

[0037] The correction cropping in step S2 is because the actual production line is more complicated. The position of the tableware may be tilted due to external interference factors. It is necessary to use image processing technology to detect the outline of the object. Then, the rotation angle and rotation matrix of the object are calculated based on the outline information. The image is then corrected using the rotation matrix to keep the object in a vertical position. After the processing is completed, the image is cropped at a fixed position, retaining the information of the object to be measured at the specified position and removing useless background information.

[0038] It should be noted that due to the long shape of wooden spoons, when a camera captures an image, N wooden spoons will be included in the image. The distance between the camera and the scrapping device is fixed. For example, if the distance between the camera and the scrapping device is T wooden spoons, then after obtaining an image, only the image of a single wooden spoon that is T intervals away from the scrapping device needs to be captured. Not all wooden spoons in the image need to be predicted. This ensures that if a defect is detected in the wooden spoon, it can be scrapped after T cycles.

[0039] The purpose of adjusting the brightness and contrast of the image in step S2 is to make the defect features in the image to be tested clearer and more conducive to defect detection. Secondly, the image is scaled so that its size meets the input requirements of the detection model.

[0040] It should be noted that in actual production, the probability of large cracks in wooden spoons is relatively small, and most cracks are small in size. Therefore, adjusting the brightness and contrast of the image to be tested can make these small defects more obvious.

[0041] Step S3: sending the pre-processed image to be tested to the defect detection module, where the defect types detected include material defects and processing defects;

[0042] The defect detection module in step S3: sends the image to be detected after data preprocessing to the defect detection model controlled by each process. The model result of each process is the type and location information of the detected defect;

[0043] Furthermore, before the defect detection model performs defect location information detection and type information identification, the method further includes:

[0044] Deep learning defect detection model training: Collect images of disposable tableware containing different types of defects and divide them into three categories according to the defect type. Perform the following operations on each data set: Annotate all data with rectangular box labels that represent the location of the defect; Divide the annotated images into training set, validation set, and test set according to material defects and processing defects; Use the divided training set and validation set to train the defect detection model respectively, input the test set into the model to obtain the type and location information of the defect, and perform model evaluation at the same time, and finally obtain a detection model that matches material defects and processing defects.

[0045] Material defect detection in step S3: Material defects include knot defects and decay defects. The main process is responsible for detecting material defects. It inputs the pre-processed image to the material defect detection model, obtains information such as the defect type and confidence level, and performs post-processing. When the defect confidence score exceeds the previously set judgment threshold, the material defect detection module outputs a result indicating the presence of a material defect. This result is finally stored in a designated location in shared memory, and the result flag of the first process is set high, indicating that the first process has completed detection within the current cycle.

[0046] It should be noted that: for the judgment threshold, for example, the judgment threshold of the crack defect is set to T, and the defect type confidence information finally output by the defect detection and recognition module is post-processed, that is, the recognition score information of the defect is extracted and represented as S. When the recognition score of the crack defect is greater than the previously set defect judgment threshold, that is, S>=T, the crack defect recognition module determines that the defect exists in this wooden spoon, otherwise it determines that there is no crack defect in the wooden spoon;

[0047] Processing defect detection in step S3: Processing defects include cracks, missing pieces, and oil stains. The two sub-processes initiated by the main process are responsible for detecting processing defects. The pre-processed image to be tested is input into the processing defect detection model, and information such as the defect type and confidence level is obtained for post-processing. When the defect recognition score exceeds the set judgment threshold, the processing defect detection module outputs a processing defect presence result, stores this result in shared memory, and sets the result flags of the second and third processes high, indicating that the second and third processes have completed detection within the current cycle.

[0048] It should be noted that the first process detects material defects in the wooden spoon, the second process detects machining defects on the front of the spoon, and the third process detects machining defects on the back of the spoon. All three processes begin reasoning simultaneously. When the first process completes reasoning, the first process's flag is set to 1. When the second process completes reasoning, the second process's flag is set to 1. When the third process completes reasoning, the third process's flag is set to 1. The results of each of the three processes are stored in shared memory.

[0049] Step S4: Post-process the detection results put into the shared memory by the three processes, including reducing duplicate information storage and merging result encoding.

[0050] To reduce duplicate storage in step 4, the main process first reads the three result flags in the shared memory. If all are high, it indicates that all three processes have completed detection in this cycle. Then, the detection results of each process are read from the shared memory and statistically merged. The results of each process are analyzed with the cycle of the distance between the control camera and the waste ejection device. The results are calculated to see if there are duplicate results for the same tableware. If there are duplicate results, no secondary storage is performed to reduce the pressure on the server and industrial computer.

[0051] The merged result code in step S4 refers to the result of the main process merging the three processes. After the merged code is sent to the industrial computer, the industrial computer determines whether the current disposable tableware needs to be discarded based on the signal.

[0052] It should be noted that at this time, the result of the main process integration is the results of three different spoons, which are merged into one bit of data for easy transmission to the industrial computer and the waste kicking machine.

[0053] Taking a certain period as an example, the distribution of production lines is as follows: Figure 2 ;

[0054] At this point, three processes are responsible for controlling the three cameras to capture images simultaneously. The main process controls camera 1 to capture an image of wooden spoon number 10. After the image is captured, it begins model inference for material defects and stores the results in shared memory. The first sub-process controls camera 2 to capture an image of wooden spoon number 4. After the image is captured, it begins model inference for processing defects and stores the results in shared memory. A flipper is installed between cameras 2 and 3 to flip the spoon from the front to the back. The second sub-process controls camera 3 to capture an image of wooden spoon number 1. After the image is captured, it begins processing defect detection on the back of the spoon and stores the results in shared memory.

[0055] The main process combines the results of the main process and the two subprocesses into a combined encoding. For example, (1, 0, 0) indicates that only the wooden spoon under camera 1 is defective, (0, 1, 0) indicates that only the wooden spoon under camera 2 is defective, (0, 0, 1) indicates that only the wooden spoon under camera 3 is defective, and (1, 1, 1) indicates that the wooden spoons under all three cameras are defective. Taking the result (1, 0, 0) as an example, after the industrial computer receives this signal and detects that the first bit is high, it controls the ejection device to eject the wooden spoon after the 12th cycle, at which point wooden spoon 10 has just reached the ejection device. Similarly, (1, 1, 1) results in the following: after receiving this signal, the IPC detects that the first signal is high and controls the ejection device to eject after the 12th cycle, at which point wooden spoon 10 has just arrived at the ejection device. Simultaneously, if the second signal is high, the ejection device is also activated after the 6th cycle, at which point the defective wooden spoon 4 has arrived at the ejection device. Simultaneously, if the third signal is high, the ejection device is activated after the 3rd cycle, at which point wooden spoon 1 has arrived at the ejection device. This ensures that within one cycle, the three processes can complete the detection and ejection signal transmission for all three wooden spoons.

[0056] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Any changes that may be made to certain parts thereof by those skilled in the art all reflect the principles of the present invention and fall within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects of disposable tableware based on deep learning, characterized in that: The following steps are involved: Step S1: multiple processes acquire video streams in real time, and acquire the disposable tableware images to be processed by each process according to the hardware trigger signal; Step S2: Each process performs data preprocessing operations on the image to be processed, including correction and cropping, adjustment of image brightness and contrast, and scaling; Step S3: sending the pre-processed image to be tested to the defect detection module, where the defect types detected include material defects and processing defects; Step S4: Post-processing the detection results put into the shared memory by the three processes, including reducing repeated push and storage, and merging the result encoding; The defect detection module in step S3: sends the image to be detected after data preprocessing to the defect detection model controlled by each process, and the output result of each process is the type and location information of the defect detected by the model; In step S3, before the defect detection model performs defect location information detection and type information identification, the method further includes: Deep learning defect detection model training: Images of disposable tableware containing different types of defects are collected and divided into three categories based on defect type. For each data set, the following operations are performed: All data are annotated with rectangular box labels indicating the location of the defects; the annotated images are divided into training, validation, and test sets based on material defects and processing defects; the defect detection model is trained using the training and validation sets, and the test set is input into the model to obtain defect type and location information. The model is also evaluated, ultimately resulting in a detection model that matches material defects and processing defects. The material defect detection using the defect detection model in step S3 includes: material defects include dead knot defects and decay defects, the main process is responsible for detecting material defects, the pre-processed image to be tested is input into the model responsible for material defect detection, the defect type and confidence information are obtained for post-processing, when the confidence score of the defect is greater than the set judgment threshold, the material defect detection module outputs a result that the material defect exists, otherwise the material defect detection module outputs a result that the material defect does not exist, and finally the result is stored in a specified location in the shared memory, and the result flag of the first process is set high, indicating that the first process has completed the detection within this cycle; The use of the defect detection model for processing defect detection in step S3 includes: processing defects include cracks, lack of meat, and oil stains. The two sub-processes started by the main process are responsible for detecting processing defects. The image to be tested after the above preprocessing is input into the processing defect detection model, and the defect type and confidence information are obtained for post-processing. When at least one of the defect recognition scores output by the two sub-processes is greater than the set judgment threshold, the processing defect detection module outputs a result that a processing defect exists, otherwise the processing defect detection module outputs a result that no processing defect exists, and then stores this result in the shared memory, and sets the result flags of the second process and the third process to high, indicating that the second process and the third process have completed the detection within this cycle.

2. The method for detecting surface defects of disposable tableware based on deep learning according to claim 1, characterized in that: In step S1, the disposable tableware to be inspected is placed in a stable light environment, and the ambient light source for each camera to capture images is configured according to the characteristics of the defects to be inspected; Configuring different ambient light sources is to obtain better image information; The camera takes the original image frame according to the trigger signal, and each process obtains the image to be processed.

3. The method for detecting surface defects of disposable tableware based on deep learning according to claim 1, characterized in that: The correction cropping in step S2 is because the actual production line situation is more complicated. The position of the tableware may be tilted due to external interference factors. It is necessary to first correct the position of the object to be measured in the image. After the processing is completed, cropping is performed at a fixed position to retain the information of the object to be measured at the specified position and remove useless background information.

4. The method for detecting surface defects of disposable tableware based on deep learning according to claim 1, characterized in that: The purpose of adjusting the brightness and contrast of the image in step S2 is to make the defect features in the image to be tested clearer and more conducive to defect detection. Secondly, the image is scaled so that its size meets the input requirements of the detection model.

5. The method for detecting surface defects of disposable tableware based on deep learning according to claim 1, characterized in that: To reduce repeated push and storage in step S4, first the main process reads the three result flags in the shared memory. If all are high, it indicates that all three processes have completed detection in this cycle; secondly, the detection results of each process are read and statistically merged, and the results of each process and the cycle of the distance between the control camera and the waste kicking device are analyzed to calculate whether there are repeated results for the same tableware. If repeated results occur, no secondary storage is performed to reduce the pressure on the server and industrial computer.

6. The method for detecting surface defects of disposable tableware based on deep learning according to claim 1, characterized in that: The merged result encoding in step S4 refers to merging the output results of the main process and the two sub-processes, encoding the results and sending them to the industrial computer. The industrial computer determines whether to discard the current disposable tableware based on the signal.

Images