Product edge sealing defect detection method and device and storage medium

Abstract

The invention discloses a product edge sealing defect detection method and device and a storage medium. The method comprises the following steps: acquiring an edge sealing image of a to-be-detected product; inputting the edge sealing image of the to-be-detected product into the target edge sealing defect detection model to obtain an initial edge sealing defect detection result; if the initial edge sealing defect detection result is defective, performing template matching on the edge sealing image of the to-be-detected product by utilizing an edge sealing template to obtain a defect matching result; if the initial edge sealing defect detection result is no defect, inputting the edge sealing image of the to-be-detected product into a target classification network model to obtain a classification result; if the defect matching result and the classification result are both defect-free, performing strategy judgment on the edge sealing image of the to-be-detected product by utilizing a defect identification strategy to obtain a strategy judgment result; and determining a target edge sealing defect detection result according to the target identification result. According to the invention, edge sealing defect detection is realized, and the accuracy and applicability are improved. The method can be widely applied to the technical field of image recognition.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a product edge defect detection method, device and storage medium. Background Art

[0002] The edge sealing of product packaging involves aspects such as material properties, edge sealing technology, edge sealing machinery, automatic control, defect recognition algorithms and ultrasonic technology, and edge sealing defects may occur. Traditional edge sealing defect detection is difficult to identify very small defects or defects in complex backgrounds. For example, on a high-speed production line, the high-speed moving packaging materials make it difficult to capture high-resolution images and the detection accuracy is low. At the same time, it is often only targeted at specific types of edge sealing materials and edge sealing machinery, and has low applicability to new materials or different production environments.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The embodiments of the present invention provide a method, device and storage medium for detecting edge defects of products, which effectively improve the accuracy and applicability.

[0005] In one aspect, an embodiment of the present invention provides a method for detecting edge defects of a product, comprising the following steps:

[0006] Acquire the edge banding image of the product to be inspected;

[0007] Inputting the edge banding image of the product to be inspected into a target edge banding defect detection model to obtain an initial edge banding defect detection result;

[0008] If the initial edge sealing defect detection result is defective, template matching is performed on the edge sealing image of the product to be detected using an edge sealing template to obtain a defect matching result;

[0009] If the initial edge banding defect detection result is no defect, inputting the edge banding image of the product to be inspected into the target classification network model to obtain a classification result;

[0010] If both the defect matching result and the classification result are defect-free, a defect recognition strategy is used to perform a strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result;

[0011] According to the target recognition result, the target edge sealing defect detection result is determined, the target recognition result includes the defect matching result, the classification result or the strategy determination result, and the target edge sealing defect detection result includes edge sealing rupture, wire pulling, edge sealing explosion, edge sealing indentation or no defect.

[0012] In some embodiments, the target edge banding defect detection model is obtained by the following steps:

[0013] Acquire a product edge sealing image set, wherein the product edge sealing image set includes a qualified edge sealing image and an edge sealing defect image;

[0014] generating an initial edge banding defect detection model according to the product edge banding image set;

[0015] Collecting an edge banding update image set, wherein the edge banding update image set includes a newly added defect image and an edge banding image to be identified, wherein the newly added defect image has an untrained defect category;

[0016] The target edge banding defect detection model is generated according to the edge banding update image set and the initial edge banding defect detection model.

[0017] In some embodiments, generating an initial edge banding defect detection model according to the product edge banding image set includes:

[0018] Performing grayscale conversion on the product edge banding image set to obtain a grayscale data set;

[0019] Cutting out the defective area in the edge sealing defect image to obtain a defective area image;

[0020] Overlaying the defective area image onto the qualified edge sealing image to obtain an augmented data set;

[0021] Using a preset edge detection method to perform edge detection on the product edge banding image set to obtain a contour data set;

[0022] The product edge banding image set, the grayscale data set, the amplified data set and the contour data set are input into a preset edge banding defect detection model so that the preset edge banding defect detection model is trained to obtain the initial edge banding defect detection model.

[0023] In some embodiments, generating the target edge banding defect detection model according to the edge banding update image set and the initial edge banding defect detection model includes:

[0024] Performing category labeling processing on the newly added defect image to obtain a newly added labeled image;

[0025] According to the prediction threshold, the edge banding image to be identified is input into the initial edge banding defect detection model to obtain an edge banding defect identification result, and the edge banding defect identification result is used as a pseudo label;

[0026] Combining the newly added annotated image with the edge banding defect recognition result to obtain a new task data set;

[0027] The new task data set, the product edge banding image set, the grayscale data set, the augmented data set and the contour data set are input into the initial edge banding defect detection model through incremental learning, so that the initial edge banding defect detection model is retrained to obtain the target edge banding defect detection model.

[0028] In some embodiments, the edge-sealing template is obtained by the following steps:

[0029] Use white light source to capture product images;

[0030] Identify the edge color of the edge area in the product image through a three-channel color space;

[0031] If the edge banding color is black, re-capture is performed using a red light source to update the product image;

[0032] If the edge banding color is white, re-capture is performed using a blue light source to update the product image;

[0033] If the edge banding color is yellow, re-capture is performed using a yellow light source to update the product image;

[0034] The edge banding area is cut out from the updated product image as the edge banding template.

[0035] In some embodiments, the target classification network model is obtained by the following steps:

[0036] The product edge banding image set is input into a preset classification network model so that the preset classification network model is trained to obtain the target classification network model.

[0037] In some embodiments, the using of the defect recognition strategy to perform strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result includes:

[0038] If a long strip protrudes from the edge of the edge banding in the edge banding image of the product to be detected, the length of the long strip is greater than a preset length threshold, and the angle between the long strip and the horizontal plane is greater than a preset angle threshold, the pull line is used as the strategy determination result;

[0039] If there is an ellipse in the edge sealing of the product to be detected and the area of ​​the ellipse is greater than a preset area threshold, the edge sealing is exploded as the strategy determination result;

[0040] If there are multiple linear objects on the edge of the product to be inspected in the edge image and the multiple linear objects intersect each other, the edge rupture is taken as the strategy determination result;

[0041] If there are concave blocks and convex blocks in the edge sealing of the product to be detected, the total area of ​​the concave blocks and the convex blocks is greater than the preset concave-convex threshold and the average distance between the concave blocks and the convex blocks is less than the preset distance threshold, the edge sealing indentation is taken as the strategy judgment result.

[0042] On the other hand, an embodiment of the present invention provides a device for detecting edge defects of a product, comprising:

[0043] The first module is used to obtain the edge banding image of the product to be inspected;

[0044] The second module is used to input the edge banding image of the product to be inspected into the target edge banding defect detection model to obtain an initial edge banding defect detection result;

[0045] The third module is used for, if the initial edge sealing defect detection result is defective, using the edge sealing template to perform template matching on the edge sealing image of the product to be detected to obtain a defect matching result;

[0046] The fourth module is used for inputting the edge banding image of the product to be inspected into the target classification network model to obtain a classification result if the initial edge banding defect detection result is no defect;

[0047] The fifth module is used to perform a strategy determination on the edge banding image of the product to be inspected by using a defect recognition strategy to obtain a strategy determination result if both the defect matching result and the classification result are defect-free;

[0048] The sixth module is used to determine the target edge banding defect detection result according to the target recognition result, wherein the target recognition result includes the defect matching result, the classification result or the strategy determination result, and the target edge banding defect detection result includes edge banding rupture, wire pulling, edge banding explosion, edge banding indentation or no defect.

[0049] In another aspect, an embodiment of the present invention provides a computer device, comprising:

[0050] at least one processor;

[0051] at least one memory for storing at least one program;

[0052] When the at least one program is executed by the at least one processor, the at least one processor implements the described method.

[0053] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0054] The beneficial effects of the present invention are as follows:

[0055] The embodiment of the present invention first obtains an edge banding image of a product to be inspected, and then inputs the edge banding image of the product to be inspected into a target edge banding defect detection model to obtain an initial edge banding defect detection result; if the initial edge banding defect detection result is defective, an edge banding template is used to perform template matching on the edge banding image of the product to be inspected to obtain a defect matching result; if the initial edge banding defect detection result is not defective, the edge banding image of the product to be inspected is input into a target classification network model to obtain a classification result; if both the defect matching result and the classification result are not defective, a defect recognition strategy is used to perform a strategy judgment on the edge banding image of the product to be inspected to obtain a strategy judgment result; finally, according to the target recognition result, the target edge banding defect detection result is determined, thereby realizing edge banding defect detection and improving accuracy and applicability.

[0056] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 This is a flow chart of a method for detecting edge defects of a product according to an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of a multi-modal edge banding defect detection model structure according to an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of a wire defect according to an embodiment of the present invention;

[0061] Figure 4 A schematic diagram of an edge-sealing explosion defect according to an embodiment of the present invention;

[0062] Figure 5 A schematic diagram of an edge sealing crack defect according to an embodiment of the present invention;

[0063] Figure 6 A schematic diagram of an edge sealing inward defect according to an embodiment of the present invention;

[0064] Figure 7 This is an overall decision diagram for defect detection according to an embodiment of the present invention;

[0065] Figure 8This is a schematic structural diagram of a device for detecting edge defects of a product according to an embodiment of the present invention;

[0066] Fig. 9 The figure is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0068] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0069] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0072] Incremental learning: refers to a learning system that can continuously learn new knowledge from new samples and preserve most of the previously learned knowledge. Incremental learning is very similar to the learning mode of human beings.

[0073] Among the related technologies, edge sealing defect detection of product packaging is a highly specialized research field involving multiple disciplines such as materials science, mechanical engineering, image processing and artificial intelligence. In terms of the material science basis of edge sealing technology, product packaging materials usually include polymers such as polyethylene (PE) and polypropylene (PP). The sealing performance of these materials at different temperatures and pressures is the key to edge sealing quality; heat sealing is the most commonly used edge sealing technology, which requires precise control of temperature, pressure and sealing time to achieve the best sealing effect. From the perspective of mechanical engineering, the edge sealing mechanical design must ensure uniform pressure distribution and temperature control, and the accuracy of the machinery directly affects the edge sealing quality; modern edge sealing equipment usually integrates advanced control systems to achieve automatic adjustment of process parameters and improve the consistency and repeatability of edge sealing. The edge sealing part is pressure tested to ensure that the seal will not break or leak under a certain pressure, which is particularly important for liquid or gas packaging. In terms of image processing and machine vision, image processing techniques such as edge detection, texture analysis, and pattern recognition are used to automatically detect defects in the edge banding area; machine learning and deep learning models, such as convolutional neural networks (CNNs), are used to learn and identify complex defect patterns from edge banding images. High-resolution cameras are used to capture images of the edge banding area to detect any visible defects, such as uneven sealing, bubbles, wrinkles, or unsealed areas. In terms of ultrasonic and non-destructive testing, ultrasonic technology is used to check the uniformity and integrity of the edge banding seal, and the difference in ultrasonic reflection and transmission is used to detect tiny defects in the edge banding, such as micropores and unsealed areas; the collected ultrasonic signal data is analyzed and processed to evaluate the integrity and consistency of the edge banding.

[0074] In the edge defect detection of product packaging bags, the existing technology still has some shortcomings and technical problems to be solved, such as unknown learned knowledge samples and few sample defect detection tasks. The shortcomings of the existing technology are as follows: Although modern image processing and machine vision technology can identify a variety of edge defects, it is still difficult to identify very small defects or defects in complex backgrounds, especially on high-speed production lines. High-speed moving packaging materials make it more difficult to capture high-resolution images, with low sensitivity and accuracy; the existing technology is often optimized for specific types of edge materials and edge machinery, and is not adaptable to new materials or different production environments, with low adaptability and flexibility; although automated detection technology improves detection efficiency, the cost of high-end equipment is high, which may not be economical for small enterprises or low-cost production lines. The integration of advanced AI and machine learning technologies requires professional knowledge, and the skills requirements for operators are high, which is difficult to implement. The real-time data processing and feedback capabilities in the existing system are still limited, and it is impossible to adjust the parameters in the production process in real time to reduce the generation of defects, and there is a lack of real-time monitoring and feedback mechanisms. Therefore, the technical problems that need to be solved include: developing detection technology that can adapt to different edge banding materials and diversified production environments, enhancing the flexibility and adaptability of the system to improve the generalization ability of the detection technology; adopting higher-resolution image capture devices and more sophisticated image processing algorithms to improve the system's ability to detect tiny defects; developing more cost-effective detection technology, simplifying the technology implementation process, making it easier to be adopted by companies of all sizes to reduce the cost of technology implementation, developing more user-friendly operating interfaces and automated systems, reducing dependence on operator skills and reducing complexity; using real-time data analysis and machine learning to develop intelligent systems that can instantly identify production deviations and automatically adjust production parameters to achieve higher production efficiency and reduce defect rates, and achieve more efficient real-time feedback and automatic adjustment, thereby improving the efficiency and accuracy of edge banding defect detection, and expanding its application range in various production environments, thereby improving overall product quality.

[0075] In view of this, this embodiment is based on machine vision and combined with deep learning, which can significantly improve the accuracy and efficiency of product packaging edge defect detection. This embodiment uses traditional machine vision technology to capture images and combines deep learning models to analyze and identify defects in images. First, image capture and preprocessing are performed, and high-resolution industrial cameras are used to capture images of edge-sealed areas in real time on the production line. These cameras can provide clear images under different ambient lighting conditions, generate data streams, and transmit the captured data to the image processing module for further processing. Image preprocessing includes steps such as grayscale, binarization, noise removal, and edge enhancement. These processes help reduce background interference and highlight defect features. Feature extraction is then performed, and image edge information is extracted using edge detection algorithms such as Sobel or Canny. These traditional algorithms work well in extracting obvious boundaries. Deep learning models (such as convolutional neural networks, CNN) are used to automatically learn and extract complex features in images. These features include not only edges and shapes, but also textures and possible defect patterns. Traditional machine vision is used to generate edge texture feature obvious data and grayscale data, and enhanced data sets are generated based on common data enhancement methods and deep learning models such as gan, as well as original rgb visible light data sets. In the defect detection model, the network model structure uses three different feature data (RGB data set, grayscale data set, edge information feature data set) as input, and uses three backbone networks for feature extraction and concatenation of operations such as concat. Among them, neck and task head use common convolutional neural network algorithms such as yoloV10. The deep learning model is trained using the labeled and processed edge defect image data set. This step is critical and requires a large amount of high-quality and diverse data to ensure the generalization ability of the model. The trained model is deployed on the production line. The model analyzes the images captured by the camera in real time and quickly identifies edge defects to achieve real-time detection, which can greatly improve the accuracy and speed of detection, reduce the need for manual inspection, and ultimately improve product quality and production efficiency.

[0076] A product edge defect detection method provided in an embodiment of the present application relates to the field of image recognition technology. A product edge defect detection method provided in an embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a product edge defect detection method, etc., but is not limited to the above forms.

[0077] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0078] The following is a detailed explanation of the embodiments of the present application in conjunction with the accompanying drawings:

[0079] Figure 1 is an optional flow chart of a product edge defect detection method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0080] Step S101, obtaining an edge banding image of a product to be inspected;

[0081] Step S102, inputting the edge banding image of the product to be inspected into the target edge banding defect detection model to obtain an initial edge banding defect detection result;

[0082] Step S103: If the initial edge banding defect detection result is defective, use the edge banding template to perform template matching on the edge banding image of the product to be inspected to obtain a defect matching result;

[0083] Step S104: if the initial edge banding defect detection result is no defect, the edge banding image of the product to be inspected is input into the target classification network model to obtain a classification result;

[0084] Step S105: If both the defect matching result and the classification result are no defects, a defect recognition strategy is used to perform a strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result;

[0085] Step S106, determining the target edge sealing defect detection result according to the target recognition result, the target recognition result includes a defect matching result, a classification result or a strategy determination result, and the target edge sealing defect detection result includes edge sealing rupture, line pulling, edge sealing explosion, edge sealing indentation or no defect.

[0086] Steps S101 to S106 shown in the embodiment of the present application realize edge banding defect detection and improve accuracy and applicability.

[0087] In some embodiments, the edge banding image of the product to be inspected can be first obtained on the production line through a high-resolution industrial camera. The edge banding image of the product to be inspected can also be obtained by other means, not limited thereto. Then the edge banding image of the product to be inspected is input into the target edge banding defect detection model to obtain the initial edge banding defect detection result, and then the initial edge banding defect detection result is judged. If the initial edge banding defect detection result is defective, the edge banding template is used to perform template matching on the edge banding image of the product to be inspected to obtain a defect matching result; if the initial edge banding defect detection result is not defective, the edge banding image of the product to be inspected is input into the target classification network model to obtain a classification result. Then the defect matching result and the classification result are comprehensively judged. If both the defect matching result and the classification result are not defective, the defect recognition strategy is used to perform strategy judgment on the edge banding image of the product to be inspected to obtain a strategy judgment result, and finally the target edge banding defect detection result is determined according to the target recognition result. Among them, the target recognition result includes a defect matching result, a classification result or a strategy judgment result, and the target edge banding defect detection result includes edge banding rupture, wire pulling, edge banding explosion, edge banding indentation or no defect. For example, when the defect matching result is defective, the edge defect category and defect location can be obtained and used as the target edge defect detection result; when the classification result is defective, the edge defect category can be obtained and used as the target edge defect detection result; when the strategy determination result is defective, the edge defect category can be obtained and used as the target edge defect detection result. Furthermore, the strategy determination result can also identify no defect as the target edge defect detection result.

[0088] In some embodiments, in step S102, the target edge banding defect detection model is obtained through steps S201 to S204:

[0089] Step S201, obtaining a product edge sealing image set, wherein the product edge sealing image set includes edge sealing qualified images and edge sealing defective images;

[0090] Step S202: Generate an initial edge banding defect detection model based on the product edge banding image set;

[0091] Step S203, collecting an edge banding update image set, the edge banding update image set including a newly added defect image and an edge banding image to be identified, the newly added defect image having an untrained defect category;

[0092] Step S204: Generate a target edge banding defect detection model according to the edge banding update image set and the initial edge banding defect detection model.

[0093] In some embodiments, in step S202, generating an initial edge banding defect detection model based on the product edge banding image set may include but is not limited to the following steps:

[0094] Perform grayscale conversion on the product edge banding image set to obtain a grayscale data set;

[0095] Cutting out the defective area in the edge-sealing defect image to obtain a defective area image;

[0096] Overlay the defective area image onto the qualified edge sealing image to obtain an augmented data set;

[0097] The preset edge detection method is used to perform edge detection on the product edge banding image set to obtain a contour data set;

[0098] The product edge banding image set, the grayscale data set, the amplified data set and the contour data set are input into the preset edge banding defect detection model so that the preset edge banding defect detection model is trained to obtain an initial edge banding defect detection model.

[0099] In some embodiments, in step S201-step S202, a product edge sealing image set can be first obtained, wherein the product edge sealing image set can be in a three-channel RGB (red, green, blue) format, and the product edge sealing image set includes qualified edge sealing images and defective edge sealing images. The qualified edge sealing images have no target detection labels, and the defective edge sealing images have target detection labels and defect category labels. The defect categories include edge sealing rupture, wire pulling, edge sealing explosion, and edge sealing indentation. The target detection label contains the defect frame position for secondary judgment and classification. It can be understood that this embodiment assumes that a data set containing a small number of labeled images and a large number of unlabeled images is used to train a target edge sealing defect detection model. Then, the product edge sealing image set is converted to grayscale to obtain a grayscale data set. For example, the existing RGB channel data can be directly converted into grayscale data using an open source method. Then the defective area in the edge-sealing defect image is cropped to obtain a defective area image, and the defective area image is overlaid on the qualified edge-sealing image to obtain an augmented data set. For example, the data defect (i.e., defective area) of the defect NG (NO GO, not passed) in the edge-sealing defect image can be cropped out and then pasted on the edge of the normal data product (i.e., qualified edge-sealing image). The template matching method can be used to find the edge-sealing contour position, and the maximum circumscribed area is used as the mapping area. After pasting the defective data, it is merged into enhanced defect data (i.e., augmented data set). Then, the preset edge detection method is used to perform edge detection on the product edge-sealing image set to obtain a contour data set. For example, feature pyramids or traditional canny methods can be used to extract image detail features to obtain a contour data set. Furthermore, in data enhancement, random cropping, rotation and other methods can be used to increase data diversity to increase the generalization ability of the model. Finally, the product edge-sealing image set, grayscale data set, augmented data set and contour data set are input into the preset edge-sealing defect detection model to train the preset edge-sealing defect detection model to obtain an initial edge-sealing defect detection model. For example, the preset edge defect detection model may include yolo class or ViT-H / 14, etc. The structure of the multimodal edge defect detection model is as follows: Figure 2 As shown, it includes a multi-backbone feature network extraction module and a feature layer fusion module, which can extract features from the grayscale dataset, the amplified dataset, and the contour dataset respectively and perform feature fusion, and then pre-train the model. After adding sample data, the model can be optimized through multi-stage continuous learning. Furthermore, a labeled dataset can be used to train an initial edge defect detection model, which can be a convolutional neural network (CNN). It can be understood that unlabeled means that there is no defect category label. During training, normal data with an empty background can be added to improve the model domain adaptability. The grayscale dataset, the amplified dataset, and the contour dataset are all generated through the product edge image set, and their defect category labels are the same as the labels of the corresponding edge defect images in the product edge image set.

[0100] In some embodiments, in step S204, generating a target edge banding defect detection model according to the edge banding update image set and the initial edge banding defect detection model may include but is not limited to the following steps:

[0101] Perform category labeling on the newly added defect image to obtain a newly added labeled image;

[0102] According to the prediction threshold, the edge banding image to be identified is input into the initial edge banding defect detection model to obtain the edge banding defect recognition result, which is used as a pseudo label;

[0103] Combine the newly added annotated images and the edge defect recognition results to obtain a new task dataset;

[0104] Through incremental learning, the new task dataset, product edge banding image set, grayscale dataset, augmented dataset and contour dataset are input into the initial edge banding defect detection model so that the initial edge banding defect detection model is retrained to obtain the target edge banding defect detection model.

[0105] In some embodiments, in step S203-step S204, an edge banding update image set may be first collected, wherein the edge banding update image set includes a newly added defect image and an edge banding image to be identified, the newly added defect image has an untrained defect category, that is, the newly added defect category has not been trained in the initial edge banding defect detection model, the initial edge banding defect detection model cannot identify the newly added defect category, and the defect category in the edge banding image to be identified can be identified in the initial edge banding defect detection model. Then, the newly added defect image is subjected to category labeling processing to obtain a newly added labeled image. It can be understood that only the newly added defect category needs to be labeled, while the others are similar defect data that already exist in the original data set and are not labeled. Then, according to the prediction threshold, the edge banding image to be identified is input into the initial edge banding defect detection model to obtain an edge banding defect recognition result, wherein the edge banding defect recognition result is used as a pseudo label. Exemplarily, for each edge banding image to be identified, the initial edge banding defect detection model will output the edge banding defect recognition result as a pseudo label. At the same time, in order to ensure the quality of the pseudo label, the prediction threshold can be set to 0.6, and the label with a model prediction probability greater than 0.6 is selected as the pseudo label to reduce noise. Then, the newly added annotated images and the edge defect recognition results are combined to obtain a new task data set. Finally, the new task data set, product edge image set, grayscale data set, amplified data set and contour data set are input into the initial edge defect detection model through incremental learning, so that the initial edge defect detection model is retrained to obtain the target edge defect detection model. Exemplarily, the product edge image set, grayscale data set, amplified data set, contour data set and the newly added new task data set are merged, and the initial edge defect detection model is retrained. The generation of a new task data set and the retraining process can be iterative, and each iteration may introduce new high-confidence untrained defect category data. It can be understood that the initial edge defect detection model is trained using a public data set and is a three-channel input. The defect target detection in this embodiment is usually a multi-modal input. When each modality loads pre-trained weights, the pre-trained weights that have been trained are loaded, and fine-tuned training is performed on this basis until the algorithm converges.

[0106] In incremental learning iterative optimization, each iteration evaluates the performance of the model on an independent validation set. If the performance improves, more iterations can be performed, each time using an updated model to generate more accurate pseudo labels. In essence, it is a balancing act between being able to learn new tasks (i.e., plasticity) and maintaining the performance of previously learned concepts (i.e., stability). In order to solve the stability-plasticity trade-off problem, the model parameters of the previous task model and the current task can be weighted integrated. For example, in the first iteration, assuming that the current first session is a 10-category defect detection task, the first session model is first trained to the best, and the model weights and parameter gradients are saved. In the second iteration, the second session adds 3 different defect categories to reach 13-category defect detection tasks. When training the second session task model, the gradient values ​​corresponding to the weight parameters of the first session and the weight gradient values ​​obtained by the second session training are used to calculate the ratio, and then the ratio is added as the weighted value of each weight parameter to obtain the new model parameters and update the saved model. In the third iteration, the third session continued to add 3 different defect categories, reaching 16 types of defect detection tasks. The model gradient ratio of the third session and the second session was used as the weighted value, and the weight parameters of the weighted sum of the two were updated and used as the final defect detection model. This method of weighted integration model, called continuous model averaging, achieves high accuracy on the current task by utilizing plasticity while not deviating too far from the previous weight configuration to ensure stability. Incremental learning iterative optimization is easy to implement and can effectively improve model performance. Moreover, incremental learning can use different learning rates, different continuous learning strategy training iterations and other hyperparameters to find the best training configuration. In this way, even with limited new labeled data, the accuracy and generalization ability of the model for sample defects that have not been learned and trained can be significantly improved, which is very valuable in practical applications, especially in scenarios where resources are limited or data is difficult to collect more defects.

[0107] In some embodiments, in step S103, the edge-sealing template is obtained by the following steps:

[0108] Use white light source to capture product images;

[0109] Identify the edge color of the edge area in the product image through a three-channel color space;

[0110] If the edge color is black, use a red light source to recapture and update the product image;

[0111] If the edge color is white, use a blue light source to recapture and update the product image;

[0112] If the edge color is yellow, use a yellow light source to recapture and update the product image;

[0113] Cut out the edge banding area from the updated product image to use as the edge banding template.

[0114] In some embodiments, an image acquisition device may be used to first acquire a product image using a white light source, and then the edge color of the edge area in the product image and the color of the packaging may be identified through a three-channel RGB color space, and then defect detection hardware image acquisition conditions may be automatically generated based on the light source selection conditions. If the edge color is black or dark, a red light source may be used to re-acquire and update the product image; if the edge color is white or light, a blue light source may be used to re-acquire and update the product image; if the edge color is yellow, a yellow light source may be used to re-acquire and update the product image; if the edge color is purple, a white light source may be used to re-acquire and update the product image. In addition, this embodiment uses two left and right bar light sources to illuminate the product packaging at a 45° angle to reduce shadows and reflections and improve the image quality. Finally, the edge area is cut out from the updated product image as an edge template. For example, edge sealing templates can be created by cutting out different packaging edge sealing areas, so that the templates can be used for template matching using correlation coefficient matching (Correlation Coefficient Matching) and SIFT (Scale-Invariant Feature Transform) to output the coordinate position area.

[0115] In some embodiments, in step S104, the target classification network model is obtained by the following steps:

[0116] The product edge banding image set is input into a preset classification network model so that the preset classification network model is trained to obtain a target classification network model.

[0117] In some embodiments, the product edge banding image set can be input into a preset classification network model so that the preset classification network model is trained to obtain a target classification network model. Exemplarily, the preset classification network model can include algorithms such as clip or convnext, and can be trained using edge banding qualified images and edge banding defect images until the model converges to obtain a target classification network model, so as to accurately obtain edge banding defect screening in subsequent decision-making.

[0118] In some embodiments, in step S105, the defect recognition strategy is used to perform strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result, which may include but is not limited to the following steps:

[0119] If a long strip protrudes from the edge of the edge banding image of the product to be detected, the length of the long strip is greater than a preset length threshold, and the angle between the long strip and the horizontal plane is greater than a preset angle threshold, the pull line is used as the strategy determination result;

[0120] If there is an ellipse in the edge of the product to be inspected and the area of ​​the ellipse is greater than the preset area threshold, the edge explosion is taken as the strategy determination result;

[0121] If there are multiple linear objects on the edge of the product to be inspected and the multiple linear objects intersect each other, the edge rupture is taken as the strategy judgment result;

[0122] If there are concave and convex blocks in the edge of the product to be inspected in the edge image, the total area of ​​the concave and convex blocks is greater than the preset concave and convex threshold, and the average distance between the concave and convex blocks is less than the preset distance threshold, the edge indentation is taken as the strategy judgment result.

[0123] In some embodiments, the edge sealing area position is output according to the edge sealing template matching, and image preprocessing is performed. In the preprocessing, the color image can be converted into a grayscale image to simplify the domain adaptation feature problem, and filtering, denoising and contrast enhancement can be performed. The edge of the object can be identified using edge detection algorithms such as Sobel and Canny, the defect can be separated from the background based on the grayscale threshold, and morphological operations such as corrosion, expansion, opening and closing can be used to further clarify the boundary and shape of the defect. Then, the defect recognition strategy is used to make a strategic decision on the edge sealing image of the product to be inspected.

[0124] In the identification of the pull line, Figure 3 As shown, if a long strip protrudes from the edge of the edge banding image of the product to be detected, the length of the long strip is greater than the preset length threshold, and the angle between the long strip and the horizontal plane is greater than the preset angle threshold, then the pull line is used as the strategy determination result. Exemplarily, the preset length threshold can be set to 120 pixels in length, and the preset angle threshold can be set to 10°. If a long strip protrudes from the edge of the edge banding image of the product to be detected, the length of the long strip is greater than 120 pixels in length, and the angle between the long strip and the horizontal plane is greater than 10°, then the pull line is used as the strategy determination result. It can be understood that the pull line usually appears as a long and thin straight line shape, which can be identified by the linear extension of the edge and the texture directionality. The size of the connected domain is usually less than 160, the position is at the edge of the edge banding, the shape is a long strip, and a long line protrudes from the edge relatively.

[0125] In the identification of edge burst, Figure 4As shown, if there is an ellipse in the edge of the edge image of the product to be inspected and the area of ​​the ellipse is greater than the preset area threshold, the edge burst is taken as the strategy judgment result. It can be understood that the edge burst may be due to material defects or poor sealing, which is manifested as a local circular or elliptical dark area with high texture uniformity, a connected domain size usually greater than 20 and less than 100, and a position at any position, but usually close to the middle of the edge, with an irregular shape and uneven edges.

[0126] In the identification of edge cracks, Figure 5 As shown in the figure, if there are multiple linear objects in the edge of the edge image of the product to be inspected and the multiple linear objects intersect each other, the edge rupture is taken as the strategy judgment result. It can be understood that the edge rupture usually has bifurcated lines and double straight line intersections, the size of the connected domain is usually greater than 10 and less than 250, the position is at the beginning or end of the edge, and in the middle of the edge template, the shape is extremely irregular, and there is obvious torn edge.

[0127] In the identification of edge indentation, Figure 6 As shown, if there are concave blocks and convex blocks in the edge sealing image of the product to be detected, the total area of ​​the concave blocks and convex blocks is greater than the preset concave-convex threshold and the average distance between the concave blocks and the convex blocks is less than the preset distance threshold, the edge sealing indentation is used as the strategy determination result. Exemplarily, the preset concave-convex threshold can be set to 1200 pixel values, and the preset distance threshold can be set to 10 pixel distances. If there are concave blocks and convex blocks in the edge sealing image of the product to be detected, the total area of ​​the concave blocks and convex blocks is greater than 1200 pixel values ​​and the average distance between the concave blocks and the convex blocks is less than 10 pixel distances, the edge sealing indentation is used as the strategy determination result. More, the area of ​​the concave blocks or convex blocks can also be used as a determination condition. It can be understood that the edge sealing indentation usually has irregular protrusions or indentations between the edge sealing strips, the size of the connected domain is usually greater than 10 and less than 120, the position is in any position area, the shape is extremely irregular, and there are obvious unevenness.

[0128] In some embodiments, the overall defect detection decision is as follows: Figure 7As shown, the image data stream can be collected first, and then identified through the target edge banding defect detection model to determine whether there are defects and the defect category and location, and then combined with the edge banding template to search for the specific location area (usually x1, y1; x2, y2, are the coordinates of the upper left corner and lower right corner of the image). If the target edge banding defect detection model outputs the defect category and the position (x3, y3, w, h, are the center coordinates and length and width) is in the area matched by the edge banding template, it is determined to be an edge banding defect. If the target edge banding defect detection model determines that there is no defect, it is identified again through the target classification network model. If the target classification network model identifies that there is no defect, the defect identification strategy is used and combined with parameters such as connected domain, area and contour size to perform strategy determination to obtain the strategy determination result. If the strategy determination result is defective, the corresponding product is transported to the manual verification area by a conveyor belt to determine whether there is a misjudgment. If there is no misjudgment, the product is recycled.

[0129] In some embodiments, after obtaining the target edge banding defect detection result, the result analysis can be performed, and the target edge banding defect detection result can be used to directly feed back to the production line control system to automatically adjust the parameters of the edge banding machinery (such as temperature and pressure) to correct the impending defects. Continuous learning can be performed. For flexible production lines with rapid production, when a large amount of defect data cannot be obtained in time, a model with continuous learning capabilities can be designed to continuously optimize and update the model through new defect instances. The ability of continuous learning using the edge banding defect detection model can not only keep the previous knowledge from being forgotten, but also increase the learning of new defect knowledge points. A friendly user interface can be developed to allow operators to easily set detection parameters, view detection results, and perform system maintenance. The detection device can be integrated into the production line and seamlessly integrated with existing production line equipment and control systems to ensure the continuity and efficiency of operations.

[0130] In some embodiments, this embodiment uses the convolutional neural network pre-training weights obtained by the collected data set and annotated training. Since the real-time data of the production line may be inconsistent, the pre-training weights are obtained by training with visible light RGB and grayscale and contour data. Multimodal images are usually obtained by processing RGB picture data. In order to quickly reuse the pre-training weights, a convolutional neural network for multi-backbone feature extraction is designed, and the pre-training model of the RGB channel is loaded. The multimodal algorithm uses multimodal image data to input the backbone network or feature fusion and other forms for design. It can quickly train to obtain a multimodal target edge defect detection model, and does not require manual annotation, meeting the requirements of low cost and high precision. Among them, the diversity approximation of different modal features and the difference of detail layers effectively represent image feature extraction at multiple feature levels, so that in image processing tasks, such as image style changes or changes in lighting, the product edge defect detail features can be extracted more flexibly and effectively. At the same time, this embodiment combines deep learning and incremental learning, identifies defects through real-time image capture, high-resolution preprocessing and deep learning models, and uses incremental learning strategies such as online learning and small batch updates to continuously optimize the model, maintain high accuracy and adaptability, and ensure production quality and efficiency.

[0131] The beneficial effects of implementing the embodiments of the present invention include: the embodiments of the present invention first obtain an edge banding image of the product to be inspected, and then input the edge banding image of the product to be inspected into a target edge banding defect detection model to obtain an initial edge banding defect detection result; if the initial edge banding defect detection result is defective, the edge banding template is used to perform template matching on the edge banding image of the product to be inspected to obtain a defect matching result; if the initial edge banding defect detection result is not defective, the edge banding image of the product to be inspected is input into a target classification network model to obtain a classification result; if both the defect matching result and the classification result are not defective, the defect recognition strategy is used to perform strategy judgment on the edge banding image of the product to be inspected to obtain a strategy judgment result; finally, according to the target recognition result, the target edge banding defect detection result is determined, thereby realizing edge banding defect detection and improving accuracy and applicability.

[0132] In some embodiments, this embodiment adds a multimodal target edge defect detection model, and different processing quantity methods are used as model inputs. For example, RGB images have good detection effects on bright color defects, while grayscale and contour images have good detection effects on edge information sensitive feature defects. When single mode or multiple single mode models weightedly predict defect features, they are far less effective than multimodal input models. This greatly improves the prediction accuracy of unknown data. In actual production, there are usually different lighting and product defects, and features that have not been added to the training set. This embodiment relies on the edge defect detection performed by the multimodal target edge defect detection model (network structure), as well as the use details of each subtask data processing and training. At the same time, this embodiment combines incremental learning, and different data forms are used as model inputs. There is no need to rely on a large amount of prior sample knowledge and sample annotations, and the corresponding subtask algorithm module is selected according to the existing small amount of data for incremental learning and training. It greatly saves computing power and improves accuracy. When adding unknown defect detection tasks, it can quickly implement decisions, improve intelligence and work efficiency. Using incremental learning to solve the problem of training with few samples in a multimodal target detection model greatly reduces forgetting and improves the accuracy of new sample defect detection and saves training time.

[0133] like Figure 8 As shown, an embodiment of the present invention further provides a device for detecting edge defects of a product, comprising:

[0134] The first module 801 is used to obtain an edge banding image of a product to be inspected;

[0135] The second module 802 is used to input the edge banding image of the product to be inspected into the target edge banding defect detection model to obtain an initial edge banding defect detection result;

[0136] The third module 803 is used to perform template matching on the edge image of the product to be inspected using the edge template to obtain a defect matching result if the initial edge defect detection result is defective;

[0137] The fourth module 804 is used to input the edge banding image of the product to be inspected into the target classification network model to obtain a classification result if the initial edge banding defect detection result is no defect;

[0138] The fifth module 805 is used to use the defect recognition strategy to perform strategy determination on the edge banding image of the product to be inspected, and obtain a strategy determination result if both the defect matching result and the classification result are no defects;

[0139] The sixth module 806 is used to determine the target edge defect detection result according to the target recognition result, the target recognition result includes a defect matching result, a classification result or a strategy determination result, and the target edge defect detection result includes edge rupture, line pulling, edge bursting, edge indentation or no defect.

[0140] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0141] like Fig. 9 As shown, an embodiment of the present invention further provides a computer device, including:

[0142] at least one processor 901;

[0143] At least one memory 902, used to store at least one program;

[0144] When at least one program is executed by at least one processor, the at least one processor implements Figure 1 The method shown.

[0145] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0146] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, Figure 1 The method shown.

[0147] The contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0148] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for detecting edge defects of a product, characterized in that: The following steps are involved: Acquire the edge banding image of the product to be inspected; Inputting the edge banding image of the product to be inspected into a target edge banding defect detection model to obtain an initial edge banding defect detection result; If the initial edge sealing defect detection result is defective, template matching is performed on the edge sealing image of the product to be detected using an edge sealing template to obtain a defect matching result; If the initial edge banding defect detection result is no defect, inputting the edge banding image of the product to be inspected into the target classification network model to obtain a classification result; If both the defect matching result and the classification result are defect-free, a defect recognition strategy is used to perform a strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result; According to the target recognition result, the target edge sealing defect detection result is determined, the target recognition result includes the defect matching result, the classification result or the strategy determination result, and the target edge sealing defect detection result includes edge sealing rupture, wire pulling, edge sealing explosion, edge sealing indentation or no defect.

2. The method according to claim 1, characterized in that The target edge banding defect detection model is obtained by the following steps: obtaining a product edge banding image set, wherein the product edge banding image set includes edge banding qualified images and edge banding defect images; generating an initial edge banding defect detection model according to the product edge banding image set; Collecting an edge banding update image set, wherein the edge banding update image set includes a newly added defect image and an edge banding image to be identified, wherein the newly added defect image has an untrained defect category; The target edge banding defect detection model is generated according to the edge banding update image set and the initial edge banding defect detection model.

3. The method according to claim 2, characterized in that The step of generating an initial edge banding defect detection model according to the product edge banding image set comprises: Performing grayscale conversion on the product edge banding image set to obtain a grayscale data set; Cutting out the defective area in the edge sealing defect image to obtain a defective area image; Overlaying the defective area image onto the qualified edge sealing image to obtain an augmented data set; Using a preset edge detection method to perform edge detection on the product edge banding image set to obtain a contour data set; The product edge banding image set, the grayscale data set, the amplified data set and the contour data set are input into a preset edge banding defect detection model so that the preset edge banding defect detection model is trained to obtain the initial edge banding defect detection model.

4. The method according to claim 3, characterized in that The step of generating the target edge banding defect detection model according to the edge banding update image set and the initial edge banding defect detection model comprises: Performing category labeling processing on the newly added defect image to obtain a newly added labeled image; According to the prediction threshold, the edge banding image to be identified is input into the initial edge banding defect detection model to obtain an edge banding defect identification result, and the edge banding defect identification result is used as a pseudo label; Combining the newly added annotated image with the edge banding defect recognition result to obtain a new task data set; The new task data set, the product edge banding image set, the grayscale data set, the augmented data set and the contour data set are input into the initial edge banding defect detection model through incremental learning, so that the initial edge banding defect detection model is retrained to obtain the target edge banding defect detection model.

5. The method according to claim 1, characterized in that The edge banding template is obtained by the following steps: Use white light source to capture product images; Identify the edge color of the edge area in the product image through a three-channel color space; If the edge banding color is black, re-capture is performed using a red light source to update the product image; If the edge banding color is white, re-capture is performed using a blue light source to update the product image; If the edge banding color is yellow, re-capture is performed using a yellow light source to update the product image; The edge banding area is cut out from the updated product image as the edge banding template.

6. The method according to claim 1, characterized in that The target classification network model is obtained by the following steps: The product edge banding image set is input into a preset classification network model so that the preset classification network model is trained to obtain the target classification network model.

7. The method according to claim 1, characterized in that The method of using the defect recognition strategy to perform strategy determination on the edge banding image of the product to be inspected to obtain a strategy determination result includes: If a long strip protrudes from the edge of the edge banding in the edge banding image of the product to be detected, the length of the long strip is greater than a preset length threshold, and the angle between the long strip and the horizontal plane is greater than a preset angle threshold, the pull line is used as the strategy determination result; If there is an ellipse in the edge sealing of the product to be detected and the area of ​​the ellipse is greater than a preset area threshold, the edge sealing is exploded as the strategy determination result; If there are multiple linear objects on the edge of the product to be inspected in the edge image and the multiple linear objects intersect each other, the edge rupture is taken as the strategy determination result; If there are concave blocks and convex blocks in the edge sealing of the product to be detected, the total area of ​​the concave blocks and the convex blocks is greater than the preset concave-convex threshold and the average distance between the concave blocks and the convex blocks is less than the preset distance threshold, the edge sealing indentation is taken as the strategy judgment result.

8. A device for detecting defects in product edge banding, characterized in that: include: The first module is used to obtain the edge banding image of the product to be inspected; The second module is used to input the edge banding image of the product to be inspected into the target edge banding defect detection model to obtain an initial edge banding defect detection result; The third module is used for, if the initial edge sealing defect detection result is defective, using the edge sealing template to perform template matching on the edge sealing image of the product to be detected to obtain a defect matching result; The fourth module is used for inputting the edge banding image of the product to be inspected into the target classification network model to obtain a classification result if the initial edge banding defect detection result is no defect; The fifth module is used to perform a strategy determination on the edge banding image of the product to be inspected by using a defect recognition strategy to obtain a strategy determination result if both the defect matching result and the classification result are defect-free; The sixth module is used to determine the target edge banding defect detection result according to the target recognition result, wherein the target recognition result includes the defect matching result, the classification result or the strategy determination result, and the target edge banding defect detection result includes edge banding rupture, wire pulling, edge banding explosion, edge banding indentation or no defect.

9. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images