Cosmetic bottle defect automatic detection system based on machine vision

Abstract

The invention relates to the technical field of defect detection, in particular to an automatic cosmetic bottle defect detection system based on machine vision, which comprises a bottle body acquisition module for performing image acquisition on cosmetic bottles on an assembly line, and performing matching and image correction according to cosmetic bottle features in a detection area and preset cosmetic bottle features; obtaining a to-be-detected cosmetic bottle image; the surface defect detection module performs appearance feature recognition on the image of the cosmetic bottle to be detected in combination with the features of the cosmetic bottle, and performs bottle body surface defect detection according to different appearance features; if the defects exist, calculating the overall defect degree, judging whether sales is affected or not according to the overall defect degree, performing early warning and marking cosmetic bottles; the sealing defect detection module is used for processing the to-be-detected cosmetic bottle image by using a bottle opening feature matching algorithm to obtain a bottle opening image, and performing defect detection to judge whether a sealing defect exists or not; if yes, early warning is conducted, and the cosmetic bottles are marked.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and particularly to an automatic defect detection system for cosmetic bottles based on machine vision. Background Art

[0002] With the rapid development of the cosmetic industry, as the core carrier of product packaging, the appearance quality and sealing performance of cosmetic bottles directly affect the consumer experience and brand reputation. Traditional manual inspection methods rely on visual inspection, which has problems such as low efficiency, poor stability, and being easily affected by subjective factors. Especially on high-precision and high-throughput production lines, it is difficult for manual inspection to meet the refined identification requirements for defects such as micro-scratches, air bubbles, and bottle mouth deformation. Although machine vision technology has been applied in the bottle inspection of fields such as food and medicine, existing inspection systems are mostly designed for single materials or standardized bottle shapes, and lack adaptability to the variable material characteristics (such as transparent, semi-transparent, pearlescent spraying), complex surface textures (such as anti-slip patterns, relief patterns), and sealing structures of cosmetic bottles, resulting in it being difficult to balance detection accuracy and efficiency, and restricting the whole-process control of cosmetic production quality.

[0003] The existing machine vision-based bottle defect detection technologies generally have the following limitations: Firstly, when detecting impurities inside transparent bottles, misjudgment is easily caused due to the interference of light refraction, and the surface reflection characteristics of matte sprayed bottles make it difficult for traditional algorithms to accurately locate scratches; Secondly, the detection of bottle mouth sealing performance mostly relies on contact physical tests, which cannot be efficiently integrated with the vision detection system, and the recognition sensitivity to micron-level structural defects such as thread precision and sealing ring slot positions is insufficient; Thirdly, decorative areas such as anti-slip patterns and reliefs are prone to forming texture interference, and existing methods are prone to missing detections during segmentation and defect marking. Especially under the dynamic imaging conditions of high-speed production lines, the ability to extract multi-scale features and perform real-time analysis is insufficient. In addition, there is a lack of systematic detection for composite indicators such as liquid level consistency and label adhesion after the cosmetic bottles are filled, and there is an urgent need to develop an automatic solution that can take into account multi-dimensional defect detection.

[0004] Therefore, an automatic defect detection system for cosmetic bottles based on machine vision is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide an automated defect detection system for cosmetic bottles based on machine vision to improve the quality detection efficiency and accuracy of cosmetic bottles on the assembly line. The bottle body acquisition module acquires images of cosmetic bottles on the assembly line, and matches and corrects the images according to the characteristics of the cosmetic bottles in the detection area and the preset cosmetic bottle characteristics to obtain the images of the cosmetic bottles to be detected; the surface defect detection module combines the characteristics of the cosmetic bottles to identify the appearance characteristics of the images of the cosmetic bottles to be detected, and detects the surface defects of the bottle body for different appearance characteristics; if there are defects, the overall defect degree is calculated, and it is judged whether it affects sales according to the overall defect degree, and a warning is issued and the cosmetic bottle is marked; the sealing defect detection module is used to process the images of the cosmetic bottles to be detected by using the bottle mouth feature matching algorithm to obtain the bottle mouth images, and perform defect detection to judge whether there are sealing defects; if there are, a warning is issued and the cosmetic bottle is marked.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An automated defect detection system for cosmetic bottles based on machine vision, comprising:

[0008] The bottle body acquisition module acquires images of cosmetic bottles on the assembly line, and matches and corrects the images according to the characteristics of the cosmetic bottles in the detection area and the preset cosmetic bottle characteristics to obtain the images of the cosmetic bottles to be detected;

[0009] Further, the process of matching according to the characteristics of the cosmetic bottles in the detection area and the preset cosmetic bottle characteristics includes:

[0010] After the laser sensor on the assembly line detects that the cosmetic bottle reaches the shooting position, the camera is triggered synchronously to acquire images, and a number of first images are obtained; the features of the first images are extracted to obtain the cosmetic bottle features, and further the first cosmetic bottle feature set of all the first images is obtained; the cosmetic bottle features in the first cosmetic bottle set are obtained and matched with the preset cosmetic bottle characteristics, and sorted according to the matching results, and the first image corresponding to the cosmetic bottle feature that best matches the preset cosmetic bottle characteristics is the initial image of the cosmetic bottle to be detected.

[0011] Further, the process of image correction includes:

[0012] The initial image of the cosmetic bottle to be detected is obtained, the cosmetic bottle features corresponding to the preset cosmetic bottle characteristics are extracted and feature-matched to obtain the matching information, including the matching feature point pairs and the contour correspondence;

[0013] The OpenCV algorithm is used to obtain and process the matching information to obtain the geometric transformation matrix, including the affine transformation matrix and the perspective transformation matrix;

[0014] Process the initial image of the cosmetic bottle to be detected using a geometric transformation matrix to obtain a corrected image; perform feature extraction and feature matching on the corrected image. If the matching score exceeds the threshold, obtain the image of the cosmetic bottle to be detected; if the matching score does not exceed the threshold, repeat the image correction.

[0015] The surface defect detection module combines the characteristics of the cosmetic bottle to identify the appearance features of the image of the cosmetic bottle to be detected, and performs bottle body surface defect detection for different appearance features; if there are defects, calculate the overall defect degree, and determine whether it affects sales based on the overall defect degree, issue a warning and mark the cosmetic bottle.

[0016] Further, the process of performing bottle body surface defect detection includes:

[0017] The appearance features include the label pasting area, the key area of the bottle body, the text printing area, the bottle shoulder and the material joint; use machine vision technology to identify and locate the appearance features, and call the detection algorithms and parameters for each area. The calling method is determined according to the historical detection results of the algorithm in the area.

[0018] Perform defect detection on each area and obtain the defect detection results, including defect type, defect area, defect depth and defect position data.

[0019] Further, the process of calculating the overall defect degree includes:

[0020] Calculate the overall defect degree of the cosmetic bottle according to the defect area, defect depth, defect position data obtained by the surface defect detection module and the corresponding weight coefficients. Among them, the weight coefficients are obtained according to the defect occurrence frequency in historical data and the importance of the position where the cosmetic bottle defect is located. The importance of the position where the cosmetic bottle defect is located is obtained according to the regional importance division of the cosmetic bottle body.

[0021] Further, if the overall defect degree of the cosmetic bottle is greater than the preset defect degree threshold, it means that the defect of the cosmetic bottle affects the sales of the cosmetic bottle, mark the corresponding cosmetic bottle and issue a warning.

[0022] The seal defect detection module is used to process the image of the cosmetic bottle to be detected using the bottle mouth feature matching algorithm to obtain the bottle mouth image, and perform defect detection to judge whether there is a seal defect; if there is, issue a warning and mark the cosmetic bottle.

[0023] Further, the process of processing the image of the cosmetic bottle to be detected using the bottle mouth feature matching algorithm includes:

[0024] Obtain the image of the cosmetic bottle to be detected and perform feature extraction to obtain geometric shape features, structural features, and position features; obtain the shape features and process them through the Hough transform to obtain the first bottle mouth image, obtain the structural features and process them through the contour analysis algorithm to obtain the second bottle mouth image, obtain the position features and process them through the deep learning algorithm to obtain the third bottle mouth image; obtain the first bottle mouth image, the second bottle mouth image, and the third bottle mouth image and perform intersection processing of the image positions to obtain the bottle mouth image.

[0025] Further, the process of defect detection for the bottle mouth image includes:

[0026] Perform white balance correction, histogram equalization, and Gaussian filtering or median filtering on the bottle mouth image to eliminate uneven illumination and noise, and obtain the processed bottle mouth image; process the processed bottle mouth image to obtain the skeleton features of the cosmetic bottle mouth, including the number of breakpoints feature, the total skeleton length feature, the distance between adjacent breakpoints feature, the number of skeleton branches feature, the number of breakpoints in a specific area feature, the skeleton curvature feature, and the breakpoint distance feature; among them, if a pixel has only 1 non-zero pixel in its 8-neighborhood, the pixel is considered a breakpoint; if the number of non-zero pixels in the 8-neighborhood of a pixel is greater than 2, the pixel is a branch point; calculate the sum of the Euclidean distances between all connected pixels in the skeleton graph to obtain the total skeleton length; if there are multiple breakpoints in the skeleton graph, calculate the average distance between adjacent breakpoints to obtain the distance between adjacent breakpoints feature; for the sealed contact area, count the number of breakpoints in the area to obtain the number of breakpoints in a specific area feature; estimate the local curvature of each part of the skeleton to obtain the skeleton curvature feature; calculate the distance between a breakpoint and the nearest branch point to obtain the breakpoint distance feature;

[0027] Perform weighted processing based on the skeleton features of the cosmetic bottle mouth to construct a sealing defect index. If it exceeds the sealing threshold, it indicates that there is a sealing defect in the cosmetic bottle mouth, and a warning is issued and the cosmetic bottle is marked.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] 1. Through the synchronous triggering of the laser sensor and the camera, multiple images are collected and the features of the cosmetic bottle are extracted respectively, and then they are matched and sorted with the preset cosmetic bottle features, so as to accurately determine the initial image to be detected; subsequently, the algorithm is used to calculate the geometric transformation matrix for image correction, continuously optimizing the image quality until the matching score reaches the preset threshold, thereby ensuring that the finally obtained detection image has a stable and consistent geometric structure and appearance features, greatly improving the accuracy and robustness of the detection, and reducing the misjudgment caused by image distortion.

[0030] 2. By identifying defects in key parts of the cosmetic bottle surface in different regions, such as the label pasting area, the key areas of the bottle body, the text printing area, the bottle shoulder and the material joint, and combining the weight coefficients determined according to the historical detection data of the defect areas, the area, depth and position data of each defect area are accurately quantified, and then weighted and summed to calculate the overall defect degree. This can not only more accurately reflect the impact of defects on the overall appearance and function, but also improve the robustness of detection.

[0031] 3. The bottle mouth feature matching algorithm can accurately locate the bottle mouth area, effectively exclude other interference factors in the image, provide an accurate operation object for subsequent defect detection, thereby improving the detection accuracy and reducing the false detection rate; secondly, by skeletonizing the bottle mouth image and extracting multi-dimensional features, the tiny structure of the bottle mouth edge can be analyzed in detail, so as to effectively identify various potential sealing defects; finally, by weighted construction of the sealing defect index, automated defect judgment and early warning can be realized, timely discover and mark the cosmetic bottles with sealing risks, and improve the quality detection efficiency of cosmetic bottles. Description of the Drawings

[0032] Figure 1 It is a schematic structural diagram of an automatic defect detection system for cosmetic bottles based on machine vision provided by an embodiment of the present invention;

[0033] Figure 2 It is a flowchart for obtaining an initial image of a cosmetic bottle to be detected provided by an embodiment of the present invention;

[0034] Figure 3 It is a flowchart of the working process of the sealing defect detection module provided by an embodiment of the present invention. Detailed Embodiments

[0035] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0036] Embodiment 1:

[0037] Cosmetics products have very high requirements for sealing, especially in high-end products, where the bottle mouth and bottle cap often adopt a composite sealing design, which not only prevents the internal contents from leaking, volatilizing or being contaminated by the outside world, but also maintains the appearance and user experience. Traditional manual inspection methods are easily affected by subjective factors and fatigue, while machine vision inspection has the characteristics of high speed, high precision, non-contact, and objective consistency, and can achieve efficient inspection on automated production lines. By detecting whether there are defects such as tiny cracks, bubbles, misalignment or uneven sealing in the sealing area, the product quality and safety can be effectively improved.

[0038] In order to improve the defect detection efficiency of cosmetic bottles on the production line, a cosmetics company introduced a cosmetic bottle defect automatic detection system based on machine vision provided by the present invention. The system structure is as follows: Figure 1 As shown, the specific implementation is as follows:

[0039] First, the bottle body acquisition module acquires images of the cosmetic bottles on the production line, and matches and corrects the images according to the image features in the detection area and the preset cosmetic bottle features to obtain the image of the cosmetic bottle to be detected;

[0040] Furthermore, the process of matching the cosmetic bottle features in the detection area with the preset cosmetic bottle features includes:

[0041] After the laser sensor on the assembly line detects that the cosmetic bottle has arrived at the shooting position, it synchronously triggers the camera to collect images to obtain a number of first images; feature extraction is performed on the first images to obtain cosmetic bottle features, and further a first cosmetic bottle feature set of all first images is obtained; the cosmetic bottle features in the first cosmetic bottle set are obtained and matched with preset cosmetic bottle features, and sorted according to the matching results, and the first image corresponding to the cosmetic bottle feature that best matches the preset cosmetic bottle feature is the initial cosmetic bottle image to be detected.

[0042] Furthermore, according to the needs of product design, standard cosmetic bottle images are collected in advance under multiple angles and lighting conditions, and feature information such as their geometric contours, spatial structures, color distributions and texture patterns are extracted. Common features such as the outer contour of the bottle, the shape of the bottle mouth, the position of the label and other iconic patterns are also included. These are important bases for subsequent matching, forming a preset template or feature descriptor library, and further obtaining the preset cosmetic bottle features.

[0043] Furthermore, using local feature point detection methods such as SIFT, SURF, and ORB, feature point matching is performed between the captured image and the preset cosmetic bottle features, or the candidate area extracted from the current captured image is matched with the preset feature template, and the similarity score is calculated. The first image corresponding to the cosmetic bottle feature with the highest score is selected as the initial cosmetic bottle image to be detected. The process is as follows:Figure 2 as shown

[0044] Through synchronous triggering and acquisition by a laser sensor and a high-speed camera, it is ensured that the acquired images meet the requirements in terms of timeliness and clarity; secondly, algorithms are used to extract key features and match them with preset templates, so that the most standard-compliant target images can be automatically selected under complex lighting, angle change or background interference conditions, reducing the risks of false detection and missed detection; this method not only ensures high-quality and standardized initial images in the subsequent defect detection stage, but also improves the real-time performance and stability of the entire automated detection process, providing solid data support for product sorting and quality control.

[0045] Furthermore, the process of image correction includes:

[0046] Obtain the initial image of the cosmetic bottle to be detected, extract the cosmetic bottle features corresponding to the preset cosmetic bottle features and perform feature matching to obtain matching information, including pairs of matching feature points and contour correspondence;

[0047] Use the OpenCV algorithm to obtain and process the matching information to obtain geometric transformation matrices, including an affine transformation matrix and a perspective transformation matrix;

[0048] Use the geometric transformation matrices to process the initial image of the cosmetic bottle to be detected to obtain a corrected image; perform feature extraction and feature matching on the corrected image. If the matching score exceeds the threshold, the image of the cosmetic bottle to be detected is obtained; if the matching score does not exceed the threshold, repeat the image correction until the matching score exceeds the threshold.

[0049] Furthermore, the affine transformation matrix is used to correct the geometric distortions of translation, scaling and rotation in the image, and the perspective transformation matrix is used to eliminate the perspective distortion caused by the shooting angle, so that the geometric shape of the cosmetic bottle in the image is close to the actual appearance of the standard template. Use relevant algorithms (such as RANSAC) in the OpenCV library to perform robust estimation between the initial image and the preset template, and calculate a stable geometric transformation matrix from the matching information.

[0050] Furthermore, perform feature extraction on the corrected image again and calculate the matching score with the preset cosmetic bottle features. If the matching score exceeds the preset threshold, the image is confirmed as the image of the cosmetic bottle to be detected and enters the subsequent defect detection process. If the matching score does not meet the requirements, readjust the geometric transformation parameters and repeat the image correction and matching verification process until an image that meets the standards is obtained.

[0051] By matching the features with a preset template and performing geometric correction, it is ensured that the corrected image highly coincides with the standard template, effectively eliminating errors caused by shooting angles, perspective distortions, etc., thus providing high-quality and standardized input images for subsequent defect detection. In addition, multiple matching verifications and threshold judgments can significantly improve the robustness and accuracy of detection, reducing the risks of false positives and missed detections.

[0052] The working process of the surface defect detection module is as Figure 3 shown. It combines the characteristics of cosmetic bottles to identify appearance features and detects surface defects on the bottle body for different appearance features. If there are defects, the overall defect degree is calculated, and it is determined whether it affects sales based on the overall defect degree, and a warning is given and the cosmetic bottle is marked.

[0053] Furthermore, the process of detecting surface defects on the bottle body includes:

[0054] Appearance features include the label pasting area, key areas of the bottle body, text printing areas, bottle shoulders, and material joints. Machine vision technology is used to identify and locate appearance features, and detection algorithms and parameters are called for each area. The calling method is determined according to the historical detection results of the algorithm in the area.

[0055] Defect detection is performed on each area, and defect detection results are obtained, including defect type, defect area, defect depth, and defect position data.

[0056] Furthermore, for the appearance of the bottle body, first, according to product design requirements and historical detection data, the bottle body is divided into multiple key areas, each area corresponding to different appearance features. These areas usually include the label pasting area, which is the area on the bottle body for pasting brand labels or product descriptions, the key areas of the bottle body, including the overall appearance contour and the main visual focus area, which determine the beauty and symmetry of the bottle, the text printing area, including areas for printing product names, ingredient descriptions, warning signs, etc., the bottle shoulder area, the transitional part between the upper and lower parts of the bottle body, which is usually prone to local defects due to the molding process, and the material joint, the connection area between different materials of the bottle body (such as transparent and frosted, between the bottle body and the bottle cap), which is prone to interface irregularities or delamination.

[0057] Furthermore, by using preset information such as geometric contours, spatial structures, color distributions, and texture patterns, and combining a preset template (or feature descriptor library) established from standardized images collected from multiple angles and under multiple lighting conditions, automatic positioning and identification of the above areas can be achieved.

[0058] Further, using deep learning object detection or traditional morphological algorithms, each part of the cosmetic bottle in the image is initially segmented, and feature matching is performed in combination with a preset template. Then, the ROI of each key area is delimited, such as the label area, printing area, bottle shoulder and material junction, etc. According to historical detection data and statistical characteristics, the corresponding detection algorithm is called for each area, and special detection parameters (such as threshold, filtering window size, edge sensitivity, etc.) are set. This adaptive calling method can be optimized and updated according to the performance of the area in historical detections (such as error rate or feature distribution) to ensure that the detection of each area is more in line with the actual working conditions. For example, for the label pasting area, color consistency detection, edge continuity analysis, and texture uniformity detection can be used; for the text printing area, high-contrast edge extraction and OCR are used to assist in verifying printing clarity; for the bottle shoulder and material junction, more attention is paid to structural abnormalities, uneven dividing lines, or local depressions and protrusions in the area, and local template matching and morphological segmentation can be used; for the key area of the bottle body, the detection focus is usually on overall appearance defects, such as large-area scratches, abrasions, bubbles, or poor molding. When the detection algorithm for each area is called, it will read the historical detection results (such as defect occurrence frequency, error statistics, etc.) to adjust the threshold parameters and processing window size, so as to obtain the algorithm configuration most suitable for the current detection scenario.

[0059] Further, in each area, the detection algorithm will output detailed defect detection results, including specific defects that may be detected according to different areas, such as scratches, bubbles, glue detachment, printing blurring, discontinuous edges, local depressions or protrusions, etc.; it also outputs the area occupied by the defect in this area, usually expressed in pixels or the actual area after calibration, which is used to judge the severity of the defect; it also outputs information such as defect depth and defect location.

[0060] By performing segmented detections on multiple different appearance feature areas such as the label pasting area, key area of the bottle body, printing area, bottle shoulder, and material junction, the proprietary information of each area and historical detection data can be fully utilized to achieve the adaptive optimization of the algorithm. This not only greatly improves the accuracy and robustness of the detection, but also enables fine quantitative analysis of different types of defects, providing detailed and reliable data support for subsequent product quality scoring, automatic sorting, and production process improvement.

[0061] Further, the process of calculating the overall defect degree includes:

[0062] Calculating the overall defect degree of the cosmetic bottle according to the defect area, defect depth, defect location data obtained by the surface defect detection module and the corresponding weight coefficients, where the weight coefficients are obtained according to the defect occurrence frequency in historical data and the importance of the location of the cosmetic bottle defect, and the importance of the location of the cosmetic bottle defect is obtained according to the regional importance division of the cosmetic bottle body;

[0063] If the overall defect degree of the cosmetic bottle is greater than the preset defect degree threshold, it indicates that the defects of the cosmetic bottle affect the sales of the cosmetic bottle. Mark the corresponding cosmetic bottle and give an early warning.

[0064] Furthermore, the calculation formula for the overall defect degree is:

[0065]

[0066] where Defect represents the overall defect degree, M represents the number of detected defects in the image of the cosmetic to be detected, f m represents the frequency of occurrence of the m-th defect in the historical data, p m represents the regional importance of the area where the defect is located, A m represents the area of the m-th defect, B m represents the depth of the m-th defect, P m represents the defect importance of the m-th defect, and α, β, and γ are scaling factors for the data, used to normalize or unify the dimensions of each index, so that each index can be weighted and summed in the same formula. Using data such as the defect area, depth, and position information, and combining the weights determined from historical data, makes the quantitative evaluation of defects more comprehensive, refined, and targeted.

[0067] As shown in Table 1, the regional importance of the cosmetic bottle in areas such as the label pasting area, the key area of the bottle body, the text printing area, the bottle shoulder, and the material joint is shown.

[0068] After normalizing and weighted summing each index, it not only improves the objectivity and robustness of the detection, but also can dynamically adjust the detection sensitivity according to the importance of different areas in product quality, so as to more accurately judge whether the defect will affect the sales of the cosmetic bottle, and realize automatic early warning and marking on the product sorting line, providing a quantitative basis and data support for production quality control.

[0069] Table 1. Regional importance of each area of the cosmetic bottle

[0070] Region Region Importance Label Pasting Area 0.30 Key Area of Bottle Body 0.35 Text Printing Area 0.20 Bottle Shoulder 0.10 Material Junction 0.05

[0071] As shown in Table 2, the defect importance of some defects is shown.

[0072] Table 2. Importance of some defects

[0073] Defect Type Defect Importance Scratch 0.20 Bubble 0.15 Decal Off 0.25 Poor Printing 0.35

[0074] The seal defect detection module is used to process the image of the cosmetic bottle to be detected by using the bottle mouth feature matching algorithm, obtain the bottle mouth image, and perform defect detection to judge whether there is a seal defect; if so, give an early warning and mark the cosmetic bottle.

[0075] Furthermore, this step occurs after the overall surface defect detection of the bottle body (if passed). Its core objective is to precisely focus attention on the key area of the bottle mouth for subsequent more refined defect detection related to sealing performance.

[0076] Furthermore, the "bottle mouth feature matching algorithm" is not a single algorithm but a collective term for a class of methods. Its core idea is to utilize the relatively stable and unique features of the bottle mouth area to find its exact position and range in the image. The process of processing the bottle mouth using the bottle mouth feature matching algorithm includes:

[0077] Obtain the image of the cosmetic bottle to be detected and perform feature extraction to obtain geometric shape features, structural features, and position features; obtain the shape features and process them through the Hough transform to obtain the first bottle mouth image, obtain the structural features and process them through the contour analysis algorithm to obtain the second bottle mouth image, obtain the position features and process them through the deep learning algorithm to obtain the third bottle mouth image; obtain the first bottle mouth image, the second bottle mouth image, and the third bottle mouth image and perform intersection processing of the image positions to obtain the bottle mouth image.

[0078] Furthermore, the geometric shape features describe the overall contour and shape of the bottle mouth. For cosmetic bottle mouths, common geometric shapes include circles, ellipses, rectangles, or special-shaped ones with specific arcs. Extracting these features helps to preliminarily determine whether there is a potential bottle mouth area in the image. The Hough transform is a classic method for detecting specific shapes (such as lines, circles, ellipses) in an image. For cosmetic bottle mouths, especially circular or elliptical ones, the Hough circle transform or the Hough ellipse transform can effectively detect the potential bottle mouth contour.

[0079] Furthermore, the structural features focus on the unique structural elements of the bottle mouth, such as threads, bayonets, sealing rings, inner plugs, etc. These structural features are the key to distinguishing different types of bottle mouths and are directly related to sealing performance. Extracting the structural features helps to more precisely locate the bottle mouth and provides structured information for subsequent defect detection; use the contour analysis algorithm for the extracted structural features to analyze the continuous edges and texture distributions in the image. Contour analysis can identify the detailed edges, dividing lines, and other structural information of the bottle mouth, generating the second bottle mouth image, which mainly reflects the detailed structure of the bottle mouth.

[0080] Furthermore, the position feature describes the position and orientation information of the bottle mouth in the entire image. Since the cosmetic bottle usually has a relatively fixed placement pattern in the image, using the position feature can narrow the search range and improve the detection efficiency. Common position features include image coordinates and orientation information. The position information of the bottle mouth in the image is input into the object localization model trained by deep learning. This model learns using a large amount of previously annotated data and can accurately identify the position of the bottle mouth under complex backgrounds and varying lighting conditions, and finally generates the third bottle mouth image, emphasizing the accurate position of the bottle mouth and the overall layout.

[0081] Furthermore, in order to more accurately locate the bottle mouth, it is necessary to fuse the three intermediate results obtained through different feature processing. The first bottle mouth image, the second bottle mouth image, and the third bottle mouth image are regarded as binary mask images, where the pixel value of the area marked as the bottle mouth is 1, and the pixel value of the background area is 0. Then, a pixel-level logical AND operation is performed on these three mask images. Only when the pixel values at the corresponding positions in all three mask images are 1, the pixel value at this position in the result image is 1.

[0082] The final image obtained after the intersection processing is the image of the bottle mouth area accurately extracted by us. This image integrates feature information in multiple aspects such as geometric shape, structure, and position, has high accuracy, and can be used as the input for subsequent defect detection. Through the above steps, using the bottle mouth feature matching algorithm, the accurate bottle mouth area can be effectively extracted from the cosmetic bottle image, laying a solid foundation for subsequent sealability defect detection, improving the accuracy and robustness of bottle mouth localization, and reducing the false detection rate.

[0083] Furthermore, detecting the continuity of the edge of the cosmetic bottle mouth is one of the key steps in judging whether there is a sealability defect. A continuous and complete bottle mouth edge is the basis for achieving good sealing. The process of defect detection for the bottle mouth image includes:

[0084] Perform white balance correction, histogram equalization, and Gaussian filtering or median filtering on the collected bottle mouth images to eliminate uneven illumination and noise, obtaining the processed bottle mouth images; process the processed bottle mouth images to obtain the cosmetic bottle mouth skeleton features, including the number of breakpoints feature, total skeleton length feature, distance between adjacent breakpoints feature, number of skeleton branches feature, number of breakpoints in a specific area feature, skeleton curvature feature, and breakpoint distance feature; among them, if a pixel has only 1 non-zero pixel in its 8-neighborhood, then this pixel is considered a breakpoint; if the number of non-zero pixels in the 8-neighborhood of a pixel is greater than 2, then this pixel is considered a branch point; calculate the sum of the Euclidean distances between all connected pixels in the skeleton graph to obtain the total skeleton length; if there are multiple breakpoints in the skeleton graph, calculate the average distance between adjacent breakpoints to obtain the distance between adjacent breakpoints feature; for the sealed contact area, count the number of breakpoints in the area to obtain the number of breakpoints in a specific area feature; estimate the local curvature for each part of the skeleton to obtain the skeleton curvature feature; calculate the distance between a breakpoint and the nearest branch point to obtain the breakpoint distance feature;

[0085] Perform weighted processing based on the cosmetic bottle mouth skeleton features to construct a sealing defect index. If it exceeds the sealing threshold, it indicates that there is a sealing defect in the cosmetic bottle mouth, and an alarm is issued and the cosmetic bottle is marked.

[0086] Furthermore, the calculation formula for the sealing defect index is:

[0087]

[0088] where SDI represents the sealing defect index, ω 1 represents the weight coefficient of the number of breakpoints feature N ep in the skeleton, L total represents the total skeleton length, ω 2 represents the weight coefficient of the distance between adjacent breakpoints feature d avg in the skeleton, ω 3 represents the weight coefficient of the number of skeleton branches feature N branch in the skeleton, represents the number of branches in the standard skeleton, ω 4 represents the weight coefficient of the number of breakpoints in a specific area feature N reg in the skeleton, ω 5 represents the weight coefficient of the skeleton curvature feature, C i represents the actual curvature of the i-th skeleton, represents the standard curvature of the i-th skeleton, I represents the total number of skeletons, ω 6 represents the weight coefficient of the breakpoint distance feature D avg in the skeleton.

[0089] By comprehensively considering multiple aspects such as the number of breakpoints, the density of breakpoints, edge branches, local curvature anomalies, and key area defects, different features are weighted according to their actual importance, achieving a scientific evaluation of sealing defects. It can not only accurately capture subtle defects, reduce misjudgment and missed detection, but also dynamically adjust the weight of each parameter according to historical data, thereby greatly improving the robustness and accuracy of the detection system.

[0090] An automated detection system for cosmetic bottle defects based on machine vision provided by the present invention realizes standardized and high-quality input of images through automated acquisition and preset template matching; greatly improves the robustness and accuracy of detection through surface defect detection and sealing defect detection; effectively reduces the risks of misjudgment and missed detection through automatic warning and marking of unqualified products, and improves production efficiency and product quality. Through image acquisition, feature matching, image correction, and multi-region defect detection technologies, it can comprehensively evaluate the appearance and sealing performance of cosmetic bottles in real time and accurately, improving the defect detection efficiency of cosmetic bottles on the production line.

[0091] Embodiment 2:

[0092] A certain cosmetic company introduced an automated detection system for cosmetic bottle defects based on machine vision provided by the present invention to improve the defect detection efficiency of cosmetic bottles. The specific implementation method is as follows:

[0093] The bottle body acquisition module acquires images of cosmetic bottles on the production line, and performs matching and image correction according to the features of the cosmetic bottles in the detection area and the preset cosmetic bottle features to obtain the images of the cosmetic bottles to be detected.

[0094] Further, the process of matching according to the features of the cosmetic bottles in the detection area and the preset cosmetic bottle features includes:

[0095] After the laser sensor on the production line detects that the cosmetic bottle reaches the shooting position, it synchronously triggers the camera to acquire images, obtaining a number of first images; extracting features from the first images to obtain the features of the cosmetic bottles, and further obtaining the first cosmetic bottle feature set of all the first images; acquiring the features of the cosmetic bottles in the first cosmetic bottle set and matching them with the preset cosmetic bottle features, and sorting according to the matching results. The first image corresponding to the cosmetic bottle feature that best matches the preset cosmetic bottle feature is the initial image of the cosmetic bottle to be detected.

[0096] Further, the process of image correction includes:

[0097] Obtaining the initial image of the cosmetic bottle to be detected, extracting the cosmetic bottle features corresponding to the preset cosmetic bottle features and performing feature matching to obtain matching information, including the matching feature point pairs and contour correspondence relationships.

[0098] Use the OpenCV algorithm to obtain and process the matching information to get the geometric transformation matrix, including the affine transformation matrix and the perspective transformation matrix;

[0099] Use the geometric transformation matrix to process the initial image of the cosmetic bottle to be detected to obtain the corrected image; perform feature extraction and feature matching on the corrected image. If the matching score exceeds the threshold, the image of the cosmetic bottle to be detected is obtained; if the matching score does not exceed the threshold, repeat the image correction.

[0100] Furthermore, as shown in Table 3, the correction results of some images are presented. Only when it exceeds the threshold of 85 after image correction can it be regarded as the image of the cosmetic bottle to be detected.

[0101] Table 3. Image Correction Results

[0102] Image Number Revision Times Matching Score Whether Exceeds Threshold TP5023 First Revision 87 Yes TP5024 First Revision 88 Yes TP5025 First Revision 84 No TP5025 Second Revision 89 Yes

[0103] The surface defect detection module combines the characteristics of the cosmetic bottle to identify the appearance features of the image of the cosmetic bottle to be detected, and performs bottle body surface defect detection for different appearance features; if there are defects, calculate the overall defect degree, and judge whether it affects sales according to the overall defect degree, issue a warning and mark the cosmetic bottle;

[0104] Furthermore, the process of performing bottle body surface defect detection includes:

[0105] The appearance features include the label pasting area, the key area of the bottle body, the text printing area, the bottle shoulder and the material joint; use machine vision technology to identify and locate the appearance features, and call the detection algorithms and parameters for each area. The calling method is determined according to the historical detection results of the algorithm in the area;

[0106] Perform defect detection on each area and obtain the defect detection results, including the defect type, the defect area, the defect depth and the defect position data.

[0107] Furthermore, the process of calculating the overall defect degree includes:

[0108] Calculate the overall defect degree of the cosmetic bottle according to the defect area, defect depth, defect position data obtained by the surface defect detection module and the corresponding weight coefficients. Among them, the weight coefficients are obtained according to the defect occurrence frequency in the historical data and the importance of the position where the cosmetic bottle defect is located. The importance of the position where the cosmetic bottle defect is located is obtained according to the division of the regional importance of the cosmetic bottle body;

[0109] Furthermore, if the overall defect degree of the cosmetic bottle is greater than the preset defect degree threshold of 0.1, it indicates that the defects of the cosmetic bottle affect the sales of the cosmetic bottle. Mark the corresponding cosmetic bottle and give an early warning. Table 4 shows the results of the overall defect degrees of some cosmetic bottles.

[0110] Table 4. Overall Defect Degrees of Some Cosmetic Bottles

[0111] Cosmetic Bottle Number Overall Defect Degree Whether Exceeds Defect Degree Threshold HZP48727 0.08 No HZP48728 0.07 No HZP48729 0.13 Yes HZP48730 0.05 No

[0112] The seal defect detection module is used to process the image of the cosmetic bottle to be detected by using the bottle mouth feature matching algorithm, obtain the bottle mouth image, and perform defect detection to determine whether there is a seal defect; if so, give an early warning and mark the cosmetic bottle.

[0113] Furthermore, the process of processing the image of the cosmetic bottle to be detected by using the bottle mouth feature matching algorithm includes:

[0114] Obtain the image of the cosmetic bottle to be detected and perform feature extraction to obtain geometric shape features, structural features, and position features; obtain the shape features and process them through the Hough transform to obtain the first bottle mouth image, obtain the structural features and process them through the contour analysis algorithm to obtain the second bottle mouth image, obtain the position features and process them through the deep learning algorithm to obtain the third bottle mouth image; obtain the first bottle mouth image, the second bottle mouth image, and the third bottle mouth image and perform intersection processing of the image positions to obtain the bottle mouth image.

[0115] Furthermore, the process of defect detection for the bottle mouth image includes:

[0116] Perform white balance correction, histogram equalization, and Gaussian filtering or median filtering on the bottle mouth image to eliminate uneven illumination and noise, and obtain the processed bottle mouth image; process the processed bottle mouth image to obtain the skeleton features of the cosmetic bottle mouth, including the number of breakpoints feature, the total length of the skeleton feature, the distance between adjacent breakpoints feature, the number of skeleton branches feature, the number of breakpoints in a specific area feature, the skeleton curvature feature, and the breakpoint distance feature; among them, if a pixel has only 1 non-zero pixel in its 8-neighborhood, the pixel is considered a breakpoint; if the number of non-zero pixels in the 8-neighborhood of a pixel is greater than 2, the pixel is a branch point; calculate the sum of the Euclidean distances between all connected pixels in the skeleton graph to obtain the total length of the skeleton; if there are multiple breakpoints in the skeleton graph, calculate the average distance between adjacent breakpoints to obtain the distance between adjacent breakpoints feature; for the seal contact area, count the number of breakpoints in the area to obtain the number of breakpoints in a specific area feature; estimate the local curvature of each part of the skeleton to obtain the skeleton curvature feature; calculate the distance between a breakpoint and the nearest branch point to obtain the breakpoint distance feature;

[0117] Perform weighted processing according to the characteristics of the cosmetic bottle mouth skeleton, construct a sealing defect index. If it exceeds the sealing threshold, it indicates that there is a sealing defect in the cosmetic bottle mouth, and give an early warning and mark the cosmetic bottle.

[0118] Furthermore, as shown in Table 5 are the specific data of the sealing defect indexes of some cosmetics. Through the detection of the sealing performance of the cosmetic bottle mouth, it is possible to timely discover and mark the cosmetic bottles with sealing risks, improving the quality inspection efficiency of cosmetic bottles.

[0119] Table 5. Sealing defect indexes of some cosmetic bottles

[0120] Cosmetic Number Sealing Defect Index Whether Exceeds Sealing Threshold HZP19483 0.02 No HZP19484 0.04 No HZP19485 0.03 No HZP19486 0.02 No

[0121] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cosmetic bottle defect automatic detection system based on machine vision, characterized in that: include: The bottle body acquisition module acquires images of the cosmetic bottles on the production line, and matches and corrects the images according to the cosmetic bottle features in the detection area and the preset cosmetic bottle features to obtain the image of the cosmetic bottle to be detected; The surface defect detection module recognizes the appearance features of the cosmetic bottle image to be detected based on the characteristics of the cosmetic bottle, and detects surface defects on the bottle body according to different appearance features; if there are defects, the overall defect degree is calculated, and it is determined whether it affects sales based on the overall defect degree, and an early warning is issued and the cosmetic bottle is marked; The sealing defect detection module is used to process the image of the cosmetic bottle to be inspected using the bottle mouth feature matching algorithm to obtain the bottle mouth image and perform defect detection to determine whether there is a sealing defect; if so, an early warning is issued and the cosmetic bottle is marked.

2. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1 is characterized in that: The process of matching the cosmetic bottle features in the detection area with the preset cosmetic bottle features includes: After the laser sensor on the assembly line detects that the cosmetic bottle has arrived at the shooting position, it synchronously triggers the camera to collect images to obtain a number of first images; feature extraction is performed on the first images to obtain cosmetic bottle features, and further a first cosmetic bottle feature set of all first images is obtained; the cosmetic bottle features in the first cosmetic bottle set are obtained and matched with preset cosmetic bottle features, and sorted according to the matching results, and the first image corresponding to the cosmetic bottle feature that best matches the preset cosmetic bottle feature is the initial cosmetic bottle image to be detected.

3. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1, characterized in that: The image correction process includes: Acquire an initial cosmetic bottle image to be detected, extract cosmetic bottle features corresponding to preset cosmetic bottle features and perform feature matching to obtain matching information, including matching feature point pairs and contour correspondences; Use OpenCV algorithm to obtain matching information and process it to obtain geometric transformation matrix, including affine transformation matrix and perspective transformation matrix; The initial image of the cosmetic bottle to be detected is processed using a geometric transformation matrix to obtain a corrected image; feature extraction and feature matching are performed on the corrected image. If the matching score exceeds a threshold, the image of the cosmetic bottle to be detected is obtained; if the matching score does not exceed the threshold, the image correction is repeated.

4. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1 is characterized in that: The process of bottle surface defect detection include: Appearance features include label sticking area, key areas of bottle body, text printing area, bottle shoulder and material junction; Use machine vision technology to identify and locate appearance features, and call detection algorithms and parameters for each area. The calling method is determined based on the algorithm's historical detection results in the area. Defect detection is performed on each area, and defect detection results are obtained, including defect type, defect area, defect depth and defect location data.

5. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1 is characterized in that: The process of calculating overall defectivity includes: The overall defect degree of the cosmetic bottle is calculated based on the defect area, defect depth, defect location data and corresponding weight coefficients obtained by the surface defect detection module, wherein the weight coefficient is obtained based on the defect occurrence frequency in the historical data and the importance of the location of the cosmetic bottle defect, and the importance of the location of the cosmetic bottle defect is obtained based on the regional importance division of the cosmetic bottle body; If the overall defect degree of the cosmetic bottle is greater than the preset defect degree threshold, it means that the defect of the cosmetic bottle affects the sales of the cosmetic bottle, and the corresponding cosmetic bottle is marked and an early warning is issued.

6. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1, characterized in that: The process of processing the image of the cosmetic bottle to be detected using the bottle mouth feature matching algorithm includes: The image of the cosmetic bottle to be inspected is obtained and feature extraction is performed to obtain geometric shape features, structural features, and position features; the shape features are obtained and processed through Hough transform to obtain a first bottle mouth image, the structural features are obtained and processed through a contour analysis algorithm to obtain a second bottle mouth image, the position features are obtained and processed through a deep learning algorithm to obtain a third bottle mouth image; the first bottle mouth image, the second bottle mouth image, and the third bottle mouth image are obtained and the intersection processing of the image positions is performed to obtain a bottle mouth image.

7. The automatic defect detection system for cosmetic bottles based on machine vision according to claim 1, characterized in that: The process of defect detection on bottle mouth images includes: The bottle mouth image is subjected to white balance correction, histogram equalization, and Gaussian filtering or median filtering to eliminate uneven illumination and noise, and obtain a processed bottle mouth image; the processed bottle mouth image is processed to obtain the skeleton features of the cosmetic bottle mouth, including the number of breakpoints feature, the total length of the skeleton feature, the distance feature between adjacent breakpoints, the number of skeleton branches feature, the number of breakpoints in a specific area feature, the skeleton curvature feature, and the breakpoint distance feature; wherein, if a pixel point has only one non-zero pixel in the 8-neighborhood, the pixel point is considered to be a breakpoint; if the number of non-zero pixels in the 8-neighborhood of the pixel point is greater than 2, the pixel point is a branch point; the sum of the Euclidean distances between all connected pixels in the skeleton image is calculated to obtain the total length of the skeleton; if there are multiple breakpoints in the skeleton image, the average distance between adjacent breakpoints is calculated to obtain the distance feature between adjacent breakpoints; for the sealing contact area, the number of breakpoints in the area is counted to obtain the number of breakpoints in the specific area feature; the local curvature of each part of the skeleton is estimated to obtain the skeleton curvature feature; the distance between the breakpoint and the nearest branch point is calculated to obtain the breakpoint distance feature; Weighted processing is performed based on the skeleton features of the cosmetic bottle mouth to construct a sealing defect index. If the sealing threshold is exceeded, it means that there is a sealing defect in the cosmetic bottle mouth, and an early warning is issued and the cosmetic bottle is marked.

Images