Glue path detection method, electronic equipment and storage medium

Abstract

The invention provides a glue path detection method, electronic equipment and a storage medium. The glue path detection method comprises the following steps: acquiring a product image of a product subjected to glue dispensing; respectively determining a corresponding frame area and a glue path area in the product image according to the frame and the glue path of the product after glue dispensing; based on the glue path area, determining a glue path outer contour of the glue path area; correcting the frame area by using a template matching algorithm, and determining a frame outer contour of the corrected frame area; and determining the distance between the outer contour of the glue path and the outer contour of the frame, and determining a detection result of the glue path according to the distance. According to the invention, the accuracy of determining whether the product frame glue path position is abnormal can be improved.

Description

Technical Field

[0001] This application relates to the technical field of fault detection, and particularly to a glue path detection method and related equipment. Background Art

[0002] In modern manufacturing, the dispensing process is a key link to ensure the assembly quality and sealing performance of products. Therefore, the position accuracy of the dispensing glue path is of great significance to the performance, safety, etc. of products. However, the manual detection method used in related glue path detection methods is inefficient, and it is difficult to ensure the accuracy of the glue path detection results, which may lead to product function failure, shortened service life, and even potential safety hazards. Summary of the Invention

[0003] In view of the above, it is necessary to propose a glue path detection method and related equipment that can solve the problem of difficult to quickly and accurately detect the glue path.

[0004] In the first aspect of the embodiments of this application, a glue path detection method is provided, including: obtaining a product image of the product after dispensing; respectively determining a corresponding frame area and a glue path area in the product image according to the frame and the glue path of the product after dispensing; determining an outer contour of the glue path of the glue path area based on the glue path area; using a template matching algorithm to correct the frame area, and determining an outer contour of the frame of the corrected frame area; determining a distance between the outer contour of the glue path and the outer contour of the frame, and determining a detection result of the glue path according to the distance.

[0005] According to the embodiments of this application, the determining the distance between the outer contour of the glue path and the outer contour of the frame includes: determining a minimum distance from each point in the outer contour of the glue path to the outer contour of the frame, and taking the minimum distance as the distance between each point in the outer contour of the glue path and the outer contour of the frame.

[0006] According to the embodiments of this application, the determining the detection result of the glue path according to the distance includes: dividing the outer contour of the frame into multiple sub-regions, and determining a corresponding standard distance range for each sub-region; determining a target point on the outer contour of the frame corresponding to any point in the outer contour of the glue path according to the minimum distance from the point in the outer contour of the glue path to the outer contour of the frame; taking the standard distance range corresponding to the sub-region where the target point is located as the target distance range, and if the minimum distance exceeds the target distance range, determining that the detection result is abnormal.

[0007] According to the embodiments of this application, the determining method of the glue path area and the outer contour of the glue path includes: identifying, based on a semantic segmentation algorithm, the area where the glue path is located in the product image as the glue path area; and extracting, based on an edge detection algorithm, the outer contour of the glue path area as the outer contour of the glue path.

[0008] According to an embodiment of the present application, the method for determining the glue path area and the outer contour of the glue path includes: based on a semantic segmentation algorithm, identifying the area where the glue path is located in the product image as the glue path area; and extracting the outer contour of the glue path area as the outer contour of the glue path based on an edge detection algorithm.

[0009] According to an embodiment of the present application, the method for determining the glue path area and the outer contour of the glue path includes: based on a semantic segmentation algorithm, identifying the area where the glue path is located in the product image as the glue path area; and extracting the outer contour of the glue path area as the outer contour of the glue path based on an edge detection algorithm.

[0010] According to an embodiment of the present application, the correction of the border area by using the template matching algorithm includes: traversing the template images in the matching template library level by level in a preset hierarchical order, determining the matching value between the traversed template image and the border area; determining the maximum matching value from the matching values greater than a preset matching threshold, selecting the corresponding template image according to the maximum matching value, and correcting the border area.

[0011] According to an embodiment of the present application, determining the outer contour of the border of the corrected border area includes: determining the corrected border area according to the position, angle, and size of the original border in the selected template image, and the corrected border area represents the border area corresponding to the original border.

[0012] In a second aspect of the embodiments of the present application, a glue path detection device is provided. The glue path detection device includes: an acquisition module for acquiring a product image of the product after dispensing; a determination module for respectively determining a corresponding border area and a glue path area in the product image according to the border and the glue path of the product after dispensing; the determination module is further configured to determine the outer contour of the glue path area based on the glue path area; a correction module for correcting the border area by using a template matching algorithm and determining the outer contour of the border of the corrected border area; the determination module is further configured to determine the distance between the outer contour of the glue path and the outer contour of the border, and determine the detection result of the glue path according to the distance.

[0013] In a third aspect of the embodiments of the present application, an electronic device is provided, including a memory and at least one processor. At least one instruction is stored in the memory, and when the at least one instruction is executed by the at least one processor, the glue path detection method is implemented.

[0014] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the glue path detection method is implemented.

[0015] The glue path detection method provided by the embodiment of the present application collects the product image after dispensing; in the product image, according to the characteristics of the border and the glue path, the corresponding border area and glue path area are respectively determined; based on the glue path area, the outer contour of the glue path is extracted; the template matching algorithm is used to correct the border area to determine the corrected outer contour of the border, providing a basis for subsequent distance calculation; the distance between the outer contour of the glue path and the outer contour of the border is calculated, and the detection result of the glue path is judged according to the preset standard. It can be applied to the automatic detection and analysis of the glue path quality of various products, improving the efficiency and accuracy of glue path detection, not only improving the product quality and production efficiency, but also reducing the labor detection cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0017] Figure 1 It is a schematic flow chart of the glue path detection method provided by the embodiment of the present application.

[0018] Figure 2 It is a schematic structural diagram of an electronic device provided by the embodiment of the present application.

[0019] Figure 3 It is an example diagram of a product image provided by an embodiment of the present application.

[0020] Figure 4 It is an example diagram of the border segmentation result and the glue path segmentation result provided by an embodiment of the present application.

[0021] Figure 5 It is an example diagram of a template image provided by an embodiment of the present application.

[0022] Figure 6 It is a schematic flow chart of the template matching algorithm provided by an embodiment of the present application.

[0023] Figure 7 It is a schematic flow chart of the method for determining the detection result of the glue path provided by an embodiment of the present application.

[0024] Figure 8 It is an example diagram of a sub-region of the outer contour of the border provided by an embodiment of the present application.

[0025] Figure 9 It is a schematic flow chart of the glue path detection method provided by another embodiment of the present application.

[0026] Figure 10 A flowchart of a method for constructing a matching template library provided by an embodiment of the present application.

[0027] Figure 11 A schematic diagram of constructing a matching template library provided by an embodiment of the present application.

[0028] Figure 12 A flowchart of a template matching algorithm provided by another embodiment of the present application.

[0029] Figure 13 A principle block diagram of a glue path detection device provided by an embodiment of the present application. Detailed implementation manners

[0030] In order to more clearly understand the above objects, features, and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the specification of the present application herein are only for the purpose of describing an embodiment, and are not intended to limit the present application.

[0032] It should be noted that "at least one" in the present application means one or more, and "a plurality" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0033] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner. Without conflict, the following embodiments and the features in the embodiments may be combined with each other.

[0034] Figure 1 It is a flowchart of a glue path detection method provided by an embodiment of the present application. The glue path detection method is applied to an electronic device, such as Figure 2The electronic device 10 therein specifically includes the following steps. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0035] Step S201: Obtain the product image of the product after dispensing.

[0036] In one embodiment, the dispensing process is a process of coating glue or other adhesives on specific parts of the product, which is widely used in various manufacturing fields. In one example, the dispensing process can be used to fix different components of the product together and provide a sealing effect to prevent external factors such as dust and moisture from damaging the inside of the product. Some types of glue have conductive or heat-conductive properties, and these properties can be introduced into the product through the dispensing process to meet specific electrical or thermal management requirements. In addition, the dispensing process can form a buffer layer inside the product to reduce the damage caused by vibration and impact to the product. In some cases, the dispensing process can also be used for the aesthetic design of the product or as a label.

[0037] In one example, the product can include but is not limited to electronic products, automotive parts, medical devices, aerospace products, and other products. For example, electronic products can include but are not limited to internal components of electronic products such as mobile phones, tablet computers, and laptop computers, such as batteries, cameras, speakers, etc., which usually require dispensing for fixation and sealing. In addition, the outer shells, circuit boards, etc. of electronic products may also require dispensing treatment. Automotive parts can include but are not limited to many components in automobiles, such as headlights, sensors, wire harnesses, etc., which all require dispensing for fixation and sealing to ensure their stability and reliability. Many components in medical devices also require dispensing treatment to meet their specific electrical, thermal management, or sealing requirements. Aerospace products have extremely high requirements for reliability and stability, so many components require dispensing for fixation and sealing. In addition to the above types of products, there are also many other products that require dispensing treatment, such as toys, furniture, building materials, etc.

[0038] In one embodiment, the dispensing process can coat glue or other adhesives on the inner periphery of the outer frame of the product to form a glue path. If there are defects in the quality of the dispensing process, it may coat the glue or other adhesives on the outer frame of the product, causing some areas of the glue path to cover the outer frame of the product, resulting in a series of problems such as appearance defects, functional failures, decreased structural strength, difficulties in cleaning and maintenance, or safety hazards of the product.

[0039] In one embodiment, a camera device can be used to obtain a product image of the product after dispensing, and quality control of the dispensing process and defect detection of the product can be performed based on the product image. Among them, the camera device can include, but is not limited to, a Charge-Coupled Device (CCD). In another example, the product image of the product after dispensing can also be obtained by receiving user input. For example Figure 3 As shown, it is an example diagram of the product image provided by an embodiment of the application. Among them, the blue area represents the product, and the dark gray rectangle represents the glue path.

[0040] Step S202: Based on the border of the product after dispensing and the glue path, determine the corresponding border area and glue path area in the product image respectively.

[0041] In one embodiment, based on a semantic segmentation algorithm, the area where the border of the product after dispensing is located in the product image can be identified as the border area, and the area where the glue path is located in the product image can be identified as the glue path area. Among them, the semantic segmentation algorithm can assign a semantic label to each pixel in the product image, that is, perform an accurate classification and division of each area in the product image, so as to identify the border of the product, the glue area, and other possible objects or backgrounds in the product image.

[0042] In one example, when identifying the border area and the glue path area based on the semantic segmentation algorithm, the product image can be preprocessed. For example, the preprocessing includes, but is not limited to, denoising, enhancing contrast, etc., to improve the image quality and reduce the difficulty of subsequent processing. Use a pre-trained semantic segmentation model to perform semantic segmentation on the product image to obtain the semantic category corresponding to each pixel point in the product image, and output a segmentation result map with the same size as the product image. In the segmentation result map, the pixel points belonging to the border area will be marked as a specific category (for example, "border"), and the pixel points belonging to the glue path area will be marked as a specific category (for example, "glue path"). Post-process the segmentation result map. For example, the post-processing includes, but is not limited to, dilation, erosion, etc., to optimize the recognition effect of the border area and the glue path area. For example Figure 4 As shown, it is an example diagram of the border segmentation result and the glue path segmentation result provided by an embodiment of the present application.

[0043] In one example, when training the semantic segmentation model, a large amount of labeled data can be used to perform supervised training on the semantic segmentation model. The labeled data should include product images of the product after dispensing corresponding to various dispensing situations and different product border styles, as well as the known positions of the border area and the glue path area in the product image. Through supervised training, the semantic segmentation model can learn how to classify the pixel points in the image into the border area, the glue area, or other categories.

[0044] Through the above embodiments, by applying a semantic segmentation algorithm, the border area and the glue path area of the product after dispensing can be recognized in the product image. This algorithm assigns semantic labels to each pixel in the image, thereby improving the accuracy and automation of border and glue path recognition, and providing a basis for subsequent glue path detection.

[0045] Step S203: Based on the glue path area, determine the outer contour of the glue path in the glue path area.

[0046] In one embodiment, the outer contour of the glue path area can be extracted based on an edge detection algorithm as the outer contour of the glue path. In an example, based on the glue path area determined from the product image, a preset edge detection algorithm can be used to extract the outer contour of the glue path. For example, an edge detection algorithm can be selected from an image processing library such as OpenCV, including but not limited to edge detection algorithms such as Laplacian, Sobel, Scharr, or Canny. After obtaining the result of edge detection, a contour extraction algorithm can be used to extract the outer contour of the glue path. For example, the contour extraction algorithm includes but is not limited to the cv2.findContours() function in the OpenCV image processing library, which can identify the outer contour of the glue path and return the coordinate point set of the outer contour of the glue path.

[0047] Based on the above embodiments, by adopting an edge detection algorithm and a contour extraction algorithm, the outer contour of the glue path area can be determined, which not only improves the recognition accuracy of the outer contour of the glue path, but also provides strong support for subsequent image analysis and processing. In practical applications, this method can significantly improve the efficiency and accuracy of product detection and quality control.

[0048] Step S204: Use a template matching algorithm to correct the border area and determine the outer contour of the border in the corrected border area.

[0049] In one embodiment, in the dispensing process, the glue often flows along the border of the product to form a continuous glue path. However, during this process, the border of the product may be partially covered by the glue path, resulting in the fact that in the product image, the border area recognized by the above method is not completely accurate but is affected by the glue path area. This inaccuracy may interfere with subsequent image analysis and processing, especially the precise detection of whether the position of the glue path is qualified. To solve the above problems, a template matching algorithm can be used to correct the border area, thereby eliminating the influence of the glue path area on the recognition of the border area and determining the actual shape of the original border of the product before dispensing.

[0050] In one embodiment, before correcting the border area using the template matching algorithm, a matching template library corresponding to the original border of the product before dispensing can be constructed based on the scale space pyramid algorithm. The matching template library includes template images at multiple levels, each level includes multiple template images, and the template images at each level correspond to a preset angular step size, scaling ratio step size, and number of feature points. The total number of pyramid levels can be set according to actual needs. For example, 4 levels, and the present application does not make specific limitations on this.

[0051] In one example, the angular transformation range, angular step size, scaling ratio range, scaling ratio step size, and number of feature points corresponding to the 0th level in multiple levels can be set. For example, the angular transformation range corresponding to the 0th level can be (-30°, 30°), and the angular step size is 0.3°, indicating that within this angular transformation range, a template image will be created for the 0th level every 0.3° of angle. For example, the original image of the original border of the product before dispensing is adjusted at an interval of 0.3° of angle to obtain a template image at the corresponding angle.

[0052] The scaling ratio range corresponding to the 0th level can be (0.8, 1.2), and the scaling ratio step size is 0.001, indicating that within this scaling ratio range, a template image will be created for the 0th level every 0.001 of scaling ratio. In practical applications, the scaling ratio step size of the 0th level can be dynamically adjusted according to the maximum distance L from the centroid of the original border of the product to the points on the original border. For example, it is 1 / L. For example, the original image of the original border of the product before dispensing is adjusted at an interval of 0.001 of scaling ratio to obtain a template image at the corresponding scaling ratio.

[0053] The number of feature points corresponding to the 0th level can be set to 80. For example, the outer contour of the border in the original image of the original border includes 3000 contour points, and 80 contour points can be extracted from the 3000 contour points as feature points through the feature point extraction algorithm. For example, each template image in the 0th level includes 80 feature points. For example Figure 5 As shown, the template image of the 0th level includes the outer contour of the border of the original border, where the red dots represent the feature points obtained by extracting the outer contour of the border.

[0054] Based on the above settings, a total of 12000 template images can be included in the 0th level. Among the multiple levels of the scale space pyramid, the 0th level includes the largest number of template images, the angular step size and scaling ratio step size between the template images are the smallest, and the number of feature points in the template images is also the largest. The template images of each subsequent level can be constructed in a decreasing manner based on the above parameters of the 0th level. Among them, the angular transformation range and scaling ratio range remain unchanged.

[0055] In one example, based on the above parameters of layer 0, the angle transformation step size corresponding to layer 1 among multiple layers can be 0.3°×2 = 0.6°, the scaling ratio step size corresponding to layer 1 can be 0.001×2 = 0.002, the number of feature points can be set to 80 / 2 = 40, and the number of template images for layer 1 can be 3000.

[0056] The angle transformation step size corresponding to layer 2 can be 0.6°×2 = 1.2°, the scaling ratio step size corresponding to layer 2 can be 0.002×2 = 0.004, the number of feature points can be set to 40 / 2 = 20, and the number of template images for layer 2 can be 750.

[0057] The angle transformation step size corresponding to layer 3 can be 1.2°×2 = 2.4°, the scaling ratio step size corresponding to layer 3 can be 0.004×2 = 0.008, the number of feature points can be set to 20 / 2 = 10, and the number of template images for layer 2 can be 187.

[0058] According to the above settings, a total of 4 layers with more than 16,000 template images are generated, and the angle step size, scaling ratio step size, and number of feature points decrease layer by layer from layer 0 to layer 3. Among them, the gradual decrease in the number of feature points can refer to Figure 5 as shown. It can be seen that the number of feature points shown as red dots in layer 0 is the largest, and the number of red dots shown in layers 0 to 3 decreases layer by layer.

[0059] Based on the above embodiments, using the scale space pyramid algorithm to construct a multi-level matching template library can generate rich template images at different angles, scaling ratios, and numbers of feature points. The angle step size, scaling ratio step size, and number of feature points decrease layer by layer in the template images, which can ensure the matching accuracy while improving the efficiency and flexibility of template matching, and can effectively meet the complex requirements of product border correction after dispensing.

[0060] In one embodiment, referring to Figure 6 as shown, the method for correcting the border area using the template matching algorithm includes the following process.

[0061] Step S601, traverse the template images in the matching template library layer by layer in the preset hierarchical order, and determine the matching value between the traversed template image and the border area.

[0062] In one embodiment, all levels in the matching template library can be sorted according to the number of feature points, and the level with the fewest number of feature points will be placed at the starting position of the sequence. Continuing with the above example, the sorted hierarchical order can be {layer 3, layer 2, layer 1, layer 0}.

[0063] Traverse the template images in each layer one by one according to the sorted hierarchical order. During the traversal, compare the current template image with the border area, and use a preset matching algorithm to calculate the matching value between them. For example, the matching algorithm includes but is not limited to the feature point-based matching algorithm. The calculation of the matching value can take into account various factors, such as the coincidence degree of feature points, the fitting degree of edges, and the similarity of the overall shape, etc. These factors can be integrated to obtain a value that can accurately reflect the matching degree between the template image and the border area as the matching value.

[0064] For the template images in each layer, the template image with the largest matching value with the border area can be determined as the target image. Based on the angle and scaling ratio of the target image, select the template images to be traversed in the next adjacent layer. In one example, according to the angle and size ratio of the target image, an angle selection range and a size ratio selection range can be set, and traverse the template images within the angle selection range and the size ratio selection range in the next adjacent layer. For example, the angle range can be set as the target angle ±3°, and the size ratio range can be set as the target ratio ±0.1.

[0065] In one example, if the angle corresponding to the template image with the highest matching degree with the border area determined in the 3rd layer is 21.3°, and the corresponding size ratio is 0.9111. When traversing the template images in the 2nd layer, select the template images within the range of 21.3° ± 3° in angle and 0.9111 ± 0.1 in size ratio as the template images to be traversed.

[0066] Repeat the above steps, traverse the template images in each layer layer by layer according to the sorted hierarchical order until all layers are traversed, and obtain the matching values corresponding to each traversed template image.

[0067] Step S602, determine the maximum matching value from the matching values greater than the preset matching threshold, and select the corresponding matching template image according to the maximum matching value to correct the border area.

[0068] In one embodiment, among the matching values corresponding to each traversed template image, there may be multiple matching values that meet the required matching accuracy. For example, there may be multiple matching values greater than the preset matching threshold. The matching image corresponding to the maximum matching value can be selected from them, and the border area can be corrected based on this matching image.

[0069] Based on the above embodiments, by combining hierarchical traversal and feature-based screening, the template image that best matches the border area can be found more efficiently. At the same time, by restricting the selection range of candidate template images in the next layer according to the angle and size ratio of the target image, the calculation amount can be significantly reduced and the matching speed can be improved.

[0070] In one embodiment, the outer contour of the corrected border region can be determined by selecting the angle, size, position, etc. of the outer contour of the matching image corresponding to the maximum matching value.

[0071] Based on the above embodiment, by adopting the scale space pyramid algorithm to construct a multi-level matching template library and combining the template matching algorithm to correct the border region, the influence of the glue path region on the border region recognition in the dispensing process can be efficiently and accurately eliminated. By gradually decreasing the angle step size, scaling ratio step size, and the number of feature points, while ensuring the matching accuracy, the efficiency and flexibility of template matching are significantly improved. Finally, based on the matching image corresponding to the maximum matching value, the outer contour of the corrected border region can be accurately determined, providing a reliable basis for subsequent image analysis and processing, and effectively meeting the complex requirements of product border correction after dispensing.

[0072] Step S205: Determine the distance between the outer contour of the glue path and the outer contour of the border, and determine the detection result of the glue path according to the distance.

[0073] In one embodiment, it is possible to determine whether the glue path is abnormal according to the distance between the outer contour of the glue path and the outer contour of the border. Among them, the minimum distance from each point in the outer contour of the glue path to the outer contour of the border can be determined, and the minimum distance is used as the distance between each point in the outer contour of the glue path and the outer contour of the border.

[0074] In one embodiment, since the product border is not necessarily a regular figure, for example, not necessarily a rectangle, therefore, for different regions of the product border, the judgment criteria for whether the position of the glue path is abnormal may be different. In one example, refer to Figure 7 As shown, the method for determining the detection result of the glue path according to the distance includes the following process.

[0075] Step S701: Divide the outer contour of the border into multiple sub-regions and determine the standard distance range corresponding to each sub-region.

[0076] In one embodiment, since the product border may have an irregular shape and the requirements for the position of the glue path in different regions may be different. Therefore, in order to more accurately judge whether the position of the glue path is abnormal, the outer contour of the border can be divided into multiple sub-regions, and a reasonable standard distance range can be set for each sub-region.

[0077] In one example, an image processing software can be used to divide the outer contour of the border into multiple sub-regions. For example, the sub-regions can be divided according to the shape, feature points, or design requirements of the border. For example Figure 8 As shown, it is an example diagram of the sub-regions of the outer contour of the border provided by an embodiment of the present application.

[0078] In one example, according to actual requirements such as product design and manufacturing requirements, a standard distance range is set for each sub-region, and this range represents the allowable distance from the points on the outer contour of the glue path to the outer contour of the border within the sub-region.

[0079] Step S702: Determine the target point on the outer contour of the border corresponding to any point in the outer contour of the glue path according to the minimum distance from any point in the outer contour of the glue path to the outer contour of the border.

[0080] In one embodiment, in order to determine whether the position of the glue path is abnormal, the minimum distance from the points on the outer contour of the glue path to the outer contour of the border can be calculated, and the point on the outer contour of the border corresponding to this minimum distance can be found, such as the target point.

[0081] In one example, an image processing algorithm can be used to calculate the minimum distance from each point on the outer contour of the glue path to the outer contour of the border. Based on this minimum distance, the corresponding point on the outer contour of the border is found as the target point. For example, the first coordinates of each point on the outer contour of the glue path and the second coordinates of each point on the outer contour of the border can be determined in the product image coordinate system. Based on the Euclidean distance between the first coordinates and the second coordinates, the minimum distance from each point on the outer contour of the glue path to each point on the outer contour of the border is determined, and the target point corresponding to the minimum distance is determined.

[0082] Step S703: Use the standard distance range corresponding to the sub-region where the target point is located as the target distance range. If the minimum distance exceeds the target distance range, determine that the detection result is abnormal.

[0083] In one embodiment, it is possible to determine whether the position of the glue path is abnormal by comparing the calculated minimum distance with the standard distance range of the sub-region where the target point is located. In one example, the sub-region where the target point is located is determined, and the corresponding standard distance range of this sub-region is found. The calculated minimum distance is compared with this standard distance range. If the minimum distance exceeds the standard distance range, it is considered that the position of the glue path is abnormal and the detection result is abnormal. If the minimum distance is within the standard distance range, it is considered that the position of the glue path is normal and the detection result is normal.

[0084] In another example, the distance between each point on the outer contour of the glue path and each point on the outer contour of the border can be positive or negative. All positive distances indicate that the points on the outer contour of the glue path are completely inside the outer contour of the border, meaning that no glue overflow is found in the current detection, that is, there is no glue overflow phenomenon. The existence of negative distances indicates that some or all of the points on the outer contour of the glue path are outside the outer contour of the border, meaning that there is a glue overflow phenomenon, that is, the glue path exceeds the predetermined product border range. Positive and negative values do not directly determine whether the position of the glue path is in an abnormal state, because in different production areas or scenarios, the tolerance for glue overflow (i.e., the maximum allowed negative value) may be different. To determine whether the position of the glue path is abnormal, the calculated shortest distance from each point to the border can be compared with the standard distance range of the corresponding sub-region. If the calculated distance meets the standard distance range of the region, the position of the glue path is normal. If the calculated distance exceeds the standard distance range of the region, the position of the glue path is abnormal.

[0085] Based on the above embodiments, by dividing the sub-regions of the outer contour of the border, determining the standard distance range, calculating the minimum distance and comparing, etc., an accurate judgment on whether the position of the glue path of the product border is abnormal is achieved, which helps to improve product quality and reduce production costs.

[0086] The glue path detection method provided by the embodiments of the present application collects the product image after dispensing; in the product image, according to the characteristics of the border and the glue path, the corresponding border region and glue path region are determined respectively; based on the glue path region, the outer contour of the glue path is extracted; the template matching algorithm is used to correct the border region to determine the corrected outer contour of the border, providing a basis for subsequent distance calculation; the distance between the outer contour of the glue path and the outer contour of the border is calculated, and the detection result of the glue path is judged according to the preset standard. It can be applied to the automatic detection and analysis of the glue path quality of various products, improving the efficiency and accuracy of glue path detection, not only can improve product quality and production efficiency, but also can reduce the labor detection cost.

[0087] Reference Figure 9 As shown, it is a flowchart of the glue path detection method provided by another embodiment of the present application. The glue path detection method may include: obtaining the product image after dispensing through a CCD, performing semantic segmentation on the product image in reverse, and obtaining the semantic segmentation result of the product border and the semantic segmentation result of the glue path. Extracting the outer contour of the border based on the semantic segmentation result of the product border, using the template matching algorithm to correct the outer contour of the border, and positioning to obtain the corrected outer contour of the border. Extracting the outer contour of the glue path based on the semantic segmentation result of the glue path, and determining whether the glue path is abnormal according to the margin between the outer contour of the glue path and the outer contour of the border.

[0088] Based on the above embodiments, an advanced CCD technology is adopted to obtain the product image after dispensing, and semantic segmentation technology is used to finely process the product image to accurately distinguish the regions of the product border and the glue path. Subsequently, based on the semantic segmentation result of the border, the outer contour of the border is extracted, and a template matching algorithm is used for correction to ensure the accuracy of the border position. At the same time, according to the semantic segmentation result of the glue path, the outer contour of the glue path is extracted, and by calculating the margin between the outer contour of the glue path and the outer contour of the border, it is accurately determined whether there is an abnormality in the glue path. This not only improves the accuracy and efficiency of detection but also significantly reduces human errors, providing a strong guarantee for product quality control.

[0089] Refer to Figure 10 As shown, it is a flowchart of a method for constructing a matching template library provided by an embodiment of the present application. Among them, the template image corresponding to the original border of the product before dispensing can be obtained, and a template mask image is made to focus on the information of the range of the outer contour of the border in the template image. The border contour in the template image is extracted, and the contour is filtered according to the template mask image to retain the area of interest. The candidate feature points of the filtered contour are extracted, and the pyramid transformation technology is used to process these feature points to determine the number of feature points at each layer. The feature points are transformed according to the preset angle and scaling range, such as rotation and scaling transformation, and the transformed points and their related information are saved, thereby generating a template. For example Figure 11 As shown, it is a schematic diagram of a method for constructing a matching template library provided by an embodiment of the present application.

[0090] Based on the above embodiments, by obtaining the template image corresponding to the original border of the product before dispensing and making a template mask image to focus on the information of the outer contour of the border. Subsequently, the border contour in the template image is extracted, and the contour is filtered using the template mask image to only retain the area of interest. Then, the candidate feature points are extracted from the filtered contour, and the pyramid transformation technology is used to process these feature points to determine the number of feature points at each level. Finally, the feature points are subjected to rotation and scaling transformation according to the preset angle and scaling range, and the transformed points and their related information are saved, thus successfully constructing a template. This method can efficiently generate an accurately matching template, providing strong support for subsequent image matching and product quality detection.

[0091] Refer to Figure 12As shown in the figure, it is a flowchart of a method for correcting a border area using a template matching algorithm provided by another embodiment of the present application. Among them, semantic segmentation is performed on the product image after dispensing to obtain the outer contour of the product and the glue path. The contour of the segmented product image is extracted, especially the outer contour of the border. A 4-layer pyramid transformation is used, and the size of each layer of the image is 1 / 2 of the previous layer. The size of the image of the topmost pyramid is 1 / 8 of the size of the original image. In the topmost pyramid, all top-layer template images are moved to perform matching on the entire image, and the best matching position is recorded. According to the similarity of the matching of the template image compared with a preset threshold, templates with low similarity are filtered out to reduce the subsequent calculation amount. In the next layer of the pyramid, according to the position of the matching template in the previous layer, it is transformed into the position of the current pyramid layer, and the area around this position is selected for local matching. According to the threshold, templates that do not meet the conditions are continuously filtered, and this process is repeated until the bottommost pyramid (i.e., the original image). In the current layer, if there are multiple templates that meet the threshold conditions, non-maximum suppression processing is performed. For example, among the overlapping templates, the template with the largest similarity is selected as the finally matched template, and the position, angle, and ratio of the outer contour of the border in this template are the outer contour of the corrected border area.

[0092] Based on the above embodiments, semantic segmentation is performed on the product image after dispensing through a deep learning model to accurately extract the outer contour of the product and the glue path information. Subsequently, using the pyramid transformation technology, the image size is gradually reduced layer by layer to improve the matching efficiency. In the topmost pyramid, the best matching position is determined through global matching, and templates with low similarity are filtered according to the preset threshold. As the pyramid level decreases, the matching result of the previous layer is used to guide the local matching of the current layer, further streamlining the calculation amount. Finally, in the bottommost pyramid (i.e., the original image layer), the most matching template is selected through non-maximum suppression processing. After the position, angle, and ratio of the outer contour of its border are corrected, they are the required outer contour of the product border. This not only greatly improves the matching accuracy but also effectively reduces the computational complexity.

[0093] Figure 13 It is a structural diagram of a glue path detection device provided by an embodiment of the present application.

[0094] In some embodiments, the glue path detection device 8 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the glue path detection device 8 can be stored in the memory of the electronic device and executed by at least one processor to execute (see the details in Figure 1 the description) the function of glue path detection.

[0095] In this embodiment, the glue path detection device 8 can be divided into multiple functional modules according to its executed functions. The functional modules may include: an acquisition module 81, a determination module 82, and a calibration module 83. A module referred to in this application means a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and is stored in a memory. In this embodiment, for the implementation manners of the functions of each module in the glue path detection device 8, reference may be made to the limitations on the glue path detection method above, and details will not be repeated here.

[0096] The acquisition module 81 is configured to acquire a product image of the product after dispensing glue.

[0097] The determination module 82 is configured to respectively determine a corresponding border region and a glue path region in the product image according to the border of the product after dispensing glue and the glue path.

[0098] The determination module 82 is further configured to determine an outer contour of the glue path of the glue path region based on the glue path region.

[0099] The calibration module 83 is configured to calibrate the border region by using a template matching algorithm, and determine an outer contour of the border of the calibrated border region.

[0100] The determination module 82 is further configured to determine a distance between the outer contour of the glue path and the outer contour of the border, and determine a detection result of the glue path according to the distance.

[0101] Figure 2 It is a schematic structural diagram of an electronic device provided by an embodiment of this application. The electronic device 10 can be a mobile phone, a tablet computer, a smart wearable device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, a netbook, an energy storage device, a power distribution device, a vehicle-mounted device, a self-mobile device, etc. The electronic device can also be a production and manufacturing device, such as a product detection device, etc. The embodiment of this application does not impose any limitation on the specific type of the electronic device.

[0102] As Figure 2 shown, the electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is respectively coupled to the communication module 101, the memory 102, and the I / O interface 104 through the bus 105.

[0103] The communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more of the solutions for wired communication such as universal serial bus (USB), Controller Area Network (CAN), etc. The wireless communication module may provide one or more of the solutions for wireless communication such as wireless fidelity (Wi-Fi), bluetooth (BT), mobile communication network, frequency modulation (FM), near field communication (NFC), infrared (IR), etc.

[0104] The memory 102 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 103, can be used to store the operating system or executable programs of other running programs (such as machine instructions), and can also be used to store user and application data, etc. The random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0105] The non-volatile memory can also store executable programs and store user and application data, etc., and can be pre-loaded into the random access memory for the processor 103 to directly read and write. The non-volatile memory may include disk storage devices, flash memory.

[0106] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions, and when the plurality of instructions are executed by the processor 103, a glue path detection method executable on the electronic device 10 can be implemented.

[0107] In other embodiments, the electronic device 10 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10.

[0108] The processor 103 may include one or more processing units. For example, the processor 103 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0109] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute the computer program stored in the memory 102 to implement the above-mentioned glue path detection method.

[0110] The I / O interface 104 is used to provide a channel for user input or output. For example, the I / O interface 104 can be used to connect various input and output devices, such as a mouse, a keyboard, a touch device, a display screen, etc., so that the user can enter information or visualize the information.

[0111] The bus 105 is at least used to provide a communication channel between the communication module 101, the memory 102, the processor 103, and the I / O interface 104 in the electronic device 10.

[0112] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and the method implemented when the program instructions are executed may refer to the methods in the above various embodiments of the present application.

[0114] Among them, the computer-readable storage medium may be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc.

[0115] In some embodiments, the computer-readable storage medium may include a storage program area and a storage data area. Among them, the storage program area may store an operating system, application programs required for at least one function, etc.; the storage data area may store data created according to the use of the electronic device, etc.

[0116] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0117] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be determined to exceed the scope of this application.

[0118] In the embodiments provided in this application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0119] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] The foregoing embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A glue path detection method, characterized in that: The method comprises: Acquire a product image of the product after dispensing; According to the frame and glue path of the product after glue dispensing, respectively determine the corresponding frame area and glue path area in the product image; Based on the glue path area, determining a glue path outer contour of the glue path area; Correcting the frame area using a template matching algorithm, and determining the outer frame contour of the corrected frame area; The distance between the outer contour of the glue path and the outer contour of the frame is determined, and the detection result of the glue path is determined according to the distance.

2. The glue path detection method according to claim 1, characterized in that: Determining the distance between the outer contour of the glue path and the outer contour of the frame includes: The minimum distance from each point in the outer contour of the glue path to the outer contour of the frame is determined, and the minimum distance is used as the distance between each point in the outer contour of the glue path and the outer contour of the frame.

3. The glue path detection method according to claim 1, characterized in that: The step of determining the detection result of the glue path according to the distance includes: Dividing the outer contour of the frame into a plurality of sub-areas, and determining a standard distance range corresponding to each sub-area; Determine a target point on the outer contour of the frame corresponding to any point in the outer contour of the glue path according to the minimum distance from the outer contour of the frame to the outer contour of the frame; The standard distance range corresponding to the sub-region where the target point is located is used as the target distance range. If the minimum distance exceeds the target distance range, it is determined that the detection result is abnormal.

4. The glue path detection method according to claim 1, characterized in that: The method for determining the glue path area and the outer contour of the glue path includes: Based on a semantic segmentation algorithm, identifying the area where the glue path is located in the product image as the glue path area; and The outer contour of the glue path area is extracted based on an edge detection algorithm as the outer contour of the glue path.

5. The glue path detection method according to claim 1, characterized in that: The method for determining the border area includes: Based on a semantic segmentation algorithm, an area where the border of the glued product is located is identified in the product image as the border area.

6. The glue path detection method according to claim 1, characterized in that: The method further comprises: Based on the scale space pyramid algorithm, a matching template library corresponding to the original border of the product before dispensing is constructed, and the matching template library includes multiple levels of template images, each level includes multiple template images, and the template images of each level correspond to preset angle steps, scaling steps and number of feature points.

7. The method for detecting a glue path according to claim 6, characterized in that: The correcting the frame area by using a template matching algorithm comprises: Traversing the template images in the matching template library layer by layer according to a preset hierarchical order, and determining a matching value between the traversed template images and the border area; A maximum matching value is determined from the matching values ​​that are greater than a preset matching threshold, a corresponding template image is selected according to the maximum matching value, and the frame area is corrected.

8. The method for detecting a glue path according to claim 7, characterized in that: Determining the outer contour of the frame of the corrected frame area includes: The corrected frame area is determined according to the position, angle and size of the original frame in the selected template image, and the corrected frame area represents the frame area corresponding to the original frame.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory, and the processor is used to implement the glue path detection method as described in any one of claims 1 to 8 when executing a computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the glue path detection method as described in any one of claims 1 to 8 is implemented.

Images