Module lens dirt detection and glue line detection method

Abstract

The present application provides a method for detecting dirt and glue lines on a module lens. The method includes: classifying the input image to obtain a classified image, which includes a glue line category image and a dirt category image; then, locating the detection area of ​​the classified image. If it is a regular rectangular area, the template set of the area to be detected that introduces a retrieval area threshold is applied for positioning; if it is an irregular area, the minimum circumscribed rectangle is first framed and then a pre-trained deep learning segmentation model is used for deep segmentation to perform pixel-level positioning; finally, dirt detection is performed on the dirt detection area, and the dirt detection results are quantified; glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified. This solution solves the problem that the current detection algorithm is difficult to meet the detection needs of tiny dirt and key glue lines on the module lens.

Description

Technical Field

[0001] The present application relates to computer vision and deep learning technologies, and specifically to a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the continuous development of module lens technology, high-resolution imaging requirements have put forward higher standards for module defect detection during the production process, especially the inspection of tiny dirt that affects motor operation and critical glue lines that ensure sealing.

[0003] When faced with increasingly large amounts of image data, the processing speed and recognition accuracy of current detection algorithms are unable to adapt to the fast pace of modern efficient production lines. Summary of the Invention

[0004] The main purpose of this application is to provide a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium and an electronic device, so as to at least solve the problem that the current detection algorithm is difficult to meet the detection requirements of minor dirt and key glue lines on the module lens.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a module lens dirt detection and glue line detection method is provided, including: an image classification step: classifying the input image to obtain a classified image, and the classified image includes a glue line category image and a dirt category image; a detection area positioning step: positioning the detection area of ​​the classified image, if it is a regular rectangular area, applying a template set of the area to be detected that introduces a retrieval area threshold for positioning; if it is an irregular area, first frame the minimum circumscribed rectangle and then use a pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning; a dirt detection step: performing dirt detection on the dirt detection area, and quantifying the dirt detection results; a glue line detection step: performing glue line detection on the glue line detection area, and quantifying glue lines of different shapes.

[0006] Optionally, if it is a regular rectangular area, the template set of the area to be detected that introduces the retrieval area threshold is used for positioning, including: setting the template set and its corresponding position coordinate set , for each position The upper left corner coordinates and lower right corner coordinates of the rectangular area are defined; according to the position coordinate set , limiting the global search to a local area , and its position coordinate set after position adjustment , where the adjustment rule is ; Only in the reduced area Execute on and The pixel-level convolution calculation between Calculate the convolution score and get each The highest score in ; Through the conversion function , map the coordinates of the highest scoring point back to the original image coordinate system to obtain the final accurate matching position ,in, , , The coordinates of the highest-scoring point are mapped back to the upper-left corner coordinates and the lower-right corner coordinates of the original image coordinate system.

[0007] Optionally, if it is an irregular area, first perform a minimum bounding rectangle frame and then use a pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning, including: As the input of the deep learning segmentation model, the updated region set is obtained after processing by the deep learning segmentation model , each Described as a set of coordinate points , wherein the method for determining the minimum circumscribed rectangle is the same as the method for determining the regular rectangular area.

[0008] Optionally, dirt detection is performed on the dirt detection area, and the dirt detection results are quantified, including: horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas; performing deep identification and fine classification based on the set of dirt detection areas to obtain accurate identification and classification of suspected dirty areas; and performing fine post-processing and indicator quantification on the suspected dirty areas.

[0009] Optionally, horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a dirt detection area set includes: and irregular rectangular dirt detection area set Merge to form a preliminary comprehensive regional set ;use Function, according to the size of the horizontal coordinate x, the preliminary comprehensive area set is sorted in ascending order, ensuring that the order of each area is consistent with its horizontal position in the original image, and obtaining the sorted set ;Through horizontal splicing operation Function, the sorted area set All the areas in the horizontal direction are spliced ​​together to generate a unified and positionally ordered set of the dirt detection areas. .

[0010] Optionally, deep recognition and refined classification are performed based on the set of dirt detection areas to obtain accurate recognition and classification of suspected dirt areas, including: using a pre-trained recognition neural network function , the dirt detection area set Perform in-depth analysis and output a set of suspected dirty areas , where each element Zn contains the dirt type identifier and the coordinate information of the dirt area, .

[0011] Optionally, the suspected dirty area is subjected to refined post-processing and index quantification, including: for the suspected dirty area set Identification according to the type of dirt , through intelligent functions Dynamically calculate adaptive thresholds Based on the calculated adaptive threshold , using the binarization function The suspected dirty area is collected Each area in the image is segmented to generate a binary image, and the final output is a set of precise dirty areas. , where each element Corresponding to The Zn region in which .

[0012] Optionally, glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, including: horizontally splicing the regular rectangular glue line detection area and the irregular rectangular glue line detection area obtained by depth segmentation to obtain a set of glue line detection areas; using an advanced segmentation network to perform image processing on the set of glue line detection areas, outputting a mask of the glue line area, and the segmentation network automatically identifies and highlights all glue line areas.

[0013] Optionally, an advanced segmentation network is used to perform image processing on the set of glue line detection areas, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for long straight glue lines, a specific area selection and preprocessing step, a minimum circumscribed rectangle and contour acquisition step, and a glue width quantization step are successively performed to achieve quantification of the long straight glue lines, wherein the mask of the glue line area is applied in the process of performing the specific area selection, and the preprocessing includes thresholding processing and closing operation processing.

[0014] Optionally, an advanced segmentation network is used to perform image processing on the set of glue line detection areas, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for circular glue lines, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein the mask of the glue line area is applied in the process of selecting the specific area, and the preprocessing includes thresholding processing and closing operation processing.

[0015] According to another aspect of the present application, a module lens dirt detection and glue line detection device is provided, including: a classification unit, used to execute the image classification step: classify the input image to obtain a classified image, the classified image including a glue line category image and a dirt category image; an area positioning unit, used to execute the detection area positioning step: locate the detection area of ​​the classified image, if it is a regular rectangular area, apply the template set of the area to be detected that introduces the retrieval area threshold for positioning, if it is an irregular area, first frame the minimum circumscribed rectangle and then use the pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning; a dirt detection unit, used to execute the dirt detection step: perform dirt detection on the dirt detection area, and quantify the dirt detection result; a glue line detection unit, used to execute the glue line detection step: perform glue line detection on the glue line detection area, and quantify glue lines of different shapes.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the module lens dirt detection and glue line detection methods.

[0017] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the module lens dirt detection and glue line detection methods.

[0018] By applying the technical solution of the present application, first, the input image is classified and processed to obtain a classified image, which includes a glue line category image and a dirt category image; then, the detection area of ​​the classified image is located; if it is a regular rectangular area, a template set of the area to be detected that introduces a retrieval area threshold is used for positioning; if it is an irregular area, the minimum circumscribed rectangle is first framed and then a pre-trained deep learning segmentation model is used for deep segmentation for pixel-level positioning; finally, dirt detection is performed on the dirt detection area, and the dirt detection results are quantified; glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, thereby solving the problem that the current detection algorithm is difficult to meet the detection requirements of tiny dirt and key glue lines on the module lens. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0020] Figure 1 A schematic diagram of a flow chart of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a dirty image template for a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of a regular rectangular area template for a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram showing the regular rectangular area matching results of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram showing the irregular area positioning results of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of horizontal stitching results of regular areas and irregular areas of a module lens dirt detection and glue line detection method provided in an embodiment of the present application is shown;

[0026] Figure 7 A schematic diagram showing preliminary classification and positioning results of a dirt detection area of ​​a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0027] Figure 8 A schematic diagram showing precise positioning and quantification results of specific types of dirt in a module lens dirt detection and glue line detection method provided in an embodiment of the present application is shown;

[0028] Figure 9 A schematic diagram of a glue line detection area of ​​a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0029] Figure 10 The figure shows a long straight glue detection area and a quantitative result diagram of a module lens dirt detection and glue line detection method provided in an embodiment of the present application;

[0030] Figure 11 The figure shows a circular glue detection area and a quantitative result diagram of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application;

[0031] Figure 12 The figure shows an overall flow chart of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application;

[0032] Figure 13 A structural block diagram of a module lens dirt detection and glue line detection device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] As introduced in the background technology, the processing speed and recognition accuracy of the detection algorithms in the existing technology are difficult to adapt to the fast pace of modern efficient production lines when faced with increasingly large image data. In order to solve the problem that the current detection algorithms are difficult to meet the detection needs of tiny dirt and key glue lines on module lenses, the embodiments of the present application provide a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium and an electronic device.

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0038] In this embodiment, a module lens dirt detection and glue line detection method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 1 FIG. 1 is a flow chart of a method for detecting module lens dirt and glue lines according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0040] Step S101, image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line image and a dirt image;

[0041] Specifically, during the production of module lenses, images of the lenses on the production line are captured in real time or periodically. These images contain various types of defects, such as glue line distribution problems or residual dirt. These images are fed into the subsequent classification process as input. A pre-trained deep learning model (such as a convolutional neural network (CNN)) analyzes the input images. By learning from a large amount of labeled image data, the deep learning model automatically extracts image features and classifies them based on these features. The deep learning model processes each input image separately, calculating its similarity to pre-set glue line and dirt image features. If the input image features are more similar to glue line image features, the image is classified as a glue line image; conversely, if the features are similar to dirt image features, the image is classified as a dirt image. This process typically involves calculating the distance or similarity score between the two images, such as using methods such as Euclidean distance, cosine similarity, or the output probability of a convolutional neural network. The final output is two categories of images: glue line images and dirt images. These two types of images have been preliminarily screened to contain only specific types of defects, facilitating more targeted processing and analysis in subsequent steps. In this embodiment, the input image is compared with a preset dirty image template by convolution operation. The dirty image template is shown in the figure below. Figure 2 As shown, if the calculation result exceeds a preset threshold (eg, 0.8), the image is determined to be a dirty image; if it does not reach this threshold, it is considered to be a glue line image.

[0042] The image classification step is the starting point of the entire inspection process. It allows subsequent processing steps to be divided into two branches: one for glue line detection and the other for contamination detection. This ensures the processing algorithm's targetedness and efficiency, avoiding redundant analysis of all content in the image. This effectively reduces computing resource consumption and speeds up the overall inspection process. Accurate classification allows subsequent detection and quantification steps to be more focused and efficient, improving detection accuracy and reliability.

[0043] Step S102, detection area positioning step: the detection area of ​​the above-mentioned classification image is positioned. If it is a regular rectangular area, the detection area template set to be detected is introduced with the search area threshold for positioning. If it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used to perform deep segmentation for pixel-level positioning;

[0044] Whether the area to be inspected is a regular rectangular area or an irregular area has been determined before the dirt detection and glue line detection. Specifically, it is determined based on the characteristics of the module lens image to be processed (in fact, it is determined based on the structural composition of the module lens itself, which is related to the model). For details, see Figure 2If the area to be detected is the area framed by the yellow frame, then the area to be detected is an irregular area. If the area to be detected is the area framed by the blue frame, then the area to be detected is a regular rectangular area.

[0045] Specifically, for regular rectangular region localization, if the region to be detected in the classified image is a regular rectangle, the system first utilizes a preset set of region templates, which contain rectangular templates of varying sizes and positions, along with their corresponding location information. When searching for these regions, a retrieval region threshold is introduced to reduce the search range and improve localization speed. Based on the template set, location information, and the retrieval region threshold, the system quickly searches for and locates regular rectangular regions within the image.

[0046] For locating irregular areas, since irregularly shaped detection areas do not adhere to fixed templates and positional information, a different approach is required. First, traditional algorithms (such as edge detection and contour extraction) are used to preliminarily define the minimum bounding rectangle of the irregular area. This step provides a preliminary positioning reference for the deep learning model. The preliminarily defined area is then fed into a pretrained deep learning segmentation model, which performs pixel-by-pixel segmentation, separating the irregular area from the background and achieving precise pixel-by-pixel positioning.

[0047] Through these steps, accurate positioning is achieved for both regular rectangular and irregular areas, providing precise target areas for subsequent specific area detection (dirt detection and glue line detection). This positioning method, which combines deep learning with traditional algorithms, not only improves detection speed but also ensures accuracy.

[0048] Step S103, dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection result;

[0049] Specifically, after completing the accurate classification of the module lens image and positioning of the detection area, step S103 focuses on in-depth analysis and quantification of the located dirty areas. In this step, all identified dirt detection areas, whether regular rectangular areas or complex irregular areas, are first comprehensively processed to form a unified set of dirt detection areas. Subsequently, deep learning technology is used to perform detailed analysis of these areas to identify various types of dirt. Immediately afterwards, the refined post-processing stage is entered, and the segmentation threshold is dynamically adjusted through an intelligent algorithm to perform more accurate segmentation of the identified dirty areas. This process is carried out at the pixel level, greatly improving the accuracy and reliability of dirt detection. Finally, a detailed quantitative analysis is performed on each dirty area, including but not limited to measuring its area, evaluating its brightness, and classifying its type. These quantitative results can provide a scientific basis for quality control on the production line, help to promptly discover and solve potential dirt problems, and ensure that each module lens can meet high-standard production requirements.

[0050] Step S104, glue line detection step: glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified.

[0051] Specifically, the glue line detection step, after image classification and contamination detection, utilizes advanced image analysis technology to precisely detect and quantify glue lines in the specific glue line detection area. This step automatically identifies and analyzes features such as the shape, width, and length of the glue line. Specific quantification algorithms are used to perform detailed evaluations of glue lines of different shapes (such as long straight lines and circular lines), including detection of glue breaks and insufficient glue, thereby ensuring glue line quality and the sealing performance of the module lens. This glue line detection step not only allows for rapid location of glue line areas but also accurately quantifies their status, providing critical data support for quality control during production.

[0052] In a specific embodiment of the present application, if it is a regular rectangular area, the template set of the area to be detected that introduces the search area threshold is used for positioning, including: setting the template set and its corresponding position coordinate set , for each position The upper left corner coordinates and lower right corner coordinates of the rectangular area are defined, where: is the horizontal coordinate of the upper left corner of the rectangular area, is the ordinate of the upper left corner of the rectangular area, is the horizontal coordinate of the lower right corner of the rectangular area, is the vertical coordinate of the lower right corner of the rectangular area; according to the above position coordinate set , limiting the global search to a local area , and its position coordinate set after position adjustment , where the adjustment rule is ,in, It is the adjusted detection area coordinate set, which represents the coordinate information after the boundary is expanded based on the retrieval area threshold on the basis of the original detection area. The default threshold for the search area is 20 pixels. Indicates that the left boundary of the original detection area is extended to the left pixels, Indicates that the upper boundary of the original detection area is extended upward pixels, Indicates that the right boundary of the original detection area is extended to the right pixels, Indicates that the lower boundary of the original detection area is extended downward pixels; only in the reduced area Execute on and The pixel-level convolution calculation between Calculate the convolution score, where x represents the horizontal coordinate position of any pixel point in the detection area, and y represents the vertical coordinate position of any pixel point in the detection area. Indicates that it is in the detection area The matching score at position (x,y) in , Is the result of the convolution calculation, used to evaluate the detection template and detection area And get each The highest score in ,in, is the set of detection areas The nth local detection area in , which is considered to be the most likely to contain target features such as dirt or glue lines, and Respectively represent the horizontal and vertical coordinates of the position with the highest convolution score in the detection area; through the conversion function , map the coordinates of the highest scoring point back to the original image coordinate system to obtain the final accurate matching position ,in, , , The coordinates of the highest scoring point above are converted through the conversion function Map back to the upper left corner coordinates and lower right corner coordinates of the original image coordinate system.

[0053] Specifically, the traditional matching algorithm is used to instantly locate the regular rectangular area to improve the processing speed. The schematic diagram of the regular rectangular area template is as follows: Figure 3As shown. First, initialize the template set and position coordinates, and preset a set of template sets , which contains regular rectangular templates of various sizes and shapes, as well as the position coordinate sets corresponding to these templates Specifically, Is a template collection that contains multiple preset templates ,Each template is an image region with a specific shape and size, which is used to compare the corresponding part of the input image.,Each template has its specific geometric definition, which can be rectangular, circular or other shapes, depending on the features to be identified, Is the position coordinate set corresponding to these templates, including the template set The expected position information of each template in the image, for each template, Contains the horizontal and vertical coordinates of the upper left corner and lower right corner of the template on the image, which is used to guide the matching position of the template on the input image. In short, A collection is a set of standard templates used to compare dirt or glue line features. The coordinate information of the expected location of these templates in the image is used to guide the algorithm to quickly and accurately find and identify target features in the image. The position coordinates of each template The coordinates of the upper left corner and lower right corner of the rectangle are defined, which provides a reference standard for subsequent area positioning. It is the position coordinate of the nth template within its target detection area, which is used to guide the matching of the template with the corresponding area in the input image.

[0054] In order to reduce unnecessary calculations and improve positioning speed, a retrieval area threshold is introduced to narrow the global retrieval range to a local area based on the template position information. The coordinates of these local areas are adjusted. The adjustment rule is to expand left, right, up and down based on the coordinates of the upper left corner and lower right corner of the template. pixels (the default setting is 20 pixels), forming a new position coordinate set This design ensures the targeted search range, avoids unnecessary scanning of the entire image, and saves computing resources. Represents a set of multiple local detection areas after preliminary positioning, including multiple detection areas ,Each detection region is a local area determined from the original input ,image for further analysis and detection of features such as dirt or glue ,lines. It is a local area A set of position adjustments, where the position information of each detection area is adjusted to more accurately locate the feature.

[0055] Only in the reduced area Execute on and Pixel-level convolution calculation between A template set representing the normal state, which contains multiple template images , It is a set of local detection areas, including multiple local areas after preliminary positioning ,Will The template in Perform pixel-level convolution calculation on each detection area in the image to determine the matching degree between the template and the detection area. The matching score of each local area is obtained through convolution operation. , the higher the score, the higher the similarity between the local area and the template. Highest scoring point in the area , the schematic diagram of the regular rectangular area matching result is as follows Figure 4 As shown, that is, the best matching area, through the specific formula Sure.

[0056] After obtaining the highest score point, through the conversion function Map these coordinate points from the local area coordinate system back to the original image coordinate system to obtain the final accurate matching position of each template in the original image This process is done through The function is executed according to the local area coordinate adjustment algorithm rules, ensuring that the positioning result can accurately correspond to the specific position on the original image.

[0057] In the specific embodiment of the present application, if it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used to perform deep segmentation for pixel-level positioning, including: the determined minimum bounding rectangle area set As the input of the above deep learning segmentation model, the updated region set is obtained after processing by the above deep learning segmentation model , each Described as a set of coordinate points , wherein the method for determining the above-mentioned minimum circumscribed rectangle is the same as the method for determining the above-mentioned regular rectangular area.

[0058] Specifically, for the detection area with irregular shape, the detection and positioning are further refined. The schematic diagram of the positioning result of irregular area is as follows: Figure 5As shown, the first step is to determine its minimum bounding rectangle (MBR), the smallest rectangle that can completely encompass the area. This MBR is similar to the method used to locate regular rectangular regions, describing the region by locating the coordinates of the rectangle's top-left and bottom-right corners. However, locating irregular regions further relies on deep learning models to achieve pixel-level accuracy. This step confines the irregular region to a relatively regular rectangle, facilitating subsequent depth segmentation.

[0059] The minimum bounding rectangle area set determined above The input is fed into a pre-trained deep learning segmentation model for processing. Deep learning models, such as U-Net and Mask R-CNN, possess powerful image segmentation capabilities, capable of identifying and distinguishing different objects in an image. During this process, the model classifies the pixels within each minimum bounding rectangle based on the region's characteristics, thereby accurately segmenting the boundaries of irregular regions. Is a set containing multiple minimum bounding rectangle areas ,These rectangular areas are used for the detection and positioning of irregular shape features, Each element in the set is the minimum bounding rectangle determined by the previous processing for the suspected dirt or glue line features of irregular shapes in the input image, which means Each element in the set tightly surrounds one or more irregular-shaped features in the image, including the coordinates of the upper left corner and the lower right corner, thereby defining a specific image area for subsequent depth segmentation and feature recognition.

[0060] After processing by the deep learning segmentation model, the updated region set is obtained , It is a new set of regions that contains more accurate irregular shape regions formed after processing by deep learning segmentation model , these areas are the minimum bounding rectangle area set The boundaries obtained by refinement are more consistent with the actual dirt or glue line features, so they can provide more accurate detection areas, reduce false detections and missed detections in subsequent processing, and improve the overall detection accuracy. is a collection The nth irregular shape detection area in each area A set of coordinate points composition, and The nth boundary point represents the horizontal and vertical coordinates, respectively. These coordinates accurately describe the contour of the irregular area. In this way, the transition from coarse positioning of the minimum bounding rectangle to precise positioning of the interior of the irregular area at the pixel level is achieved.

[0061] In a specific embodiment of the present application, dirt detection is performed on the dirt detection area, and the dirt detection results are quantified, including: horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas; performing deep recognition and fine classification based on the above-mentioned dirt detection area set to obtain accurate recognition and classification of suspected dirty areas; and performing fine post-processing and indicator quantification on the above-mentioned suspected dirty areas.

[0062] Specifically, for the comprehensive processing of the dirt detection area, such as Figure 6 As shown, Figure 6 A schematic diagram shows the results of horizontally stitching regular and irregular areas. First, the regular rectangular dirt detection areas precisely located by traditional algorithms are horizontally stitched together with the irregular dirt detection areas obtained by deep segmentation technology to form a unified set of dirt detection areas. This process ensures that all potential dirt areas are included in subsequent detection and analysis. Regardless of whether the dirt is regular or irregular in shape, they are all processed within a unified framework, improving the comprehensiveness and efficiency of detection.

[0063] Based on the collection of contamination detection areas after this comprehensive processing, each detection area is carefully analyzed to determine whether it is actually contaminated. The contamination is then further refined and classified, including but not limited to determining the type (e.g., dust, oil stains), location, size, and other information. After identifying suspected contamination areas, further post-processing is performed to improve the accuracy and reliability of contamination detection.

[0064] After identifying suspected contaminated areas, the system undergoes refined post-processing and quantification. This means that after initial identification, deep learning and image processing techniques are used to further optimize the boundaries of these areas, eliminate noise, and ensure the accuracy of the detection results. Subsequently, the characteristics of the contamination, such as location, size, shape, and brightness, are quantitatively analyzed to provide specific data for production quality control and help determine whether the contamination exceeds acceptable standards.

[0065] In a specific embodiment, the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation are horizontally spliced ​​to obtain a dirt detection area set, including: and irregular rectangular dirt detection area set Merge to form a preliminary comprehensive regional set ;use Function, according to the size of the horizontal coordinate x, the above preliminary comprehensive area set is sorted in ascending order, ensuring that the order of each area is consistent with its horizontal position in the original image, and obtaining the sorted set ;Through horizontal splicing operation Function, the above sorted area set All the areas in the horizontal direction are spliced ​​together to generate a unified and positionally ordered set of the above-mentioned dirt detection areas. .

[0066] Specifically, the first step is to initialize and merge the region set, and then the regular rectangular dirt detection region set accurately located by the traditional algorithm is The set of irregular rectangular dirt detection areas obtained by deep segmentation technology Merge to form a preliminary comprehensive regional set This step ensures that all potential contamination detection areas are included in the subsequent processing flow, regardless of their regular or irregular shapes. It is a set of regular regions, including multiple detection regions of regular shapes , Represents a set The first, second, and up to the nth regular detection areas in the image are obtained by accurately locating regular rectangles in the input image through traditional matching algorithms. Their shapes and boundaries are rectangular, so standard image processing techniques can be used directly for further detection and analysis. It is a set of irregular regions, including multiple irregular shape regions that have been optimized by deep learning segmentation models. , Represents a set The boundaries of the first, second, and nth irregular detection areas in the image are processed by the deep learning model to more accurately fit the actual shape of the dirt or glue line in the image, making them more suitable as the precise range for feature detection. is achieved by merging the rule region set and irregular region collections The comprehensive regional collection formed by The generation of means that both regular and irregular regions are integrated into one set, so that subsequent steps can uniformly process all detection regions, whether regular or irregular in shape, providing a comprehensive regional basis for subsequent comprehensive detection and analysis. The symbol U represents the union operation of sets, which means merging all elements of two sets into one set, removing duplicates and forming a new set. .

[0067] Use the Sort function to sort the initial integrated area Sort by x. The sorting is based on the size of the horizontal coordinate x, and the regions are arranged in ascending order from left to right, ensuring that the order of each region during processing matches its actual horizontal position in the original image. The sorted set is recorded as ,This operation helps the accuracy and efficiency of subsequent horizontal splicing. Indicates applying the Sort function to the comprehensive area set Sort all elements (i.e., all detection areas) in . In this scenario, the sorting is based on the x-value of the upper left corner of each detection area. That is, the Sort function extracts the x-value of the upper left corner of each area and sorts all detection areas in ascending order based on the size of these values. It is a comprehensive area set obtained by sorting the Sort function. In , each detection region is arranged in ascending order according to the value of the horizontal coordinate x of its upper left corner, which ensures that the order of regions in the set reflects their horizontal position order in the original image, thus facilitating unified processing and management of subsequent operations.

[0068] Horizontal splicing processing: through the horizontal splicing operation function For the sorted region set Processing is performed to splice all detection areas horizontally to form a continuous, positionally ordered set of dirt detection areas. The purpose of this step is to create conditions for subsequent unified analysis and processing, ensuring seamless and continuous detection and analysis of all inspection areas, avoiding processing breakpoints, and improving the consistency and accuracy of the entire process.

[0069] Through the above process, both regular rectangular areas and irregular detection areas can be accurately and orderly processed and analyzed, providing a unified framework for dirt detection, simplifying the subsequent deep identification and quantitative analysis process, and improving the efficiency and accuracy of detection.

[0070] In a specific embodiment, deep recognition and refined classification are performed based on the above-mentioned dirt detection area set. The schematic diagram of the preliminary classification and positioning results of the dirt detection area is shown in FIG. Figure 7 As shown, accurate identification and classification of suspected dirty areas are obtained, including: using pre-trained recognition neural network functions , for the above dirt detection area set Perform in-depth analysis and output a set of suspected dirty areas , where each element Zn contains the dirt type identifier and the coordinate information of the dirt area, .

[0071] Specifically, using the pre-trained neural network function Collection of dirt detection areas This neural network performs in-depth analysis, identifying and classifying each detection area based on image features, determining the presence and type of dirt. The use of a pre-trained model means it has been trained on extensive data and can effectively identify different types of dirt.

[0072] After in-depth analysis, The function outputs a set of suspected dirty areas Each element in the set not only contains the type of dirt identification, but also accurately gives the coordinate information of the dirty area, including the upper left corner coordinates and the lower right corner coordinates. These coordinate information provides accurate location data for subsequent dirt quantitative analysis.

[0073] pass Identification of functions, sets It not only identifies suspected areas of contamination but also clearly identifies the specific type and location of the contamination. This dual confirmation mechanism increases the accuracy and reliability of contamination detection, providing a data foundation for subsequent refined post-processing and indicator quantification.

[0074] In summary, through deep recognition and refined classification, the system can accurately identify areas suspected of being dirty from a pre-processed set of dirt detection areas, classify the types of dirt, and provide precise coordinate information of the dirty areas, providing key recognition results for subsequent processing steps, ensuring the accuracy and efficiency of dirt detection.

[0075] In a specific embodiment, the above-mentioned suspected dirty areas are subjected to refined post-processing and index quantification, including: for the above-mentioned suspected dirty area set According to the above dirt type identification , through intelligent functions Dynamically calculate adaptive thresholds ; Based on the calculated adaptive threshold , using the binarization function Collect the suspected dirty areas Each area in the image is segmented to generate a binary image, and the final output is a set of precise dirty areas. , where each element Corresponding to The Zn region in which .

[0076] Specifically, the suspected dirty areas are processed and the indicators are quantified. The schematic diagram of the precise positioning and quantification results of specific types of dirt is shown in the following figure. Figure 8 As shown, for the suspected dirty area collection The dirt type identification of each area Zn in , using smart functions Dynamically calculate an adaptive threshold The key to this step is that the threshold is not fixed, but is intelligently adjusted according to the type of dirt to adapt to different types of dirt characteristics and improve the accuracy and pertinence of segmentation.

[0077] Based on the adaptive threshold calculated above , using the binarization function Gather suspected dirty areas Each region in the image is subjected to in-depth post-processing, known as precise segmentation. This process converts each region into a binary image, a two-state image consisting of only the dirty area and the background. The dirty area is highlighted, while the background is ignored. This process clearly defines the boundaries of the dirty area, providing a more accurate basis for subsequent quantification of indicators.

[0078] Through the above-mentioned refined post-processing, a set of precise dirty areas is generated. , each element in the set Corresponding to However, after deep post-processing, it provides more accurate boundaries and internal details of the dirty area, which facilitates subsequent quantitative analysis, such as measuring the area of ​​dirt, evaluating brightness, and judging shape.

[0079] Through adaptive threshold calculation and binarization processing, the suspected dirty areas are finely segmented and their boundaries are defined, generating a set of precise dirty areas. This provides more accurate quantitative data for subsequent in-depth analysis of the dirt properties and product quality control, enhancing the intelligence and accuracy of the entire detection process.

[0080] In a specific embodiment of the present application, glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, including: horizontally splicing the regular rectangular glue line detection area and the irregular rectangular glue line detection area obtained by depth segmentation to obtain a set of glue line detection areas; using an advanced segmentation network to perform image processing on the above-mentioned set of glue line detection areas, outputting a mask of the glue line area, and the segmentation network automatically identifies and highlights all glue line areas.

[0081] Specifically, glue line detection is performed in the glue line detection area, and glue lines of different shapes are quantified, including but not limited to glue width, glue length, whether glue is broken, whether glue is insufficient, etc. The schematic diagram of the glue line detection area is as follows Figure 9As shown in the figure, the set of regular rectangular glue line detection regions located by the traditional algorithm and the set of irregular rectangular glue line detection regions obtained by the deep segmentation technique are horizontally spliced ​​to form a unified set of glue line detection regions. The horizontal splicing operation ensures that all potential glue line detection regions, whether regular or irregular in shape, are integrated into a continuous detection framework, laying the foundation for subsequent comprehensive glue line detection and analysis.

[0082] Using a pre-trained advanced image segmentation network (such as U-Net or Mask R-CNN), we perform deep image processing on the set of glue line detection areas after comprehensive processing. This segmentation network automatically identifies glue line regions in the image and generates a mask for the glue line region—a binary image where 1 represents the glue line and 0 represents the background. This mask not only highlights the glue line region but also clearly demarcates the boundary between the glue line and the background, facilitating accurate quantification of the glue line.

[0083] After the segmentation network is processed, all glue line areas are automatically identified and highlighted in the generated mask. This feature is particularly critical for accurate glue line detection on automated production lines, as the mask eliminates background interference and focuses only on the glue line itself, thereby improving the accuracy and efficiency of glue line detection.

[0084] Based on the generated glue line area mask, further quantitative analysis of the glue line can be performed, including but not limited to measurement of glue line width, length, continuity, and other indicators. The introduction of the segmentation network not only accurately identifies the glue line but also provides highly precise glue line shape information, which is crucial for subsequent quantitative analysis.

[0085] Through the above process, both regular and irregularly shaped glue lines can be accurately detected, identified, and quantified, providing strong technical support for quality control during module production and ensuring that the performance of the glue lines meets high standards.

[0086] In a specific embodiment of the present application, an advanced segmentation network is used to perform image processing on the above-mentioned glue line detection area set, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for long straight glue lines, a specific area selection and preprocessing step, a minimum circumscribed rectangle and contour acquisition step, and a glue width quantization step are successively performed to realize the quantization of the long straight glue lines, wherein the mask of the above-mentioned glue line area is applied in the process of performing the above-mentioned specific area selection, and the above-mentioned preprocessing includes thresholding processing and closing operation processing.

[0087] Specifically, the process begins with the selection and preprocessing of a specific region. Based on the glue line region mask generated by the segmentation network, a specific rectangular region containing the long straight glue line is automatically or manually selected. Next, the selected rectangular region undergoes preprocessing, including thresholding and closing operations. Thresholding converts the image into a binary image, highlighting the contrast between the glue line region and the background. Closing eliminates holes and breaks in the image, ensuring the continuity and integrity of the glue line and providing a clear outline for subsequent measurements.

[0088] Next, the minimum enclosing rectangle (MBR) and outline are obtained. Based on the preprocessed image, the minimum enclosing rectangle (MBR) of a specific rectangular area is calculated, and its outline is obtained. The MAR helps accurately locate the glue line boundary, while the outline information is used to more carefully delineate the glue line edge. These two steps provide the geometric framework for accurate glue line width measurement.

[0089] Finally, the glue width is quantified, the long straight glue detection area and the quantification results are shown in the figure below Figure 10 As shown, the long straight glue quantification method includes:

[0090] 1. Specific area selection and preprocessing:

[0091] Although the glue line region set Det_glue can automatically identify all glue lines, it is still necessary to focus on a specific rectangular region ROI_re when analyzing long straight glue lines. ROI_re can be manually or automatically selected from the mask output by Det_glue. This region is then thresholded and closed to eliminate noise and discontinuities, ultimately extracting the largest connected region, ROI_con.

[0092] 2. Minimum bounding rectangle and outline acquisition:

[0093] Calculate the minimum bounding rectangle Smallest_re of ROI_con and obtain its contour Contour_con. This step is crucial for subsequent glue width measurement.

[0094] 3. Quantification of glue width:

[0095] Based on the coordinates of the four corners of Smallest_re (Re_lt, Re_lb, Re_rt, Re_rb), draw equidistant lines Re|_step along the direction (Re_lb – Re_lt). The intersection points of Re|_step and Contour_con are {con_t, con_b}, and the distance between them is the width of the glue line at that location.

[0096] In summary, the segmentation network first identifies and highlights the glue line area. Then, through specific region selection, preprocessing, minimum bounding rectangle and outline extraction, and glue width quantification, it achieves accurate measurement of the width of long, straight glue lines. This series of operations not only improves the accuracy of glue line detection but also greatly enhances the efficiency of glue line quality control on automated production lines.

[0097] The circular glue detection area and quantification results are shown in the figure below. Figure 11 As shown, the circular glue quantification method includes:

[0098] 1. Specific area selection and preprocessing:

[0099] Although Det_glue can automatically identify all glue lines, it is still necessary to focus on a specific circular region (ROI_cir) when analyzing circular glue. ROI_cir can be manually or automatically selected from the mask output by Det_glue. This region is then thresholded and closed to eliminate noise and discontinuous areas, ultimately extracting the largest connected region to obtain ROI_con^cir.

[0100] 2. Glue line quantization: Based on the center of ROI_cir (center_x, center_y), set a line Cir|step according to the rotation angle (step). The intersection of Cir|step and ROI_con^cir is {con_t^cir, con_b^cir} respectively, and the distance between them is the width of the glue line at that angle.

[0101] Through the above process, high-precision quantitative measurement can be achieved for both long straight glue lines and circular glue areas, providing strong technical support for quality control on automated production lines.

[0102] In a specific embodiment of the present application, an advanced segmentation network is used to perform image processing on the above-mentioned glue line detection area set, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for circular glue lines, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein the mask of the above-mentioned glue line area is applied in the process of performing the above-mentioned specific area selection, and the above-mentioned preprocessing includes thresholding processing and closing operation processing.

[0103] Specifically, based on the glue line area mask generated by the segmentation network, the specific circular area where the circular glue line is located is automatically or manually selected. This selection process is based on the circular glue line area highlighted in the mask, ensuring the accuracy and specificity of the target area. Subsequently, the selected circular area is preprocessed, including thresholding and closing operations. Thresholding can convert the image into a binary image to highlight the contrast between the circular glue line and the background; closing operations are used to eliminate small holes and discontinuities in the image to ensure the integrity and coherence of the circular glue line, which is conducive to subsequent accurate measurement.

[0104] Based on the preprocessed image, a quantitative analysis of the circular glue lines is performed. The specific steps include: first, determining the center point of the circular area; then, drawing a series of rays from 0 to 360 degrees at predetermined angular intervals, centered at the center point. These rays intersect with the edges of the circular glue line area to form an ordered set; finally, by measuring the distance from the center point to the intersection point, the glue line width at different angular directions can be calculated. This process generates a complete distribution of circular glue line widths, providing comprehensive data for evaluating circular glue line quality.

[0105] By performing specific preprocessing and detailed quantitative analysis of the circular glue line area, combined with a mask generated by the segmentation network, we achieve precise measurement of the circular glue line width. This series of steps not only improves detection accuracy but also optimizes the efficiency of circular glue line quality control on automated production lines, providing technical support for high-precision module lens production.

[0106] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the module lens dirt detection and glue line detection method of the present application will be described in detail below with reference to specific embodiments.

[0107] This embodiment relates to a specific method for detecting module lens dirt and glue lines, such as Figure 12 As shown, the following steps are included:

[0108] Input the image to be detected;

[0109] Calculate the similarity between the image to be detected and the template set;

[0110] Determine whether the similarity calculation result meets the set threshold;

[0111] When the similarity calculation result meets the set threshold, the dirty image is located in a regular rectangular area;

[0112] Determine whether the detection area is regular;

[0113] In the case of regular detection areas, the regular areas and irregular areas are horizontally spliced;

[0114] The recognition network performs rough positioning and classification;

[0115] Determine whether the recognition area is white dirt;

[0116] In the case where the identified area is white dirt, a white dirt quantification method is implemented;

[0117] Make a comprehensive judgment and end the process;

[0118] If the identified area is not white contamination, the scattered contamination quantification method is implemented and the step of "making a comprehensive judgment and ending the process" is executed;

[0119] In the case where the detection area is irregular, the segmentation network segments the area and performs the step of “horizontally splicing regular areas with irregular areas”;

[0120] When the similarity calculation result does not meet the set threshold, the glue line image is used to locate the irregular rectangular area;

[0121] Manually divide the glue line image into regions;

[0122] Determine whether it is a long straight glue area;

[0123] In the case of a long straight glue area, the long straight glue quantification method is implemented and the step of "comprehensive judgment and end of process" is executed;

[0124] In the case of a non-long straight glue area, the circular glue quantification method is implemented and the step of "making a comprehensive judgment and ending the process" is executed;

[0125] The embodiment of the present application also provides a module lens dirt detection and glue line detection device. It should be noted that the module lens dirt detection and glue line detection device of the embodiment of the present application can be used to execute the module lens dirt detection and glue line detection method provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation methods, and the details that have been explained will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] The following introduces the module lens dirt detection and glue line detection device provided in the embodiment of the present application.

[0127] Figure 13 : is a structural block diagram of a module lens dirt detection and glue line detection device according to an embodiment of the present application. Figure 13 As shown, the device includes:

[0128] The classification unit 10 is configured to perform an image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line image and a dirt image;

[0129] Specifically, the classification unit can split subsequent processing steps into two branches: one for glue line detection and the other for contamination detection. This ensures the targeted and efficient processing algorithm and avoids redundant analysis of all content in the image, effectively reducing computing resource consumption and speeding up the overall inspection process. Through precise classification, subsequent detection and quantification steps can be more focused and efficient, improving detection accuracy and reliability.

[0130] The region positioning unit 20 is configured to perform the detection region positioning step: locating the detection region of the classified image. If the detection region is a regular rectangular region, the detection region template set with a search region threshold is used for positioning. If the detection region is an irregular region, the minimum bounding rectangle is first defined and then depth segmentation is performed using a pre-trained deep learning segmentation model for pixel-level positioning.

[0131] Specifically, the region localization unit is responsible for detecting and locating regular rectangular regions. For regular rectangular region localization, if the region to be detected in the classified image is a regular rectangle, the system first utilizes a preset set of detection region templates, which contains rectangular templates of varying sizes and positions, along with their corresponding location information. When searching for these regions, a retrieval region threshold is introduced to reduce the search range and improve localization speed. Based on the template set, location information, and the retrieval region threshold, the system quickly searches for and locates regular rectangular regions in the image. For irregular region localization, since irregularly shaped detection regions do not adhere to fixed templates and location information, a different approach is required. First, traditional algorithms (such as edge detection and contour extraction) are used to preliminarily define the minimum bounding rectangle of the irregular region. This step provides a preliminary positioning reference for the deep learning model. The preliminarily defined region is then fed into a pretrained deep learning segmentation model, which performs pixel-level segmentation, separating the irregular region from the background and achieving pixel-level precision localization.

[0132] The dirt detection unit 30 is used to perform a dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection result;

[0133] Specifically, the dirt detection unit is used to conduct in-depth analysis and quantification of the located dirty areas. In the dirt detection unit, all identified dirt detection areas, whether they are regular rectangular areas or complex irregular areas, are comprehensively processed to form a unified set of dirt detection areas. Deep learning technology is used to perform detailed analysis of these areas to identify various types of dirt. Then, the refined post-processing stage is entered. The segmentation threshold is dynamically adjusted through an intelligent algorithm to perform more accurate segmentation of the identified dirty areas. This process is carried out at the pixel level, which greatly improves the accuracy and reliability of dirt detection. A detailed quantitative analysis is performed on each dirty area, including but not limited to measuring its area, evaluating its brightness, and classifying its type. These quantitative results can provide a scientific basis for quality control on the production line, help to promptly discover and solve potential dirt problems, and ensure that each module lens can meet high-standard production requirements.

[0134] The glue line detection unit 40 is used to perform the glue line detection step: perform glue line detection on the glue line detection area and quantify glue lines of different shapes.

[0135] Specifically, the glue line detection unit utilizes advanced image analysis technology to accurately detect and quantify glue lines within the glue line detection area. The unit automatically identifies and analyzes features such as the shape, width, and length of the glue line. For glue lines of different shapes (such as long straight lines and circular lines), a specific quantification algorithm is used to perform a detailed assessment, including detecting glue breaks and insufficient glue, thereby ensuring glue line quality and the sealing performance of the module lens.

[0136] The aforementioned module lens dirt detection and glue line detection device includes a processor and memory. The classification unit, area positioning unit, dirt detection unit, glue line detection unit, and the like are all stored as program units in the memory. The processor executes these program units stored in the memory to implement the corresponding functions. All of these modules are located in the same processor; alternatively, the modules can be located in different processors in any combination.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0138] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is run, the device where the computer-readable storage medium is located is controlled to execute the module lens dirt detection and glue line detection method.

[0139] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the module lens dirt detection and glue line detection method when running.

[0140] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and runnable on the processor. When the processor executes the program, at least the steps of the above-mentioned module lens dirt detection and glue line detection methods are implemented.

[0141] The present application also provides a computer program product which, when executed on a data processing device, is suitable for executing a program that initializes at least the steps of the above-mentioned module lens dirt detection and glue line detection method.

[0142] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0143] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0147] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0148] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0149] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0150] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0151] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting module lens dirt and glue lines, characterized in that: include: Image classification step: performing classification processing on the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image; Detection area positioning step: positioning the detection area of ​​the classified image. If it is a regular rectangular area, the template set of the area to be detected that introduces the retrieval area threshold is applied for positioning. If it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used for deep segmentation to perform pixel-level positioning. The template set of the area to be detected is a set of templates Dnor = {D1, D2, ..., Dn}, and the template set Dnor = {D1, D2, ..., Dn} includes regular rectangular templates of various sizes and shapes, and a position coordinate set corresponding to the regular rectangular template. The search area threshold is a preset pixel value expanded in four directions, left, right, up and down, based on the coordinates of the upper left corner and lower right corner of the regular rectangular template; Dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection results; Glue line detection steps: Glue line detection is performed in the glue line detection area, and glue lines of different shapes are quantified; Performing dirt detection on the dirt detection area and quantifying the dirt detection results includes: horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas; performing deep recognition and refined classification based on the set of dirt detection areas to obtain accurate recognition and classification of suspected dirt areas; and performing refined post-processing and indicator quantification on the suspected dirt areas. The regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation are horizontally spliced ​​to obtain a set of dirt detection areas, including: Irregular rectangular dirt detection area set Forming a preliminary comprehensive regional collection Use the Sort(·) function to sort the preliminary integrated region set in ascending order according to the size of the horizontal coordinate x, ensuring that the order of each region is consistent with its horizontal position in the original image, and obtain the sorted set By horizontal stitching operation horiz concat (·) function, the sorted set All the areas in the horizontal direction are spliced ​​together to generate a unified and positionally ordered set of the dirt detection areas.

2. The method according to claim 1, characterized in that If it is a regular rectangular area, the template set of the area to be detected that introduces the retrieval area threshold is applied for positioning, including: Set template set Dnor = {D1.D2.....Dn} and its corresponding position coordinate set For each position Dn dis ={xn l ,yn l ,xn r ,yn r } defines the coordinates of the upper left corner and lower right corner of the rectangular area; According to the position coordinate set Limit the global search to the local area D det ={D1 det , D2 det ,...,Dn det } and its adjusted position coordinate set The adjustment rules are Only in the reduced area D det Execute D nor With D det The pixel-level convolution calculation between Calculate the convolution score and get each Dn det The highest score in Through the transformation function Trans(·), the coordinates of the highest scoring point are mapped back to the original image coordinate system to obtain the final accurate matching position in, The coordinates of the highest scoring point are mapped back to the upper left corner coordinates and the lower right corner coordinates of the original image coordinate system.

3. The method according to claim 1, characterized in that If it is an irregular area, first frame the area with the minimum bounding rectangle and then use the pre-trained deep learning segmentation model to perform depth segmentation for pixel-level positioning, including: The minimum bounding rectangle area set As the input of the deep learning segmentation model, the updated region set is obtained after processing by the deep learning segmentation model Each Described as a set of coordinate points {Dn x1 , Dn y1 , Dn x2 ,Dn y2 ,...,Dn xn ,Dn yn }Wherein, the method for determining the minimum circumscribed rectangle is the same as the method for determining the regular rectangular area.

4. The method according to claim 1, wherein Based on the set of dirt detection areas, deep recognition and refined classification are performed to obtain accurate recognition and classification of suspected dirt areas, including: Using the pre-trained recognition neural network function f det (·), the dirt detection area set Det z Perform in-depth analysis and output the suspected dirty area set Z back = {Z1, Z2, ..., Zn}, where each element Zn contains the dirt type identifier and the coordinate information of the dirt area, where Z back =f det (Det z ) 5. The method according to claim 4, characterized in that Perform refined post-processing and index quantification on the suspected dirty areas, including: For the suspected dirty area set Z back According to the dirt type identification cat, through the intelligent function Adaρt thres (·) Dynamically calculate the adaptive threshold Bin threshold =Adapt thres (Z back )| cat Based on the calculated adaptive threshold Bin threshold , using the binarization function Bin process (·) for the suspected dirty area set Z back Each area in the image is segmented to generate a binary image, and the final output is the precise dirty area set Out Z ={Out Z1 ,Out Z2 ,...,Out Zn }, where each element Out Zn Corresponding to Z back The Zn region in which 6. The method according to claim 1, characterized in that Glue line detection is performed in the glue line detection area, and glue lines of different shapes are quantified, including: The regular rectangular glue line detection area and the irregular rectangular glue line detection area obtained by depth segmentation are horizontally spliced ​​to obtain a glue line detection area set; An advanced segmentation network is used to perform image processing on the set of glue line detection areas and output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas.

7. The method according to claim 6, characterized in that An advanced segmentation network is used to process the image of the glue line detection area set and output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas, including: For long straight glue lines, the specific area selection and preprocessing steps, the minimum circumscribed rectangle and contour acquisition steps, and the glue width quantization steps are performed successively to achieve quantization of the long straight glue lines. Among them, the mask of the glue line area is applied in the process of performing the specific area selection, and the preprocessing includes threshold processing and closing operation processing.

8. The method according to claim 6, characterized in that An advanced segmentation network is used to process the image of the glue line detection area set and output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas, including: For the circular glue line, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein a mask of the glue line area is applied in the process of the specific area selection, and the preprocessing includes thresholding processing and closing operation processing.

9. A module lens dirt detection and glue line detection device, characterized in that: include: A classification unit is configured to perform an image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line image and a dirt image; The region positioning unit is used to perform the detection region positioning step: positioning the detection region of the classified image, if it is a regular rectangular region, applying the template set of the region to be detected that introduces the retrieval region threshold for positioning, if it is an irregular region, first frame the minimum bounding rectangle and then use the pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning, wherein the template set of the region to be detected is a set of template sets Dnor = {D1, D2, ..., Dn}, and the template set Dnor = {D1, D2, ..., Dn} includes regular rectangular templates of various sizes and shapes, and a position coordinate set corresponding to the regular rectangular template The search area threshold is a preset pixel value expanded in four directions, left, right, up and down, based on the coordinates of the upper left corner and lower right corner of the regular rectangular template; A dirt detection unit, configured to perform a dirt detection step: performing dirt detection on a dirt detection area and quantifying the dirt detection result; A glue line detection unit is used to perform glue line detection steps: performing glue line detection on a glue line detection area and quantifying glue lines of different shapes; The dirt detection unit is specifically used to: horizontally splice the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a dirt detection area set; perform depth recognition and fine classification based on the dirt detection area set to obtain accurate recognition and classification of suspected dirt areas; perform fine post-processing and index quantification on the suspected dirt areas; wherein, horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a dirt detection area set includes: Irregular rectangular dirt detection area set , forming a preliminary comprehensive regional set Use the Sort(·) function to sort the preliminary integrated region set in ascending order according to the size of the horizontal coordinate x, ensuring that the order of each region is consistent with its horizontal position in the original image, and obtain the sorted set By horizontal stitching operation horiz concat (·) function, the sorted set All the areas in the horizontal direction are spliced ​​together to generate a unified and positionally ordered set of the dirt detection areas.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the module lens dirt detection and glue line detection method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the module lens dirt detection and glue line detection method described in any one of claims 1 to 8.

Images