Module lens smudginess detection and glue line detection method

Abstract

The invention provides a smudginess detection and glue line detection method for a module lens. The method comprises the steps that input images are classified, classified images are obtained, and the classified images comprise a glue line type image and a smudginess type image; then, positioning a detection region of the classified image, if the detection region is a regular rectangular region, positioning by applying a to-be-detected region template set into which a retrieval region threshold is introduced, and if the detection region is an irregular region, framing a minimum bounding rectangle and then performing deep segmentation by using a pre-trained deep learning segmentation model to perform pixel-level positioning; and finally, carrying out smudginess detection on the smudginess detection area, and quantifying a smudginess detection result. And carrying out glue line detection on the glue line detection area, and quantifying the glue lines in different shapes. According to the scheme, the problem that the current detection algorithm is difficult to meet the detection requirements of tiny dirt and key glue lines of the module lens is solved.

Description

Technical Field

[0001] The present application relates to computer vision and deep learning technologies, and more specifically, to a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the continuous development of module lens technology, high-resolution imaging requirements have put forward higher standards for module defect detection during the production process, especially the inspection of tiny dirt that affects the operation of the motor and the key glue lines that ensure sealing.

[0003] When faced with increasingly large amounts of image data, the processing speed and recognition accuracy of current detection algorithms are unable to adapt to the fast pace of modern efficient production lines. Summary of the invention

[0004] The main purpose of the present application is to provide a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium and an electronic device, so as to at least solve the problem that the current detection algorithm is difficult to meet the detection requirements of slight dirt and key glue lines on the module lens.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a module lens dirt detection and glue line detection method is provided, including: an image classification step: classifying the input image to obtain a classified image, the classified image including a glue line category image and a dirt category image; a detection area positioning step: positioning the detection area of ​​the classified image, if it is a regular rectangular area, applying a template set of the area to be detected that introduces a retrieval area threshold for positioning, if it is an irregular area, first frame the minimum circumscribed rectangle and then use a pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning; a dirt detection step: performing dirt detection on the dirt detection area, and quantifying the dirt detection result; a glue line detection step: performing glue line detection on the glue line detection area, and quantifying glue lines of different shapes.

[0006] Optionally, if it is a regular rectangular area, a template set of the area to be detected that introduces a retrieval area threshold is applied for positioning, including: setting the template set and its corresponding position coordinate set , for each location The upper left corner coordinates and the lower right corner coordinates of the rectangular area are defined; according to the position coordinate set , limiting the global search to a local area , and its position coordinate set after position adjustment , where the adjustment rule is ; Only in the reduced area Execute on and The pixel-level convolution calculation between Calculate the convolution score and get each The highest score in ; Through the conversion function , map the coordinates of the highest scoring point back to the original image coordinate system to obtain the final accurate matching position ,in, , , The coordinates of the highest scoring point are mapped back to the upper left corner coordinates and the lower right corner coordinates of the original image coordinate system.

[0007] Optionally, if it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used to perform deep segmentation for pixel-level positioning, including: the determined minimum bounding rectangle area set As the input of the deep learning segmentation model, the updated region set is obtained after being processed by the deep learning segmentation model , each Described as a set of coordinate points , wherein the method for determining the minimum circumscribed rectangle is the same as the method for determining the regular rectangular area.

[0008] Optionally, dirt detection is performed on the dirt detection area, and the dirt detection result is quantified, including: horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas; performing deep recognition and refined classification based on the set of dirt detection areas to obtain accurate recognition and classification of suspected dirty areas; and performing refined post-processing and indicator quantification on the suspected dirty areas.

[0009] Optionally, horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas includes: Set of irregular rectangular dirt detection areas Merge to form a preliminary comprehensive regional set ;use Function, according to the size of the horizontal coordinate x, the preliminary comprehensive area set is sorted in ascending order to ensure that the order of each area is consistent with its horizontal position in the original image, and the sorted set is obtained ; Through horizontal splicing operation Function, the sorted region set All the areas in the horizontal direction are spliced ​​to generate a unified and positionally ordered set of the dirt detection areas. .

[0010] Optionally, deep recognition and refined classification are performed based on the set of dirt detection areas to obtain accurate recognition and classification of suspected dirt areas, including: using a pre-trained recognition neural network function , for the dirt detection area set Perform in-depth analysis and output a set of suspected dirty areas , where each element Zn contains the dirt type identifier and the coordinate information of the dirty area, .

[0011] Optionally, the suspected dirty area is subjected to refined post-processing and index quantification, including: for the suspected dirty area set Identification according to the type of soiling , through intelligent functions Dynamically calculate adaptive thresholds Based on the calculated adaptive threshold , using the binarization function The suspected dirty area is collected Each area in is segmented to generate a binary image, and the final output is a set of precise dirty areas. , where each element Corresponds to The Zn region in .

[0012] Optionally, glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, including: horizontally splicing regular rectangular glue line detection areas and irregular rectangular glue line detection areas obtained by depth segmentation to obtain a set of glue line detection areas; using an advanced segmentation network to perform image processing on the set of glue line detection areas, outputting masks of the glue line areas, and the segmentation network automatically identifies and highlights all glue line areas.

[0013] Optionally, an advanced segmentation network is used to perform image processing on the set of glue line detection areas, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for long straight glue lines, a specific area selection and preprocessing step, a minimum circumscribed rectangle and contour acquisition step, and a glue width quantization step are successively performed to achieve quantification of the long straight glue lines, wherein the mask of the glue line area is applied in the process of performing the specific area selection, and the preprocessing includes thresholding processing and closing operation processing.

[0014] Optionally, an advanced segmentation network is used to perform image processing on the set of glue line detection areas, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for circular glue lines, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein the mask of the glue line area is applied in the process of selecting the specific area, and the preprocessing includes thresholding processing and closing operation processing.

[0015] According to another aspect of the present application, a module lens dirt detection and glue line detection device is provided, including: a classification unit, used to execute an image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image; an area positioning unit, used to execute a detection area positioning step: locate the detection area of ​​the classified image, if it is a regular rectangular area, apply a template set of the area to be detected that introduces a retrieval area threshold for positioning, if it is an irregular area, first frame the minimum circumscribed rectangle and then use a pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning; a dirt detection unit, used to execute a dirt detection step: perform dirt detection on the dirt detection area, and quantify the dirt detection result; a glue line detection unit, used to execute a glue line detection step: perform glue line detection on the glue line detection area, and quantify glue lines of different shapes.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the module lens dirt detection and glue line detection methods.

[0017] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include functions for executing any one of the module lens dirt detection and glue line detection methods.

[0018] By applying the technical solution of the present application, first, the input image is classified to obtain a classified image, which includes a glue line category image and a dirt category image; then, the detection area of ​​the classified image is located; if it is a regular rectangular area, a template set of the area to be detected that introduces a retrieval area threshold is used for positioning; if it is an irregular area, the minimum circumscribed rectangle is first framed and then deep segmentation is performed using a pre-trained deep learning segmentation model for pixel-level positioning; finally, dirt detection is performed on the dirt detection area, and the dirt detection result is quantified; glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, thereby solving the problem that the current detection algorithm is difficult to meet the detection requirements of tiny dirt and key glue lines of the module lens. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic flow chart of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a dirty image template of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of a regular rectangular area template of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of a regular rectangular area matching result of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram showing the irregular area positioning result of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of horizontal stitching results of regular areas and irregular areas of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0026] Figure 7 A schematic diagram showing preliminary classification and positioning results of a dirt detection area of ​​a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0027] Figure 8 A schematic diagram showing precise positioning and quantification results of specific types of dirt in a module lens dirt detection and glue line detection method provided in an embodiment of the present application is shown;

[0028] Fig. 9 A schematic diagram of a glue line detection area of ​​a module lens dirt detection and glue line detection method provided according to an embodiment of the present application is shown;

[0029] Fig.10 The figure shows a long straight glue detection area and a quantitative result diagram of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application;

[0030] Fig.11 A circular glue detection area and a quantitative result diagram of a module lens dirt detection and glue line detection method provided according to an embodiment of the present application are shown;

[0031] Fig.12 The overall flow chart of a module lens dirt detection and glue line detection method provided in accordance with an embodiment of the present application is shown;

[0032] Fig.13 A structural block diagram of a module lens dirt detection and glue line detection device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] As introduced in the background technology, when faced with increasingly large image data, the processing speed and recognition accuracy of the detection algorithms in the prior art are difficult to adapt to the fast pace of modern efficient production lines. In order to solve the problem that the current detection algorithms are difficult to meet the detection needs of tiny dirt and key glue lines on the module lens, the embodiments of the present application provide a module lens dirt detection and glue line detection method, a module lens dirt detection and glue line detection device, a computer-readable storage medium and an electronic device.

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0038] In this embodiment, a module lens dirt detection and glue line detection method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 1 FIG. 1 is a flow chart of a method for detecting module lens dirt and glue line according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0040] Step S101, image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image;

[0041] Specifically, during the production process of module lenses, images of lenses on the production line are taken in real time or regularly. These images contain various types of defects, such as glue line distribution problems or dirt left behind. These images are used as input and sent to the subsequent classification process. The input images are analyzed using a pre-trained deep learning model (such as a convolutional neural network CNN). The deep learning model can automatically extract features in the image by learning a large amount of labeled image data and classify the image based on these features. The deep learning model processes each input image separately and calculates its similarity with the preset glue line and dirt image features. If the features of the input image are more similar to the glue line image features, the image will be classified as a glue line category image; conversely, if the features are similar to the dirt image features, it will be classified as a dirt category image. This process usually involves calculating the distance or similarity score between the two images, such as using methods such as Euclidean distance, cosine similarity, or the output probability of a convolutional neural network. Finally, two types of images will be output: glue line category images and dirt category images. These two types of images have been preliminarily screened and only contain specific types of defects, which facilitates more targeted processing and analysis in subsequent steps. In this embodiment, the input image is compared with a preset dirty image template by convolution operation. The dirty image template schematic diagram is shown in FIG. Figure 2 As shown, if the calculation result exceeds a preset threshold (eg, 0.8), the image is determined to be a dirty image; if it does not reach this threshold, it is considered to be a glue line image.

[0042] The image classification step is the starting point of the entire inspection process, and the subsequent processing steps can be divided into two branches: one for glue line detection and the other for dirt detection. This ensures the pertinence and efficiency of the processing algorithm and avoids redundant analysis of all content in the image, thereby effectively reducing the consumption of computing resources and speeding up the overall inspection process. Through accurate classification, the subsequent inspection and quantification steps can be more centralized and effective, improving the accuracy and reliability of the inspection.

[0043] Step S102, detection area positioning step: positioning the detection area of ​​the above-mentioned classified image. If it is a regular rectangular area, the detection area template set with the retrieval area threshold is used for positioning. If it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used for deep segmentation to perform pixel-level positioning.

[0044] Whether the area to be detected is a regular rectangular area or an irregular area has been determined before the dirt detection and glue line detection. Specifically, it is determined based on the characteristics of the module lens image to be processed (in fact, it is determined based on the structural composition of the module lens itself, and the structural composition is related to the model). For details, see Figure 2If the area to be detected is the area framed by the yellow box, then the area to be detected is an irregular area. If the area to be detected is the area framed by the blue box, then the area to be detected is a regular rectangular area.

[0045] Specifically, for regular rectangular area positioning, if the area to be detected contained in the classified image is a regular rectangle, the preset template set of the area to be detected will be used first, which contains rectangular templates of different sizes and positions and their corresponding position information. When searching for these areas, in order to reduce the search range and increase the positioning speed, the retrieval area threshold is introduced. The system will quickly search and locate the regular rectangular area in the image based on the template set and position information, as well as the retrieval area threshold.

[0046] For irregular area positioning, since the detection area of ​​irregular shapes does not follow fixed templates and position information, another method is needed for positioning. First, traditional algorithms (such as edge detection, contour extraction, etc.) are used to preliminarily frame the minimum enclosing rectangle of the irregular area. This step provides a preliminary positioning reference for the processing of the deep learning model. Then, the preliminarily framed area is sent to the pre-trained deep learning segmentation model, which can accurately segment the image at the pixel level, separate the irregular area from the background, and achieve pixel-level precise positioning.

[0047] Through the above steps, accurate positioning can be achieved in both regular rectangular areas and irregular areas, providing precise target areas for subsequent specific area detection (dirt detection and glue line detection). This positioning method that combines deep learning with traditional algorithms not only improves the detection speed, but also ensures the accuracy of detection.

[0048] Step S103, dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection result;

[0049] Specifically, after completing the accurate classification of the module lens image and positioning of the detection area, step S103 focuses on the in-depth analysis and quantification of the located dirty areas. In this step, all the identified dirty detection areas, whether they are regular rectangular areas or complex irregular areas, are first processed comprehensively to form a unified set of dirty detection areas. Subsequently, these areas are analyzed in detail using deep learning technology to identify various types of dirt. Next, the refined post-processing stage is entered, and the segmentation threshold is dynamically adjusted through an intelligent algorithm to perform more accurate segmentation on the identified dirty areas. This process is performed at the pixel level, which greatly improves the accuracy and reliability of dirt detection. Finally, a detailed quantitative analysis is performed on each dirty area, including but not limited to measuring its area, evaluating its brightness, and classifying its type. These quantitative results can provide a scientific basis for quality control on the production line, help to promptly discover and solve potential dirt problems, and ensure that each module lens can meet high-standard production requirements.

[0050] Step S104, glue line detection step: perform glue line detection on the glue line detection area and quantify glue lines of different shapes.

[0051] Specifically, the glue line detection step is a process of accurately detecting and quantifying the glue line using advanced image analysis technology specifically for the glue line detection area after image classification and dirt detection. In this step, the shape, width, length and other characteristics of the glue line are automatically identified and analyzed. For glue lines of different shapes (such as long straight glue lines and round glue lines), specific quantification algorithms are used for detailed evaluation, including detection of whether the glue line is broken or lacking glue, so as to ensure the quality of the glue line and the sealing performance of the module lens. Through the glue line detection step, not only can the glue line area be quickly located, but the glue line status can also be accurately quantified, providing key data support for quality control in the production process.

[0052] In a specific embodiment of the present application, if it is a regular rectangular area, a template set of the area to be detected that introduces a search area threshold is applied for positioning, including: setting the template set and its corresponding position coordinate set , for each location The upper left corner coordinates and lower right corner coordinates of the rectangular area are defined, where: is the horizontal coordinate of the upper left corner of the rectangular area, is the ordinate of the upper left corner of the rectangular area. is the horizontal coordinate of the lower right corner of the rectangular area, is the ordinate of the lower right corner of the rectangular area; according to the above position coordinate set , limiting the global search to a local area , and its position coordinate set after position adjustment , where the adjustment rule is ,in, It is the adjusted detection area coordinate set, which indicates the coordinate information after the boundary is expanded according to the retrieval area threshold based on the original detection area. The default threshold for the search area is 20 pixels. Indicates that the left boundary of the original detection area is extended to the left pixels, Indicates that the upper boundary of the original detection area is extended upward pixels, Indicates that the right boundary of the original detection area is extended to the right pixels, Indicates that the lower boundary of the original detection area is extended downward pixels; only in the reduced area Execute on and The pixel-level convolution calculation between Calculate the convolution score, where x represents the horizontal coordinate position of any pixel point in the detection area, and y represents the vertical coordinate position of any pixel point in the detection area. Indicates that the detection area The matching score at position (x, y) in , Is the result of the convolution calculation, used to evaluate the detection template With detection area And get each The highest score in ,in, is the set of detection areas The nth local detection area in , which is considered to be the most likely to contain target features such as dirt or glue lines, and Respectively represent the horizontal and vertical coordinates of the position with the highest convolution score in the detection area; through the conversion function , map the coordinates of the highest scoring point back to the original image coordinate system to obtain the final accurate matching position ,in, , , The coordinates of the highest scoring point above are converted through the conversion function Map back to the upper left corner coordinates and lower right corner coordinates of the original image coordinate system.

[0053] Specifically, the conventional matching algorithm is used to locate the regular rectangular area in real time to improve the processing speed. The schematic diagram of the regular rectangular area template is shown in Figure 3As shown. First, the template set and position coordinates are initialized, and a set of template sets are preset. , which contains regular rectangular templates of various sizes and shapes, as well as the position coordinate sets corresponding to these templates Specifically, Is a template collection, including multiple preset templates ,Each template is an image region with a specific shape and size, which is used to compare the corresponding part of the input image. Each template has its specific geometric definition, which can be a rectangle, a circle or other shape, depending on the features to be recognized. is the set of position coordinates corresponding to these templates, including the template set The expected position information of each template in the image. For each template, contains the horizontal and vertical coordinates of the upper left and lower right corners of the template on the image, which is used to guide the matching position of the template on the input image. In short, A collection is a set of standard templates used to match dirt or glue line features. The coordinate information of the expected location of these templates in the image is used to guide the algorithm to quickly and accurately find and identify target features in the image. The position coordinates of each template The coordinates of the upper left corner and lower right corner of the rectangle are defined, which provides a reference standard for subsequent area positioning. It is the position coordinate of the nth template within its target detection area, which is used to guide the matching of the template with the corresponding area in the input image.

[0054] In order to reduce unnecessary calculations and improve positioning speed, a retrieval area threshold is introduced to narrow the global retrieval range to a local area based on the template position information. The coordinates of these local areas are adjusted. The adjustment rule is to expand left, right, up and down based on the coordinates of the upper left corner and lower right corner of the template. pixels (the default setting is 20 pixels), forming a new position coordinate set This design ensures the targeted search range, avoids unnecessary scanning of the entire image, and saves computing resources. Represents a set of multiple local detection areas after preliminary positioning, including multiple detection areas ,Each detection region is a local area determined from the original input image for ,further analysis and detection of features such as dirt or glue lines. It is a local area A set of position adjustments, where the position information of each detection area is adjusted to more accurately locate the feature.

[0055] Only in the reduced area Execute on and The pixel-level convolution calculation between A template set representing the normal state, which contains multiple template images , It is a set of local detection areas, including multiple local areas after preliminary positioning ,Will The template in Perform pixel-level convolution calculations on each detection area in the image to determine the degree of match between the template and the detection area. The matching score of each local area is obtained through convolution operation. , the higher the score, the higher the similarity between the local area and the template. Highest scoring point in the area , the schematic diagram of the regular rectangular area matching result is as follows Figure 4 As shown, the best matching area is Sure.

[0056] After obtaining the highest score, the conversion function Map these coordinate points from the local area coordinate system back to the original image coordinate system to obtain the final accurate matching position of each template in the original image This process is done through The function is executed according to the local area coordinate adjustment algorithm rules, ensuring that the positioning result can accurately correspond to the specific position on the original image.

[0057] In a specific embodiment of the present application, if it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used to perform deep segmentation for pixel-level positioning, including: the determined minimum bounding rectangle area set As the input of the above deep learning segmentation model, the updated region set is obtained after being processed by the above deep learning segmentation model , each Described as a set of coordinate points , wherein the method for determining the above-mentioned minimum circumscribed rectangle is the same as the method for determining the above-mentioned regular rectangular area.

[0058] Specifically, for the detection area with irregular shape, the detection and positioning are further refined. The schematic diagram of the positioning result of the irregular area is as follows: Figure 5As shown, we first need to determine its minimum enclosing rectangle, that is, the smallest rectangle that can completely contain the area. The method of determining the minimum enclosing rectangle here is similar to the method of locating a regular rectangular area. Both describe the area by locating the coordinates of the upper left and lower right corners of the rectangle, but the positioning of irregular areas further relies on the processing of the deep learning model to achieve pixel-level accuracy. Through this step, the irregular area is framed in a relatively regular rectangle, which is convenient for subsequent deep segmentation processing.

[0059] The minimum bounding rectangle area set determined above As input, it is sent to the pre-trained deep learning segmentation model for processing. Deep learning models, such as U-Net and Mask R-CNN, have powerful image segmentation capabilities and can identify and distinguish different objects in the image. In this process, the model will classify the pixels in each minimum bounding rectangle based on the characteristics of the area, thereby accurately segmenting the boundaries of irregular areas. Is a set containing multiple minimum enclosing rectangular areas ,These rectangular areas are used for the detection and location of irregular shape features, Each element in the set is the minimum bounding rectangle determined by the previous processing for the suspected dirt or glue line features of irregular shapes in the input image, which means Each element in the set tightly surrounds one or more irregular-shaped features in the image, including the coordinates of the upper left corner and the lower right corner, thereby defining a specific image area for subsequent depth segmentation and feature recognition.

[0060] After processing by the deep learning segmentation model, an updated set of regions is obtained , It is a new set of regions that contains more accurate irregular shape regions processed by the deep learning segmentation model. , these areas are the minimum bounding rectangle area set The boundaries obtained by refinement are more in line with the actual dirt or glue line features, so a more accurate detection area can be provided, which reduces false detection and missed detection in subsequent processing and improves the overall detection accuracy. Is a collection The nth irregular shape detection area in each area A set of coordinate points composition, and Represent the horizontal and vertical coordinates of the nth boundary point, respectively, and these coordinate points accurately describe the outline of the irregular area. In this way, the transition from the coarse positioning of the minimum bounding rectangle to the precise positioning of the pixel level inside the irregular area is achieved.

[0061] In a specific embodiment of the present application, dirt detection is performed on a dirt detection area, and the dirt detection result is quantified, including: horizontally splicing the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation to obtain a set of dirt detection areas; performing deep recognition and refined classification based on the above-mentioned set of dirt detection areas to obtain accurate recognition and classification of suspected dirty areas; and performing refined post-processing and indicator quantification on the above-mentioned suspected dirty areas.

[0062] Specifically, for the comprehensive processing of the dirt detection area, such as Figure 6 As shown, Figure 6 The schematic diagram of the horizontal splicing results of regular areas and irregular areas is shown. First, the regular rectangular dirt detection area accurately located by the traditional algorithm and the irregular dirt detection area obtained by the deep segmentation technology are horizontally spliced ​​to form a unified set of dirt detection areas. This process ensures that all potential dirt areas are included in the subsequent detection and analysis. Whether it is regular or irregular dirt, it is processed under a unified framework, which improves the comprehensiveness and efficiency of detection.

[0063] Based on the above-mentioned set of dirt detection areas after comprehensive processing, each detection area is carefully analyzed to identify whether there is dirt in the area, and the dirt is further refined, including but not limited to determining the type of dirt (such as dust, oil stains, etc.), location, size and other information. After identifying the suspected dirty area, further post-processing is performed to improve the accuracy and reliability of dirt detection.

[0064] After the suspected dirty areas are identified, the suspected dirty areas are refined post-processed and index quantified, which means that after the dirty areas are initially identified, deep learning and image processing technology are used to further optimize the boundaries of these areas, eliminate noise, and ensure the accuracy of the detection results. Subsequently, the characteristics of the dirt, such as location, size, shape, brightness, etc., will be quantitatively analyzed to provide specific data for production quality control and help determine whether the dirt exceeds the acceptable standard.

[0065] In a specific embodiment, the regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation are horizontally spliced ​​to obtain a set of dirt detection areas, including: Set of irregular rectangular dirt detection areas Merge to form a preliminary comprehensive regional collection ;use Function, according to the size of the horizontal coordinate x, the above preliminary comprehensive area set is sorted in ascending order to ensure that the order of each area is consistent with its horizontal position in the original image, and the sorted set is obtained ; Through horizontal splicing operation Function, the above sorted area set All the areas in the horizontal direction are spliced ​​together to generate a unified and positionally ordered set of the above-mentioned dirt detection areas. .

[0066] Specifically, the first step is to initialize and merge the region set, and then merge the regular rectangular dirt detection region set accurately located by the traditional algorithm. The irregular rectangular dirt detection area set obtained by deep segmentation technology Merge to form a preliminary comprehensive regional set This step ensures that all potential contamination detection areas are included in the subsequent processing flow, regardless of their regular or irregular shapes. It is a set of regular regions, including multiple detection regions of regular shapes. , Represents a set The first, second, up to the nth regular detection areas in the image are obtained by accurately locating the regular rectangles in the input image through traditional matching algorithms. Their shapes and boundaries are rectangular, so standard image processing techniques can be used directly for further detection and analysis. It is a set of irregular regions, including multiple irregular shape regions that have been optimized by deep learning segmentation models. , Represents a set The boundaries of the first, second, up to the nth irregular detection areas in the image are processed by the deep learning model to more accurately fit the actual shape of the dirt or glue line in the image, so they are more suitable as the precise range of feature detection. is a set of merged rule regions and irregular region collection The comprehensive regional collection formed by The generation of means that both regular and irregular regions are integrated into one set, so that the subsequent steps can uniformly process all detection regions, whether regular or irregular in shape, providing a comprehensive regional basis for subsequent comprehensive detection and analysis. The symbol U represents the union operation of sets, which means that all elements in two sets are merged into one set, and a new set is formed after removing duplicates. .

[0067] Use the Sort function to sort the initial integrated area collection The sorting is based on the size of the horizontal coordinate x, and the regions are arranged in ascending order from left to right, ensuring that the order of each region during processing matches its actual horizontal position in the original image. The sorted set is recorded as ,This operation helps the accuracy and efficiency of the subsequent horizontal ,stitching. Indicates the application of the Sort function to the comprehensive area set In this scenario, the sorting is performed based on the x-value of the upper left corner of each detection area. That is, the Sort function extracts the x-value of the upper left corner of each area and sorts all detection areas in ascending order according to the size of these values. It is a comprehensive set of regions sorted by the Sort function. In , each detection region is arranged in ascending order according to the value of the horizontal coordinate x of its upper left corner, which ensures that the order of regions in the set reflects their horizontal position order in the original image, thus facilitating the unified processing and management of subsequent operations.

[0068] Horizontal splicing processing: through the horizontal splicing operation function For the sorted region set Processing is performed to splice all detection areas horizontally to form a continuous, positionally ordered set of dirt detection areas. The purpose of this step is to create conditions for subsequent unified analysis and processing, ensuring seamless and continuous detection and analysis of all detection areas, avoiding processing breakpoints, and improving the consistency and accuracy of the entire process.

[0069] Through the above process, both regular rectangular areas and irregular detection areas can be accurately and orderly processed and analyzed, providing a unified framework for dirt detection, simplifying the subsequent deep identification and quantitative analysis process, and improving the efficiency and accuracy of detection.

[0070] In a specific embodiment, deep recognition and refined classification are performed based on the above-mentioned dirt detection area set. The schematic diagram of the preliminary classification and positioning results of the dirt detection area is as follows: Figure 7 As shown, accurate identification and classification of suspected dirty areas are obtained, including: using pre-trained recognition neural network functions , for the above dirt detection area set Perform in-depth analysis and output a set of suspected dirty areas , where each element Zn contains the dirt type identifier and the coordinate information of the dirty area, .

[0071] Specifically, using the pre-trained neural network function Collection of dirty detection areas The neural network function can identify and classify each detection area based on image features, and determine whether there is dirt in the area and the type of dirt. The use of pre-trained models means that the model has been trained with a large amount of data and can effectively identify different types of dirt.

[0072] After in-depth analysis, The function outputs a set of suspected dirty areas Each element in the set not only contains the type of dirt, but also accurately gives the coordinate information of the dirty area, including the upper left corner coordinates and the lower right corner coordinates. These coordinate information provides accurate location data for subsequent dirt quantitative analysis.

[0073] pass Identification of functions, sets It not only points out which areas are suspected of being contaminated, but also clearly marks the specific type and location of the contamination. This double confirmation mechanism increases the accuracy and reliability of contamination detection and provides a data basis for subsequent refined post-processing and indicator quantification.

[0074] In summary, through deep recognition and refined classification, the system can accurately identify areas suspected of being dirty from the pre-processed set of dirt detection areas, classify the types of dirt, and provide precise coordinate information of the dirty areas, providing key recognition results for subsequent processing steps, ensuring the accuracy and efficiency of dirt detection.

[0075] In a specific embodiment, the above-mentioned suspected dirty area is subjected to refined post-processing and index quantification, including: for the above-mentioned suspected dirty area set According to the above dirt type identification , through intelligent functions Dynamically calculate adaptive thresholds ; Based on the calculated adaptive threshold , using the binarization function Collect the suspected dirty areas above Each area in is segmented to generate a binary image, and the final output is a set of precise dirty areas. , where each element Corresponds to The Zn region in which .

[0076] Specifically, the suspected dirty areas are processed in detail and the indicators are quantified. The schematic diagram of the precise positioning and quantification results of specific types of dirt is shown in the figure. Figure 8 As shown, for the suspected dirty area collection The dirt type identification of each area Zn in , using smart functions Dynamically calculate an adaptive threshold The key to this step is that the threshold is not fixed, but is intelligently adjusted according to the type of dirt to adapt to different types of dirt characteristics and improve the accuracy and pertinence of segmentation.

[0077] Based on the adaptive threshold calculated above , using the binarization function Gather suspected dirty areas Each region in the image is subjected to in-depth post-processing, i.e., precise segmentation. This process converts each region into a binary image, i.e., a binary image with only dirty regions and backgrounds, where dirty regions are highlighted and backgrounds are ignored. Through such processing, the boundaries of dirty regions can be clearly defined, providing a more accurate basis for subsequent indicator quantification.

[0078] Through the above-mentioned refined post-processing, a set of precise dirty areas is generated. , each element in the set Corresponds to However, after deep post-processing, it provides more accurate boundaries and internal details of the dirty area, which is convenient for subsequent quantitative analysis, such as measuring the area of ​​dirt, evaluating brightness, and judging shape.

[0079] Through adaptive threshold calculation and binarization processing, the suspected dirty areas are segmented and bounded in a refined manner, generating a set of precise dirty areas. This provides more accurate quantitative data for subsequent in-depth analysis of the dirt properties and product quality control, and enhances the intelligence and accuracy of the entire detection process.

[0080] In a specific embodiment of the present application, glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, including: horizontally splicing the regular rectangular glue line detection area and the irregular rectangular glue line detection area obtained by depth segmentation to obtain a set of glue line detection areas; using an advanced segmentation network to perform image processing on the above-mentioned set of glue line detection areas, outputting a mask of the glue line area, and the segmentation network automatically identifies and highlights all glue line areas.

[0081] Specifically, glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified, including but not limited to glue width, glue length, whether glue is broken, whether glue is insufficient, etc. The schematic diagram of the glue line detection area is as follows Fig. 9As shown in the figure, the regular rectangular glue line detection area set located by the traditional algorithm and the irregular rectangular glue line detection area set obtained by the deep segmentation technology are horizontally spliced ​​to form a unified glue line detection area set. The horizontal splicing operation ensures that all potential glue line detection areas, whether regular or irregular in shape, are integrated into a continuous detection framework, laying the foundation for subsequent comprehensive glue line detection and analysis.

[0082] Using a pre-trained advanced image segmentation network (such as U-Net, Mask R-CNN, etc.), deep image processing is performed on the set of glue line detection areas after comprehensive processing. This segmentation network can automatically identify the glue line area in the image and generate a mask of the glue line area, that is, a binary image where 1 represents the glue line and 0 represents the background. The generation of the mask not only highlights the glue line area, but also clearly divides the boundary between the glue line and the background, providing conditions for the accurate quantification of the glue line.

[0083] After the segmentation network is processed, all glue line areas are automatically identified and highlighted in the generated mask. This feature is particularly critical for accurate detection of glue lines on automated production lines, because the mask can eliminate background interference and focus only on the glue line itself, thereby improving the accuracy and efficiency of glue line detection.

[0084] Based on the generated glue line area mask, the glue line can be further quantitatively analyzed, including but not limited to the measurement of glue line width, length, continuity and other indicators. The introduction of the segmentation network can not only accurately identify the glue line, but also provide high-precision glue line shape information, which is crucial for subsequent quantitative analysis.

[0085] Through the above process, both regular and irregular shaped glue lines can be accurately detected, identified and quantified, providing strong technical support for quality control in the module production process and ensuring that the performance of the glue lines meets high standards.

[0086] In a specific embodiment of the present application, an advanced segmentation network is used to perform image processing on the above-mentioned glue line detection area set, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for long straight glue lines, specific area selection and preprocessing steps, minimum circumscribed rectangle and contour acquisition steps, and glue width quantization steps are successively performed to achieve quantification of the long straight glue lines, wherein the mask of the above-mentioned glue line area is applied in the process of performing the above-mentioned specific area selection, and the above-mentioned preprocessing includes thresholding processing and closing operation processing.

[0087] Specifically, the first step is to select and preprocess a specific area. Based on the glue line area mask generated by the segmentation network, the specific rectangular area where the long straight glue line is located is automatically or manually selected. Then, the selected specific rectangular area is preprocessed, including thresholding and closing operations. The purpose of thresholding is to convert the image into a binary image to highlight the contrast between the glue line area and the background; the closing operation is used to eliminate holes and breaks in the image, ensure the continuity and integrity of the glue line, and provide a clear outline for subsequent measurements.

[0088] Next is to obtain the minimum enclosing rectangle and contour. Based on the preprocessed image, the minimum enclosing rectangle of a specific rectangular area is calculated, and the contour of the specific rectangular area is obtained. The minimum enclosing rectangle can help accurately locate the boundary of the glue line, while the contour information is used to more carefully depict the edge of the glue line. These two steps provide a geometric framework for the accurate measurement of the glue line width.

[0089] Finally, the glue width is quantified, the long straight glue detection area and the quantification result are shown in the figure Fig.10 As shown, the long straight glue quantification method includes:

[0090] 1. Specific area selection and preprocessing:

[0091] Although the glue line region set Det_glue can automatically identify all glue lines, it is still necessary to focus on a specific rectangular region ROI_re when analyzing long straight glue. ROI_re can be manually or automatically selected from the mask output by Det_glue, and then this area is thresholded and closed to eliminate noise and discontinuous parts, and finally the maximum connected domain is extracted to obtain ROI_con.

[0092] 2. Minimum bounding rectangle and outline acquisition:

[0093] Calculate the minimum enclosing rectangle Smallest_re of ROI_con and obtain its contour Contour_con. This step is crucial for the subsequent glue width measurement.

[0094] 3. Quantification of glue width:

[0095] Based on the four corner coordinates of Smallest_re (Re_lt, Re_lb, Re_rt, Re_rb), draw equidistant straight lines Re|_step along the (Re_lb–Re_lt) direction. The intersection points of Re|_step and Contour_con are {con_t, con_b}, and the distance between them is the width of the glue line at that position.

[0096] In summary, the segmentation network first identifies and highlights the glue line area, and then achieves accurate measurement of the width of the long straight glue line through steps such as specific area selection, preprocessing, minimum bounding rectangle and contour acquisition, and glue width quantification. This series of operations not only improves the accuracy of glue line detection, but also greatly improves the efficiency of glue line quality control on the automated production line.

[0097] The circular glue detection area and quantification results are shown in the figure below. Fig.11 As shown, the circular glue quantification method includes:

[0098] 1. Specific area selection and preprocessing:

[0099] Although Det_glue can automatically identify all glue lines, it is still necessary to focus on a specific circular area ROI_cir when analyzing circular glue. ROI_cir can be manually or automatically selected from the mask output by Det_glue, and then this area is thresholded and closed to eliminate noise and discontinuous parts, and finally the maximum connected domain is extracted to obtain ROI_con^cir.

[0100] 2. Glue line quantification: Based on the center (center_x, center_y) of ROI_cir, set the straight line Cir|step according to the rotation angle (step). The intersection points of Cir|step and ROI_con^cir are {con_t^cir, con_b^cir}, and the distance between them is the width of the glue line in the angle direction.

[0101] Through the above process, high-precision quantitative measurement can be achieved for both long straight glue lines and circular glue areas, providing strong technical support for quality control on automated production lines.

[0102] In a specific embodiment of the present application, an advanced segmentation network is used to perform image processing on the above-mentioned glue line detection area set, and a mask of the glue line area is output. The segmentation network automatically identifies and highlights all glue line areas, including: for circular glue lines, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein the mask of the above-mentioned glue line area is applied in the process of performing the above-mentioned specific area selection, and the above-mentioned preprocessing includes thresholding processing and closing operation processing.

[0103] Specifically, based on the glue line area mask generated by the segmentation network, the specific circular area where the circular glue line is located is automatically or manually selected. This selection process is based on the circular glue line area highlighted in the mask, ensuring the accuracy and pertinence of the target area. Subsequently, the selected circular area is preprocessed, including thresholding and closing operations. Thresholding can convert the image into a binary image to highlight the contrast between the circular glue line and the background; closing operations are to eliminate small holes and discontinuous parts in the image to ensure the integrity and coherence of the circular glue line, which is conducive to subsequent accurate measurement.

[0104] Based on the preprocessed images, a quantitative analysis of the circular glue line is performed. The specific steps include: first, determine the center point of the circular area; then, draw a series of rays from 0 degrees to 360 degrees at the set angle interval with the center point as the center. These rays and the edge intersection points of the circular glue line area form an ordered set; finally, by measuring the distance from the center point to the intersection point, the width of the glue line in different angle directions can be calculated. This process can generate a complete distribution of the circular glue line width, providing comprehensive data for evaluating the quality of the circular glue line.

[0105] By performing specific preprocessing and detailed quantitative analysis on the circular glue line area, combined with the mask generated by the segmentation network, the accurate measurement of the circular glue line width is achieved. This series of steps not only improves the accuracy of detection, but also optimizes the efficiency of circular glue line quality control on the automated production line, providing technical support for high-precision module lens production.

[0106] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the module lens dirt detection and glue line detection method of the present application will be described in detail below in combination with specific embodiments.

[0107] This embodiment relates to a specific module lens dirt detection and glue line detection method, such as Fig.12 As shown, the following steps are included:

[0108] Input the image to be detected;

[0109] Calculate the similarity between the image to be detected and the template set;

[0110] Determine whether the similarity calculation result meets the set threshold;

[0111] When the similarity calculation result meets the set threshold, the dirty image is located in a regular rectangular area;

[0112] Determine whether the detection area is regular;

[0113] In the case of regular detection areas, the regular areas are horizontally spliced ​​with irregular areas;

[0114] The recognition network performs rough positioning and classification;

[0115] Determine whether the recognition area is white and dirty;

[0116] In the case where the identified area is white dirt, a white dirt quantification method is implemented;

[0117] Make a comprehensive judgment and end the process;

[0118] When the identified area is not white dirt, the scattered dirt quantification method is implemented and the step of "comprehensive judgment and end of process" is executed;

[0119] In the case where the detection area is irregular, the segmentation network segments the area and performs the step of “horizontally splicing the regular area with the irregular area”;

[0120] When the similarity calculation result does not meet the set threshold, the glue line image is used to locate the irregular rectangular area;

[0121] Manually divide the glue line image into regions;

[0122] Determine whether it is a long straight glue area;

[0123] In the case of a long straight glue area, the long straight glue quantification method is implemented, and the step of "comprehensive judgment and end of process" is executed;

[0124] In the case where it is not a long straight glue area, the circular glue quantification method is implemented and the step of "making a comprehensive judgment and ending the process" is executed;

[0125] The embodiment of the present application also provides a module lens dirt detection and glue line detection device. It should be noted that the module lens dirt detection and glue line detection device of the embodiment of the present application can be used to execute the module lens dirt detection and glue line detection method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] The following is an introduction to the module lens dirt detection and glue line detection device provided in the embodiments of the present application.

[0127] Fig.13 is a structural block diagram of a module lens dirt detection and glue line detection device according to an embodiment of the present application. Fig.13 As shown, the device comprises:

[0128] The classification unit 10 is used to perform an image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image;

[0129] Specifically, the classification unit can divide the subsequent processing steps into two branches: one for glue line detection and the other for dirt detection. This ensures the pertinence and efficiency of the processing algorithm and avoids redundant analysis of all content in the image, thereby effectively reducing the consumption of computing resources and speeding up the overall detection process. Through accurate classification, the subsequent detection and quantification steps can be more centralized and effective, improving the accuracy and reliability of detection.

[0130] The region positioning unit 20 is used to perform the detection region positioning step: positioning the detection region of the above-mentioned classification image, if it is a regular rectangular region, using the template set of the region to be detected that introduces the retrieval region threshold to position it, if it is an irregular region, first frame the minimum bounding rectangle and then use the pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning;

[0131] Specifically, the region positioning unit is used to detect region positioning. For regular rectangular region positioning, if the region to be detected contained in the classified image is a regular rectangle, the preset template set of the region to be detected will be used first, which contains rectangular templates of different sizes and positions and their corresponding position information. When searching for these areas, in order to reduce the search range and improve the positioning speed, the retrieval region threshold is introduced. The system will quickly search and locate the regular rectangular region in the image based on the template set and position information, as well as the retrieval region threshold. For irregular region positioning, since the detection area of ​​irregular shape does not follow the fixed template and position information, another method is needed for positioning. First, traditional algorithms (such as edge detection, contour extraction, etc.) are used to preliminarily frame the minimum circumscribed rectangle of the irregular region. This step provides a preliminary positioning reference for the processing of the deep learning model. Then, the preliminarily framed area is sent to the pre-trained deep learning segmentation model, which can accurately segment the image at the pixel level, separate the irregular area from the background, and achieve pixel-level precise positioning.

[0132] The dirt detection unit 30 is used to perform a dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection result;

[0133] Specifically, the dirt detection unit is used to conduct in-depth analysis and quantification of the located dirty areas. In the dirt detection unit, all identified dirt detection areas, whether they are regular rectangular areas or complex irregular areas, are comprehensively processed to form a unified set of dirt detection areas. Deep learning technology is used to perform detailed analysis on these areas to identify various types of dirt. Then, the refined post-processing stage is entered. The segmentation threshold is dynamically adjusted through an intelligent algorithm to perform more accurate segmentation on the identified dirty areas. This process is carried out at the pixel level, which greatly improves the accuracy and reliability of dirt detection. A detailed quantitative analysis is performed on each dirty area, including but not limited to measuring its area, evaluating its brightness, and classifying its type. These quantitative results can provide a scientific basis for quality control on the production line, help to promptly discover and solve potential dirt problems, and ensure that each module lens meets high-standard production requirements.

[0134] The glue line detection unit 40 is used to execute the glue line detection step: perform glue line detection on the glue line detection area and quantify glue lines of different shapes.

[0135] Specifically, the glue line detection unit is used to accurately detect and quantify the glue line in the glue line detection area using advanced image analysis technology. In the glue line detection unit, the shape, width, length and other characteristics of the glue line are automatically identified and analyzed. For glue lines of different shapes (such as long straight glue lines and round glue lines), a specific quantification algorithm is used for detailed evaluation, including detection of whether the glue line is broken or insufficient, so as to ensure the quality of the glue line and the sealing performance of the module lens.

[0136] The above module lens dirt detection and glue line detection device includes a processor and a memory. The above classification unit, area positioning unit, dirt detection unit, glue line detection unit, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to realize the corresponding functions. The above modules are all located in the same processor; or, the above modules are located in different processors in any combination.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0138] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is running, the device where the computer-readable storage medium is located is controlled to execute the module lens dirt detection and glue line detection method.

[0139] An embodiment of the present invention provides a processor, which is used to run a program, wherein the module lens dirt detection and glue line detection method is executed when the program is running.

[0140] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the steps of the above-mentioned module lens dirt detection and glue line detection methods are implemented.

[0141] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes at least the steps of the above-mentioned module lens dirt detection and glue line detection method.

[0142] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0143] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0147] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0148] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0149] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0150] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0151] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A module lens dirt detection and glue line detection method, characterized in that: include: Image classification step: classifying the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image; Detection area positioning step: positioning the detection area of ​​the classified image. If it is a regular rectangular area, the detection area template set with the retrieval area threshold is used for positioning. If it is an irregular area, the minimum bounding rectangle is first framed and then the pre-trained deep learning segmentation model is used for deep segmentation to perform pixel-level positioning. Dirt detection step: performing dirt detection on the dirt detection area and quantifying the dirt detection result; Glue line detection steps: Glue line detection is performed on the glue line detection area, and glue lines of different shapes are quantified.

2. The method according to claim 1, characterized in that If it is a regular rectangular area, the template set of the area to be detected that introduces the retrieval area threshold is used for positioning, including: Set Template Collection and its corresponding position coordinate set , for each location Defines the coordinates of the upper left corner and lower right corner of the rectangular area; According to the position coordinate set , limiting the global search to a local area , and its position coordinate set after position adjustment , where the adjustment rule is ; Only in the reduced area Execute on and The pixel-level convolution calculation between Calculate the convolution score and get each The highest score in ; Through the conversion function , map the coordinates of the highest scoring point back to the original image coordinate system to obtain the final accurate matching position ,in, , , The coordinates of the highest scoring point are mapped back to the upper left corner coordinates and the lower right corner coordinates of the original image coordinate system.

3. The method according to claim 1, characterized in that: If it is an irregular area, first frame the minimum bounding rectangle and then use the pre-trained deep learning segmentation model to perform deep segmentation for pixel-level positioning, including: The determined minimum bounding rectangle area set As the input of the deep learning segmentation model, the updated region set is obtained after being processed by the deep learning segmentation model , each Described as a set of coordinate points , wherein the method for determining the minimum circumscribed rectangle is the same as the method for determining the regular rectangular area.

4. The method according to claim 1, characterized in that: Perform dirt detection on the dirt detection area and quantify the dirt detection results, including: The regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation are horizontally spliced ​​to obtain a set of dirt detection areas; Based on the set of dirt detection areas, deep recognition and refined classification are performed to obtain accurate recognition and classification of suspected dirt areas; The suspected dirty area is subjected to refined post-processing and indicator quantification.

5. The method according to claim 4, characterized in that The regular rectangular dirt detection area and the irregular rectangular dirt detection area obtained by depth segmentation are horizontally spliced ​​to obtain a set of dirt detection areas, including: Set the regular rectangular dirt detection area Set of irregular rectangular dirt detection areas Merge to form a preliminary comprehensive regional collection ; use Function, according to the size of the horizontal coordinate x, the preliminary comprehensive area set is sorted in ascending order to ensure that the order of each area is consistent with its horizontal position in the original image, and the sorted set is obtained ; Through horizontal splicing operation Function, the sorted collection All the areas in the horizontal direction are spliced ​​to generate a unified and positionally ordered set of the dirt detection areas. .

6. The method according to claim 5, characterized in that Based on the set of dirt detection areas, deep recognition and refined classification are performed to obtain accurate recognition and classification of suspected dirt areas, including: Using pre-trained recognition neural network functions , for the dirt detection area set Perform in-depth analysis and output a set of suspected dirty areas , where each element Zn contains the dirt type identifier and the coordinate information of the dirty area, .

7. The method according to claim 6, characterized in that The suspected dirty area is subjected to refined post-processing and index quantification, including: For the suspected dirty area collection Identification according to the type of soiling , through intelligent functions Dynamically calculate adaptive thresholds ; Based on the calculated adaptive threshold , using the binarization function The suspected dirty area is collected Each area in is segmented to generate a binary image, and the final output is a set of precise dirty areas. , where each element Corresponds to The Zn region in .

8. The method according to claim 1, characterized in that Carry out glue line detection in the glue line detection area and quantify glue lines of different shapes, including: The regular rectangular glue line detection area and the irregular rectangular glue line detection area obtained by depth segmentation are horizontally spliced ​​to obtain a glue line detection area set; An advanced segmentation network is used to perform image processing on the set of glue line detection areas and output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas.

9. The method according to claim 8, characterized in that The image of the glue line detection area set is processed using an advanced segmentation network to output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas, including: For long straight glue lines, the specific area selection and preprocessing steps, the minimum circumscribed rectangle and contour acquisition steps, and the glue width quantization steps are performed successively to achieve quantization of the long straight glue lines, wherein a mask of the glue line area is applied in the process of the specific area selection, and the preprocessing includes thresholding processing and closing operation processing.

10. The method according to claim 8, characterized in that The image of the glue line detection area set is processed using an advanced segmentation network to output a mask of the glue line area. The segmentation network automatically identifies and highlights all glue line areas, including: For the circular glue line, a specific area selection and preprocessing step and a circular glue line quantization step are performed successively, wherein a mask of the glue line area is applied in the process of the specific area selection, and the preprocessing includes thresholding processing and closing operation processing.

11. A module lens dirt detection and glue line detection device, characterized in that: include: A classification unit is used to perform an image classification step: classify the input image to obtain a classified image, wherein the classified image includes a glue line category image and a dirt category image; A region positioning unit is used to perform the detection region positioning step: positioning the detection region of the classified image, if it is a regular rectangular region, positioning it by applying a template set of the region to be detected that introduces a retrieval region threshold, and if it is an irregular region, first framing the minimum bounding rectangle and then performing deep segmentation using a pre-trained deep learning segmentation model to perform pixel-level positioning; A dirt detection unit, used to perform a dirt detection step: performing dirt detection on a dirt detection area and quantifying the dirt detection result; The glue line detection unit is used to perform the glue line detection step: perform glue line detection on the glue line detection area and quantify glue lines of different shapes.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the module lens dirt detection and glue line detection method according to any one of claims 1 to 10.

13. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing the module lens dirt detection and glue line detection method described in any one of claims 1 to 10.

Images