Character defect adaptability card control method

Abstract

The invention discloses a character defect adaptive control method, which comprises the following steps of: inputting a template image, and identifying character pixels through threshold segmentation; creating card control parameters, adjusting numerical values, and generating character objects; creating a character model, performing template matching to obtain a single template image, and optimizing the character model; configuring a card control strategy; collecting an input image; obtaining an OK image or an NG image through a template matching algorithm; content defect detection is carried out on the NG image, a defect image is obtained through gray threshold segmentation, and the defect type is judged according to card control parameters. Through the above mode, the character defect adaptive control method provided by the invention aims at the defect of poor detection effect of an existing surface defect detection algorithm, the surface characters or patterns of a product are divided into a plurality of single templates, then template matching is performed, and parameters are adjusted according to different defect requirements of each character, so that the detection effect of the character defect adaptive control method is improved. Therefore, the effect of accurate detection is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing methods, and in particular to a character defect adaptive card control method. Background Art

[0002] Existing matching defect detection algorithms are generally based on template matching methods, which match the input image with a pre-designed template, find the area similar to the template through similarity scoring, and further analyze the matching results by combining threshold setting and morphological processing methods. This enables the detection of product surface defects. However, it is easily affected by factors such as lighting, noise, and complex defect shapes, resulting in poor detection results. Summary of the invention

[0003] The main technical problem solved by the present invention is to provide a character defect adaptive card control method. For defects that are not well detected by the existing surface defect detection algorithm, the characters or patterns on the surface of the product are divided into several single templates, and then template matching is performed. According to the different defect requirements of each character, the parameters are adjusted to achieve the effect of accurate detection.

[0004] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a character defect adaptive card control method, comprising the following steps:

[0005] S1, character modeling:

[0006] S1.1, input the template image, perform smoothing through mask operation, and generate a grayscale image;

[0007] S1.2, threshold segmentation, adjust the gray threshold range and form the foreground and background areas, calculate the gray difference between the areas to identify the character pixels;

[0008] S1.3, creating card control parameters for the characters, adaptively adjusting the card control parameters, and generating character objects; the card control parameters include but are not limited to the printing thickness value of the character pixels, and the printing thickness value is defined by the pixel expansion operation;

[0009] S1.4, creating a character model based on the character object, and performing template matching on the grayscale image to obtain a single template image. If the matched single template image does not meet the card control expectations, repeat operation S1.3 to optimize the character model;

[0010] S2, Defect Detection:

[0011] S2.1, configuring a card control strategy, which includes but is not limited to screening of template matching scores;

[0012] S2.2, collect the input image, perform smoothing through mask operation, and generate a grayscale image;

[0013] S2.3, obtaining an OK image or a NG image through a template matching algorithm;

[0014] S2.4.1. Perform content defect detection on the NG image, remove the single template image pixels from the NG image pixels to obtain difference pixels, i.e., content defect pixels; use the grayscale threshold to segment the content defect pixels and the surrounding excessive pixels to obtain a defective image, and determine the defect type based on the card control parameters. The card control parameters include but are not limited to the minimum length comparison, minimum width comparison, minimum grayscale comparison, and minimum area comparison of the defective image.

[0015] In a preferred embodiment of the present invention, the single template image of S1.4 includes a predefined MARK point template image and several character template images. After the template matching operation, the character template image carries template offset information about the relative position relationship of the MARK point template image. The template offset information is based on the center point coordinates of each image, including x-axis offset, y-axis offset, and θ angle offset.

[0016] In a preferred embodiment of the present invention, in S1.3, the adaptive numerical adjustment includes adjusting the number of dilated pixels used to generate the MARK point template image, and the number of dilated pixels used to generate the MARK point template image is greater than the number of dilated pixels used to generate the character template image.

[0017] In a preferred embodiment of the present invention, the S1.1 further includes planning of the ROI region of interest before smoothing, and there are at least two sets of template images with different fonts, and each set of template images learns S1 once.

[0018] In a preferred embodiment of the present invention, the OK image of S2.3 carries a matching score and input offset information after template matching. The input offset information is based on the center point coordinates of each image, including an x-axis offset, a y-axis offset, and a θ angle offset.

[0019] In a preferred embodiment of the present invention, the S2 also includes S2.4.2, which performs position offset defect detection on the OK image: taking the absolute value of the difference between the "input offset" and the "template offset" to obtain the offset difference; the S2.1 card control strategy also includes a screening operation of the offset difference, and according to the card control strategy, the offset NG image is screened out from the OK image, and the offset NG image is affine transformed back to the input image and marked.

[0020] The beneficial effects of the present invention are as follows: the character defect adaptive card control method provided by the present invention aims at defects that are poorly detected by existing surface defect detection algorithms. The characters or patterns on the surface of the product are divided into several single templates, and then template matching is performed. Parameters are adjusted according to the different defect requirements of each character, thereby achieving the effect of accurate detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0022] Figure 1 It is a schematic diagram of the card control change from the original form of the input image to the difference pixels matched by the template;

[0023] Figure 2 It is a schematic diagram of multiple printing defects;

[0024] Figure 3 It is a schematic diagram of missing printing defects;

[0025] Figure 4 is a schematic diagram of contrast defects;

[0026] Figure 5 It is a schematic diagram of generating a grayscale image;

[0027] Figure 6 This is the effect picture after the foreground and background are separated;

[0028] Figure 7 It is a schematic diagram of two printing directions. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figure 1-7 As shown, the embodiment of the present invention includes:

[0031] A character defect adaptive card control method comprises the following steps:

[0032] S1, character modeling:

[0033] S1.1, input the template image, perform smoothing through mask operation, and generate a grayscale image;

[0034] S1.2, threshold segmentation, adjust the gray threshold range and form the foreground and background areas, calculate the gray difference between the areas to identify the character pixels;

[0035] S1.3, creating card control parameters for the characters, adaptively adjusting the card control parameters, and generating character objects; the card control parameters include but are not limited to the printing thickness value of the character pixels, and the printing thickness value is defined by the pixel expansion operation;

[0036] S1.4, creating a character model based on the character object, and performing template matching on the grayscale image to obtain a single template image. If the matched single template image does not meet the card control expectations, repeat operation S1.3 to optimize the character model;

[0037] S2, Defect Detection:

[0038] S2.1, configuring a card control strategy, which includes but is not limited to screening of template matching scores;

[0039] S2.2, collect the input image, perform smoothing through mask operation, and generate a grayscale image;

[0040] S2.3, obtaining an OK image or a NG image through a template matching algorithm;

[0041] S2.4.1. Perform content defect detection on the NG image, remove the single template image pixels from the NG image pixels to obtain difference pixels, i.e., content defect pixels; use the grayscale threshold to segment the content defect pixels and the surrounding excessive pixels to obtain a defective image, and determine the defect type based on the card control parameters. The card control parameters include but are not limited to the minimum length comparison, minimum width comparison, minimum grayscale comparison, and minimum area comparison of the defective image.

[0042] Among them, the single template image of S1.4 includes a predefined MARK point template image and several character template images. After the template matching operation, the character template image carries template offset information about the relative position relationship of the MARK point template image. The template offset information is based on the center point coordinates of each image, including x-axis offset, y-axis offset, and θ angle offset.

[0043] In addition, in S1.3, the adaptive numerical adjustment includes adjusting the number of dilated pixels used to generate the MARK point template image, and the number of dilated pixels used to generate the MARK point template image is greater than the number of dilated pixels used to generate the character template image.

[0044] Furthermore, the S1.1 also includes planning of the ROI region of interest before smoothing. There are at least two sets of template images with different fonts, and each set of template images learns S1 once.

[0045] On the other hand, the OK image of S2.3 carries a matching score and input offset information after template matching. The input offset information is based on the center point coordinates of each image, including an x-axis offset, a y-axis offset, and a θ angle offset.

[0046] At the same time, the S2 also includes S2.4.2, which performs position offset defect detection on the OK image: taking the absolute value of the difference between the "input offset" and the "template offset" to obtain the offset difference; the S2.1 card control strategy also includes a screening operation of the offset difference, and according to the card control strategy, the offset NG image is screened out from the OK image, and the offset NG image is affine transformed back to the input image and marked.

[0047] The present invention relates to chip printing defect detection technology, including a detection method for chip surface character defects, mainly matching, analyzing and classifying chip surface character images. This solution involves image processing, pattern recognition, matching algorithms and other related fields, and can be applied to surface defect detection and quality control in industries such as electronic manufacturing and semiconductor production.

[0048] Existing matching defect detection algorithms are generally based on template matching methods, which match the input image with a pre-designed template, find the area similar to the template through similarity scoring, and further analyze the matching results by combining threshold setting and morphological processing methods. This enables the detection of product surface defects. However, it is easily affected by factors such as lighting, noise, and complex defect shapes, resulting in poor detection results.

[0049] like Figure 2 The above figure shows the overprint defect (micro defect). Overprint indicates the quality of the area around the printed characters. It usually indicates blurred characters, overprinting and extra spots. Overprint can be detected within the character ROI. The unit of measurement is area, i.e. the number of pixels. In general, ordinary detection technology will cause a large number of false detections. And such false detections cannot be adaptively controlled according to the severity of the defect.

[0050] like Figure 3 The figure shows a missing print defect (minor defect). In the screen printing of the chip surface, the missing print defect refers to a blank space in the middle of the printed pattern or text, and the printing is not complete. This defect is usually caused by an incomplete pattern on the printing plate or screen or a shallow engraving. It is easy to appear when the printing pressure is insufficient or the screen has a long service life. In general, ordinary detection technology will cause a large number of false detections. And this false detection cannot be adaptively controlled according to the severity of the defect.

[0051] like Figure 4The figure shows a contrast defect (micro defect). In the screen printing of the chip surface, the contrast defect of printing mainly refers to the insufficient contrast between the printed characters and the background of the chip surface. This defect may be manifested as the color of the characters being too close to the background color, or the color of the characters being different in depth and blurred, making it difficult to accurately identify the character information. In general, ordinary detection technology will cause a large number of false detections. And this false detection cannot be adaptively controlled according to the severity of the defect.

[0052] The defect detection method of the present invention is divided into two parts: modeling and detection. The user provides a template image and models the template image; batches of input images are collected to perform defect detection.

[0053] The first step is to input the template image, define the ROI area, perform smoothing through mask operation, and generate a grayscale image. Figure 5 .

[0054] The second step is threshold segmentation, judging black text on a white background or white text on a black background, and obtaining the foreground background:

[0055] Specific methods:

[0056] Adjust the grayscale threshold range of the grayscale image (pull grayscale) to obtain two pixel areas a and b with obvious grayscale differences. The method for determining obvious grayscale differences is:

[0057] Calculate the grayscale mean of pixels in area a and area b respectively.

[0058] is the white foreground pixel area, and b is the black background pixel area ( Figure 6 )

[0059] (x is the grayscale difference between the foreground and background. For example, a grayscale value of 30 units different from b can distinguish the foreground from the background.)

[0060] (For example, due to interference from lighting factors, the grayscale stretching operation cannot effectively separate two pixel areas a and b with obvious grayscale differences)

[0061] The third step is to remove small white dots after setting the grayscale threshold in the previous step. Then create a character object, inject parameters, and define the threshold conditions required by the template matching algorithm for each parameter.

[0062] For example:

[0063] like Figure 7 As shown in the figure, different printing directions may appear in actual production, which will affect the determination of threshold conditions. Therefore, when creating character objects, the following parameters should be predefined to prevent BUGs and significantly improve card control accuracy:

[0064] Printing direction: horizontal / vertical;

[0065] Character color: white / black (foreground grayscale);

[0066] Minimum character width: m1;

[0067] Minimum character height: m2.

[0068] In addition, other parameters include:

[0069] Search range: defines the specific dilation pixel value

[0070] Matching score: defines the screening results of template matching

[0071] Step 4:

[0072] Create a character model using the character object: (one model is created for each character)

[0073] According to the character model, a template matching algorithm is used to perform template matching on the pixel content in the white character foreground pixel area to obtain a single template image (one character obtains a single template). Each single template image carries the offset information of the character based on the mark point position (xyθ offset from center point to center point) (referred to as template offset), as well as matching score information. In the modeling process, the accuracy of the character model can be verified through the above template matching operation to prevent errors in the subsequent actual detection process. If the matching score is lower than 0.8 (full score 1), it is determined that the character matching has failed and an error is reported, and re-modeling is required. In the actual detection process, when the single template image obtained by modeling is used to perform template matching on the input image, if the matching score is lower than 0.8, it means that the character has not been detected and there is a missing print defect.

[0074] In order to quickly find the mark point and determine the reference position in the first step of template matching, when modeling, when setting the matching threshold conditions of the mark point separately, the search range value of the mark point is expanded (the number of expanded pixels is expanded from 3 pixels of ordinary characters to 55 pixels), which can improve the recognition sensitivity of the mark point, quickly find the area of ​​the mark point template image, define the pixels in the area, and define it as the mark point. After the mark point is obtained, the offset from the mark point to each character is calculated based on the coordinates of the mark point, and the offset is written into each character model.

[0075] After the above steps, a set of templates is obtained. In order to further solve the problem that the same chip has different printed fonts, resulting in template matching failure and being detected as NG, multiple sets of templates need to be established. In order to solve the problem of fast modeling of different fonts for the same product, the same set of ROI area and threshold segmentation parameters can be used to process the spare template images and generate multiple sets of single template images. In this way, when the font changes due to chip batch changes, defect detection of chips from different batches can be automatically compatible.

[0076] After introducing the modeling method, the following introduces the specific method of defect detection. Defect detection is divided into defect detection of character content and defect detection of character position. Before starting the detection, set the template matching parameters first, which determines whether the input image will be judged as NG under the tolerance condition. For example, set the defect score threshold, and it is NG if it is less than the threshold. For example, set the offset tolerance threshold, and it is NG if it is greater than the threshold.

[0077] The detection process is implemented using the following basic principles:

[0078] In chip surface defect detection, parameters are set separately according to the different defect requirements of each character. In multiple printing and missed printing detection, printing thickness adjustment method and threshold sensitivity control method are introduced to ensure that each character meets different defect requirements.

[0079] Take the multiple printing defect as an example:

[0080] Although there are multiple printing defects in the characters shown in the figure, the degree of the defect is still within a tolerable range (they can be normally identified as the numbers 0 and F in the figure).

[0081] In order to relax the defect recognition strategy for certain specified characters, parameters for controlling tolerance are introduced here - printing thickness and threshold sensitivity. By adjusting the printing thickness and threshold sensitivity, the image can be processed as normal under the condition of "not meeting the defect requirements".

[0082] On the contrary, if defects are judged by the template matching method adopted by the prior art for the entire input image, since the template matching method cannot adjust the tolerance for a specific character separately, the template image obviously does not match the input image due to multiple printing defects, so the entire input image will be judged as a defective image (NG-OP), but this input image can actually be identified as OK, which causes a false detection.

[0083] Based on the above ideas, the beneficial effect that can be achieved is: the rules can be adjusted flexibly and conveniently according to the actual situation and adapted to local conditions, thereby reducing false detections.

[0084] The specific method is:

[0085] Image preprocessing, input image, generate grayscale image according to smoothing coefficient (same as modeling), and denoise. Then template match reference point to find the position of mark reference point. Then use mark point as reference to make each character model do template matching at the correct position. Regarding printing thickness processing, it means to expand x pixels, input image-expanded template to get a batch of difference pixels. Before the next threshold segmentation processing, for the difference pixels obtained above, the threshold sensitivity processing method is used here to automatically generate grayscale range value, and then determine the grayscale range of the surrounding excessive pixels. The method is: before template matching, first judge the color of the character (black spots on white background or white characters on black background).

[0086] like Figure 1 As shown in the figure, ① the input image to be tested; ② the template image; ③ the template image after dilation (erosion) of n pixels; ④ the image obtained by subtracting the dilated (eroded) template image from the image to be tested; ⑤ the area obtained by subtracting the dilated (eroded) template image from the image to be tested. During detection, the dilated template image is used to perform template matching with the input image. Then, by setting thresholds, including area, length and width, these pixels are blocked so that they cannot be judged as defects (this part of the defect is tolerable). In this way, an OK image is obtained.

[0087] When the image content is OK, offset detection is still required for the position. After template matching, the actual x / y / θ offset (referred to as input offset) and matching score of each character on the input image will be obtained. Calculate the offset difference: take the absolute value of the difference between [input offset] and [template offset], and output the judgment conclusion based on the template matching parameters to determine whether there is an offset defect. When an offset defect occurs, in order to accurately report the defect and fully present the defective character image content, the defective image that deviates from the ROI must be affine transformed back to the input image.

[0088] When the image content is NG, it is necessary to further identify the NG type. First, the pixels within the grayscale threshold range of the expanded template should be removed from the pixels within the grayscale threshold range of the input image, so that the difference pixels are obtained.

[0089] Then, threshold segmentation is performed on these difference pixels. The specific method is to perform threshold segmentation on the difference pixel with a grayscale of 255 together with the surrounding excessive pixel points with a grayscale value range of 220 to 254 to segment the difference image.

[0090] Then determine the defect type:

[0091] The user needs to preset parameters that are used to determine whether the difference image is a defect, for example:

[0092] Threshold MAX / MIN threshold segmentation range

[0093] Union distance expansion scale

[0094] miniLength(mm) Minimum length

[0095] MinimumArea(mm^2) minimum area

[0096] miniWidth(mm) minimum width

[0097] Minimum contrast minimum grayscale

[0098] wait

[0099] Based on the above parameters, the difference images are compared one by one. When any indicator of the difference image exceeds the preset parameter value, it will be judged as a defect. When it exceeds the minimum length, it will be identified as a scratch.

[0100] As shown in ⑤, when the actual length of the defect image exceeds miniLength (the minimum tolerable length), a defect is found, and the defect can be determined as a scratch. When the actual width of the defect image exceeds miniWidth (the minimum tolerable width), a defect is found, and the defect can be determined as a stain / reflection. If the length and width are both OK, the actual area will be compared. If the actual defect area is larger than miniArea, a defect will also be found, and the defect can be determined as an overprint defect. The smaller the parameter value, the higher the sensitivity of defect detection and the greater the number of defects. Users can freely adjust the sensitivity of defect detection according to actual production needs. The above is the threshold sensitivity method.

[0101] The beneficial effects of the present invention are:

[0102] In order to solve the defects that the existing surface defect detection algorithm has poor detection effect, the characters or patterns on the product surface are divided into several single templates, and then template matching is performed. According to the different defect requirements of each character, the parameters are adjusted to achieve accurate detection.

[0103] Using small templates of single characters to match characters one by one can achieve higher accuracy than the traditional method of matching all characters at the same time.

[0104] For a single template: you can set matching conditions for individual characters (for example, define a search range (expanded number) for each character), so that the template matching algorithm can be more accurate and reliable, and reduce false positives. When a character has a high false positive rate, you can adjust the character model separately.

[0105] For multiple templates: accurate recognition can be achieved for different fonts.

[0106] In summary, the present invention provides a character defect adaptive card control method. For defects that are not well detected by existing surface defect detection algorithms, the product surface characters or patterns are divided into several single templates, and then template matching is performed. Parameters are adjusted according to the different defect requirements of each character, so as to achieve accurate detection effects.

[0107] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A character defect adaptive card control method, characterized in that: The following steps are involved: S1, character modeling: S1.1, input the template image, perform smoothing through mask operation, and generate a grayscale image; S1.2, threshold segmentation, adjust the gray threshold range and form the foreground and background areas, calculate the gray difference between the areas to identify the character pixels; S1.3, creating card control parameters for the characters, making adaptive numerical adjustments to the card control parameters, and generating character objects; the card control parameters include but are not limited to the printing thickness value of the character pixels, and the printing thickness value is defined by the pixel expansion operation; S1.4, creating a character model based on the character object, and performing template matching on the grayscale image to obtain a single template image. If the matched single template image does not meet the card control expectations, repeat operation S1.3 to optimize the character model; S2, Defect Detection: S2.1, configuring a card control strategy, which includes but is not limited to screening of template matching scores; S2.2, collect the input image, perform smoothing through mask operation, and generate a grayscale image; S2.3, obtaining an OK image or a NG image through a template matching algorithm; S2.4.

1. Perform content defect detection on the NG image, remove the single template image pixels from the NG image pixels to obtain difference pixels, i.e., content defect pixels; use the grayscale threshold to segment the content defect pixels and the surrounding excessive pixels to obtain a defective image, and determine the defect type based on the card control parameters. The card control parameters include but are not limited to the minimum length comparison, minimum width comparison, minimum grayscale comparison, and minimum area comparison of the defective image.

2. The character defect adaptive card control method according to claim 1, characterized in that: The single template image of S1.4 includes a predefined MARK point template image and several character template images. After the template matching operation, the character template image carries template offset information about the relative position relationship of the MARK point template image. The template offset information is based on the center point coordinates of each image, including x-axis offset, y-axis offset, and θ angle offset.

3. The character defect adaptive card control method according to claim 2, characterized in that: In S1.3, the adaptive numerical adjustment includes adjusting the number of dilated pixels used to generate the MARK point template image, and the number of dilated pixels used to generate the MARK point template image is greater than the number of dilated pixels used to generate the character template image.

4. The character defect adaptive card control method according to claim 1, characterized in that: The S1.1 also includes the planning of the ROI region of interest before the smoothing process. There are at least two sets of template images with different fonts, and each set of template images learns S1 once.

5. The character defect adaptive card control method according to claim 1, characterized in that: The OK image of S2.3 carries a matching score and input offset information after template matching. The input offset information is based on the center point coordinates of each image, including an x-axis offset, a y-axis offset, and a θ angle offset.

6. The character defect adaptive card control method according to claim 5, characterized in that: The S2 also includes S2.4.2, performing position offset defect detection on the OK image: taking the absolute value of the difference between the "input offset" and the "template offset" to obtain the offset difference; the S2.1 card control strategy also includes a screening operation of the offset difference, screening out the offset NG image from the OK image according to the card control strategy, and affine transforming the offset NG image back to the input image and marking it.

Images