Intelligent card embossment surface defect detection method based on photometric stereo method

Abstract

The invention relates to an intelligent card embossment surface defect detection method based on a photometric stereo method, and belongs to the technical field of intelligent card surface defect detection. According to the method, multi-direction strip light is sequentially lightened to obtain a low-angle image, after edge position information is obtained through a photometric stereo method, the edge position information is combined with an image which is shot by coaxial light and has defect characteristics to extract an embossed area, then defect proportion quantification is carried out through a visual algorithm, and whether a current card is a defective piece or not is judged according to a preset threshold value of a client. The method effectively solves the problem of reflective positioning of a convex bright surface and surface detection of unknown characters of a bank card form, remarkably improves the accuracy of defect detection and the applicability of characters in a non-standard form, avoids poor quality such as gold stamping and silver stamping of a convex area in a card, and ensures the quality of the card delivered to a terminal client.

Description

Technical Field

[0001] The invention relates to the technical field of smart card embossed character surface defect detection, and in particular to a smart card embossed character surface defect detection method based on photometric stereo method, which is suitable for printing defect detection of embossed character bright surface areas of various smart cards. Background Art

[0002] Smart cards such as bank cards play an important role in modern financial transactions, where the embossed area is used to provide touch recognition and mechanical reading functions. In order to ensure the quality of bank cards delivered to customers, the quality of the embossed area on the card surface needs to be tested during the production process. At present, in the production process of smart cards, the hot stamping and silver stamping of the embossed area are prone to incompleteness, resulting in a lower yield rate of the card. The best solution is to add a quality inspection process during the production process, use an industrial camera with a light source to take photos of the embossed surface, and use a visual inspection algorithm to determine the defects of the embossed area.

[0003] However, the current visual detection algorithm has the following problems: 1. The current detection difficulty mainly lies in the difficulty of clearly imaging the edges of the embossed bright area and the non-standardization of the personalized shape of the embossed characters. Because the embossed characters have different degrees of curvature, their reflection angles vary under different lighting conditions and their brightness is uneven, making it difficult to capture complete and clear embossed characters; 2. Due to the lack of a unified production standard, it is impossible to predetermine the standard shape of the characters, so the customized algorithm has poor adaptability when processing a variety of characters, making it difficult to ensure the accuracy and consistency of the recognition effect; 3. The hot stamping in the embossed area has a very high glossiness, which can easily cause overexposure of the image under the illumination of the light source, and the surface features of the embossed characters cannot be recognized, making it difficult to clearly judge the edges of the characters and locate the embossed bright area. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting surface defects of embossed characters on smart cards based on photometric stereo method, which is easy to operate and reliable to use. The method aims to solve the problem that existing detection means cannot effectively distinguish whether the detected defects are real features of embossed characters on the card surface or surface flaws, significantly improve the accuracy of defect detection, and ensure the delivery quality of smart cards such as bank cards.

[0005] The technical solution of the present invention is:

[0006] A method for detecting surface defects of embossed characters on smart cards based on photometric stereo method is provided. The technical key points are to use photometric stereo method to determine the embossed character area on the card surface and extract the embossed character contour. The method specifically includes the following steps:

[0007] Step 1: Place the finished cards on the conveyor belt for transmission. When the cards are transported to the shooting position, the strip lights in four directions are lit in sequence, and four images under low-angle illumination are taken in sequence;

[0008] Step 2, using a coaxial light located directly above the card to capture a complete embossed glossy image;

[0009] Step 3, according to the relative height and relative angle between the four strip lights and the card, combined with the four low-angle images, four groups of light matrices corresponding to the card position of the light source are calculated, and stored in the matrix groups corresponding to 0°, 90°, 270°, and 360°, and all normal vectors of the four low-angle shooting images are extracted by calculating the inverse matrix of the light matrix, and the initial contour image of the embossed area in the card is obtained after normalization and morphological processing;

[0010] Step 4: Based on the initial contour image of the embossed area, classify according to the parent-child hierarchy of all contours to determine the type of character. If necessary, complete the hierarchical grouping by recursion, remove multiple sub-contour areas in the parent contour, calculate the accurate edge and create a mask area. Combined with the overall image taken with coaxial light, extract the image of the embossed area, retain only the bright surface area of ​​the target embossed area, and complete the complete extraction of the embossed area.

[0011] Step 5: Defect judgment is performed on the extracted embossed area. If the detected defect ratio exceeds the set threshold, the card is judged as a defective card and cannot be delivered to the customer.

[0012] In the above-mentioned smart card embossed surface defect detection method based on photometric stereo, in step 3, a metal ball is placed at the bank card position, and the light matrix is ​​calculated according to the relative position of the convergence point reflected on the ball surface and the center of the circle. The light angle calculation formula is as follows:

[0013]

[0014] Among them, m00 represents the area of ​​the region (zero-order moment), m10 and m01 represent the first-order moments in the x and y directions respectively, and radius represents the radius of the circular area of ​​the image. The values ​​of x, y, and z in the matrix are obtained by subtracting radius and dividing by radius for normalization.

[0015] In the above-mentioned smart card embossed surface defect detection method based on photometric stereo, in step 3, the inverse matrix of the light matrix is ​​calculated by singular value decomposition SVD. The light matrix of the four images is a matrix with one row and three columns, that is, According to Lights=U∑V T Calculate the orthogonal matrices U and V;

[0016] Specifically: Lights T Lights, calculate the eigenvalues ​​λ1, λ2, λ3 to form the matrix V, and then calculate the singular values ​​σ1, v2, v3 based on λ1, λ2, λ3, Form a diagonal matrix Next, calculate U, U = normalized (LightsV∑ -1 ), and finally calculate the inverse matrix LightsInv = V∑ -1 U T , summarize the light matrix calculated by the four images, perform inverse matrix calculation to get three rows and four columns of LightsInv, and multiply it with one row and four columns of pixel matrix on the four images to get the normal vector of each pixel (n x , n y , n z ), using the normalized n z The component approximately replaces the gradient vector in the Z direction, and the initial contour image of the embossed area in the card is obtained after image morphology processing.

[0017] In the above-mentioned method for detecting surface defects of embossed characters on smart cards based on photometric stereo, the strip light is used to photograph the convex edge at a low angle, and the coaxial light is used to photograph the surface morphology of the embossed area.

[0018] In the above-mentioned smart card embossed surface defect detection method based on photometric stereo, the four strip lights have the same relative height to the card, the same relative angle to the card, but different orientations, and the difference between two adjacent strip lights is 90°.

[0019] The above-mentioned smart card embossed character surface defect detection method based on photometric stereo method, for embossed characters with blank areas in the middle, recursively completes hierarchical grouping and eliminates multiple sub-contour areas in the parent contour.

[0020] The beneficial effects of the present invention are:

[0021] This method obtains low-angle images by lighting up multi-directional strip lights in sequence, obtains edge position information using photometric stereo, and then extracts the embossed area by combining it with the image of defective features taken with coaxial light. Then, the defect ratio is quantified through a visual algorithm, and the customer's preset threshold is used to determine whether the current card is a defective part. This method effectively solves the problem of reflective positioning on the bright surface of embossed characters and surface detection of unknown characters in bank card form, significantly improving the accuracy of defect detection and the applicability of non-standard characters, avoiding poor quality such as gold and silver hot stamping in the embossed area of ​​the card, thereby ensuring the quality of the cards delivered to end customers. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a detection flow chart of the present invention. DETAILED DESCRIPTION

[0023] The present invention is described in detail with reference to the accompanying drawings.

[0024] Applied to the defect detection of embossed characters on the card surface during the production process of cards with bright embossed characters.

[0025] The specific detection steps are as follows:

[0026] Step 1: Place the finished cards on the conveyor belt one by one for transmission. When the cards are transported to the shooting position, the strip lights in four directions are lit one by one to shoot four images under low-angle illumination.

[0027] Step 2, use the coaxial light located directly above the card to capture the complete embossed glossy image with defect features.

[0028] Step 3: Based on the relative heights and angles between the four strip lights and the card, and in combination with the four low-angle images, calculate the four light matrices corresponding to the card position. Place a metal ball at the bank card position and calculate the light matrix based on the relative position of the convergence point reflected by the ball surface and the center of the circle. The light angle calculation formulas are as follows: In the formula, m00 represents the area of ​​the region (zero-order moment), m10 and m01 represent the first-order moments in the x and y directions respectively, and radius is the radius of the circular area of ​​the image. Normalization is performed by subtracting radius and dividing by radius to obtain the values ​​of x, y, and z in the matrix.

[0029] The four groups of light matrices are stored in the matrix groups corresponding to 0°, 90°, 270°, and 360°. After calculating the inverse matrix of the light matrix, all the normal vectors of the four low-angle images are extracted.

[0030] The inverse matrix of the light matrix is ​​calculated by SVD (singular value decomposition). The light matrix of the four images is a matrix with one row and three columns. According to Lights=U∑V T Calculate the orthogonal matrices U and V. The steps are: Lights T Lights, calculate the eigenvalues ​​λ1, λ2, λ3 to form the matrix V, and then calculate the singular values ​​σ1, σ2, σs based on λ1, λ2, λ3, Form a diagonal matrix Next, calculate U, U = normalized (LightsV∑ -1 ), and finally calculate the inverse matrix LightsInv = V∑ -1 U T , summarize the light matrix calculated by the four images, perform inverse matrix calculation to get three rows and four columns of LightsInv, and multiply it with one row and four columns of pixel matrix on the four images to get the normal vector of each pixel (n x , n y , nz ), using the normalized n z The component approximately replaces the gradient vector in the Z direction, and the initial contour image of the embossed area in the card is obtained after image morphology processing.

[0031] Step 4: Based on the initial contour image of the embossed area, classify according to the parent-child hierarchy of all contours to determine the type of character. For example, for characters with blank areas in the middle such as the number "8" and the English word "B", it is also necessary to complete the hierarchical grouping through recursion, eliminate multiple sub-contour areas in the parent contour, calculate the accurate edge and create a mask area. Combined with the overall image taken with coaxial light, the image of the embossed area is extracted, and only the bright area position of the target embossed character is retained.

[0032] Step 5: Use the defect detection algorithm to determine the defects of the extracted embossed area. If the detected defect ratio exceeds the set threshold, the card is determined to be a defective card and cannot be delivered to the customer. The algorithm uses photometric stereo combined with multi-angle illumination image processing to obtain an image of the embossed bright area, and completes the defect program determination of the bright area through a visual algorithm based on the surface characteristics of the characters.

[0033] Step 6: sort the abnormal cards that cannot be delivered to customers into the waste area, and feed back their corresponding card numbers to the production database for record as a basis for re-production. The qualified cards enter the packaging process at the end of the conveyor belt.

[0034] In the present invention, the two shooting light sources are a strip light source and a coaxial light source. The strip light is used to shoot raised edges at a low angle, and the coaxial light is used to shoot the surface morphology of embossed areas such as gold and silver stamping.

[0035] The embodiments of the present invention are described in detail above, but the contents described are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A method for detecting surface defects of embossed characters on smart cards based on photometric stereo, characterized in that: The photometric stereo method is used to determine the embossed area on the card surface and extract the embossed outline, which specifically includes the following steps: Step 1: Place the finished cards on the conveyor belt for transmission. When the cards are transported to the shooting position, the strip lights in four directions are lit in sequence, and four images under low-angle illumination are taken in sequence; Step 2, using a coaxial light located directly above the card to capture a complete embossed glossy image; Step 3, according to the relative height and relative angle between the four strip lights and the card, combined with the four low-angle images, four groups of light matrices corresponding to the card position of the light source are calculated, and stored in the matrix groups corresponding to 0°, 90°, 270°, and 360°, and all normal vectors of the four low-angle shooting images are extracted by calculating the inverse matrix of the light matrix, and the initial contour image of the embossed area in the card is obtained after normalization and morphological processing; Step 4: Based on the initial contour image of the embossed area, classify according to the parent-child hierarchy of all contours to determine the type of character. If necessary, complete the hierarchical grouping by recursion, remove multiple sub-contour areas in the parent contour, calculate the accurate edge and create a mask area. Combined with the overall image taken with coaxial light, extract the image of the embossed area, retain only the bright surface area of ​​the target embossed area, and complete the complete extraction of the embossed area. Step 5: Defect judgment is performed on the extracted embossed area. If the detected defect ratio exceeds the set threshold, the card is judged as a defective card and cannot be delivered to the customer.

2. The method for detecting surface defects of embossed characters on smart cards based on photometric stereo method according to claim 1, characterized in that: In step 3, a metal ball is placed at the bank card position, and the light matrix is ​​calculated based on the relative position of the convergence point reflected by the ball surface and the center of the circle. The light angle calculation formula is: Among them, m00 represents the area of ​​the region (zero-order moment), m10 and m01 represent the first-order moments in the x and y directions respectively, and radius represents the radius of the circular area of ​​the image. The values ​​of x, y, and z in the matrix are obtained by subtracting radius and dividing by radius for normalization.

3. The method for detecting surface defects of embossed characters on smart cards based on photometric stereo method according to claim 1, characterized in that: In step 3, the inverse matrix of the light matrix is ​​calculated by singular value decomposition (SVD). The light matrix of the four images is a matrix with one row and three columns, that is, According to Lights=U∑V T Calculate the orthogonal matrices U and V; Specifically: Lights T Lights, calculate the eigenvalues ​​λ1, λ2, λ3 to form the matrix V, and then calculate the singular values ​​σ1, σ2, σ3 based on λ1, λ2, λ3, Form a diagonal matrix Next, calculate U, U = normalized (LightsV∑ -1 ), and finally calculate the inverse matrix LightsInv = V∑ -1 U T , summarize the light matrix calculated by the four images, perform inverse matrix calculation to get three rows and four columns of LightsInv, and multiply it with one row and four columns of pixel matrix on the four images to get the normal vector of each pixel (n x , n y , n z ), using the normalized n z The component approximately replaces the gradient vector in the Z direction, and the initial contour image of the embossed area in the card is obtained after image morphology processing.

4. The method for detecting surface defects of embossed characters on smart cards based on photometric stereo method according to claim 1, characterized in that: The strip light is used for photographing the raised edge at a low angle, and the coaxial light is used for photographing the surface morphology of the embossed area.

5. The method for detecting surface defects of embossed characters on smart cards based on photometric stereo method according to claim 1, characterized in that: The four strip lights have the same relative height to the card, the same relative angle to the card, but different orientations, and the difference between two adjacent strip lights is 90°.

6. The method for detecting surface defects of embossed characters on smart cards based on photometric stereo method according to claim 1, characterized in that: For convex characters with blank areas in the middle, hierarchical grouping is completed recursively to eliminate multiple sub-contour areas in the parent contour.

Images