A method for segmenting and recognizing characters adhered to a surface of a part

Abstract

The application discloses a part surface adhesion character segmentation and recognition method, and belongs to the field of industrial recognition and image processing. The application analyzes the part surface adhesion character, innovatively applies image processing technology and artificial intelligence technology to the industrial character recognition scene, and successfully completes segmentation and recognition of the part surface adhesion character. The application firstly analyzes the adhesion point position of the adhesion character by using a graph model algorithm, then finds out the segmentation path of the character by using an improved drip algorithm, and finally performs character recognition by using a simplified VGG16 network. The method can accurately locate the position of the adhesion point of the adhesion character, can find out a suitable segmentation path to prevent over segmentation of the character, can accurately recognize the single character after segmentation, and solves the problem of low recognition accuracy of the part surface adhesion character.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial identification and image processing, and particularly relates to a segmentation and identification method for characters adhered on the surface of a part. BACKGROUND

[0002] In the industrial production process, characters on the surface of a part are very common, because many parts are cast, so the characters on the surface of the part are prone to adhesion; the parameters on the surface of the part include strength grade information, model number and batch number. These parameters not only provide a basis for intelligent assembly of the part in the early stage, but also provide convenience for a series of work such as product tracking, quality control, recall management and the like in the later stage. How to accurately identify the adhered characters on the surface of the part is very important.

[0003] In the existing technology, the identification of the adhered characters on the surface of the part is mainly divided into two kinds, a traditional image processing method and a deep learning method. The traditional image processing method includes image preprocessing, character positioning, character segmentation and character recognition. All traditional algorithms have been very widely applied in the industrial field due to their good effect in a specific scene and high explainability and the like.

[0004] The deep learning method has also achieved good results in the field of character recognition in recent years due to its strong adaptability, convenient operation and the like. However, deep learning needs a large amount of original data for training, and if the data volume is too small, the model is prone to overfitting; and the adhered characters on the surface of the part have too many types, some of which are single-point adhesion, some of which are multi-point adhesion, some of which are single-character adhesion, and some of which are multi-character adhesion; and all of them are not ideal in effect when the deep learning method is directly used for the adhered characters on the surface of the part.

[0005] Therefore, it is planned to combine the traditional image processing algorithm and the deep learning method to realize the identification of the adhered characters on the surface of the part; first, the traditional image processing algorithm is used to segment the adhered characters into single characters, and then the deep learning algorithm is used to identify the single characters; so as to solve the problems of great difficulty in identification of the adhered characters on the surface of the part and low identification accuracy. SUMMARY

[0006] The present application provides a segmentation and identification method for characters adhered on the surface of a part. The method mainly solves the problem of low identification accuracy of the adhered characters on the surface of the part in the prior art method. The technical problem proposed in the present application is solved as follows:

[0007] A segmentation and identification method for characters adhered on the surface of a part, comprising the following steps:

[0008] Step 1, first find the adhesion point of the adhered characters, and the detailed steps can be divided into the following 8 steps;

[0009] Step 1-1, first convert the character image into a binary image, and use the connected domain feature to extract each block of characters, and use Z n to represent;

[0010] Step 1-2, Z n is skeletonized, and the positions of all straight lines are initially considered to represent adhesion; if there is no straight line in the skeleton, it is considered that there is no adhesion;

[0011] Step 1-3, determine whether all straight lines in the skeleton are true adhesion points, and use a graph model to analyze the complexity of the skeleton structure; the skeleton is considered to be a graph G, and the intersection points and end points in the skeleton correspond to branch points and end points in the graph G;

[0012] Step 1-4, add nodes to the graph G, rather than how they are connected; in order to reflect the connection state of each point in the graph G, draw the skeleton structure on the graph G and perform incremental point operation; starting from the end point of the skeleton, use the pixels of the skeleton to fit the skeleton into multiple straight lines; the angle between the straight lines is made to be Ω; if Ω is less than 135 degrees, it is considered that the skeleton has a turning point, and a new point is added at the turning point;

[0013] Step 1-5, then perform "noise removal operation" on the graph model G after adding points, because there are many short branches in the extracted skeleton, which are caused by the burrs of the characters; the end point deletion operation of the graph G is used to delete all end points to avoid interference with the character segmentation points;

[0014] Step 1-6, determine whether the known straight line is a true adhesion point, and each straight line is sequentially determined from left to right; use the midline of the straight line to split the skeleton graph G to generate two new graphs G1 and G2;

[0015] Step 1-7, then analyze the complexity of G1 and G2 according to Table 1; if the complexity of the two graphs is 1, then the cut line corresponds to the adhesion skeleton; all straight lines are sequentially analyzed, and if the previous straight line is determined to represent adhesion, then G1 on the left side of the straight line will not participate in the subsequent complexity analysis;

[0016] Step 1-8, the coordinates of the median of the corresponding straight line at the adhesion point are recorded as the initial position of the adhesion character segmentation line;

[0017] Table 1 Skeleton complexity determination method

[0018] G1 (G2) skeleton structure Complexity One crossing and one supplemental point 1 One loop 1 Two crossing points 1

[0019] Step 2, according to the found coordinates of the sticking point, find the starting point of the segmentation, use the water drop algorithm to simulate the rolling of the water drop, and segment the character on the rolling path of the water drop, optimize the segmentation path rule of the character; After entering the recessed area, the simulated liquid drop will not immediately choose a breakthrough at the recessed place, but will store the liquid drop; Instead, generate another new water drop from the drop point, execute the same path selection algorithm to reach the depression, store the water drop again, and then choose a breakthrough when the stored water amount reaches a certain threshold; If the amount of water stored in the depression does not reach the threshold, and the water drop finds a new depression, it can be determined that the previous depression does not meet the segmentation condition; The specific implementation steps are divided into the following 3 steps:

[0020] Step 2-1, set the character image as I, whose size is N*M; Use n j to represent the pixel value around the water drop, let (x i ,y i ) be the coordinate position of the water drop point, and T be the water drop path:

[0021] T(x i+1 , y i+1 ) = t(x i , y i , W i ) i = 0, 1, 2,...

[0022] Where (x (i+1) , y (i+1) ) is the coordinate of the next drop point, (x0, y0) represents the starting drop point, and the coordinate of the starting point of the water drop is the coordinate of the median of the sticking point found in step 1; W i is the gravitational potential energy of the water drop at the current position, which is represented as:

[0023]

[0024] Where v j represents the pixel value of n j points, 0 and 1 represent black and white respectively, m j represents the weight of the n j point selected as the next drop point, m j = 6 - j;

[0025]

[0026] Step 2-2, the traditional algorithm rolls left and right when it encounters a water pit, and finally chooses to drop vertically downward on the right side of the recess; The improved water drop algorithm records the area of the recessed area and filters the smaller recessed areas;

[0027] Steps 2-3: Use the segmentation path to separate the connected characters into individual characters for subsequent recognition;

[0028] Step 3: Use the VGG16 neural network to recognize single characters. The VGG-16 network was pruned, including modifying the image input to 112×112×1, removing one max pooling layer and three convolutional layers, and making the final fully connected output a vector of length 21, because the iron cap character only contains 9 digits and 12 letters. Attached Figure Description

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention above or in other aspects will become clearer.

[0030] Figure 1 This is a flowchart illustrating the overall process of segmenting and recognizing characters adhered to the surface of a part according to an embodiment of the present invention.

[0031] Figure 2 This is a flowchart illustrating the location of adhesion points according to an embodiment of the present invention;

[0032] Figure 3 This is a flowchart illustrating the overall process of determining the character segmentation path according to an embodiment of the present invention.

[0033] Figure 4 This is a schematic diagram of the binarized adhering characters according to an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram showing the skeleton of the adhering characters after extraction, according to an embodiment of the present invention.

[0035] Figure 6 This is a schematic diagram of the skeleton after being converted into a graphical model according to an embodiment of the present invention;

[0036] Figure 7 This is a schematic diagram of the graphical model after adding points according to an embodiment of the present invention;

[0037] Figure 8 This is a schematic diagram of the graph model after edge deletion according to an embodiment of the present invention;

[0038] Figure 9 This is a schematic diagram of an effective dividing line according to an embodiment of the present invention;

[0039] Figure 10 The diagram illustrates the conventional and improved methods of this invention.

[0040] Figure 11 This is a schematic diagram of a single-character recognition VGG16 network according to an embodiment of the present invention; Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] The overall process constructed by the method of this invention is as follows: Figure 1 As shown; the overall flowchart for locating adhesion points is as follows. Figure 2 As shown; the flowchart for determining the character segmentation path is as follows. Figure 3 As shown; the image shows the binarized representation of the characters adhered to the surface of the part, as shown below. Figure 4 As shown; the image after skeleton extraction, as shown. Figure 5 As shown; the graph after converting the skeleton diagram into a graph model, as follows. Figure 6 As shown; the graph after adding points to the graph model, as follows. Figure 7 As shown; the graph after edge removal from the graph model, as follows. Figure 8 As shown; finally, a straight line diagram is provided, as follows. Figure 9 As shown; a schematic diagram of path segmentation using the traditional dripping method and the improved dripping algorithm, as follows. Figure 10 As shown; the VGG16 network structure diagram for single-character recognition, as follows. Figure 11 As shown;

[0043] The specific construction steps of the method for segmenting and recognizing characters adhered to the surface of parts in this embodiment of the invention are as follows:

[0044] A method for segmenting and recognizing characters adhered to the surface of a part, characterized by comprising the following steps, to... Figure 4 For example:

[0045] Step 1: First, find the connection points of the connected characters. The detailed steps can be divided into the following 8 steps.

[0046] Step 1-1: First, convert the character image into a binary image, and then extract each character block using the connected component properties, and use Z-axis... n Indicate; such as Figure 4 As shown;

[0047] Steps 1-2, Z n Skeleton extraction is performed, and the positions of all straight lines are initially considered to represent adhesion; if there are no straight lines in the skeleton, it is considered that there is no adhesion; for example... Figure 5 As shown;

[0048] Steps 1-3 determine whether all straight lines in the skeleton are actual adhesion points. A graphical model is used to analyze the complexity of the skeleton structure. The skeleton is considered a graph G, where intersections and endpoints correspond to branch points and endpoints in graph G, respectively. Branch points are represented in red, and endpoints in green. Figure 6 As shown;

[0049] Steps 1-4 add nodes to graph G, not how they are connected; to reflect the connection state of each point in graph G, draw a skeleton structure on graph G and perform incremental point operations; starting from the endpoints of the skeleton, fit the skeleton to multiple straight lines using the pixels of the skeleton; the angle between the straight lines is represented as Ω; if Ω is less than 135 degrees, the skeleton is considered to have a turning point, and a new point is added at the turning point; the added point is represented in blue, such as... Figure 7 As shown;

[0050] Steps 1-5 are followed by a "denoising operation" on the graph model G after the addition of points. This is because there are many short branches in the extracted skeleton, which are caused by character spikes. The endpoint deletion operation of graph G is used to remove all endpoints to avoid interference with character segmentation points. The denoising effect is shown in the figure below. Figure 8 As shown;

[0051] Steps 1-6: Determine whether the known straight lines are real adhesion points. Each straight line is judged sequentially from left to right. Use the midline of the straight lines to split the skeleton graph G and generate two new graphs G1 and G2.

[0052] Steps 1-7 are then performed, followed by an analysis of the complexity of G1 and G2 based on Table 1. If the complexity of both graphs is 1, then the cut lines correspond to the skeleton of the adhesion. All straight lines are analyzed sequentially. If a previous straight line is determined to represent adhesion, then the G1 portion to the left of that line will not participate in subsequent complexity analysis. Straight lines that satisfy the adhesion point are marked with green circles, while those that do not are marked with orange circles, such as... Figure 9 As shown;

[0053] In steps 1-8, the coordinates of the median of the corresponding straight line at the point of adhesion are recorded as the initial position of the character separation line.

[0054] Table 1. Skeleton Complexity Determination Method

[0055] G1 (G2) skeleton structure Complexity One crossing and one supplemental point 1 One loop 1 Two crossing points 1

[0056] Step 2: Based on the coordinates of the identified adhesion points, find the starting point for segmentation. Use the dripping algorithm to simulate the rolling of a water droplet, and segment the characters along the rolling path of the droplet. This may result in... Figure 10 In case (b), this paper optimizes the character segmentation path rules. After entering the depression, the simulated droplet does not immediately choose a breakthrough point at the depression, but stores the droplet. Instead, it generates another new droplet, starts from the landing point, executes the same path selection algorithm to reach the depression, stores the droplet again, and selects a breakthrough point only after the stored water volume reaches a certain threshold. If the stored water volume in the depression does not reach the threshold, and the droplet finds a new depression, then it can be determined that the previous depression does not meet the segmentation conditions. The specific implementation steps are divided into the following 3 steps.

[0057] Step 2-1: Set the character image to I, with a size of N★M; use n j Represents the pixel values ​​surrounding the water droplet, let (x i ,y i () represents the coordinates of the water droplet, and T represents the path of the water droplet.

[0058] T(x i+1 y i+1 )=t(x i y i W i i = 0, 1, 2, ...

[0059] Where (x) (i+1) ,y (i+1) (x0, y0) represents the coordinates of the next landing point of the water droplet, and (x0, y0) represents the initial landing point. The coordinates of the initial landing point of the water droplet are the coordinates of the median of the adhesion points found in step 1; W i It is the gravitational potential energy of the water droplet at its current position, expressed as:

[0060]

[0061]

[0062] Where v j Represents n j The pixel value of a point, 0 and 1 represent black and white respectively, m j n represents the point selected as the next landing point. j The weight of the point, m j =6–j;

[0063]

[0064] Step 2-2: The traditional algorithm rolls left and right when encountering a puddle, eventually choosing to drip vertically downwards on the right side of the depression; the improved dripping algorithm records the area of ​​the depression and filters out smaller depressions; the final segmentation result is as follows. Figure 10 As shown in (c);

[0065] Steps 2-3: Use the segmentation path to separate the connected characters into individual characters for subsequent recognition;

[0066] Step 3: Single character recognition is performed using the VGG16 neural network. The VGG-16 network was pruned, including modifying the image input to 112×112×1, removing one max pooling layer and three convolutional layers, and making the final fully connected output a vector of length 21, because the "iron cap" character only contains 9 digits and 12 letters; the network model diagram is shown below. Figure 11 As shown.

[0067] This invention provides a method for segmenting and recognizing characters adhered to the surface of a part. Many methods and approaches exist for implementing this technical solution; the above description represents a preferred embodiment. For those skilled in the art, various improvements and refinements can be made without departing from the principles of this invention, and these improvements and refinements should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method of segmenting and recognizing a character adhered to a surface of a part, characterized by, The method comprises the following steps: Step 1, firstly find out the sticking point of the sticking character, the detailed steps can be divided into the following 8 steps; Step 1-1, first, the character image is converted into a binary image, and using the connected domain characteristics, each block of character is extracted and represented by Z n ; Steps 1-2, Z n Skeleton extraction is performed and the position of all straight lines is initially considered to represent adhesion; if there are no straight lines in the skeleton, no adhesion is considered; Step 1-3, determine whether all straight lines in the skeleton are real sticking points, quote the graph model to analyze the complexity of the skeleton structure, the skeleton is considered as a graph G, the intersection points and end points in the skeleton correspond to the branch points and end points in the graph G respectively; Step 1-4, add nodes to the graph G, in order to reflect the connection state of each point in the graph G, draw the skeleton structure on the graph G, and carry out incremental point operation; starting from the end point of the skeleton, the skeleton is fitted as a plurality of straight lines by using the skeleton pixels; the angle between the straight lines is made as Ω; if Ω is less than 135 degrees, it is considered that the skeleton has a turning, a new point is added at the turning; Step 1-5, then carry out "de-noising operation" on the graph model G after adding points, because there are many short branches in the extracted skeleton, the branches are generated by the burrs of the character; the end point deletion operation of the graph G is used to delete all end points, so as to avoid the interference to the character segmentation point; Step 1-6, judge whether the known straight line is a real sticking point, each straight line is judged from left to right in turn; the skeleton graph G is split by using the midline of the straight line, to generate two new graphs G1 and G2; Step 1-7, then analyze the complexity of G1 and G2 according to table 1; if the complexity of the two graphs is 1, the cut line is the skeleton corresponding to the sticking; all straight lines are analyzed in turn, if the previous straight line is determined to represent sticking, then G1 on the left side of the straight line will not participate in the subsequent complexity analysis; Step 1-8, the coordinates of the median of the corresponding straight line at the sticking point are recorded as the initial position of the segmentation line of the sticking character; table 1 skeleton complexity judgment method Step 2, find out the starting point of segmentation according to the found sticking point coordinates, simulate the rolling of water droplets by using the water dripping algorithm, and segment the character on the rolling path of the water droplets, the character segmentation path rule is optimized; after entering the recessed area, the simulated liquid droplet will not immediately choose a breakthrough at the recessed place, but will store the liquid droplet; instead, another new water droplet is generated to start from the falling point, the same path selection algorithm is executed to reach the depression again, the water droplet is stored again, and the breakthrough is selected after the stored water amount reaches a certain threshold; if the stored water amount in the depression does not reach the threshold, and the water droplet finds a new depression, it can be judged that the previous depression does not meet the segmentation condition; the specific implementation steps are divided into the following 3 steps; Step 2-1, set the character image as I, whose size is N*M; use n j to represent the pixel value around the water drop, let (x i , y i ) be the coordinate position of the water drop point, and T be the water drop path: T(x i+1 ,y i+1 )=t(x i ,y i ,W i ) i=0,1,2,..., where (x (i+1) ,y (i+1) ) is the coordinate of the next landing point of the water droplet, (x0,y0) represents the initial landing point, and the coordinate of the initial point of the water droplet is the coordinate of the median of the adhesion point found in step 1; W i is the gravitational potential energy of the water droplet at the current position, which is expressed as: where v j represents n j the pixel value of the point, 0 and 1 represent black and white, respectively, m j represents the weight of the n j point selected as the next drop point, m j = 6 - j: Step 2-2, the traditional algorithm rolls left and right when encountering a water pit, and finally chooses to drop vertically downward on the right side of the recess; the improved water dripping algorithm records the area of the recessed area, and carries out filtering operation on the recessed area with small area; Step 2-3, the sticking character is segmented into single characters by using the segmentation path, which is used for subsequent recognition; Step 3, single character recognition using VGG16 neural network, the VGG-16 network was pruned, including modifying the input of the image to 112x112x1, removing one max pooling layer and three convolutional layers, and making the final fully connected output a 21-length vector, as the iron cap characters only contain 9 digits and 12 letters.

Images