A method for extracting characters from surface parts based on 3D point clouds

Abstract

The present invention discloses a method for extracting characters from curved surface parts based on 3D point clouds, which belongs to the fields of industrial manufacturing and 3D image processing. Based on 3D point cloud data analysis, the present invention innovatively applies 3D image processing technology to industrial production sites, completing the task of extracting characters from curved surface parts. The invention first uses a 3D camera to collect the shape data of the curved surface parts and saves the data in a 3D point cloud format; then uses a character positioning algorithm to find the exact position of the curved surface characters, and extracts the point cloud data of the character area separately; then performs surface correction on the extracted character data one by one; finally, uses the characteristics of the normal vector of the 3D point cloud to extract the characters on the surface. The method described in the present invention can accurately locate the position of the characters on the curved surface parts and realize the extraction of the curved surface characters. It still has good robustness under conditions such as non-uniform illumination and small color difference between the character area and the non-character area, solving the problems of difficult extraction of characters from curved surface parts and poor extraction effect.

Description

Technical Field

[0001] The invention belongs to the fields of industrial manufacturing and digital image processing, and particularly relates to a method for extracting characters from curved surface parts based on 3D point clouds. Background Art

[0002] In industrial manufacturing, the geometric shapes of parts vary greatly. Part surfaces can be categorized as curved or flat based on their shape. Many parts in industry are composed of curved surfaces because they are easy to machine and help disperse stress. However, in actual use, part information cannot be directly derived from the part's shape. Therefore, during the production process, relevant parameters are marked on the surface of each part. These parameters include strength grade information, model number, and batch number. These parameters not only provide a foundation for intelligent part assembly in the early stages but also facilitate subsequent tasks such as product tracking, quality control, and recall management.

[0003] Current technologies for character recognition on part surfaces fall into two main categories: traditional image processing methods and deep learning approaches. Traditional image processing methods include image preprocessing, character location, character segmentation, and character recognition. All of these traditional algorithms have been widely used in industry due to their excellent performance in specific scenarios and high interpretability.

[0004] In recent years, deep learning methods have also achieved good results in the field of character recognition due to their strong adaptability and easy operation. However, most of the current research is based on flat characters, and very few studies are specifically aimed at the recognition of characters on part surfaces. There are two main reasons for this situation: First, traditional image acquisition methods cannot well capture the features of part surfaces. No matter from what angle, only the characters in the center of the field of view can be captured each time, while the characters at the edge of the field of view may be blurred or distorted. In order to fully capture the character image, it is inevitable to rotate the iron cap multiple times, which will introduce two other difficulties, namely complex mechanical structure design and multi-image stitching technology; second, the image features of most parts have little difference from the non-character areas, and the image features captured by traditional image acquisition systems are not conducive to subsequent character recognition.

[0005] Therefore, a new image acquisition method is needed to capture the image features of curved characters to solve the problems of difficulty in character recognition and low recognition accuracy caused by part shape and production process. Summary of the Invention

[0006] The present invention provides a method for extracting characters from curved surface parts based on 3D point clouds. This method primarily addresses the problems of existing methods, such as the inability to fully capture characters and the small difference in image features between character areas and non-character areas. The technical problem proposed by the present invention is solved as follows:

[0007] A method for extracting characters from curved surface parts based on 3D point clouds, comprising the following steps:

[0008] Step 1: Build a 3D data acquisition platform, use a 3D camera to collect the shape data of the curved part, and save the data in the form of a 3D point cloud. Use W to represent the collected point cloud image.

[0009] Step 2: Input the point cloud image W collected in step 1 into the character positioning module to obtain the exact position of each line of characters on the surface part, and extract the point cloud data of each line of characters separately. n (n=1, 2, 3, ...) represents the process of reducing the impact of non-character areas and improving the efficiency of the later algorithm. The process is divided into the following four steps:

[0010] Step 2-1: First, divide the entire point cloud image W of the part into multiple point cloud boxes with a width of 0.2 mm along the positive direction of the x-axis; D represents all points in the point cloud image W, and b represents the set of all points in a box:

[0011] D={x|x∈b1∧x∈b2...∧x∈b n}

[0012] Step 2-2, calculate the difference in z coordinates of adjacent point clouds in each small box of the point, using d n express:

[0013]

[0014] Where i represents b n A point in the i ,y i ,z i ), s represents b n The coordinates of other points in the same matrix as i are expressed as (x s ,y s ,z s ), L si Represents the projection distance between point i and point s on the XOY plane:

[0015]

[0016] If L si If it is less than 0.05 mm, then point s and point i are adjacent;

[0017] Step 2-3, d of all point cloud boxes nThe data is plotted into a broken line graph; due to the cone shape of the insulator cap, the overall trend of the broken line is increasing. As the x coordinate gradually increases, more points are included in the point cloud box of the same width, so d n The larger the point cloud data of the character area of ​​the part changes greatly, the sum of the adjacent point difference values ​​of all character parts is large; while the sum of the adjacent point difference values ​​of the non-character part is small; based on this feature, the point cloud of the character area can be distinguished from the point cloud of the non-character area;

[0018] Steps 2-4, finally use the dividing line and the width of the character to separate each line of characters separately, and use A n (n=1, 2, 3, ...) represents;

[0019] Step 3: Get the single line character A from step 2 n (n=1, 2, 3, ...) are sent to the character correction module for surface correction; wherein, the correction process is divided into the following four steps;

[0020] Step 3-1: First, the point cloud image A of the single line of characters n It is considered as a surface composed of a series of curves superimposed along the x-axis; one curve corresponds to a set of point cloud data in the point cloud image, and the entire surface can also be regarded as a part of a frustum; one curve corresponds to a set of point cloud data in the point cloud image;

[0021] Step 3-2: Find the curve with the smallest x value and the curve with the largest x value, and use their corresponding point cloud data to perform circle fitting. Two circles are obtained, denoted by C1 and C2 respectively. Since these two sets of curve data are located at the edge of the character area, they contain very little character information. Therefore, the circles fitted by these two sets of data can be approximately considered to be the real 3D data of the non-character area.

[0022] In step 3-3, C1 and C2 can be regarded as the upper and lower base surfaces of the frustum, and the height of the frustum is represented by H:

[0023] H=x 2_c -x 1_c

[0024] Among them, the center coordinates and radius of circles C1 and C2 are [x 1_c ,y 1_c ,z 1_c, r1] and [x 2_c ,y 2_c ,z 2_c, r2];

[0025] Step 3-4, calculate the circular parameters of any section of the frustum through the upper and lower bases, and let the section circle be C n Indicates that the parameter is [xn_c ,y n_c ,z n_c, r n ]:

[0026] x n_c =h n +x 1_c y n_c =y 1_c +(y 2_c -y 1_c )×h n / Hz n_c =z 1_c +(z 2_c -z 1_c )×h n / Hr n =r1+(r2-r1)×h n / H

[0027] where h n It represents the distance from the cross section circle of the truncated cone to the upper base;

[0028] Step 3-5, referring to the method of expanding a circle into a straight line, the cone table can be expanded into a plane; p represents any point on the cone, and the coordinates are (x p ,y p ,z p ), h p Indicates the distance from p to the upper base, h p =x p -x1; according to h p The conic section of point p can be determined, and circle C can be obtained according to the above formula p The parameter [x p_c ,y p_c ,z p_c, r p ]; The point with the largest z coordinate value on the cone is taken as the plane reference, represented by G, and its coordinates are (x G ,y G ,z G );y p_move and z p_move Respectively represent the y-coordinate and z-coordinate values ​​of point p to be translated after expansion:

[0029] y p_move =arcsin[(y p -y p_c ) / r p ]×π×r p ×180z p_move =z G -z p

[0030] Among them, by pressing all points in the single-line character point cloud to the corresponding y p_move and z p_move The values ​​are moved to correct the surface;

[0031] Step 4: Import the corrected character point cloud graphics into the character extraction module to extract the characters. The character extraction process is divided into three steps.

[0032] Step 4-1, first find the normal vectors of all point clouds;

[0033] Step 4-2, then unify the direction of the normal vectors so that they all point to the far end of the z coordinate; there is a transition area between the character and non-character areas, and this area is the key to extracting characters;

[0034] Step 4-3, then use the characteristic that the normal vector of the transition area is inclined, while the normal vectors of other areas are vertically upward, to find the point cloud data of the transition area;

[0035] Step 4-4, finally, the non-character area and the character area are separated according to the Z coordinate of the point cloud data of the transition area;

[0036] Step 5: Project the extracted character point cloud data into a 2D character image. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and other advantages of the present invention will become more apparent.

[0038] Figure 1 This is a general flow chart of a curved part character based on 3D point cloud according to an embodiment of the present invention;

[0039] Figure 2 This is a point cloud image of a part captured by a 3D camera according to an embodiment of the present invention, taking an insulator cap as an example;

[0040] Figure 3 The point cloud image of the iron cap according to the embodiment of the present invention is divided into several three-dimensional point cloud boxes with a width of 0.2 mm along the X-axis; (a) 3D display; (b) 2D display;

[0041] Figure 4 The D set of the embodiment of the present invention is drawn into a line graph and subjected to mean filtering; points A and D represent the boundaries between characters and non-characters, and points B and C represent the boundaries between characters;

[0042] Figure 5 It is the iron cap character positioning and character extraction of the embodiment of the present invention;

[0043] Figure 6(a) a point cloud image of a single-line character region; (b) a schematic diagram of a fitting curve according to an embodiment of the present invention;

[0044] Figure 7 Comparison of single-line characters before and after correction in an embodiment of the present invention;

[0045] Figure 8 The point cloud normal vectors of the character area and the point cloud normal vectors of the non-character area in the embodiment of the present invention;

[0046] Figure 9 The two-dimensional conversion process of characters in an embodiment of the present invention; (a) corrected characters; (b) transition area between characters and non-characters; (c) extracted three-dimensional characters; (d) two-dimensional text characters; DETAILED DESCRIPTION

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] The overall process of the method of the present invention is as follows: Figure 1 As shown in the figure, which includes the point cloud image of the insulator cap collected by the 3D camera, as shown in the figure. Figure 2 As shown in Figure 2, the point cloud image of the iron cap is divided into several three-dimensional point cloud boxes with a width of 0.2 mm along the X axis, as shown in Figure 2. Figure 3 As shown; the D set is drawn into a line graph and mean filtered; points A and D represent the boundaries between characters and non-characters, and points B and C represent the boundaries between characters, as shown Figure 4 As shown; the character positioning and extraction results are shown in Figure 5 As shown; the point cloud image and fitting curve diagram of the single-line character area are shown in Figure 6 As shown in the figure, the comparison of single-line characters before and after correction is as follows: Figure 7 As shown; the point cloud normal vector of the character area and the point cloud normal vector of the non-character area are as follows Figure 8 As shown; the two-dimensional transformation process of the character is as follows Figure 9 As shown;

[0049] The specific construction steps of the method for extracting characters from the curved surface of an insulator cap in an embodiment of the present invention are as follows:

[0050] A method for extracting characters from curved surface parts based on 3D point clouds, characterized by comprising the following steps, taking an insulator cap as an example:

[0051] Step 1: Build a 3D data acquisition platform and use a 3D camera to collect the shape data of the parts, such as Figure 2 As shown, the data is saved in the form of 3D point cloud, and W is used to represent the collected point cloud image;

[0052] Step 2: Input the point cloud image W collected in step 1 into the character positioning module to obtain the exact position of each line of characters on the part surface, and extract the point cloud data of each line of characters separately. n (n=1, 2, 3, ...) represents the process of reducing the impact of non-character areas and improving the efficiency of the later algorithm. The process is divided into the following four steps:

[0053] Step 2-1: First, divide the entire point cloud image W of the part into multiple point cloud boxes with a width of 0.2 mm along the positive direction of the x-axis, as shown in the following example: Figure 3 As shown, (a) shows the division of the point cloud box by 3D, and (b) shows the division of the point cloud box by 2D; where D represents all points in the point cloud image W, and b represents the set of all points in a box:

[0054] D={x|x∈b1∧x∈b2...∧x∈b n}

[0055] Step 2-2, calculate the difference in z coordinates of adjacent point clouds in each small box of the point, using d n express:

[0056]

[0057] Where i represents b n A point in the i ,y i ,z i ), s represents b n The coordinates of other points in the same matrix as i are expressed as (x s ,y s ,z s ), L si Represents the projection distance between point i and point s on the XOY plane:

[0058]

[0059] If L si If it is less than 0.05 mm, then point s and point i are adjacent;

[0060] Step 2-3, d of all point cloud boxes n The data is plotted as a line graph, such as Figure 4 As shown, (a) represents d n (b) shows the line graph after filtering; due to the cone shape of the insulator cap, the overall trend of the line graph is increasing. As the x coordinate increases, more points are included in the point cloud box of the same width, so d nThe larger the point cloud data of the character area of ​​the part changes greatly, the sum of the adjacent point difference values ​​of all character parts is large; while the sum of the adjacent point difference values ​​of the non-character part is small; based on this feature, the point cloud of the character area can be distinguished from the point cloud of the non-character area;

[0061] Steps 2-4, finally use the dividing line and the width of the character to separate each line of characters separately, and use A n (n=1, 2, 3, ...) means that the segmentation result is as follows Figure 5 As shown;

[0062] Step 3: Get the single line character A from step 2 n (n=1, 2, 3, ...) are sent to the character correction module for surface correction; wherein, the correction process is divided into the following four steps;

[0063] Step 3-1: First, the point cloud image A of the single line of characters n It is regarded as a surface composed of a series of curves superimposed along the x-axis; a curve corresponds to a set of point cloud data in the point cloud image, and the entire surface can also be regarded as a part of a frustum; a curve corresponds to a set of point cloud data in the point cloud image, and the specific analysis diagram is as follows Figure 6 (a)

[0064] Step 3-2, find the curve with the smallest x value and the curve with the largest x value, and use their corresponding point cloud data to perform circle fitting; get two circles, represented by C1 and C2 respectively, as shown in Figure 6 As shown in (b), since these two sets of curve data are located at the edge of the character area, they contain very little character information; that is, the circles fitted by these two sets of data can be approximately considered to be the real three-dimensional data of the non-character area;

[0065] In step 3-3, C1 and C2 can be regarded as the upper and lower base surfaces of the frustum, and the height of the frustum is represented by H:

[0066] H=x 2_c -x 1_c

[0067] Among them, the center coordinates and radius of circles C1 and C2 are [x 1_c ,y 1_c ,z 1_c, r1] and [x 2_c ,y 2_c ,z 2_c, r2];

[0068] Step 3-4, calculate the circular parameters of any section of the frustum through the upper and lower bases, and let the section circle be C n Indicates that the parameter is [x n_c ,yn_c ,z n_c, r n ]:

[0069] x n_c =h n +x 1_c

[0070] y n_c =y 1_c +(y 2_c –y 1_c )×h n / Hz n_c =z 1_c +(z 2_c -z 1_c )×h n / Hr n =r1+(r2-r1)×h n / H

[0071] where h n It represents the distance from the cross section circle of the truncated cone to the upper base;

[0072] Step 3-5, referring to the method of expanding a circle into a straight line, the cone table can be expanded into a plane; p represents any point on the cone, and the coordinates are (x p ,y p ,z p ), h p Indicates the distance from p to the upper base, h p =x p -x1; according to h p The conic section of point p can be determined, and circle C can be obtained according to the above formula p The parameter [x p_c ,y p_c ,z p_c, r p ]; The point with the largest z coordinate value on the cone is taken as the plane reference, represented by G, and its coordinates are (x G ,y G ,z G );y p_move and z p_move Respectively represent the y-coordinate and z-coordinate values ​​of point p to be translated after expansion:

[0073] y p_move =arcsin[(y p -y p_c ) / r p ]×π×r p ×180z p_move =z G -z p

[0074] Among them, by pressing all points in the single-line character point cloud to the corresponding y p_move and z p_move The values ​​are moved to correct the surface;

[0075] Step 4: Import the corrected character point cloud graphics into the character extraction module to extract the characters. The character extraction process is divided into three steps.

[0076] Step 4-1, first find the normal vectors of all point clouds, such as Figure 8 As shown;

[0077] Step 4-2, then unify the direction of the normal vectors so that they all point to the far end of the z coordinate; there is a transition area between the character and non-character areas, and this area is the key to extracting characters;

[0078] Step 4-3, then use the characteristic that the normal vector of the transition area is inclined, while the normal vectors of other areas are vertically upward, to find the point cloud data of the transition area;

[0079] Step 4-4, finally, the non-character area and the character area are separated according to the Z coordinate of the point cloud data of the transition area;

[0080] Step 5: Project the extracted character point cloud data into a 2D character image, such as Figure 9 As shown;

[0081] The present invention provides a method for extracting characters from curved surface parts based on 3D point clouds. While there are numerous methods and approaches for implementing this technical solution, the above is a preferred embodiment of the present invention. Persons skilled in the art may make improvements and refinements without departing from the principles of the present invention, and such improvements and refinements should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for extracting characters from curved surface parts based on 3D point clouds, characterized in that: The following steps are involved: Step 1: Build a 3D data acquisition platform, use a 3D camera to collect the shape data of the curved part, and save the data in the form of a 3D point cloud. Use W to represent the collected point cloud image. Step 2: Input the point cloud image W collected in step 1 into the character positioning module to obtain the exact position of each line of characters on the surface part, and extract the point cloud data of each line of characters separately. n (n=1, 2, 3, ...) represents the process of reducing the impact of non-character areas and improving the efficiency of the later algorithm. The process is divided into the following four steps: Step 2-1: First, divide the entire point cloud image W of the part into multiple point cloud boxes with a width of 0.2 mm along the positive direction of the x-axis; D represents all points in the point cloud image W, and b represents the set of all points in a box: D={x|x∈b1∧x∈b2...∧x∈b n } Step 2-2, calculate the difference in z coordinates of adjacent point clouds in each small box of the point, using d n express: Where i represents b n A point in the i ,y i ,z i ), s represents b n The coordinates of other points in the same matrix as i are expressed as (x s ,y s ,z s ), L si Represents the projection distance between point i and point s on the XOY plane: If L si If it is less than 0.05 mm, then point s and point i are adjacent; Step 2-3, d of all point cloud boxes n The data is plotted as a line graph; the overall trend of the line is increasing. As the x coordinate gradually increases, more points are included in the point cloud box of the same width, so d n The larger the point cloud data of the character area of ​​the part changes greatly, the sum of the adjacent point difference values ​​of all character parts is large; while the sum of the adjacent point difference values ​​of the non-character part is small; based on this feature, the point cloud of the character area can be distinguished from the point cloud of the non-character area; Steps 2-4, finally use the dividing line and the width of the character to separate each line of characters separately, and use A n (n=1, 2, 3, ...) represents; Step 3: Get the single line character A from step 2 n (n=1, 2, 3, ...) are sent to the character correction module for surface correction; wherein, the correction process is divided into the following four steps; Step 3-1, first take the point cloud image A of the single line of characters n It is considered as a surface composed of a series of curves superimposed along the x-axis; one curve corresponds to a set of point cloud data in the point cloud image, and the entire surface can also be regarded as a part of a frustum; one curve corresponds to a set of point cloud data in the point cloud image; Step 3-2: Find the curve with the smallest x value and the curve with the largest x value, and use their corresponding point cloud data to perform circle fitting. Two circles are obtained, denoted by C1 and C2 respectively. Since these two sets of curve data are located at the edge of the character area, they contain very little character information. Therefore, the circles fitted by these two sets of data can be approximately considered to be the real 3D data of the non-character area. In step 3-3, C1 and C2 can be regarded as the upper and lower base surfaces of the frustum, and the height of the frustum is represented by H: H=x 2_c -x 1_c Among them, the center coordinates and radius of circles C1 and C2 are [x 1_c ,y 1_c ,z 1_c ,r1] and [x 2_c ,y 2_c ,z 2_c ,r2]; Step 3-4, calculate the circular parameters of any section of the frustum through the upper and lower bases, and let the section circle be C n Indicates that the parameter is [x n_c ,y n_c , z n_c ,r n ]: x n_c =h n +x 1_c and n_c =and 1_c +(and 2_c -and 1_c )×h n / H z n_c =z 1_c +(z 2_c -z 1_c )×h n / H r n =r1+(r2-r1)×h n / H where h n It represents the distance from the cross section circle of the truncated cone to the upper base; Step 3-5, referring to the method of expanding a circle into a straight line, the cone table can be expanded into a plane; p represents any point on the cone, and the coordinates are (x p ,y p ,z p ), h p Indicates the distance from p to the upper base, h p =x p -x1; according to h p The conic section of point p can be determined, and circle C can be obtained according to the above formula p The parameter [x p_c ,y p_c ,z p_c ,r p ]; the point with the largest z coordinate value on the cone is taken as the plane reference, represented by G, and its coordinates are (x G ,y G ,z G );y p_move and z p_move Respectively represent the y-coordinate and z-coordinate values ​​of point p to be translated after expansion: and p_move =arcsin[(and p -and p_c ) / r p ]×π×r p ×180 With p_move =z G -With p Among them, by pressing all points in the single-line character point cloud to the corresponding y p_move and z p_move The values ​​are moved to correct the surface; Step 4: Import the corrected character point cloud graphics into the character extraction module to extract the characters. The character extraction process is divided into three steps. Step 4-1, first find the normal vectors of all point clouds; Step 4-2, then unify the direction of the normal vectors so that they all point to the far end of the z coordinate; there is a transition area between the character and non-character areas, and this area is the key to extracting characters; Step 4-3, then use the characteristic that the normal vector of the transition area is inclined, while the normal vectors of other areas are vertically upward, to find the point cloud data of the transition area; Step 4-4, finally, the non-character area and the character area are separated according to the Z coordinate of the point cloud data of the transition area; Step 5: Project the extracted character point cloud data into a 2D character image.

Images