Robot vision welding fillet weld identification method and system and storage medium

Abstract

The invention provides a robot vision welding fillet weld identification method. The method comprises the following steps: setting a region of interest; solving the sum of gray values of each row of pixels in the RO I, and solving the row with the maximum cumulative sum; searching a row with the minimum gray value accumulated sum; searching a coarse coordinate point of the welding seam; the line with the maximum gray value accumulated sum on the two sides of the weld joint is obtained; the centers of light strip sections on the two sides of the weld joint are obtained respectively; and center fitting and weld joint identification. The method can quickly identify the fillet weld.

Description

Technical Field

[0001] The present invention relates to the field of welding technology, and in particular to a robot vision welding fillet weld recognition method, system and storage medium. Background Art

[0002] Intelligent manufacturing technology is particularly important for the development of the manufacturing industry. Welding technology is an important technology in the manufacturing field. How to achieve automation, efficiency and intelligence in welding is particularly important for the realization of intelligent manufacturing. Welding robots are gradually being used in manufacturing. Adding vision to automatically identify the position of welds to welding robots makes them more automated. Intelligence is a hot topic in the current research of intelligent manufacturing.

[0003] Fillet welds are a type of weld that is widely used in manufacturing. When the welding material that makes up the fillet weld is a highly reflective metal material, the weld image captured by the line structured light vision sensor will have strong reflective noise, especially for fillet welds composed of bright metal surfaces such as stainless steel, aluminum alloy, and titanium alloy. There will be a lot of reflective noise. However, many existing fillet weld recognition methods can only recognize fillet welds without reflective noise, with large center extraction errors, slow speed, and complex methods. Summary of the invention

[0004] The present invention provides a robot vision welding fillet weld recognition method, system and storage medium, which adopts a line structured light vision sensor composed of a line laser (or called line structured light) and a camera to collect the fillet weld image irradiated by the line laser, and quickly processes the image to quickly and accurately identify the weld.

[0005] This specification discloses a robot vision welding fillet weld recognition method, comprising:

[0006] Collect weld images through line structured light vision sensor;

[0007] Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0008] Sum each row of pixel grayscale values ​​in the rectangular area of ​​interest and save each row y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum maxand the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the gray value of the light bar;

[0009] By adding the row position y1 with the largest cumulative sum, the length interval [y1-row, y1+row] is set. This length interval should include the weld position. The value of row is generally one-fourth of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min And the corresponding row position, which is the row coordinate of the rough positioning coordinate of the fillet weld, and the row is recorded as y2;

[0010] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions, and search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0011] The boundary calculation formula is:

[0012]

[0013] l is the number of columns of the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0014] By sum i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0015] Starting from the points (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar, where the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, starting from the point (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; starting from the point (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0016] When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated;

[0017] The center points of the two light strips are fitted by the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the precise position of the fillet weld.

[0018] In this manual, sum i The formula is as follows:

[0019]

[0020] Among them, x s ,y s are the minimum row and column coordinates of the rectangular region of interest, respectively; x n and m are the maximum row and column coordinates of the rectangular region of interest, respectively.

[0021] In this specification, a 1×5 vertical template is used in the process of searching for the position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of t -5,x t +b+5] to calculate the center point of the light bar.

[0022] In this manual, the calculation formula for the center of the light bar is as follows:

[0023]

[0024] (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median gray value of the pixel, and the adaptive threshold b is the light bar width.

[0025] In this specification, I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in a 3×3 window centered at t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

[0026]

[0027] In this specification, the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0028] This specification also discloses a robot vision welding fillet weld recognition system, which is used to implement any of the robot vision welding fillet weld recognition methods described above. The robot vision welding fillet weld recognition system includes:

[0029] An acquisition module, used for acquiring weld images through a line structured light vision sensor;

[0030] The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0031] The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular region of interest and save each row of y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max and the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the gray value of the light bar. The interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum. This interval must include the weld position. The value of row is generally one-fourth of the image width. The sum is compared within this length interval. iThe size of , find the sum i Minimum sum min and the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; set a "cross" template, the size of which is 1×5 pixels in both the vertical and horizontal directions, and search from left to right along the image y2 row 0 column to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap, and the triangular area is located at the fillet weld;

[0032] The boundary calculation formula is:

[0033]

[0034] l is the number of columns of the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0035] Comparison module, used to compare i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0036] The light bar center determination module is used to use the points (x0, y1) and (x0, y3) as starting points, search the position of the light bar row by row from left to right, from top to bottom or from bottom to top, and calculate the center of each row of the light bar, wherein the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, use the point (x0, y1) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y3) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, use (x0, y3) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y1) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0037] The fitting fixed point module is used to fit the center points of two light strips through the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the precise position of the fillet weld.

[0038] The present specification also discloses a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes any one of the above-mentioned robot vision welding fillet weld recognition methods.

[0039] The embodiments of this specification can at least achieve the following beneficial effects:

[0040] The present invention proposes a fast and accurate method for identifying fillet welds, which can be applied to robot intelligent welding. The present invention uses a line structured light visual sensor composed of a line laser (or line structured light) and a camera to collect images of fillet welds illuminated by the line laser, and uses this method to quickly process the images and identify the welds. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0042] Figure 1 It is a schematic diagram of the robot vision welding fillet weld recognition method involved in the present invention.

[0043] Figure 2 It is a schematic diagram of the fillet weld reflection image involved in the present invention.

[0044] Figure 3 It is a schematic diagram of the accumulation and statistics of the grayscale values ​​of pixels in each row of the image involved in the present invention. DETAILED DESCRIPTION

[0045] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0046] The disclosure below provides many different embodiments or examples to implement different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the embodiments of the present invention. In addition, the embodiments of the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplification and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed.

[0047] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] like Figure 1 As shown, this specification discloses a robot vision welding fillet weld recognition method, comprising:

[0049] Collect weld images through line structured light vision sensor;

[0050] Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0051] Sum each row of pixel grayscale values ​​in the rectangular area of ​​interest and save each row y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max and the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the gray value of the light bar;

[0052] The interval [y1-row, y1+row] is set by adding the row position y1 with the largest cumulative sum. This interval should include the weld position. The value of row is generally one-fourth of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min And the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2;

[0053] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions, and search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0054] The boundary calculation formula is:

[0055]

[0056] l is the number of columns of the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0057] By sum i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0058] Starting from the points (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar, where the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, starting from the point (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; starting from the point (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0059] When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated;

[0060] The center points of the two light strips are fitted by the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the precise position of the fillet weld.

[0061] In this manual, sum i The formula is as follows:

[0062]

[0063] Among them, xs ,y s are the minimum row and column coordinates of the rectangular region of interest, respectively; x n and m are the maximum row and column coordinates of the rectangular region of interest, respectively.

[0064] In this specification, a 1×5 vertical template is used in the process of searching for the position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of t -5,x t +b+5] to calculate the center point of the light bar.

[0065] In this manual, the calculation formula for the center of the light bar is as follows:

[0066]

[0067] (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median gray value of the pixel, and the adaptive threshold b is the light bar width.

[0068] In this specification, I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in a 3×3 window centered at t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

[0069]

[0070] In this specification, the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0071] This specification also discloses a robot vision welding fillet weld recognition system, which is used to implement any of the robot vision welding fillet weld recognition methods described above. The robot vision welding fillet weld recognition system includes:

[0072] An acquisition module, used for acquiring weld images through a line structured light vision sensor;

[0073] The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0074] The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular region of interest and save each row of y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max and the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the gray value of the light bar. The interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum. This interval must include the weld position. The value of row is generally one-fourth of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min and the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; set a "cross" template, the size of which is 1×5 pixels in both the vertical and horizontal directions, and search from left to right along the image y2 row 0 column to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap, and the triangular area is located at the fillet weld;

[0075] The boundary calculation formula is:

[0076]

[0077] l is the number of columns of the image; when B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0078] Comparison module, used to compare i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld iThere is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0079] The light bar center determination module is used to use the points (x0, y1) and (x0, y3) as starting points, search the position of the light bar row by row from left to right, from top to bottom or from bottom to top, and calculate the center of each row of the light bar, wherein the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, use the point (x0, y1) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y3) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, use (x0, y3) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y1) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0080] The fitting fixed point module is used to fit the center points of two light strips through the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the precise position of the fillet weld.

[0081] The present specification also discloses a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes any one of the above-mentioned robot vision welding fillet weld recognition methods.

[0082] In a specific embodiment, the present invention proposes a robot vision welding fillet weld recognition method based on row grayscale and statistical analysis, which can quickly identify fillet welds with reflective noise. The reflective noise of fillet welds of highly reflective materials is as follows: Figure 2 shown.

[0083] Robot vision welding fillet weld recognition methods include:

[0084] Region of Interest (ROI) Setting:

[0085] In order to improve the calculation speed, a rectangular region of interest (ROI) is set in the weld image according to prior knowledge. The region should include the weld and the light strip. The ROI is defined as the following equation, where x s ,y s are the minimum row and column coordinates of ROI, respectively. n ,y mare the maximum row and column coordinates of ROI respectively.

[0086]

[0087] Calculate the sum of the grayscale values ​​of each row of pixels in the ROI and find the row with the largest cumulative sum:

[0088] Sum the grayscale values ​​of each row of pixels in the ROI area and save each row y i The corresponding pixel gray value accumulation sum i , by comparing the grayscale value of each row of pixels I(x i ,y j ) Cumulative sum i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the largest cumulative sum of the reflection noise intensity and the gray value of the light bar. According to the statistical experimental data of hundreds of images, the position of this row in the image is a certain distance away from the row where the weld position is located, just far away from the area with the most serious noise pollution;

[0089]

[0090] Find the row with the smallest cumulative grayscale value:

[0091] In order to reduce the amount of calculation and accuracy, the interval [y1-row, y1+row] is set by summing the row position y1 with the largest sum. This interval should include the weld position. The value of row is generally one-fourth of the image width. The sum is compared in this interval. i The size of , find the sum i Minimum sum min And the corresponding row position, which is the rough positioning row position of the fillet weld, is recorded as y2. Figure 3 shown.

[0092] Find the coarse coordinate points of the weld:

[0093] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions, and search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0094] The boundary calculation formula is:

[0095]

[0096] l is the number of columns in the image.

[0097] When B takes the maximum value, the left boundary (x1, y2) of the weld position light bar is found; let the template continue to search along the y2 line to find the right boundary of the light bar. When B takes the minimum value, the right boundary (x2, y2) of the weld position light bar is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0098] Find the rows with the largest cumulative grayscale values ​​on both sides of the weld:

[0099] By sum i The constructed histogram shows that the weld is sum i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max By comparing the size of y1 and y2, determine the sum of the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0100] Find the center of the light strip segments on both sides of the weld respectively:

[0101] Starting from points (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar, where the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column. When y1>y3, starting from point (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar, starting from point (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, starting from point (x0, y3), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; starting from point (x0, y1), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar. In the process of searching for the position of the light bar, a 1×5 vertical template is used. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of t -5,x t +b+5] to calculate the center point of the light bar.

[0102] Assume that the boundary position point Pt of the light strip is found i is (x t ,y), where the central formula is:

[0103]

[0104] I(x t +k,y) is the median gray value of the pixel, and the adaptive threshold b is the light bar width.

[0105] Let the gray value of these pixels be I(x t +k,y) are all based on pixel (x t +k,y) is the median of the grayscale values ​​of the pixels in a 3×3 window centered at t +k,y) is the pixel (x t +k,y) grayscale value, where k∈[x t -5,x t +b+5], median I(x t +k,y) formula is:

[0106] I(x t + k, y) = Med{f(x t +k-1,y-1),f(x t +k-1,y),f(x t +k-1,y+1),

[0107] f(x t +k,y-1),f(x t +k,y),f(x t +k,y+1),

[0108] f(x t +k+1,y-1),f(x t +k+1,y),f(x t +k+1,y+1)};

[0109] The adaptive threshold b is the width of the light bar, and (U, y) is the new center position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , from the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0110] Center Fitting and Weld Identification:

[0111] The center points of the two light strips are fitted by the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the precise position of the fillet weld.

[0112] The above-described embodiments are used to illustrate the present invention, not to limit the present invention, so changes in the exemplified values ​​or replacement of equivalent elements should still fall within the scope of the present invention.

[0113] From the above detailed description, it can be understood by those skilled in the art that the present invention can indeed achieve the aforementioned objectives and is in compliance with the provisions of the Patent Law.

[0114] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0115] It should be noted that the above description of the relevant process is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0116] The basic concepts have been described above. Obviously, for those of ordinary skill in the art who have read this application, the above invention disclosure is only for example and does not constitute a limitation of this application. Although not explicitly stated here, those of ordinary skill in the art may make various modifications, improvements and amendments to this application. Such modifications, improvements and amendments are suggested in this application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of this application.

[0117] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0118] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful combination of processes, machines, products or substances, or any new and useful improvements thereto. Therefore, various aspects of the present application may be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "units", "modules" or "systems". In addition, various aspects of the present application may take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0119] The computer program code required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., such as C programming language, VisualBasic, Fortran2103, Perl, COBOL2102, PHP, ABAP, such as Python, Ruby and Groovy dynamic programming language or other programming languages. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on the remote computer, or run completely on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0120] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a pure software solution, for example, installation on an existing server or mobile device.

[0121] Similarly, it should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the above description of the embodiments of the application, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this method of the application should not be interpreted as reflecting the intention that the claimed object requires more features than those explicitly stated in each claim. On the contrary, the subject of the invention should have fewer features than the above single embodiment.

Claims

1. A robot vision welding fillet weld recognition method, characterized in that: include: Collect weld images through line structured light vision sensor; Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), maximum row and column coordinates (x n ,y m ); Sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save each row y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max and the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the gray value of the light bar; By adding up the row position y1 with the largest cumulative sum, set the length interval [y1-row, y1+row]. This length interval should include the weld position, and the value of row should be one-fourth of the image width. Compare sum in this length interval i The size of , find the sum i Minimum sum min And the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions, and search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld. The boundary calculation formula is: l is the number of columns of the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2; By sum i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3; Starting from the points (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar, where the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, starting from the point (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; starting from the point (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated; The center points of the two light strips are fitted by a random sampling consensus algorithm to find the intersection of the two straight lines, which is the exact position of the fillet weld.

2. The robot vision welding fillet weld recognition method according to claim 1 is characterized in that: sum i The formula is as follows: Among them, x s and s are the minimum row and column coordinates of the rectangular region of interest, respectively; x n and m are the maximum row and column coordinates of the rectangular region of interest, respectively.

3. The robot vision welding fillet weld recognition method according to claim 1 is characterized in that: In the process of searching for the position of the light bar, a 1×5 vertical template is used. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of t -5,x t +b+5] to calculate the center point of the light bar.

4. The robot vision welding fillet weld recognition method according to claim 3 is characterized in that: The calculation formula of the light bar center is as follows: (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median gray value of the pixel, and the adaptive threshold b is the light bar width.

5. The robot vision welding fillet weld recognition method according to claim 4 is characterized in that: I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in a 3×3 window centered at t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

6. The robot vision welding fillet weld recognition method according to claim 5 is characterized in that: From the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t 。 7. A robot vision welding fillet weld recognition system, characterized in that: Used to implement the robot vision welding fillet weld recognition method according to any one of claims 1 to 6, the robot vision welding fillet weld recognition system comprises: An acquisition module, used for acquiring weld images through a line structured light vision sensor; The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ); The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular region of interest and save each row of y i The corresponding pixel gray value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max and the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the gray value of the light bar. The length interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum. The length interval must include the weld position. The value of row is one-fourth of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min and the corresponding row position, which is the row coordinate of the rough positioning coordinate of the fillet weld, and is recorded as y2; set a "cross" template, the size of the "cross" template in the vertical and horizontal directions is 1×5 pixels, and search from left to right along the image y2 row 0 column to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap, and the triangular area is located at the fillet weld; The boundary calculation formula is: l is the number of columns of the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2; Comparison module, used to compare i Construct a histogram with sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each of them, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i Local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3; The light bar center determination module is used to use the points (x0, y1) and (x0, y3) as starting points, search the position of the light bar row by row from left to right, from top to bottom or from bottom to top, and calculate the center of each row of the light bar, wherein the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, use the point (x0, y1) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y3) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, use (x0, y3) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use the point (x0, y1) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; The fitting fixed point module is used to fit the center points of two light strips through a random sampling consensus algorithm to find the intersection of the two straight lines, which is the precise position of the fillet weld.

8. A computer-readable storage medium, characterized in that The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the robot vision welding fillet weld recognition method as described in any one of claims 1 to 6.

Images