A robot vision welding fillet joint identification method, system and storage medium

Abstract

The application provides a robot visual welding fillet seam identification method, which comprises the following steps: setting a region of interest; calculating the gray value sum of each row of pixels in the region of interest, and obtaining a row with the maximum accumulated sum; finding a row with the minimum accumulated sum of gray values; finding a coarse coordinate point of a weld seam; respectively calculating two rows with the maximum accumulated sum of gray values on both sides of the weld seam; respectively calculating the center of a light strip segment on both sides of the weld seam; center fitting and weld seam identification. The application can quickly identify a fillet seam.

Description

Technical Field

[0001] The present invention relates to the field of welding technology, and in particular to a robot vision welding fillet weld recognition method, system and storage medium. Background Art

[0002] Intelligent manufacturing technology is crucial to the development of the manufacturing industry. Welding technology is a key component of the manufacturing sector, and achieving automated, efficient, and intelligent welding is crucial to achieving intelligent manufacturing. Welding robots are increasingly being used in manufacturing. By adding visual recognition to weld positions, welding robots offer enhanced automation and intelligentization, a current research hotspot in intelligent manufacturing.

[0003] Fillet welds are a common type of weld in manufacturing. When made of highly reflective metals, images of these welds captured by line structured light vision sensors will contain strong reflective noise. This is particularly true for fillet welds made of bright metals such as stainless steel, aluminum alloy, and titanium alloy. However, many existing fillet weld recognition methods can only identify fillet welds without reflective noise, resulting in large center extraction errors, slow processing, and complex methods. Summary of the Invention

[0004] The present invention provides a robot vision welding fillet weld recognition method, system and storage medium, which adopts a line structured light vision sensor composed of a line laser (or called line structured light) and a camera to collect the image of the fillet weld illuminated by the line laser, and quickly processes the image to quickly and accurately identify the weld.

[0005] This specification discloses a robot vision welding fillet weld recognition method, comprising:

[0006] Collect weld seam images through line structured light vision sensor;

[0007] Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0008] Sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum maxAnd the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the light bar grayscale value;

[0009] The length interval [y1-row, y1+row] is set by adding the row position y1 with the largest cumulative sum. This length interval should include the weld position. The value of row is generally one-quarter of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min And the corresponding row position, which is the row coordinate of the rough positioning coordinate of the fillet weld, and is recorded as y2;

[0010] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions. Search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0011] The boundary calculation formula is:

[0012]

[0013] l is the number of columns in the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found. Let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2.

[0014] By sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0015] Starting from (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar. The search is bounded by column x0, and the column coordinates of the starting point of each row search are column x0. When y1>y3, starting from (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar. Starting from (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar.

[0016] When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated;

[0017] The center points of the two light strips are fitted using the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the exact location of the fillet weld.

[0018] In this manual, sum i The formula is as follows:

[0019]

[0020] Among them, x s ,y s are the minimum row and column coordinates of the rectangular region of interest; x n and y m are the maximum row and column coordinates of the rectangular region of interest, respectively.

[0021] In this specification, a 1×5 vertical template is used to search for the position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of [x t -5,x t +b+5] to calculate the center point of the light bar.

[0022] In this manual, the calculation formula for the center of the light bar is as follows:

[0023]

[0024] (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median grayscale value of the pixel, and the adaptive threshold b is the width of the light bar.

[0025] In this specification, I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in the 3×3 window centered at f(x t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

[0026]

[0027] In this specification, the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0028] This specification also discloses a robot vision welding fillet weld recognition system, which is used to implement any of the above-mentioned robot vision welding fillet weld recognition methods. The robot vision welding fillet weld recognition system includes:

[0029] An acquisition module, used for acquiring weld images through a line structured light vision sensor;

[0030] The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0031] The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row. i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the light bar grayscale value; the interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum. This interval must include the weld position. The value of row is generally taken as one-quarter of the image width. Compare sum within this length interval iThe size of , find the sum i Minimum sum min and the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; set a "cross" template with a size of 1×5 pixels in both the vertical and horizontal directions, and search from left to right along the image y2 row and 0 column to find the two boundary points of the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld;

[0032] The boundary calculation formula is:

[0033]

[0034] l is the number of columns in the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found. Let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2.

[0035] Comparison module, used to sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0036] The light bar center determination module is used to use points (x0, y1) and (x0, y3) as starting points, search the position of the light bar row by row from left to right, from top to bottom or from bottom to top, and calculate the center of each row of the light bar, wherein the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, use point (x0, y1) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use point (x0, y3) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, use (x0, y3) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use point (x0, y1) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0037] The fitting fixed point module is used to fit the center points of two light strips using the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines. The intersection is the exact location of the fillet weld.

[0038] This specification also discloses a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions, the computer executes any one of the above-mentioned robot vision welding fillet weld recognition methods.

[0039] The embodiments of this specification can achieve at least the following beneficial effects:

[0040] This paper proposes a fast and accurate fillet weld recognition method that can be applied to robotic intelligent welding. The method uses a line laser (also known as line structured light) and a camera to capture images of fillet welds illuminated by the line laser. The method then rapidly processes the images and identifies the welds. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 Schematic diagram of the robot vision welding fillet weld recognition method involved in the present invention.

[0043] Figure 2 It is a schematic diagram of the fillet weld reflection image involved in the present invention.

[0044] Figure 3 This is a schematic diagram of the accumulation and statistics of the grayscale values ​​of pixels in each row of an image involved in the present invention. DETAILED DESCRIPTION

[0045] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0046] The disclosure below provides many different embodiments or examples for implementing different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the components and configurations of specific examples are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. In addition, the embodiments of the present invention may repeat reference numerals and / or reference letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or configurations discussed.

[0047] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] like Figure 1 As shown, this specification discloses a robot vision welding fillet weld recognition method, comprising:

[0049] Collect weld seam images through line structured light vision sensor;

[0050] Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0051] Sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the light bar grayscale value;

[0052] The interval [y1-row, y1+row] is set by adding the row position y1 with the largest cumulative sum. This interval should include the weld position. The value of row is generally one-quarter of the image width. The sum is compared within this length interval. i The size of , find the sum i Minimum sum min And the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2;

[0053] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions. Search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0054] The boundary calculation formula is:

[0055]

[0056] l is the number of columns in the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found. Let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2.

[0057] By sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0058] Starting from (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar. The search is bounded by column x0, and the column coordinates of the starting point of each row search are column x0. When y1>y3, starting from (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar. Starting from (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar.

[0059] When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated;

[0060] The center points of the two light strips are fitted using the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the exact location of the fillet weld.

[0061] In this manual, sum i The formula is as follows:

[0062]

[0063] Among them, xs ,y s are the minimum row and column coordinates of the rectangular region of interest; x n and y m are the maximum row and column coordinates of the rectangular region of interest, respectively.

[0064] In this specification, a 1×5 vertical template is used to search for the position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of [x t -5,x t +b+5] to calculate the center point of the light bar.

[0065] In this manual, the calculation formula for the center of the light bar is as follows:

[0066]

[0067] (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median grayscale value of the pixel, and the adaptive threshold b is the width of the light bar.

[0068] In this specification, I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in the 3×3 window centered at f(x t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

[0069]

[0070] In this specification, the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0071] This specification also discloses a robot vision welding fillet weld recognition system, which is used to implement any of the above-mentioned robot vision welding fillet weld recognition methods. The robot vision welding fillet weld recognition system includes:

[0072] An acquisition module, used for acquiring weld images through a line structured light vision sensor;

[0073] The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), large row and column coordinates (x n ,y m );

[0074] The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row. i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the light bar grayscale value; the interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum. This interval must include the weld position. The value of row is generally taken as one-quarter of the image width. Compare sum within this length interval i The size of , find the sum i Minimum sum min and the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; set a "cross" template with a size of 1×5 pixels in both the vertical and horizontal directions, and search from left to right along the image y2 row and 0 column to find the two boundary points of the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld;

[0075] The boundary calculation formula is:

[0076]

[0077] l is the number of columns in the image; when B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0078] Comparison module, used to sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld iThere is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0079] The light bar center determination module is used to use points (x0, y1) and (x0, y3) as starting points, search the position of the light bar row by row from left to right, from top to bottom or from bottom to top, and calculate the center of each row of the light bar, wherein the search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, use point (x0, y1) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use point (x0, y3) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar; when y3>y1, use (x0, y3) as the starting point, search the position of the light bar row by row from top to bottom, and calculate the center of the light bar; use point (x0, y1) as the starting point, search the position of the light bar row by row from bottom to top, and calculate the center of the light bar;

[0080] The fitting fixed point module is used to fit the center points of two light strips using the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines. The intersection is the exact location of the fillet weld.

[0081] This specification also discloses a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions, the computer executes any one of the above-mentioned robot vision welding fillet weld recognition methods.

[0082] In a specific embodiment, the present invention proposes a robot vision welding fillet weld recognition method based on line grayscale and statistical analysis, which can quickly identify fillet welds with reflective noise. The reflective noise of fillet welds of highly reflective materials is as follows: Figure 2 shown.

[0083] Robot vision welding fillet weld recognition methods include:

[0084] Region of interest (ROI) setting:

[0085] In order to improve the calculation speed, a rectangular region of interest (ROI) is set in the weld image based on prior knowledge. The region should include the weld and the light strip. The ROI is defined as the following equation, where x s ,y s are the minimum row and column coordinates of ROI, x n ,y mare the maximum row and column coordinates of ROI respectively.

[0086]

[0087] Calculate the sum of the grayscale values ​​of each row of pixels in the ROI and find the row with the largest cumulative sum:

[0088] Sum the grayscale values ​​of each row of pixels in the ROI area and save the y value of each row i The corresponding pixel grayscale value accumulation sum i By comparing the grayscale value of each row of pixels I(x i ,y j ) Cumulative sum i The size of , find the cumulative sum i The maximum value of sum max The row with the largest cumulative sum is y1, which is the row with the largest cumulative sum of the reflected noise intensity and the grayscale value of the light stripe. Based on experimental data from hundreds of images, this row is located a certain distance from the row where the weld is located, just away from the area with the most severe noise pollution.

[0089]

[0090] Find the row with the smallest cumulative grayscale value:

[0091] In order to reduce the amount of calculation and accuracy, the interval [y1-row, y1+row] is set by adding the row position y1 with the largest sum. This interval should include the weld position. The value of row is generally one-quarter of the image width. In this interval, the sum is compared. i The size of , find the sum i Minimum sum min And the corresponding row position, which is the rough positioning row position of the fillet weld, is recorded as y2. Figure 3 shown.

[0092] Find the coarse coordinate point of the weld:

[0093] Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions. Search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld.

[0094] The boundary calculation formula is:

[0095]

[0096] l is the number of columns in the image.

[0097] When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 line to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2;

[0098] Find the rows with the largest cumulative grayscale values ​​on both sides of the weld:

[0099] By sum i The constructed histogram shows that the weld is sum i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3;

[0100] Calculate the center of the light strip segments on both sides of the weld:

[0101] Starting from (x0, y1) and (x0, y3), search the light bar position row by row from left to right, top to bottom, or bottom to top, and calculate the center of each row of the light bar. The search is bounded by column x0, and the column coordinates of the starting point of each row search are column x0. When y1>y3, start from (x0, y1) and search the light bar position row by row from top to bottom, and calculate the center of the light bar. Start from (x0, y3) and search the light bar position row by row from bottom to top, and calculate the center of the light bar. When y3>y1, start from (x0, y3) and search the light bar position row by row from top to bottom, and calculate the center of the light bar. Start from (x0, y1) and search the light bar position row by row from bottom to top, and calculate the center of the light bar. In the process of searching for the position of the light bar, a 1×5 vertical template is used. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of [x t -5,x t +b+5] to calculate the center point of the light bar.

[0102] Assume that the boundary position point Pt of the light strip is found i is (x t ,y), where the central formula is:

[0103]

[0104] I(x t +k,y) is the median grayscale value of the pixel, and the adaptive threshold b is the width of the light bar.

[0105] Let the grayscale values ​​of these pixels be I(x t +k,y) are all based on pixel (x t +k,y) is the median of the grayscale values ​​of the pixels in the 3×3 window centered at f(x t +k,y) is the pixel (x t +k,y) grayscale value, where k∈[x t -5,x t +b+5], median I(x t +k,y) formula is:

[0106] I(x t +k,y)=Med{f(x t +k-1,y-1),f(x t +k-1,y),f(x t +k-1,y+1),

[0107] f(x t +k,y-1),f(x t +k,y),f(x t +k,y+1),

[0108] f(x t +k+1,y-1),f(x t +k+1,y),f(x t +k+1,y+1)};

[0109] The adaptive threshold b is the width of the light bar, and (U, y) is the new center position of the light bar. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , from the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t .

[0110] Center fitting and weld identification:

[0111] The center points of the two light strips are fitted using the RANSAC (Random Sample Consensus) method to find the intersection of the two straight lines, which is the exact location of the fillet weld.

[0112] The above embodiments are intended to illustrate the present invention, not to limit the present invention. Therefore, changes in illustrative values ​​or substitutions of equivalent components should still fall within the scope of the present invention.

[0113] From the above detailed description, it will be clear to those skilled in the art that the present invention can indeed achieve the aforementioned objectives and is in compliance with the provisions of the Patent Law.

[0114] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all changes and modifications that fall within the scope of the invention. The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

[0115] It should be noted that the above description of the relevant processes is for illustration and purpose only and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the processes under the guidance of this specification. However, such modifications and changes are still within the scope of this specification.

[0116] The basic concepts have been described above. It will be apparent to those skilled in the art after reading this application that the above disclosures are merely illustrative and do not constitute limitations on this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and amendments to this application. Such modifications, improvements, and amendments are suggested in this application and remain within the spirit and scope of the exemplary embodiments of this application.

[0117] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or more in different places in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0118] In addition, it will be understood by those skilled in the art that various aspects of the present application can be illustrated and described by a number of patentable categories or situations, including any new and useful combination of processes, machines, products or substances, or any new and useful improvements thereto. Therefore, various aspects of the present application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can all be referred to as "units", "modules" or "systems". In addition, various aspects of the present application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0119] The computer program code required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, conventional procedural programming languages ​​such as C programming language, VisualBasic, Fortran2103, Perl, COBOL2102, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy or other programming languages. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on a remote computer, or run completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or be connected to an external computer (such as by the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0120] In addition, unless expressly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some embodiments of the invention that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a pure software solution, for example, installation on an existing server or mobile device.

[0121] Similarly, it should be noted that in order to simplify the presentation of this disclosure and thereby facilitate understanding of one or more of the invention's embodiments, the foregoing descriptions of the embodiments of this disclosure sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this approach should not be interpreted as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject matter of the invention may possess fewer features than the single embodiment described above.

Claims

1. A robot vision welding fillet weld recognition method, characterized in that: include: Collect weld seam images through line structured light vision sensor; Set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ), maximum row and column coordinates (x n ,y m ); Sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, which is recorded as y1. The row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflected noise intensity and the light bar grayscale value; Set the length interval [y1-row, y1+row] by adding the row position y1 with the largest cumulative sum. This length interval should include the weld position, and the value of row should be one-quarter of the image width. Compare sum in this length interval. i The size of , find the sum i Minimum sum min And the corresponding row position coordinates, which are the row coordinates of the rough positioning coordinates of the fillet weld, and are recorded as y2; Set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions. Search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. The triangular area is located at the fillet weld. The boundary calculation formula is: I(x i ,y j ) is the pixel grayscale value, l is the number of columns in the image, when B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found; let the template continue to search along the y2 row to find the right boundary of the light bar, when B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found, and the average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2; By sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3; Starting from (x0, y1) and (x0, y3), search the position of the light bar row by row from left to right, top to bottom or bottom to top, and calculate the center of each row of the light bar. The search is bounded by column x0, and the column coordinates of the starting point of each row search are column x0. When y1>y3, starting from (x0, y1), search the position of the light bar row by row from top to bottom, and calculate the center of the light bar. Starting from (x0, y3), search the position of the light bar row by row from bottom to top, and calculate the center of the light bar. When y3>y1, (x0, y3) is the starting point, and the position of the light bar is searched row by row from top to bottom, and the center of the light bar is calculated; with point (x0, y1) as the starting point, the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated; The center points of the two light strips are fitted by a random sampling consensus algorithm to find the intersection of the two straight lines, which is the exact position of the fillet weld.

2. The robot vision welding fillet weld recognition method according to claim 1 is characterized in that: sum i The formula is as follows: Among them, x s and y s are the minimum row and column coordinates of the rectangular region of interest; x n and y m are the maximum row and column coordinates of the rectangular region of interest, respectively.

3. The robot vision welding fillet weld recognition method according to claim 1, characterized in that: In the process of searching for the position of the light bar, a 1×5 vertical template is used. When the average grayscale value of the pixels in the template is greater than the set threshold of 200 for the first time, the boundary position point Pt of the light bar is found. i , let (x t ,y) is the boundary point Pt of the light strip i The coordinates of [x t -5,x t +b+5] to calculate the center point of the light bar.

4. The robot vision welding fillet weld recognition method according to claim 3 is characterized in that: The calculation formula for the center of the light bar is as follows: (x t ,y) is the boundary point Pt of the light strip i The coordinates of I(x t +k,y) is the median grayscale value of the pixel, and the adaptive threshold b is the width of the light bar.

5. The robot vision welding fillet weld recognition method according to claim 4, characterized in that: I(x t +k,y) is the pixel point (x t +k,y) is the median of the grayscale values ​​of the pixels in the 3×3 window centered at f(x t +k,y) is the pixel (x t +k,y) grayscale value, median I(x t +k,y) formula is:

6. The robot vision welding fillet weld recognition method according to claim 5, characterized in that: From the light strip boundary point Pt i Continue to move the vertical template. When the average grayscale value of the pixels in the template is less than the set threshold of 200 for the first time, the boundary point Pb on the other side of the light bar is found. i is (x b ,y), then b=x b -x t 。 7. A robot vision welding fillet weld recognition system, characterized in that: For implementing the robot vision welding fillet weld recognition method according to any one of claims 1 to 6, the robot vision welding fillet weld recognition system comprises: An acquisition module, used for acquiring weld images through a line structured light vision sensor; The setting module is used to set a rectangular region of interest in the weld image. The rectangular region of interest includes the weld and the light strip. The minimum row and column coordinates of the rectangular region of interest are (x s ,y s ); The summation module is used to sum the grayscale values ​​of each row of pixels in the rectangular area of ​​interest and save the y value of each row. i The corresponding pixel grayscale value accumulation sum i , by comparing the cumulative sum of the grayscale values ​​of each row of pixels i The size of , find the cumulative sum i The maximum value of sum max And the corresponding row position, the row is recorded as y1, the row with the largest cumulative sum is the row position with the maximum cumulative sum of the reflection noise intensity and the light bar grayscale value; the length interval [y1-row, y1+row] is set by the row position y1 with the largest cumulative sum, and the length interval must include the weld position. The value of row is one-quarter of the image width, and the sum is compared within this length interval. i The size of , find the sum i Minimum sum min and the corresponding row position, which is the row coordinate of the rough positioning coordinate of the fillet weld, recorded as y2; set a "cross" template with a size of 1×5 pixels in both vertical and horizontal directions, and search from left to right along row y2 and column 0 of the image to find the two boundary points of the light strip in the triangular area where the light strip and the reflection noise overlap. This triangular area is located at the fillet weld; The boundary calculation formula is: l is the number of columns in the image. When B takes the maximum value, the left boundary of the weld position light bar (x1, y2) is found. Let the template continue to search along the y2 row to find the right boundary of the light bar. When B takes the minimum value, the right boundary of the weld position light bar (x2, y2) is found. The average value of the coordinates of the left and right boundary points is the rough position coordinate of the weld (x0, y2), where x0 = (x1 + x2) / 2. Comparison module, used to sum i Construct a histogram with the sum at the weld i Minimum value, sum on both sides of the weld i There is a local maximum for each, one of which is the global maximum sum max ; By comparing the size of y1 and y2, determine the sum on the other side of the search weld i local maximum sum max2 Search direction of the row, find sum max2 and the corresponding row position, which is denoted as y3; The light bar center determination module is used to search the position of the light bar from left to right, from top to bottom or from bottom to top row by row with points (x0, y1) and (x0, y3) as starting points, and calculate the center of each row of the light bar. The search is bounded by the x0 column, and the column coordinates of the starting point of each row search are the x0 column; when y1>y3, the light bar position is searched from top to bottom row by row with point (x0, y1) as the starting point, and the light bar center is calculated; the light bar position is searched from bottom to top row by row with point (x0, y3) as the starting point, and the light bar center is calculated; when y3>y1, the light bar position is searched from top to bottom row by row with point (x0, y3) as the starting point, and the light bar center is calculated; Starting from point (x0, y1), the position of the light bar is searched row by row from bottom to top, and the center of the light bar is calculated; the fitting fixed point module is used to fit the center points of the two light bars through a random sampling consensus algorithm to find the intersection of the two straight lines. The intersection is the exact position of the fillet weld.

8. A computer-readable storage medium, characterized in that The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the robot vision welding fillet weld recognition method according to any one of claims 1 to 6.

Images