Method, device, processor and welding system for determining region of interest

Abstract

The present application provides a method, device, processor, and welding system for determining a region of interest. The method includes: obtaining an initial weld image; processing the initial weld image to obtain a binary weld image; and determining a region of interest based on the proportion of predetermined pixels in each pixel row of the binary weld image. In this method, a binary weld image of the initial weld image can be automatically obtained, and the proportion of predetermined pixels in the pixel row is first determined. The region of interest can be directly and accurately determined based on the proportion of predetermined pixels. This method does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, thereby greatly improving the accuracy of the determined region of interest, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

Description

Technical Field

[0001] The present application relates to the field of welding image processing, and in particular, to a method, device, computer-readable storage medium, processor, and welding system for determining a region of interest. Background Art

[0002] In the process of tracking welds using laser vision sensors, the processing speed of weld images has a direct impact on the responsiveness of the weld tracking system. In order to improve the speed of image processing, the region of interest should be extracted from the image first, and then the welding can be controlled according to the region of interest to achieve better welding accuracy and faster speed.

[0003] Currently, there are two commonly used methods for obtaining regions of interest: manual selection or template matching.

[0004] Extracting the region of interest through manual selection requires human participation. Manual selection of the weld inflection point area is less practical in work scenarios that require high-precision selection. The extracted region of interest is not accurate and the efficiency is low.

[0005] The region of interest is extracted by template matching. In actual welding scenes, due to the influence of arc light, metal reflection inside the weld, metal reflection on the parent material surface and surface dust, the weld in the actual image is quite different from the weld in the template, which causes the template matching to fail and the region of interest to be extracted accurately. For example, in the case of Figure 1 Better results can be achieved in Figure 2 This will cause the template matching to fail, and the region of interest cannot be accurately extracted, which will lead to inaccurate welding.

[0006] Therefore, a method that can automatically and accurately extract the region of interest is urgently needed.

[0007] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country to those skilled in the art. Summary of the Invention

[0008] The main purpose of this application is to provide a method, device, computer-readable storage medium, processor and welding system for determining a region of interest, so as to solve the problem in the prior art that the region of interest cannot be accurately extracted.

[0009] According to one aspect of an embodiment of the present invention, a method for determining a region of interest is provided, comprising: acquiring an initial weld image; processing the initial weld image to obtain a binary weld image; and determining a region of interest based on a proportion of predetermined pixel points in each pixel row of the binary weld image, wherein the predetermined pixel point is a pixel point having a grayscale value equal to a predetermined value, and the pixel row includes a plurality of pixel points having equal vertical coordinates.

[0010] Optionally, the region of interest is determined based on the proportion of predetermined pixel points in each pixel row of the binary weld image, including: determining a plurality of target pixel rows, the target pixel rows being the pixel rows in which the proportion of the predetermined pixel points is greater than a predetermined threshold; determining a target continuous region based on the plurality of target pixel rows, the target continuous region being the region formed by the target pixel rows with consecutive row numbers; and determining the position of the region of interest based on the position of the target continuous region.

[0011] Optionally, determining multiple target pixel rows includes: determining an initial pixel row, the initial pixel row including at least one predetermined pixel point; screening the pixel rows in which the proportion of the predetermined pixel points in the initial pixel row is greater than a predetermined threshold and the row number is within a predetermined range, to obtain multiple target pixel rows, wherein the proportion of the predetermined pixel points is the ratio of the number of the predetermined pixel points to the total number of the pixel points in the initial pixel row.

[0012] Optionally, determining a target continuous region based on the plurality of target pixel rows includes: determining a region where the plurality of target pixel rows with consecutive row numbers are located as an initial continuous region; and determining the initial continuous region with the largest area as the target continuous region.

[0013] Optionally, determining the position of the region of interest according to the position of the target continuous region includes: determining the position of the target continuous region as the position of the region of interest.

[0014] Optionally, after determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, the method further includes: correcting the region of interest to obtain the corrected region of interest.

[0015] Optionally, the region of interest is corrected to obtain the corrected region of interest, including: calculating the average value of the width of the region of interest in multiple frames of the binary weld image; calculating the difference between the average value and the width value of the region of interest in the current frame; and when the difference is greater than a predetermined difference, adjusting the width value of the region of interest in the current frame to the average value.

[0016] According to another aspect of an embodiment of the present invention, a device for determining a region of interest is also provided, including: an acquisition unit for acquiring an initial weld image; a processing unit for processing the initial weld image to obtain a binary weld image; a determination unit for determining a region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, wherein the predetermined pixel point is a pixel point whose grayscale value is equal to a predetermined value, and the pixel row includes a plurality of pixel points with equal vertical coordinates.

[0017] According to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the methods described above.

[0018] According to yet another aspect of the embodiments of the present invention, a processor is provided, wherein the processor is configured to run a program, wherein any one of the methods is executed when the program is run.

[0019] According to another aspect of an embodiment of the present invention, a welding system is provided, comprising: a welding device; and a device for determining a region of interest, communicatively connected to the welding device, wherein the device for determining a region of interest is configured to execute any one of the methods described above.

[0020] In an embodiment of the present invention, an initial weld image is first acquired, and then the initial weld image is processed to obtain a binary weld image. Finally, a region of interest is determined based on the proportion of predetermined pixel points in each pixel row of the binary weld image. In this method, a binary weld image of the initial weld image can be automatically acquired, and the proportion of predetermined pixel points in the pixel row where the predetermined pixel points are located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of predetermined pixel points. This method does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem of the inability to accurately extract the region of interest in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0022] Figure 1 A schematic diagram of extracting regions of interest using a template matching method is shown;

[0023] Figure 2Another schematic diagram of extracting regions of interest using a template matching method is shown;

[0024] Figure 3 A schematic flow chart of a method for determining a region of interest according to an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram showing an image of a middle weld seam is shown;

[0026] Figure 5 shows a schematic diagram of a binarized weld image;

[0027] Figure 6 A schematic diagram showing the result of the proportion of predetermined pixels;

[0028] Figure 7 A schematic diagram of the region of interest obtained according to this scheme is shown;

[0029] Figure 8 A schematic structural diagram of a device for determining a region of interest according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element or intervening elements may be present. Moreover, in the specification and claims, when it is described that an element is "connected to" another element, the element may be "directly connected to" the other element or "connected to" the other element through a third element.

[0034] As mentioned in the background technology, the existing technology cannot accurately extract the region of interest. In order to solve the above problem, a typical embodiment of the present application provides a method, device, computer-readable storage medium, processor and welding system for determining the region of interest.

[0035] According to an embodiment of the present application, a method for determining a region of interest is provided. Figure 3 FIG. 1 is a flow chart of a method for determining a region of interest according to an embodiment of the present application. Figure 3 As shown, the method includes the following steps:

[0036] Step S101, obtaining an initial weld image;

[0037] Step S102, processing the initial weld image to obtain a binary weld image;

[0038] Step S103, determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, wherein the predetermined pixel points are pixel points whose grayscale values ​​are equal to the predetermined value, and the pixel row includes a plurality of the above-mentioned pixel points with equal vertical coordinates.

[0039] In the above method, an initial weld image is first obtained, and then the initial weld image is processed to obtain a binary weld image. Finally, the region of interest is determined based on the proportion of predetermined pixel points in each pixel row of the binary weld image. In this method, the binary weld image of the initial weld image can be automatically obtained, and the proportion of predetermined pixel points in the pixel row where they are located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of predetermined pixel points. This method does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

[0040] Specifically, according to the attached Figure 1 and attached Figure 2It can be seen that the area where the inflection point in the figure is located is the area of ​​interest to be determined. In the pixel rows of the area of ​​interest, white pixels account for the largest proportion, and the area where the inflection point is located almost has white pixels distributed from left to right. Therefore, by pre-determining the proportion of pixels, the area where the inflection point is located can be accurately located, that is, the area of ​​interest can be accurately determined.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] In a specific embodiment of the present application, the initial weld image is processed to obtain a binary weld image, including: processing the initial weld image using a median filter algorithm to obtain an intermediate weld image; and processing the intermediate weld image using an adaptive threshold segmentation algorithm to obtain the binary weld image. In this embodiment, processing the initial weld image using the median filter algorithm can remove isolated noise points in the image, and then processing the intermediate weld image using the adaptive threshold segmentation algorithm can convert the intermediate weld image into a binary weld image with two pixel values. In this way, the image has only two grayscale values, ensuring that subsequent image processing is faster.

[0043] In one embodiment, the obtained middle weld image is as follows: Figure 4 As shown in the figure, the obtained binary weld image is as follows Figure 5 As shown in the figure, the binary weld image contains two grayscale values ​​0 and 255, where 0 represents black and 255 represents white. This makes it possible to more accurately determine the proportion of predetermined pixels.

[0044] In one embodiment of the present application, a region of interest is determined based on the proportion of predetermined pixel points in each pixel row of the binary weld image, including: determining multiple target pixel rows, where the target pixel rows are pixel rows where the proportion of the predetermined pixel points is greater than a predetermined threshold; determining a target continuous region based on the multiple target pixel rows, where the target continuous region is a region formed by the target pixel rows with consecutive row numbers; and determining the position of the region of interest based on the position of the target continuous region. In this embodiment, the position of the region of interest can be determined more accurately based on the position of the target continuous region, thereby further solving the problem of the inability to accurately extract the region of interest in the prior art.

[0045] In another embodiment of the present application, determining multiple target pixel rows includes: determining an initial pixel row, the initial pixel row including at least one of the predetermined pixel points; screening the pixel rows in the initial pixel row whose proportion of the predetermined pixel points is greater than a predetermined threshold and whose row numbers are within a predetermined range, to obtain multiple target pixel rows, wherein the proportion of the predetermined pixel points is the ratio of the number of the predetermined pixel points to the total number of the pixel points in the initial pixel row. In this embodiment, the number of rows and columns can be calculated from the binary weld image, and the image can be traversed row by row from the upper left corner to the lower right corner of the binary weld image, the grayscale value of the predetermined pixel points in the initial pixel row can be read, and the number of predetermined pixel points can be counted, and then the ratio of the predetermined pixel points to the total number of pixel points in the initial pixel row can be calculated, and the initial pixel rows with row numbers within the predetermined range can be used as target pixel rows, so as to obtain accurate multiple target pixel rows, thereby further ensuring that the target continuous area can be accurately determined based on the accurate multiple target pixel rows.

[0046] Specifically, calculate the number of rows img_rows and columns img_cols in the binary weld image. After traversing the binary weld image, count the number of predetermined pixels num_0_pixl. The gray value of the predetermined pixel is 0. Calculate the proportion of the predetermined pixel ratio, zero_rate, using the following formula: zero_rate = num_0_pixl / img_cols. Get the result graph of the proportion of the predetermined pixel points, as shown in the figure below. Figure 6 As shown, in Figure 6 Set a predetermined threshold T in the image, retain the row numbers whose predetermined pixel ratio is greater than T, put them into the array possible_row, and then process the array. Generally speaking, the position of the weld is distributed in the middle position of the binary weld image. Therefore, the row number may not be at the top or bottom of the image. The initial pixel rows whose row numbers are within the predetermined range are retained. The predetermined range is [img_row / 5,img_row*4 / 5] and stored in the array possible_row_2.

[0047] In another embodiment of the present application, determining a target continuous region based on multiple target pixel rows includes: determining a region containing multiple target pixel rows with consecutive row numbers as an initial continuous region; and determining the initial continuous region with the largest area as the target continuous region. In this embodiment, there are multiple initial continuous regions, and the region of interest in the binary weld image should occupy a large area. Therefore, the initial continuous region with the largest area is obtained and used as the target continuous region. This further ensures that the region of interest can be accurately determined based on the target continuous region.

[0048] Specifically, there are multiple initial continuous regions in the array possible_row_2, and the areas S of the multiple initial continuous regions are calculated respectively, such as Figure 6 As shown in , in the region greater than 0.1, there are multiple initial continuous regions. Figure 4 In the binary weld image, only the area where the weld inflection point is located has the largest number of predetermined pixels and the largest area. Therefore, the initial continuous area with the largest area is selected as the target continuous area, and the region of interest is subsequently determined based on the target continuous area.

[0049] In another embodiment of the present application, determining the position of the region of interest based on the position of the target continuous region includes: determining the position of the target continuous region as the position of the region of interest. In this embodiment, it is possible to further directly and accurately determine that the target continuous region is the region of interest, and further directly and accurately determine that the position of the target continuous region is the position of the region of interest, thereby further resolving the problem of the inability to accurately extract the region of interest in the prior art.

[0050] In one embodiment, determining the position of the target continuous region as the position of the region of interest includes: determining the position of the upper boundary line of the target continuous region as the starting position of the region of interest; determining the position of the lower boundary line of the target continuous region as the end position of the region of interest; determining the position of the left boundary line of the target continuous region as the position of the left boundary line of the region of interest; determining the position of the right boundary line of the target continuous region as the position of the right boundary line of the region of interest; determining the difference between the row number of the position of the upper boundary line of the target continuous region and the row number of the position of the lower boundary line as the width of the region of interest; determining the difference between the column number of the position of the left boundary line of the target continuous region and the column number of the position of the right boundary line as the length of the region of interest; determining the position of the region of interest based on the starting position, the end position, the position of the left boundary line of the region of interest, and the position of the right boundary line of the region of interest. In this embodiment, the position of the region of interest can be further accurately determined, thereby further solving the problem that the region of interest cannot be accurately extracted in the prior art.

[0051] Specifically, the row number of the target continuous area has a row number interval, and the row number interval includes the target pixel row with the minimum row number and the target pixel row with the maximum row number. The minimum value is the left value of the row number interval, and the maximum value is the right value of the row number interval. The left value of the row number interval is the position of the upper boundary line of the target continuous area. The position of the upper boundary line of the target continuous area is determined as the starting position of the area of ​​interest, and then the size of the area of ​​interest is determined, and the width and length of the area of ​​interest are determined. The left value of the row number interval of the target continuous area is used as the upper boundary line up_line of the area of ​​interest, and the length longth of the target continuous area is used as the width of the area of ​​interest. The area of ​​the target continuous area is obtained by the following formula: S = img_cols*longth.

[0052] In a specific embodiment, Figure 6 As shown in , the row number interval of the target continuous area is [573,617]. In fact, the target continuous area is the area of ​​interest, and the area of ​​interest also starts from the 573th row of the image and ends at the 617th row. Then, the left value of the row number interval is determined to be the upper boundary line of the area of ​​interest, that is, row 573 is the upper boundary line of the area of ​​interest, and the length of the target continuous area is the difference between the minimum and maximum values ​​of the row number interval, that is, 617-573. The length of the target continuous area is 44, and the width of the area of ​​interest is 44. Generally, the area of ​​interest should be a rectangular area. Therefore, the sum of the number of columns of all pixel rows in the target continuous area is used as the length of the area of ​​interest. In this way, an accurate area of ​​interest can be obtained.

[0053] In another embodiment, the obtained target continuous area is a rectangular area containing an inflection point, which can be as follows: Figure 7 The area formed by the two straight lines shown can be used to horizontally scale the target continuous area, that is, to scale from the left and right sides of the area to the middle to a predetermined ratio. During the scaling process, it is necessary to ensure that the two inflection points are always within the target continuous area, because the area where the two inflection points are located should be the area of ​​interest. Therefore, the target continuous area is scaled to obtain the scaled target continuous area, and the position of the scaled target continuous area is determined to be the position of the area of ​​interest.

[0054] In order to further ensure the accuracy of the region of interest, in another embodiment of the present application, after determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the above-mentioned binary weld image, the above-mentioned method also includes: correcting the above-mentioned region of interest to obtain the above-mentioned region of interest after correction.

[0055] Specifically, in the process of tracking the region of interest, there may be positioning deviations. For example, the region of interest cannot completely cover the area of ​​the weld inflection point. Therefore, after determining the region of interest, this solution also needs to correct the deviation of the region of interest to further ensure the accuracy of the region of interest.

[0056] In a specific embodiment of the present application, the above-mentioned region of interest is corrected to obtain the corrected region of interest, including: calculating the average value of the width of the above-mentioned region of interest in multiple frames of the above-mentioned binary weld images; calculating the difference between the average value and the width value of the above-mentioned region of interest in the current frame; if the difference is greater than a predetermined difference, adjusting the width value of the above-mentioned region of interest in the current frame to the average value. In this embodiment, by correcting the width of the region of interest, the accuracy of the region of interest can be further guaranteed, thereby further solving the problem of the inability to accurately extract the region of interest in the prior art.

[0057] Specifically, within 1S, the physical position of the weld and its ambient lighting conditions will not change, the camera frame rate is fps, and a new array hg with a capacity of fps*1S is created. When the array capacity is full (that is, when the number of elements in the hg array is consistent with the fps value), the array elements are cleared. Starting from the first frame, the width value of the region of interest is calculated and stored in the array hg. When there is only the first frame, the average value is the width value of the region of interest of the first frame. When there are multiple frames, the average value is the average value of the widths of the regions of interest of multiple frames. A predetermined difference value is set to Δh, and the difference between the average value and the width value of the region of interest of the current frame is calculated. If the difference is greater than Δh, the width value of the region of interest of the current frame is adjusted to the average value. If the difference is less than Δh, the width value of the region of interest of the current frame remains unchanged. When the hg array capacity is full, the elements are cleared, the width value of the region of interest of the current frame is stored back in the hg array and the calculation is performed again until the width values ​​of the regions of interest of all frames are calculated.

[0058] More specifically, the region of interest finally obtained by the solution of this application is as follows Figure 7 As shown in the figure, the framed part is the region of interest.

[0059] The present application also provides a device for determining a region of interest. It should be noted that the device for determining a region of interest in the present application can be used to execute the method for determining a region of interest provided in the present application. The device for determining a region of interest provided in the present application is described below.

[0060] Figure 8 Schematic diagram of a device for determining a region of interest according to an embodiment of the present application. Figure 8As shown, the device includes:

[0061] An acquisition unit 10 is used to acquire an initial weld image;

[0062] The processing unit 20 is used to process the initial weld image to obtain a binary weld image;

[0063] The determination unit 30 is used to determine the area of ​​interest based on the proportion of predetermined pixel points in each pixel row of the above-mentioned binary weld image, wherein the above-mentioned predetermined pixel points are pixel points whose grayscale values ​​are equal to the predetermined value, and the above-mentioned pixel row includes multiple such pixel points with equal vertical coordinates.

[0064] In the above-mentioned device, the acquisition unit acquires the initial weld image, the processing unit processes the above-mentioned initial weld image to obtain a binary weld image, and the determination unit determines the region of interest based on the proportion of predetermined pixel points in each pixel row of the above-mentioned binary weld image. In this device, the binary weld image of the initial weld image can be automatically acquired, and the proportion of predetermined pixel points in the pixel row where they are located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of predetermined pixel points. The device does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

[0065] Specifically, according to the attached Figure 1 and attached Figure 2 It can be seen that the area where the inflection point in the figure is located is the area of ​​interest to be determined. In the pixel rows of the area of ​​interest, white pixels account for the largest proportion, and the area where the inflection point is located almost has white pixels distributed from left to right. Therefore, by pre-determining the proportion of pixels, the area where the inflection point is located can be accurately located, that is, the area of ​​interest can be accurately determined.

[0066] In a specific embodiment of the present application, the processing unit includes a first processing module and a second processing module. The first processing module is configured to process the initial weld image using a median filter algorithm to obtain an intermediate weld image; the second processing module is configured to process the intermediate weld image using an adaptive threshold segmentation algorithm to obtain the binary weld image. In this embodiment, processing the initial weld image using the median filter algorithm can remove isolated noise points in the image. Then, processing the intermediate weld image using the adaptive threshold segmentation algorithm can convert the intermediate weld image into a binary weld image having two pixel values. This image has only two grayscale values, ensuring that subsequent image processing is faster.

[0067] In one embodiment, the obtained middle weld image is as follows: Figure 4 As shown in the figure, the obtained binary weld image is as follows Figure 5 As shown in the figure, the binary weld image contains two grayscale values ​​0 and 255, where 0 represents black and 255 represents white. This makes it possible to more accurately determine the proportion of predetermined pixels.

[0068] In one embodiment of the present application, the determination unit includes a first determination module, a second determination module, and a third determination module. The first determination module is used to determine multiple target pixel rows, where the target pixel rows are the pixel rows whose proportion of the predetermined pixel points is greater than a predetermined threshold; the second determination module is used to determine a target continuous area based on the multiple target pixel rows, where the target continuous area is an area formed by the target pixel rows with consecutive row numbers; and the third determination module is used to determine the position of the region of interest based on the position of the target continuous area. In this embodiment, the position of the region of interest can be determined more accurately based on the position of the target continuous area, thereby further solving the problem of the inability to accurately extract the region of interest in the prior art.

[0069] In another embodiment of the present application, the first determination module includes a first determination submodule and a screening submodule, the first determination submodule is used to determine the initial pixel row, the initial pixel row includes at least one of the predetermined pixel points; the screening submodule is used to screen the pixel rows whose ratio of the predetermined pixel points in the initial pixel row is greater than a predetermined threshold and whose row number is within a predetermined range, to obtain a plurality of the target pixel rows, the ratio of the predetermined pixel points being the ratio of the number of the predetermined pixel points to the total number of the pixel points in the initial pixel row. In this embodiment, the number of rows and columns can be calculated from the binary weld image, the image can be traversed row by row from the upper left corner to the lower right corner of the binary weld image, the grayscale value of the predetermined pixel points in the initial pixel row can be read, the number of the predetermined pixel points can be counted, and the ratio of the predetermined pixel points to the total number of the pixel points in the initial pixel row can be calculated, and the initial pixel rows whose row numbers are within the predetermined range can be used as the target pixel rows, so that the target continuous area can be accurately determined based on the accurate multiple target pixel rows.

[0070] Specifically, calculate the number of rows img_rows and columns img_cols in the binary weld image. After traversing the binary weld image, count the number of predetermined pixels num_0_pixl. The gray value of the predetermined pixel is 0. Calculate the proportion of the predetermined pixel ratio, zero_rate, using the following formula: zero_rate = num_0_pixl / img_cols. Get the result graph of the proportion of the predetermined pixel points, as shown in the figure below. Figure 6As shown, in Figure 6 Set a predetermined threshold T in the image, retain the row numbers whose predetermined pixel ratio is greater than T, put them into the array possible_row, and then process the array. Generally speaking, the position of the weld is distributed in the middle position of the binary weld image. Therefore, the row number may not be at the top or bottom of the image. The initial pixel rows whose row numbers are within the predetermined range are retained. The predetermined range is [img_row / 5,img_row*4 / 5] and stored in the array possible_row_2.

[0071] In another embodiment of the present application, the second determination module includes a second determination submodule and a third determination submodule. The second determination submodule is configured to determine that the region containing multiple target pixel rows with consecutive row numbers is the initial continuous region; and the third determination submodule is configured to determine that the initial continuous region with the largest area is the target continuous region. In this embodiment, there are multiple initial continuous regions, and the region of interest in the binary weld image should occupy a large area. Therefore, the initial continuous region with the largest area is obtained and used as the target continuous region. This further ensures that the region of interest can be accurately determined based on the target continuous region.

[0072] Specifically, there are multiple initial continuous regions in the array possible_row_2, and the areas S of the multiple initial continuous regions are calculated respectively, such as Figure 6 As shown in , in the region greater than 0.1, there are multiple initial continuous regions. Figure 4 In the binary weld image, only the area where the weld inflection point is located has the largest number of predetermined pixels and the largest area. Therefore, the initial continuous area with the largest area is selected as the target continuous area, and the region of interest is subsequently determined based on the target continuous area.

[0073] In another embodiment of the present application, the third determination module includes a fourth determination submodule, which is configured to determine that the position of the target continuous region is the position of the region of interest. In this embodiment, the target continuous region can be directly and accurately determined to be the region of interest, and the position of the target continuous region can be directly and accurately determined to be the position of the region of interest, thereby further resolving the problem of the inability to accurately extract the region of interest in the prior art.

[0074] In one embodiment, the fourth determination submodule is further used to determine the position of the upper boundary line of the above-mentioned target continuous area as the starting position of the above-mentioned region of interest; the fourth determination submodule is further used to determine the position of the lower boundary line of the above-mentioned target continuous area as the end position of the above-mentioned region of interest; the fourth determination submodule is further used to determine the position of the left boundary line of the above-mentioned target continuous area as the position of the left boundary line of the above-mentioned region of interest; the fourth determination submodule is further used to determine the position of the right boundary line of the above-mentioned target continuous area as the position of the right boundary line of the above-mentioned region of interest; the fourth determination submodule is further used to determine that the difference between the row number of the position of the upper boundary line of the above-mentioned target continuous area and the row number of the position of the lower boundary line is the width of the above-mentioned region of interest; the fourth determination submodule is further used to determine that the difference between the column number of the position of the left boundary line of the above-mentioned target continuous area and the column number of the position of the right boundary line is the length of the above-mentioned region of interest; the position of the above-mentioned region of interest is determined according to the above-mentioned starting position, the above-mentioned end position, the position of the left boundary line of the above-mentioned region of interest and the position of the right boundary line of the above-mentioned region of interest. In this embodiment, the position of the region of interest can be further accurately determined, thereby further solving the problem that the region of interest cannot be accurately extracted in the prior art.

[0075] Specifically, the row number of the target continuous area has a row number interval, and the row number interval includes the target pixel row with the minimum row number and the target pixel row with the maximum row number. The minimum value is the left value of the row number interval, and the maximum value is the right value of the row number interval. The left value of the row number interval is the position of the upper boundary line of the target continuous area. The position of the upper boundary line of the target continuous area is determined as the starting position of the area of ​​interest, and then the size of the area of ​​interest is determined, and the width and length of the area of ​​interest are determined. The left value of the row number interval of the target continuous area is used as the upper boundary line up_line of the area of ​​interest, and the length longth of the target continuous area is used as the width of the area of ​​interest. The area of ​​the target continuous area is obtained by the following formula: S = img_cols*longth.

[0076] In a specific embodiment, Figure 6As shown in , the row number interval of the target continuous area is [573,617]. In fact, the target continuous area is the area of ​​interest, and the area of ​​interest also starts from the 573th row of the image and ends at the 617th row. Then, the left value of the row number interval is determined to be the upper boundary line of the area of ​​interest, that is, row 573 is the upper boundary line of the area of ​​interest, and the length of the target continuous area is the difference between the minimum and maximum values ​​of the row number interval, that is, 617-573. The length of the target continuous area is 44, and the width of the area of ​​interest is 44. Generally, the area of ​​interest should be a rectangular area. Therefore, the sum of the number of columns of all pixel rows in the target continuous area is used as the length of the area of ​​interest. In this way, an accurate area of ​​interest can be obtained.

[0077] In another embodiment, the obtained target continuous area is a rectangular area containing an inflection point, which can be as follows: Figure 7 The area formed by the two straight lines shown can be used to horizontally scale the target continuous area, that is, to scale from the left and right sides of the area to the middle to a predetermined ratio. During the scaling process, it is necessary to ensure that the two inflection points are always within the target continuous area, because the area where the two inflection points are located should be the area of ​​interest. Therefore, the target continuous area is scaled to obtain the scaled target continuous area, and the position of the scaled target continuous area is determined to be the position of the area of ​​interest.

[0078] In order to further ensure the accuracy of the region of interest, in another embodiment of the present application, the above-mentioned device also includes a correction unit, which is used to correct the above-mentioned region of interest after determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the above-mentioned binary weld image, so as to obtain the above-mentioned region of interest after correction.

[0079] Specifically, in the process of tracking the region of interest, there may be positioning deviations. For example, the region of interest cannot completely cover the area of ​​the weld inflection point. Therefore, after determining the region of interest, this solution also needs to correct the deviation of the region of interest to further ensure the accuracy of the region of interest.

[0080] In a specific embodiment of the present application, the correction unit includes a first calculation module, a second calculation module, and an adjustment module. The first calculation module is used to calculate the average value of the width of the above-mentioned region of interest in the above-mentioned binary weld images of multiple frames; the second calculation module is used to calculate the difference between the above-mentioned average value and the width value of the above-mentioned region of interest in the current frame; and the adjustment module is used to adjust the width value of the above-mentioned region of interest in the current frame to the above-mentioned average value when the above-mentioned difference is greater than a predetermined difference. In this embodiment, by correcting the width of the region of interest, the accuracy of the region of interest can be further guaranteed, thereby further solving the problem of the inability to accurately extract the region of interest in the prior art.

[0081] Specifically, within 1S, the physical position of the weld and its ambient lighting conditions will not change, the camera frame rate is fps, and a new array hg with a capacity of fps*1S is created. When the array capacity is full (that is, when the number of elements in the hg array is consistent with the fps value), the array elements are cleared. Starting from the first frame, the width value of the region of interest is calculated and stored in the array hg. When there is only the first frame, the average value is the width value of the region of interest of the first frame. When there are multiple frames, the average value is the average value of the widths of the regions of interest of multiple frames. A predetermined difference value is set to Δh, and the difference between the average value and the width value of the region of interest of the current frame is calculated. If the difference is greater than Δh, the width value of the region of interest of the current frame is adjusted to the average value. If the difference is less than Δh, the width value of the region of interest of the current frame remains unchanged. When the hg array capacity is full, the elements are cleared, the width value of the region of interest of the current frame is stored back in the hg array and the calculation is performed again until the width values ​​of the regions of interest of all frames are calculated.

[0082] More specifically, the region of interest finally obtained by the solution of this application is as follows Figure 7 As shown in the figure, the framed part is the region of interest.

[0083] The above-mentioned device for determining the region of interest includes a processor and a memory. The above-mentioned acquisition unit, processing unit and determination unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0084] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to accurately extract the region of interest.

[0085] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0086] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for determining a region of interest is implemented.

[0087] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for determining a region of interest is executed when the program is run.

[0088] An embodiment of the present application further provides a welding system, which includes a welding transposition and a region of interest determination device, the region of interest determination device being communicatively connected to the above-mentioned welding device, and the above-mentioned region of interest determination device being used to execute any one of the above-mentioned methods.

[0089] In the above-mentioned welding system, since any one of the methods for determining the region of interest is included, in this method, a binary weld image of the initial weld image can be automatically obtained, and the proportion of the predetermined pixel points in the pixel row in which they are located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of the predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of the predetermined pixel points. This method does not require manual selection of the region of interest, nor does it require the use of a template matching scheme to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem that the region of interest cannot be accurately extracted in the prior art.

[0090] An embodiment of the present invention provides a device, comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are performed:

[0091] Step S101, obtaining an initial weld image;

[0092] Step S102, processing the initial weld image to obtain a binary weld image;

[0093] Step S103, determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, wherein the predetermined pixel points are pixel points whose grayscale values ​​are equal to the predetermined value, and the pixel row includes a plurality of the above-mentioned pixel points with equal vertical coordinates.

[0094] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0095] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0096] Step S101, obtaining an initial weld image;

[0097] Step S102, processing the initial weld image to obtain a binary weld image;

[0098] Step S103, determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, wherein the predetermined pixel points are pixel points whose grayscale values ​​are equal to the predetermined value, and the pixel row includes a plurality of the above-mentioned pixel points with equal vertical coordinates.

[0099] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the above-mentioned units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0101] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0103] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0104] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0105] 1) The method for determining the region of interest of the present application first obtains an initial weld image, then processes the initial weld image to obtain a binary weld image, and finally determines the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image. In this method, the binary weld image of the initial weld image can be automatically obtained, and the proportion of predetermined pixel points in the pixel row where it is located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of predetermined pixel points. This method does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

[0106] 2) The device for determining the region of interest of the present application comprises an acquisition unit that acquires an initial weld image, a processing unit that processes the initial weld image to obtain a binary weld image, and a determination unit that determines the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image. In this device, the binary weld image of the initial weld image can be automatically acquired, and the proportion of predetermined pixel points in the pixel row where the image is located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of predetermined pixel points. This device does not require manual selection of the region of interest, nor does it require the use of a template matching solution to obtain the region of interest, thereby greatly improving the accuracy of the determined region of interest, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

[0107] 3) The welding system of the present application includes any method for determining the region of interest. In this method, a binary weld image of the initial weld image can be automatically obtained, and the proportion of the predetermined pixel points in the pixel row in which they are located is first determined. The number of predetermined pixel points in the pixel row in the region of interest to be determined should be greater than the number of predetermined pixel points in other pixel rows, and the proportion of the predetermined pixel points should be larger. Therefore, the region of interest can be directly and accurately determined based on the proportion of the predetermined pixel points. This method does not require manual selection of the region of interest, nor does it require the use of a template matching scheme to obtain the region of interest, so that the accuracy of the determined region of interest is greatly improved, thereby solving the problem of the inability to accurately extract the region of interest in the prior art.

[0108] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for determining a region of interest, characterized in that: include: Acquire an initial weld image; Processing the initial weld image to obtain a binary weld image; Determine a plurality of target pixel rows, wherein the target pixel rows are pixel rows in which a proportion of predetermined pixel points is greater than a predetermined threshold; determine a target continuous region based on the plurality of target pixel rows, wherein the target continuous region is a region formed by the target pixel rows having consecutive row numbers; determine a position of the region of interest based on a position of the target continuous region, wherein the predetermined pixel point is a pixel point with a grayscale value equal to a predetermined value, and the pixel row includes a plurality of the pixel points having equal vertical coordinates; After determining the region of interest based on the proportion of predetermined pixel points in each pixel row of the binary weld image, the method further includes: calculating an average value of the width of the region of interest in multiple frames of the binary weld image; calculating the difference between the average value and the width value of the region of interest in the current frame; and when the difference is greater than a predetermined difference, adjusting the width value of the region of interest in the current frame to the average value.

2. The method according to claim 1, characterized in that Determine multiple target pixel rows, including: Determining an initial pixel row, where the initial pixel row includes at least one predetermined pixel point; The pixel rows in which the proportion of the predetermined pixel points in the initial pixel row is greater than a predetermined threshold and the row number is within a predetermined range are screened to obtain multiple target pixel rows, where the proportion of the predetermined pixel points is the ratio of the number of the predetermined pixel points to the total number of the pixel points in the initial pixel row.

3. The method according to claim 1, characterized in that Determining a target continuous area according to the plurality of target pixel rows includes: Determine an area where a plurality of target pixel rows with consecutive row numbers are located as an initial continuous area; The initial continuous region with the largest area is determined as the target continuous region.

4. The method according to claim 1, wherein Determining the position of the region of interest according to the position of the target continuous region includes: The position of the target continuous region is determined as the position of the region of interest.

5. A device for determining a region of interest, characterized in that: include: An acquisition unit, used for acquiring an initial weld image; a processing unit, configured to process the initial weld image to obtain a binary weld image; a determination unit, configured to determine a plurality of target pixel rows, wherein the target pixel rows are pixel rows in which a proportion of predetermined pixel points is greater than a predetermined threshold; determine a target continuous region based on the plurality of target pixel rows, wherein the target continuous region is a region formed by the target pixel rows having consecutive row numbers; and determine a position of the region of interest based on a position of the target continuous region, wherein the predetermined pixel point is a pixel point with a grayscale value equal to a predetermined value, and the pixel row includes a plurality of the pixel points having equal vertical coordinates; The deflection correction unit is configured to, after determining a region of interest based on a proportion of predetermined pixel points in each pixel row of the binary weld image, calculate an average value of widths of the region of interest in multiple frames of the binary weld image; calculate a difference between the average value and the width value of the region of interest in a current frame; and, if the difference is greater than a predetermined difference, adjust the width value of the region of interest in the current frame to the average value.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 4.

7. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 4 when running.

8. A welding system, characterized in that: include: welding equipment; A device for determining a region of interest is communicatively connected to the welding device, and the device for determining a region of interest is used to execute the method according to any one of claims 1 to 4.

Images