Weld seam appearance visual identification system for welding box type steel component

Abstract

The invention discloses a welding seam appearance visual identification system for box type steel member welding. The welding seam appearance visual identification system comprises an image acquisition module, an image preprocessing module and an image identification module. Relates to the technical field of weld joint recognition, and solves the technical problem that a weld joint area is difficult to recognize accurately under the conditions that a weld joint image has a background and weld joint surface textures are relatively complex. The method comprises the following steps: performing rotary cutting processing on an acquired welding image to obtain a plurality of processed images, performing position marking on each pixel in the image before the welding image is processed, and identifying the pixels in the processed image based on a welding seam detection model to obtain welding seam pixel position information; and finally, the weld joint image is restored based on the weld joint pixel position information corresponding to all the processed images, so that the situation of weld joint identification omission caused by interference of other contents in the image on weld joint identification can be avoided, and the accuracy degree of weld joint identification is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of weld recognition, and in particular is a weld appearance visual recognition system for welding box-type steel components. Background Art

[0002] In the traditional welding process of box-type steel components, the inspection of weld appearance quality mainly relies on manual visual inspection or simple mechanical inspection tools. This method is not only time-consuming and laborious, but also easily affected by subjective factors, resulting in inaccurate and inconsistent inspection results. With the development of industrial automation and intelligence, there is an urgent need for a system that can automatically, quickly and accurately identify the appearance quality of welds.

[0003] Most of the weld detection systems currently available on the market use a single image processing method, such as edge detection, template matching, etc. Although these methods can identify the existence of welds to a certain extent, they are easily disturbed by noise or other content in the complex background, resulting in incorrectly identified welds. Especially when there is a background in the weld image and the weld surface texture is relatively complex, it is difficult to accurately identify the weld area. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a weld appearance visual recognition system for box-type steel component welding, which is used to solve the technical problem that it is difficult to accurately identify the weld area when there is a background in the weld image and the weld surface texture is relatively complex. The present invention obtains a plurality of processed images by rotating and cropping the acquired welding image, and marks the position of each pixel in the image before processing the welding image, and then identifies the pixels in the processed image based on the weld detection model to obtain the weld pixel position information, and finally restores the weld image based on the weld pixel position information corresponding to all the processed images to solve the above-mentioned problem.

[0005] To achieve the above-mentioned object, the first aspect of the present invention provides a weld appearance visual recognition system for welding box-shaped steel components, comprising: an image acquisition module, an image preprocessing module, and an image recognition module;

[0006] The image acquisition module is used to capture the welding image of the welding area through an imaging device;

[0007] The image preprocessing module is used to process the welding image after marking the pixels in the welding image to obtain a plurality of processed images;

[0008] The image recognition module: inputs a plurality of images into the weld detection model to obtain weld pixel position information corresponding to the plurality of processed images; wherein the weld detection module is constructed based on an artificial intelligence model; and,

[0009] The weld positions corresponding to the plurality of processed images are processed to obtain weld images.

[0010] Preferably, the step of processing the welding image after marking the pixels in the welding image comprises:

[0011] A reference rectangular coordinate system is established with the center point of the welding image as the origin, pixel position information of each pixel is obtained based on the reference rectangular coordinate system, and the origin of the reference rectangular coordinate system is marked as a reference point;

[0012] The welding image is rotated n times at random angles with the origin of the rectangular coordinate system as the center, and the n rotated welding images are cut into a specified size with the origin of the rectangular coordinate system as the center to obtain n rotated images; wherein n is a positive integer, and n≤360;

[0013] The welding image is cropped to obtain a number of cropped images; the rotated image and the cropped image are marked as processed images.

[0014] It should be noted that even after the welding image is cropped and rotated, the position information of the pixels in the original welding image remains unchanged; that is, the processing of the welding image will not affect the pixel position information of the pixel mark.

[0015] Preferably, the cutting process of the welding image includes:

[0016] Step 1: Take the x-axis of the rectangular coordinate system as the dividing line, rotate the x-axis by a random angle n times, and cut the welding image with the x-axis as the dividing line to obtain 2n cut images;

[0017] Step 2: Determine whether the size of each cropped image meets the specified size; if yes, mark the cropped image as a processed image; if no, jump to step 3;

[0018] Step 3: Compare the size of the cropped image with the specified size; when the size of the cropped image is smaller than the specified size, the cropped image is repaired and the process proceeds to step 2; when the size of the cropped image is larger than the specified size, the cropped image is cropped again and the process proceeds to step 2.

[0019] It should be noted that when the cropped image is repaired, the cropped image is repaired with a white image to meet the input specifications of the subsequent model, and when cropping, the center point of the cropped image is also used as the reference point to crop it into a specified size.

[0020] Preferably, the weld detection model is constructed based on an artificial intelligence model, including:

[0021] Extracting a number of welding images from a database, establishing a plane rectangular coordinate system with the center of the welding image, and obtaining the position of welding pixels in the welding image to obtain welding pixel position information corresponding to the welding image; constructing a training set and a test set based on the number of welding images and the corresponding welding pixel position information;

[0022] A weld detection model to be trained is constructed based on the YOLO model; the weld detection model to be trained is trained through the training set, and the trained weld detection model is tested through the test set; the trained weld detection model is adjusted according to the test accuracy; and finally a weld detection model whose input is a welding image and whose output is welding pixel position information is obtained.

[0023] Preferably, the processing of the weld positions corresponding to the plurality of processed images includes:

[0024] Extract the welding pixel position information corresponding to a number of processed images respectively, and establish a blank image of a specified size;

[0025] Extract pixel information of pixels corresponding to several welding pixel position information, assign pixel information of corresponding pixels to pixels at corresponding positions in a blank image according to welding pixel positions, and obtain an initial image; correct the initial image to obtain a weld image; wherein the pixel information includes but is not limited to RGB color value, contrast, and grayscale value of the pixel.

[0026] Preferably, the correcting of the initial image includes:

[0027] Count the number of times each pixel in the initial image appears in several welding pixel position information and mark it as the pixel repetition number;

[0028] Compare the pixel repetition number Li of several pixels with the quantity threshold respectively; if the pixel repetition number Li of the pixel is less than the quantity threshold, the corresponding pixel is marked as a weak pixel; otherwise, the corresponding pixel is marked as a strong pixel; wherein Li is the number of times the i-th pixel appears;

[0029] The weak pixels in the initial image are removed to obtain the weld image.

[0030] It should be noted that the quantity threshold is determined according to the total number of processed images containing the pixel, and the quantity threshold may be 20% of the total number of processed images containing the pixel.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: by rotating and cropping the acquired welding image, a number of processed images are obtained, and before the welding image is processed, the position of each pixel in the image is marked, and then the pixels in the processed image are identified based on the weld detection model to obtain the weld pixel position information, and finally the weld image is restored based on the weld pixel position information corresponding to all the processed images. In this way, the interference of other contents in the image on weld recognition resulting in missed weld recognition can be avoided, thereby improving the accuracy of weld recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 It is a schematic diagram of the process of the present invention;

[0034] Figure 2 A logic block diagram of the present invention for cutting welding images;

[0035] Figure 3 It is a connection diagram of the modules in the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] See also Figure 1-Figure 3 , the first aspect of the present invention provides a visual recognition system for weld appearance for box-type steel member welding, including: an image acquisition module, an image preprocessing module, and an image recognition module;

[0038] See also Figure 3 As shown, the image acquisition module, the image preprocessing module and the image recognition module are progressively connected;

[0039] See also Figure 1 As shown, the image acquisition module is used to capture the welding image of the welding area through an imaging device;

[0040] Image preprocessing module: used to process the welding image after marking the pixels in the welding image to obtain several processed images;

[0041] Image recognition module: inputting several images into the weld detection model to obtain weld pixel position information corresponding to several processed images; wherein the weld detection module is constructed based on the artificial intelligence model; and,

[0042] The weld positions corresponding to the plurality of processed images are processed to obtain weld images.

[0043] The label information of each pixel remains unchanged during the processing of the welding image.

[0044] By processing the welding image, multiple processed images based on the welding image are obtained, and then the position information of the pixels of the weld in the processed image is identified through the trained weld recognition model, and then the weld pixel position information of all processed images is integrated to obtain the weld image; this method can avoid drying caused by the background to improve accuracy.

[0045] In this embodiment, the welding image is processed after marking the pixels in the welding image, including:

[0046] A reference rectangular coordinate system is established with the center point of the welding image as the reference point, i.e., the origin, and a straight line perpendicular to the edge of the image as the X-axis and the Y-axis. The pixel position information of each pixel in the welding image is obtained and marked based on the reference rectangular coordinate system.

[0047] The welding image is rotated 36 times at random angles with the origin of the rectangular coordinate system as the center, and the 36 rotated welding images are cut into a specified size with the origin of the rectangular coordinate system as the center to obtain 36 rotated images;

[0048] The welding image is cropped to obtain a number of cropped images; the rotated image and the cropped image are marked as processed images.

[0049] During implementation, even after the welding image is cropped and rotated, the position information of the pixels in the original welding image remains unchanged; that is, the processing of the welding image will not affect the pixel position information of the pixel mark.

[0050] Through pixel marking, the position information of the pixels in the original welding image can be kept unchanged even after rotation and cropping, thereby ensuring the accuracy of subsequent integration. In addition, the multiple processed images generated by rotation and cropping can increase the diversity and robustness of the image recognition model and improve the accuracy of weld recognition.

[0051] See also Figure 2As shown, in this embodiment, in order to effectively prevent the background in the welding image from interfering with subsequent recognition, the welding image is cropped, including:

[0052] Step 1: Using the x-axis of the rectangular coordinate system as the dividing line, rotate the x-axis 36 times at random angles, and cut the welding image using the x-axis as the dividing line to obtain 72 cut images;

[0053] Step 2: Determine whether the size of each cropped image meets the specified size; if yes, mark the cropped image as a processed image; if no, jump to step 3;

[0054] Step 3: Compare the size of the cropped image with the specified size; when the size of the cropped image is smaller than the specified size, the cropped image is repaired and the process proceeds to step 2; when the size of the cropped image is larger than the specified size, the cropped image is cropped again and the process proceeds to step 2.

[0055] When the cropped image is patched as described above, the cropped image is patched with a white image to meet the input specifications of the subsequent model, and when cropping, the center point of the cropped image is also used as a reference point to crop it into a specified size.

[0056] The specified size in the above description can be set according to the requirements. In this embodiment, the specified size can be set to 512×512.

[0057] The weld inspection model described in this example is built based on an artificial intelligence model, including:

[0058] Extract several welding images from the database, establish a plane rectangular coordinate system with the central pixel of the welding image as the reference point, obtain the position of the welding pixel in the welding image, and obtain the welding pixel position information of the corresponding welding image; construct a training set and a test set based on the several welding images and the corresponding welding pixel position information; wherein the ratio of the training set to the test set is 8:2;

[0059] A weld detection model to be trained is constructed based on the YOLO model; the weld detection model to be trained is trained through the training set, and the trained weld detection model is tested through the test set; the trained weld detection model is adjusted according to the test accuracy; and finally a weld detection model whose input is a welding image and whose output is welding pixel position information is obtained.

[0060] In this embodiment, the weld positions corresponding to the plurality of processed images are processed, including:

[0061] Extract the welding pixel position information corresponding to a number of processed images respectively, and establish a blank image of a specified size;

[0062] The RGB color value, contrast and grayscale value of the pixels corresponding to the position information of several welding pixels are extracted, and the pixel information of the corresponding pixels is assigned to the pixels at the corresponding position in the blank image according to the welding pixel position to obtain the initial image; the initial image is corrected to obtain the weld image.

[0063] Eliminating weak pixels can reduce the impact of noise, making the weld image clearer and facilitating subsequent weld identification and analysis.

[0064] Furthermore, the initial image is corrected, including:

[0065] Count the number of times each pixel in the initial image appears in several welding pixel position information and mark it as the pixel repetition number;

[0066] Compare the pixel repetition number Li of several pixels with the quantity threshold respectively; if the pixel repetition number Li of the pixel is less than the quantity threshold, the corresponding pixel is marked as a weak pixel; otherwise, the corresponding pixel is marked as a strong pixel; wherein Li is the number of times the i-th pixel appears;

[0067] The weak pixels in the initial image are removed to obtain the weld image.

[0068] The quantity threshold in the above description may be 5% of the total number of processed images, that is, (36+72)×5%=5.4. At this time, 5.4 may be rounded up, that is, the quantity threshold is 6.

[0069] The working principle of the present invention is as follows: the welding image of the welding area is captured by an imaging device, the pixels in the welding image are marked and then the welding image is processed to obtain a number of processed images, the several images are input into a weld detection model, the weld pixel position information corresponding to the several processed images is obtained, the welding pixel position information corresponding to the several processed images is extracted respectively, and a blank image of a specified size is established;

[0070] Pixel information of pixels corresponding to a number of welding pixel position information is extracted, and the pixel information of the corresponding pixels is assigned to the pixels at the corresponding positions in the blank image according to the welding pixel positions to obtain an initial image; the initial image is corrected to obtain a weld image.

[0071] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A weld appearance visual recognition system for box-type steel member welding, characterized in that: include: Image acquisition module, image preprocessing module, image recognition module; The image acquisition module is used to capture the welding image of the welding area through an imaging device; The image preprocessing module is used to process the welding image after marking the pixels in the welding image to obtain a plurality of processed images; The image recognition module: inputs a plurality of images into the weld detection model to obtain weld pixel position information corresponding to the plurality of processed images; wherein the weld detection module is constructed based on an artificial intelligence model; and, The weld positions corresponding to the plurality of processed images are processed to obtain weld images.

2. A weld appearance visual recognition system for box-shaped steel member welding according to claim 1, characterized in that: The step of processing the welding image after marking the pixels in the welding image comprises: A reference rectangular coordinate system is established with the center point of the welding image as the origin, pixel position information of each pixel is obtained based on the reference rectangular coordinate system, and the origin of the reference rectangular coordinate system is marked as a reference point; The welding image is rotated n times at random angles with the origin of the rectangular coordinate system as the center, and the n rotated welding images are cut into a specified size with the origin of the rectangular coordinate system as the center to obtain n rotated images; wherein n is a positive integer; The welding image is cropped to obtain a number of cropped images; the rotated image and the cropped image are marked as processed images.

3. A weld appearance visual recognition system for box-shaped steel member welding according to claim 1, characterized in that: The cutting process of the welding image comprises: Step 1: Take the x-axis of the rectangular coordinate system as the dividing line, rotate the x-axis by a random angle n times, and cut the welding image with the x-axis as the dividing line to obtain 2n cut images; Step 2: Determine whether the size of each cropped image meets the specified size; if yes, mark the cropped image as a processed image; if no, jump to step 3; Step 3: Compare the size of the cropped image with the specified size; when the size of the cropped image is smaller than the specified size, the cropped image is repaired and the process proceeds to step 2; when the size of the cropped image is larger than the specified size, the cropped image is cropped again and the process proceeds to step 2.

4. A weld appearance visual recognition system for box-shaped steel member welding according to claim 1, characterized in that: The processing of the weld positions corresponding to the plurality of processed images includes: Extract the welding pixel position information corresponding to a number of processed images respectively, and establish a blank image of a specified size; Extract pixel information of pixels corresponding to several welding pixel position information, assign pixel information of corresponding pixels to pixels at corresponding positions in a blank image according to welding pixel positions, and obtain an initial image; correct the initial image to obtain a weld image; wherein the pixel information includes but is not limited to RGB color value, contrast, and grayscale value of the pixel.

5. A weld appearance visual recognition system for box-shaped steel member welding according to claim 4, characterized in that: The correcting of the initial image comprises: Count the number of times each pixel in the initial image appears in several welding pixel position information and mark it as the pixel repetition number; Compare the pixel repetition number Li of several pixels with the quantity threshold respectively; if the pixel repetition number Li of the pixel is less than the quantity threshold, the corresponding pixel is marked as a weak pixel; otherwise, the corresponding pixel is marked as a strong pixel; wherein Li is the number of times the i-th pixel appears; The weak pixels in the initial image are removed to obtain the weld image.

6. A weld appearance visual recognition system for box-shaped steel member welding according to claim 1, characterized in that: The weld detection model is constructed based on an artificial intelligence model, including: Extracting a number of welding images from a database, establishing a plane rectangular coordinate system with the center of the welding image, and obtaining the position of welding pixels in the welding image to obtain welding pixel position information corresponding to the welding image; constructing a training set and a test set based on the number of welding images and the corresponding welding pixel position information; A weld detection model to be trained is constructed based on the YOLO model; the weld detection model to be trained is trained through the training set, and the trained weld detection model is tested through the test set; the trained weld detection model is adjusted according to the test accuracy; and finally a weld detection model whose input is a welding image and whose output is welding pixel position information is obtained.

Images