Weld seam surface defect detection method and system

Abstract

The present application relates to a weld seam surface defect detection method and system. The weld seam surface defect detection method comprises: performing multi-angle and multi-surface acquisition of target surface images; then, reconstructing normal maps of the target surface images by means of a high-performance algorithm, and clarifying normal features of defects; and finally, on the basis of the surface normal features, aiming at transverse texture defects and height mutation defects, respectively designing a transverse texture defect detection algorithm based on two stages and rotated rectangle extraction and a height mutation defect detection algorithm based on adaptive dual-threshold segmentation. The present application overcomes texture noise and pseudo-defect interference of ground surfaces, effectively solves the problems of difficulty in feature extraction and positioning of ground surfaces of steel rail weld seams, and has the system operating efficiency superior to that of existing detection methods, thus satisfying requirements of actual production.

Description

Weld surface defect detection method and system Technical Field

[0001] The present application relates to the technical field of defect detection, and in particular to a method and system for detecting surface defects of welds. Background Art

[0002] During the seamless rail production process, five 100-meter-long fixed-length rails are welded together into a 500-meter long rail. This rail then undergoes a series of milling and other processes to create a long rail ready for on-site laying and welding. To ensure the rail's safety in subsequent service, the milled rail surface must be free of transverse graining and sudden height changes. If these defects are not discovered and addressed promptly, they can delay high-speed rail construction, jeopardize operational safety, and pose a serious threat to life and property.

[0003] Currently, rail welding bases mostly rely on manual inspection methods such as visual inspection and touch. These methods are subject to significant subjective influence from inspectors, have inconsistent inspection standards, and are inefficient. Milling and grinding workshops are also prone to dust, which can be harmful to the health of inspectors. Traditional non-destructive testing techniques, such as radiographic testing, magnetic flux leakage testing, eddy current testing, and ultrasonic testing, also struggle to meet the requirements for high-efficiency and high-precision testing due to surface damage and low efficiency. Non-contact, highly efficient machine vision is gradually replacing these methods and entering the industrial field. Using advanced sensors to capture surface images or point clouds, and then using detection algorithms to analyze or locate surface defects, machine vision plays a vital role in improving production efficiency and quality.

[0004] Deep learning methods have achieved remarkable results in object detection and localization and are rapidly being applied to applications such as defect detection. Common feature extraction and classification models offer superior feature extraction and classification performance compared to traditional algorithms. Object detection models combine feature extraction and classification with defect location prediction to complete end-to-end object detection tasks, simplifying the algorithm design process and significantly improving object detection efficiency. However, deep learning methods often require extensive training data, and while they excel in scenes with simple background types, they struggle to simulate complex real-world scenarios.

[0005] Summary of the Invention

[0006] Based on this, it is necessary to provide a weld surface defect detection method and system to address the problem that the application of deep learning methods in traditional weld surface defect detection methods often requires a large amount of training data and is difficult to simulate complex scenes in reality.

[0007] In one aspect, the present application provides a method for detecting weld surface defects, the method comprising:

[0008] Exciting the target with light sources at multiple illumination angles, collecting multiple target surface images at different illumination angles, and preprocessing the multiple target surface images at different illumination angles to obtain multiple preprocessed target surface images;

[0009] The L2 algorithm is introduced into the feature fusion convolutional neural network model to improve the deep non-Lambertian photometric stereo vision algorithm, and an improved feature fusion convolutional neural network model is obtained;

[0010] Multiple pre-processed target surface images under the same illumination angle are input into the feature fusion convolutional neural network model to obtain a normal map of the target surface image under the illumination angle;

[0011] Repeatedly inputting multiple pre-processed target surface images under the same illumination angle into the feature fusion convolutional neural network model to obtain a normal map of the target surface image under the illumination angle, until normal maps of the target surface images under all illumination angles are obtained;

[0012] Select a normal map of the target surface image under a certain illumination angle;

[0013] The polishing area and the non-polishing area are divided according to the normal map of the target surface image. The rough contour of the polishing area is obtained by outer contour search. The convex hull of the contour point set of the rough contour of the polishing area is searched and the convex hull is used as the mask map of the polishing area. The mask map is multiplied with the target surface image to obtain the normal map containing only the polishing area.

[0014] The R channel of the normal map containing only the polished area is used to obtain the initial positioning image of the defect area based on Butterworth high-pass filtering. The minimum area circumscribed rectangle of the convex hull in the initial positioning image of the area is extracted based on the rotating caliper algorithm to obtain the rough positioning area of ​​the transverse grain defect. The Sobel edge detection is used to perform fine defect positioning on the rough positioning area and the image after edge detection to obtain the normal map marked with the defect positioning box.

[0015] Obtaining a grayscale histogram of the normal map of the target surface image, determining a maximum segmentation threshold and a minimum segmentation threshold based on the grayscale histogram, locating highly sudden defect areas based on the maximum segmentation threshold and the minimum segmentation threshold, and generating a normal map with the highly sudden defect areas marked;

[0016] Return the normal map of the target surface image under a selected illumination angle until the normal maps of the target surface images under all illumination angles are processed.

[0017] On the other hand, the present application also provides a weld surface defect detection system, the weld surface defect detection system comprising:

[0018] An acquisition component, wherein the acquisition component is used to acquire an image of a target surface;

[0019] A processing component is communicatively connected to the acquisition device, and the processing component is used to execute the weld surface defect detection method mentioned above.

[0020] The present application relates to a method and system for detecting weld surface defects, which collects target surface images from multiple angles and surfaces, and then reconstructs the normal map of the target surface image through a high-performance algorithm to clarify the normal characteristics of the defect; finally, based on the surface normal characteristics, a transverse grain defect detection algorithm based on dual-stage and rotated rectangle extraction and a height mutation defect detection algorithm based on adaptive dual-threshold segmentation are designed for transverse grains and height mutation defects, respectively, to overcome the texture noise and pseudo-defect interference of the polished surface, effectively solve the problem of difficult extraction and positioning of polished surface features of rail welds, and the system operation efficiency is better than that of existing detection methods, meeting actual production needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 is a flow chart of a weld surface defect detection method provided in an embodiment of the present application.

[0022] FIG2 is a schematic diagram of a rail according to a method for detecting weld surface defects provided in an embodiment of the present application.

[0023] FIG3 is a schematic diagram of a normal diagram of a weld surface defect detection method provided in an embodiment of the present application.

[0024] FIG4 is a schematic diagram of a weld surface defect detection system provided in an embodiment of the present application.

[0025] FIG5 is a schematic diagram of an acquisition component of a weld surface defect detection system provided in an embodiment of the present application.

[0026] FIG6 is a normal diagram of the rail weld grinding surface and a schematic diagram of defect magnification of a weld surface defect detection method provided in an embodiment of the present application.

[0027] FIG7 is a diagram showing mask extraction results of a partially polished area of ​​a weld surface defect detection method provided in an embodiment of the present application.

[0028] FIG8 is a schematic diagram of Butterworth high-pass filtering results of a weld surface defect detection method provided in an embodiment of the present application.

[0029] FIG9 is a schematic diagram of the Sobel edge detection results of a weld surface defect detection method provided in an embodiment of the present application.

[0030] FIG10 is a flow chart of a transverse grain defect detection algorithm of a weld surface defect detection method provided in an embodiment of the present application.

[0031] FIG11 is an output envelope mask diagram of a weld surface defect detection method provided in an embodiment of the present application.

[0032] FIG12 is a non-defect area filtering diagram of a weld surface defect detection method provided in an embodiment of the present application.

[0033] FIG13 is a location diagram of transverse grain defects in a weld surface defect detection method provided in an embodiment of the present application.

[0034] FIG14 is a grayscale histogram of the normal R channel and the polished area of ​​a weld surface defect detection method provided in an embodiment of the present application.

[0035] FIG15 is a schematic diagram of a double-threshold interception range of a grayscale histogram of a weld surface defect detection method provided in an embodiment of the present application.

[0036] FIG16 is a location diagram of highly sudden defects in a weld surface defect detection method provided in an embodiment of the present application.

[0037] FIG17 is a timing diagram of a multi-surface parallel defect detection process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0039] As shown in FIG1 , the present application provides a method for detecting weld surface defects, comprising:

[0040] S100 , performing multi-illumination angle light source excitation on a target, collecting multiple target surface images under different illumination angles, and preprocessing the multiple target surface images under different illumination angles to obtain multiple preprocessed target surface images.

[0041] Specifically, the target surface image is a Type 60 rail welding diagram, which includes two Type 60 rails and a weld between the two Type 60 rails, that is, information of three main parts.

[0042] As shown in FIG5 , this embodiment can have five illumination angles, namely, the left rail waist surface, the right rail waist surface, the left rail foot upper surface, the right rail foot upper surface and the rail bottom surface.

[0043] S200, introduces the L2 algorithm into the feature fusion convolutional neural network model to improve the deep non-Lambertian photometric stereo vision algorithm, and obtains the improved feature fusion convolutional neural network model.

[0044] S300: Input multiple pre-processed target surface images under the same illumination angle into a feature fusion convolutional neural network model to obtain a normal map of the target surface image under the illumination angle.

[0045] S400, repeatedly executing the step of inputting multiple pre-processed target surface images under the same illumination angle into the feature fusion convolutional neural network model to obtain a normal map of the target surface image under the illumination angle, until normal maps of the target surface images under all illumination angles are obtained.

[0046] S500: Select a normal map of the target surface image under a lighting angle.

[0047] Specifically, the normal refers to the vector perpendicular to the surface of an object. The normal is a crucial property of geometric surfaces. Slow changes in the normal represent slow changes in image color. When a defect occurs, a corresponding color shift will appear in the normal map.

[0048] Specifically, as shown in Figure 13, the horizontal grain defect appears in the original image as randomly distributed, side-by-side vertical bright lines of varying lengths, with scratch-like groove features on the surface morphology. This means that the x component of the normal vector in the local area of ​​the defect will undergo a more drastic left-right swing, which is mapped as a change in the grayscale value in the normal image. By observing the change in grayscale value, the subtle left-right jitter of the surface can be captured at the pixel level.

[0049] S600, divide the polishing area and the non-polishing area according to the normal map of the target surface image, obtain the rough contour of the polishing area by outer contour search, search the convex hull of the contour point set for the rough contour of the polishing area, output the convex hull as the mask map of the polishing area, multiply the mask map with the target surface image to obtain the normal map containing only the polishing area.

[0050] Specifically, in this step, the normal map of the target surface image is analyzed, and the polishing area and the non-polishing area can be divided according to the gray value change of the B channel.

[0051] S700 uses Butterworth high-pass filtering to obtain the initial positioning image of the defect area for the R channel of the normal map containing only the polished area. The minimum area circumscribed rectangle of the convex hull in the initial positioning image of the area is extracted based on the rotating caliper algorithm to obtain the coarse positioning area of ​​the transverse grain defect. Sobel edge detection is used to accurately locate the defect in the coarse positioning area and the image after edge detection to obtain a normal map marked with a defect positioning box.

[0052] S800, obtaining a grayscale histogram of a normal map of a target surface image, determining a maximum segmentation threshold and a minimum segmentation threshold according to the grayscale histogram, locating a highly sudden defect region based on the maximum segmentation threshold and the minimum segmentation threshold, and generating a normal map with the highly sudden defect region marked.

[0053] Specifically, as shown in Figure 2, the rail has a complex profile and can be divided into the rail head, rail waist, upper surface of the rail foot, and rail base. The rail head can be further divided into the rail top surface, working edge, and non-working edge. The welded area needs to be milled and polished. The weld reinforcement in the non-rail head area is longitudinally polished to be flush with the parent material. Defects such as transverse lines and sudden height changes must be avoided.

[0054] S900 , returning to S500 to select a normal map of the target surface image under a lighting angle, until the normal maps of the target surface images under all lighting angles are processed.

[0055] In this embodiment, target surface images are captured from multiple angles and surfaces, and then a high-performance algorithm is used to reconstruct the normal map of the target surface image to identify the normal features of the defect. Finally, based on the surface normal features, a transverse grain defect detection algorithm based on dual-stage and rotated rectangle extraction and a height mutation defect detection algorithm based on adaptive dual-threshold segmentation were designed for transverse grain defects and height mutation defects, respectively. These algorithms overcome the interference of texture noise and pseudo-defects on the polished surface, effectively addressing the difficulties in extracting and locating the polished surface features of rail welds. The system also outperforms existing detection methods in terms of operational efficiency, meeting actual production needs.

[0056] In one embodiment of the present application, the S100 includes:

[0057] S110, adjusting the lighting angle so that the lighting angle illuminates the target collection area.

[0058] S120: Create a target surface image database.

[0059] S130: Capture multiple target surface images at each illumination angle, and store the captured target surface images in a target surface image database. Each target surface image has a unique corresponding illumination angle used when capturing the image.

[0060] S140 , selecting a target surface image from the target surface image library, and preprocessing the target surface image using normalization to obtain multiple preprocessed target surface images, until all target surface images in the target surface image library are processed.

[0061] Specifically, this application is primarily used to detect two types of rail defects: transverse grain defects and height mutation defects. Transverse grain defects are superficial grinding marks, similar to scratches, with lengths ranging from 3mm to 20mm, and widths and depths ranging from 0.5mm to 1mm. Under illumination from a specific direction, they appear as parallel vertical bright lines of varying lengths and depths, with a certain degree of clustering.

[0062] Height mutation defects often appear in the transition area between the weld and the base material. The height changes greatly within a short distance, and the shape and orientation of the area are irregular and irregular. In scale, they are usually larger than transverse grain defects and grinding textures.

[0063] In this embodiment, before collecting the target surface image, it is necessary to adjust the illumination angle of the light source and the shooting angle of the camera in the collection component 100 so that the light source is illuminated on the waist, upper surface of the rail foot and bottom surface of the target rail. Then, the target surface images of the waist, upper surface of the rail foot and bottom surface of the target rail are collected by the camera respectively. Multiple images need to be collected at each illumination angle, and the collected target surface images are placed in the target surface image database.

[0064] The collected target surface image is then preprocessed using a normalization method to facilitate subsequent processing.

[0065] In one embodiment of the present application, the S200 includes:

[0066] S210, selecting a pixel point from the target surface image.

[0067] S220, obtain the basic imaging formula of photometric stereo, which is shown in Formula 1. I=ρ(n,l,v)max(n T ,l,0) Formula 1

[0068] Among them, I is a target surface image collected under a certain illumination angle l, ρ(n,l,v) is the BRDF model represented by the bidirectional reflectance function, max(n T ,l,0) is the attached shadow, n is the normal vector, v is the viewing direction (i.e. the shooting angle of the camera), l is the lighting angle, and T is the transposed sign.

[0069] S230, convert Formula 1 into Formula 2. I=ρn T Formula 2

[0070] Where I is an image of the target surface captured under a certain illumination angle l, ρ is a constant, n is the normal vector, l is the illumination angle, and T is the transposed sign.

[0071] Specifically, for an image, most points are close to the Lambertian body characteristics of low-frequency reflection. According to the reflection on the Lambertian body surface, Formula 1 can be written as Formula 2.

[0072] S240, calculating a preliminary normal vector according to Formula 3;

[0073] in, is the preliminary normal vector, M is the set of target surface images collected under different illumination angles, and L is the matrix composed of all illumination angles l.

[0074] Specifically, T represents matrix transposition of the matrix.

[0075] S250, according to formula 4, the initial normal vector Normalize to get the unit normal vector.

[0076] in, is the unit normal vector, is the initial normal vector, and T is the transpose sign.

[0077] S260, obtain the final accurate normal vector n through formula 4; n d =φ(M,n) Formula 5;

[0078] Among them, n d is the final accurate normal vector, φ is the feature fusion convolutional neural network model, M is the set of collected target surface images under different illumination angles, and n is the normal vector.

[0079] S270 , returning to S210 , until all pixels in the target surface image are selected, the normal map composed of the normal vectors of all pixels is the normal map of the target surface image.

[0080] Specifically, Lambertian body refers to the phenomenon that when the incident energy is reflected uniformly in all directions, that is, the incident energy is isotropically reflected in all directions in the entire hemispherical space with the incident point as the center, it is called diffuse reflection, also known as isotropic reflection. A perfect diffuse body is called a Lambertian body.

[0081] The surface normal is an important attribute of a geometric surface. In the normal map, the mapping relationship between the color space and the normal space of a point on the surface is shown in Formula 6.

[0082] Among them, x, y, z are the components of the normal vector of the point on the surface along each axis in three-dimensional space, and R, G, B are the components of the red, green and blue colors of the point in the normal map.

[0083] Figure 3 shows the surface normal map of the sphere, along with the colors of some points and their corresponding normal vectors. Slow changes in the normal map represent slow changes in the image color. When a defect occurs, a corresponding color change will appear in the normal map.

[0084] In this embodiment, as shown in Figure 6, since the transverse grain defects appear in the original image as randomly distributed, side-by-side vertical bright lines of varying lengths, and have groove features similar to scratches in the surface morphology, this means that the normal vector of the local defect area will undergo a more drastic left-right swing change in the x-component, which is mapped as a change in the grayscale value in the normal image. Then, by observing the change in grayscale value, the subtle jitter of the left and right directions of the surface can be captured at the pixel level. Therefore, the transverse grain defects can be identified and located based on this feature.

[0085] High-speed mutation defects manifest as a sharp change in the height of the polished surface, that is, in the defect area, there will be a local surface morphology that transitions sharply from one plane to another. The normal vector x or y component corresponding to the transition area will suddenly change in value to a maximum or minimum value in a certain direction, and maintain the maximum or minimum value throughout the transition section, that is, it will appear as a local irregular area with extremely large or extremely small pixel values ​​in the R or G channel grayscale image, and the scale will also be larger than the horizontal grain defect. The two types of high-speed mutation defects are more obvious in the R channel or G channel images. By finding such irregular areas, the high-speed mutation defects on the surface can be located.

[0086] In one embodiment of the present application, the step S600 includes:

[0087] S610: Select a target surface image from a target surface image library.

[0088] S620 , segmenting the target surface image into a background area and a polishing area using an OTSU threshold.

[0089] S630: Obtain a rough outline of the polishing area by searching the outer contour of the polishing area.

[0090] S640 , searching for the convex hull of the contour point set of the rough contour of the polishing area, obtaining the minimum convex polygon surrounding the contour point set of the polishing area, and outputting the convex hull as a mask map of the polishing area.

[0091] S650, returning to S610, until all target surface images in the target surface image library are selected.

[0092] Specifically, the OTSU algorithm, also known as the maximum inter-class variance method, is a threshold determination algorithm commonly used for adaptive threshold calculation in binary image segmentation. It separates an image into two components, background and object, based on the image's grayscale distribution. Segmentation is based on maximizing the inter-class variance between the two classes, essentially minimizing intra-class differences.

[0093] In this example, as shown in Figure 7, the surface of the polished rail area is relatively flat, with a relatively uniform and nearly perpendicular normal vector, a large and stable z-component, and a pixel value close to 255 in the corresponding B-channel image. In contrast, the surface of the unpolished area is relatively rough, with a chaotic, non-perpendicular normal vector direction, a large and small z-component variation, and a relatively dark image. Therefore, the polished and unpolished areas are divided by analyzing the grayscale value changes in the B-channel of the surface normal vector image.

[0094] As shown in FIG11 , FIG11 is a schematic diagram of an output envelope mask diagram.

[0095] As shown in FIG12 , this embodiment can effectively and relatively precisely extract the polishing area mask, thereby eliminating cluttered background interference for subsequent defect detection.

[0096] As shown in FIG10 , in one embodiment of the present application, the step S700 includes:

[0097] S710 , separating the defect contour corresponding to the grayscale feature of the light and dark variation defect in the normal image R channel of the surface image of the target to be measured by a second-order Butterworth high-pass filter function.

[0098] S720: Rotate the horizontal and vertical tangents of the convex hull, construct a minimum area circumscribed rectangle of the convex hull based on the horizontal and vertical tangents, output the centroid position, length, width, and rotation angle of the minimum area circumscribed rectangle, and output an envelope mask for the initial positioning of the defect area.

[0099] S730 , using a first-order Sobel operator to separate the thin line contours of the normal map R channel of the surface image of the target to be measured, to obtain an edge detection image.

[0100] S740 , performing an AND operation on the envelope mask of the initial defect area positioning and the edge detection image to obtain an image intersection of the envelope mask of the initial defect area positioning and the edge detection image.

[0101] S750: Perform a rotated rectangle fitting on the image intersection of the envelope mask for the initial positioning of the defect area and the edge detection image, extract and filter the outline of the defect area, perform an external rectangle fitting on the outline of the defect area, and output a defect positioning frame, which is the location of the transverse grain defect.

[0102] Specifically, the rotated rectangle extraction is also called the minimum area circumscribed rectangle extraction. The minimum area circumscribed rectangle fitting is performed on the contour, which can describe the contour geometric features, obtain the aspect ratio, deflection angle and other shape parameters of each contour, and screen out the defect feature contour.

[0103] The minimum area bounding rectangle can be extracted using the rotating calcaneal algorithm. The rotating calcaneal algorithm is based on the convex hull and is based on the conclusion that the minimum area bounding rectangle of a convex polygon has at least one side that coincides with a side of the convex polygon.

[0104] In this embodiment, as shown in FIG8 , the normal map R channel of the surface image of the target to be measured is processed by a second-order Butterworth high-pass filter function to preliminarily separate the defect contours corresponding to the grayscale features of the light and dark variation defects. Gaussian filtering, OSTU threshold segmentation, and morphological processing are combined to remove image noise, delete thin connections between isolated points and adjacent connected areas, and enhance the purity of the image.

[0105] Next, the defect outline is extracted from pseudo-defects and noise using a rotated rectangle extraction method. The aspect ratio threshold is set to 2.2, and the deflection angle threshold is set to ±15 degrees to filter the defect's characteristic outline and remove texture noise interference. Finally, an envelope mask is obtained for the initial positioning of the defect area, which serves as the coarse location area for the transverse grain defect.

[0106] Afterwards, as shown in FIG9 , the x-direction template of the Sobel operator is used to perform edge detection on the surface image of the target to be measured. Only one full-image convolution is required to complete the gradient calculation and edge detection, which has high computational efficiency.

[0107] Next, the envelope mask from the coarse positioning phase is ANDed with the edge-detected image to suppress artifacts and texture noise. A rotated rectangle is fitted to all contours, and the defect area contours are extracted and filtered through the rotated rectangle. The aspect ratio threshold is set to 2.2, and the deflection angle threshold is set to ±8 degrees to filter the defect feature contours.

[0108] As shown in Figure 13, the defect outline is finally fitted with an external rectangle and output as the defect positioning frame.

[0109] In one embodiment of the present application, S710 includes:

[0110] S711 , performing coarse positioning of the defect area on the normal map R channel of the surface image of the target to be measured according to Formula 7.

[0111] Among them, u, v are the coordinates of discrete points in the frequency domain, D0 is the set cutoff frequency, and D (u,v) is the distance between the frequency domain point (u, v) and the center of the frequency domain rectangle, and n is the filter order.

[0112] S712, removing image noise through Gaussian filtering.

[0113] In this embodiment, the Butterworth high-pass filter has an adjustable filter order. A higher order results in a greater attenuation rate in the stopband, and a smoother transition between the passband and the stopband. A first-order Butterworth high-pass filter does not cause ringing in the spatial domain image, while a second-order Butterworth high-pass filter only exhibits very slight ringing, achieving a good balance between effective filtering and acceptable ringing.

[0114] In one embodiment of the present application, the S720 includes:

[0115] S721, obtaining the top, bottom, leftmost and rightmost corner points of the convex hull convex polygon outline.

[0116] S722: Construct two horizontal tangent lines and two vertical tangent lines of the convex polygon through the four corner points, and use the rectangle formed by the two horizontal tangent lines and the two vertical tangent lines as the initial circumscribed rectangle.

[0117] S723: Rotate the two horizontal tangents and the two vertical tangents simultaneously clockwise until one of the tangents coincides with a side of the convex polygon, reconstruct a circumscribed rectangle based on the coincident tangents and the remaining corner points, and calculate the area of ​​the circumscribed rectangle.

[0118] S724, return to rotate the two horizontal tangents and the two vertical tangents clockwise simultaneously until the rotation angle of the tangents is greater than 90 degrees for the first time, and then stop rotating.

[0119] S725 , comparing the areas of multiple circumscribed rectangles formed during the rotation process, and outputting the centroid position, length, width, and rotation angle of the circumscribed rectangle with the smallest area.

[0120] In this embodiment, the minimum area circumscribed rectangle can be extracted using a rotating caliper algorithm. The rotating caliper algorithm is based on the convex hull and is based on the conclusion that the minimum area circumscribed rectangle of a convex polygon has at least one side that coincides with a side of the convex polygon.

[0121] The convex hull contour is fitted with the minimum area circumscribed rectangle by rotating rectangle extraction to describe the geometric characteristics of the convex hull contour, and the shape parameters such as the aspect ratio and deflection angle of each contour are obtained to screen out the defect feature contour.

[0122] In one embodiment of the present application, the S800 includes:

[0123] S810 , adaptively obtaining a large threshold and a small threshold based on a grayscale histogram of a normal map channel of the surface image of the target to be measured.

[0124] S820: Binarize the R channel and the G channel of the normal map using a large threshold and a small threshold.

[0125] S830: Based on the R channel and the G channel of the large threshold and the small threshold, the defect results of the preliminary separation are merged, and dark spots and isolated spots in the image are eliminated through morphological processing.

[0126] S840, extract all contours in the surface image of the target to be measured, screen and delete small areas, remove noise interference, obtain high-speed mutation defects, and extract the outer rectangular frame as the location of the high-speed mutation defect area.

[0127] Specifically, highly abrupt defects appear as transition areas between the weld bar and the base material. Their shape and orientation are irregular and irregular, and they are larger in scale than transverse grain defects and surface polishing textures. In the R or G channels of the normal image, these defects appear as extremely large or small irregular regions, while the normal components of the background area exhibit numerical clustering. Therefore, highly abrupt regions can be identified using grayscale histogram statistics.

[0128] In this embodiment, a dual threshold is first adaptively obtained and used to binarize the R and G channels of the current polished surface normal vector image, that is, a set of pixels within two ranges, whose grayscale values ​​are greater than the left endpoint and less than the small threshold, and whose grayscale values ​​are greater than the large threshold and less than the right endpoint, are intercepted as the preliminary segmentation of the defect area.

[0129] Secondly, based on the R channel and G channel of double threshold segmentation, the preliminary separated defect results are merged, and the dark spots and isolated points in the image are eliminated through morphological processing.

[0130] Finally, all contours in the image are extracted, small areas are screened and deleted, noise interference is removed, highly mutated defects are obtained, and the outer rectangular frame is extracted as the defect area location. The output image is shown in Figure 16.

[0131] As shown in FIG. 15 , in one embodiment of the present application, the S810 includes:

[0132] S811, obtaining the grayscale histogram of the R channel or G channel of the normal map of the target surface image to be measured, removing small extreme value burrs in the grayscale histogram through a mean smoothing operation, and excluding outlier scattered points on the left and right sides of the grayscale histogram.

[0133] S812: Determine the left endpoint l and the right endpoint r of the grayscale histogram.

[0134] S813, find all peak points in the grayscale histogram, and determine the leftmost peak point fl and the rightmost peak point fr.

[0135] S814: Set a distance threshold. Use the horizontal coordinate reached by moving the distance threshold from one end point to the peak point on the same side as the segmentation threshold. Determine the left threshold and the right threshold according to Formula 8.

[0136] Among them, F L is the left threshold, F R is the right threshold, l is the left endpoint, r is the right endpoint, f l is the leftmost peak point, f r It is the peak point on the far right.

[0137] S815, compare the left threshold F L and the right threshold F R The larger the threshold value is, the larger the threshold value is, and the smaller the threshold value is, the smaller the threshold value is,the smaller the threshold value is,.

[0138] In this embodiment, as shown in FIG14 , the surface normal component of the polishing area R channel and its corresponding grayscale histogram has a certain degree of directional aggregation, thereby forming several peaks on the histogram, which are the background area.

[0139] Subregions of the image with extremely small or large local pixel values ​​are distributed within the limited intervals on the left and right sides of the histogram. The histogram shape within these intervals is relatively low and flat, indicating that these subregions are highly abrupt defects. Therefore, finding the values ​​in the limited intervals on the left and right sides of the grayscale histogram can correspond to the grayscale abrupt regions.

[0140] By adopting double thresholds to extract rail height mutation defects, the robustness of the extracted rail height mutation defects is improved.

[0141] As shown in FIG. 4 and FIG. 5 , in one embodiment of the present application, a weld surface defect detection system is provided. The weld surface defect detection system includes an acquisition component 100 and a processing component 200 .

[0142] The acquisition component 100 is used to acquire target surface images. The processing component 200 is in communication with the acquisition component 100 and is used to execute the weld surface defect detection method as described in any of the above embodiments.

[0143] In this embodiment, the collection component 100 includes a main frame and a collection end, and the collection end includes two symmetrical side ends and a bottom end, wherein the side end includes two floating front plates for adjusting the lighting angle.

[0144] Each terminal front plate contains more than 18 point light sources and a 2000W pixel camera. Each side terminal is equipped with a linear guide to adjust the distance between the terminal (camera) and the rail surface.

[0145] The two cameras at the side end respectively capture images of the rail waist and the upper surface of the rail foot, while the bottom end captures surface data images of the rail bottom.

[0146] After executing the weld surface defect detection method described in this application, the detection accuracy is characterized by calculating the accuracy (P) and missed detection rate (O) of the detection algorithm and the F1 score through Formula 9.

[0147] TP (True Positive) refers to the number of examples correctly predicted as defects. FP (False Positive) refers to the number of examples predicted as background or other types of defects. FN (False Negative) refers to the number of examples incorrectly predicted as non-defects or other types of defects.

[0148] The Intersection over Union (IoU) is calculated using Formula 10. When evaluating the detection accuracy of the algorithm, the Intersection over Union (IoU) threshold is set to 0.5. That is, if the IoU is greater than 0.5, the position prediction is considered correct. When the category prediction is also correct, the defect sample is correctly predicted.

[0149] Among them, A and B are the true value and true value of the predicted box respectively, Area(A) is the area of ​​rectangle A, Area(B) is the area of ​​rectangle B, and Area(A∩B) is the area of ​​the intersection of rectangles A and B.

[0150] The defect detection method based on normal features is compared with commonly used deep learning object detection methods, including SSD, RetinaNet, Faster R-CNN (MobileNetV2) and Faster R-CNN (resnet50+FPN).

[0151] To adapt the deep learning method described above, the dataset was first classified into training and test sets. 165 transverse grains and 110 high-frequency mutations were selected as the training set, while 128 transverse grain defects and 119 high-frequency mutations were selected as the training set. The training set was augmented by mirroring and rotation. Before deep learning model training, the images were sliced ​​(400×400 pixels per unit) using a sliding window with a step size of 100 pixels. Slices containing complete defects were retained to achieve data augmentation. The experimental algorithm verification platform used the PyTorch framework on a desktop PC equipped with a 2.1GHz Intel Xeon(R) Gold 5218R CPU, an NVIDIA GeForce GTX 3090 GPU, and 128GB of RAM. Hyperparameters for the training process all used their default values.

[0152] During the experiment, when testing the deep learning-based method, the IoU threshold of defect detection was also set to 0.5 and the confidence threshold was 0.5. That is, the prediction value that satisfied both the IoU greater than 0.5 and the confidence greater than 0.5 was used as the final defect prediction result.

[0153] Table 1 compares the results of our proposed method and a deep learning-based method for detecting surface defects in rail weld grinding. It shows that our proposed method achieves an accuracy rate exceeding 90% for both high-frequency abrupt changes and transverse grain detection, while its missed detection rate is less than 2%. Regarding deep learning methods, with the exception of SSD, which has a lower accuracy rate for transverse grain detection, the remaining methods all achieve an accuracy rate exceeding 80%. However, all methods exhibit a missed detection rate exceeding 45%. This means that while the majority of the true positive samples predicted by the deep learning-based method are actually positive, a large proportion of defects are predicted as non-defective, resulting in a significant number of missed detections, which is unacceptable in industrial applications. However, our proposed method not only achieves high detection accuracy but also reduces the probability of predicting defects as non-defective, fully meeting industrial requirements.

[0154] Table 1 - Comparison of rail weld surface defect detection results

[0155] Table 2 compares the F1 scores of our proposed method and deep learning-based methods. The F1 score ranges from 0 to 1, with the closer it is to 1, the better the method's balance between recall and precision. As can be seen from the table, our proposed method achieves scores close to 1 for both highly mutated and horizontal grains, while deep learning-based methods generally achieve lower F1 scores, with RetinaNet achieving only 0.616 and 0.348, respectively. This is primarily due to a high missed detection rate, resulting in a very low recall rate.

[0156] Table 2 - Comparison of F1 scores for rail weld surface defect detection

[0157] Regarding the problem of high recall rate of detection by deep learning methods, analysis shows that this may be due to the large background noise interference on the polished surface of the rail weld and the small number of training samples. Feature extraction is easily interfered by the polishing noise, and defect feature extraction is easily mixed with background texture, so that the confidence of defect detection results is generally low.

[0158] To further illustrate, the advantage of this method is that it can accurately identify and locate surface defects of rail welds through experimental design feature extraction and recognition methods based on normal maps when the sample size is small.

[0159] In industrial production, improving efficiency is one of the keys to driving the core competitiveness of the manufacturing industry. Therefore, this paper analyzes the operational efficiency of the proposed method. The experimental method is to run the complete multi-surface defect detection process 10 times, recording the time from the acquisition of the image sequence of the first surface to the output of the visual inspection results for all the surfaces to be inspected. The time required for image acquisition of a single polished surface, photometric stereo vision model inference, and defect detection algorithm execution is also recorded, and the average value is taken as the final result. The experimental results are shown in Table 3.

[0160] Table 3 - System operation efficiency test results

[0161] The system's total defect detection time for all surfaces under inspection averaged 23.784 seconds, meeting the inspection time requirement and significantly improving efficiency compared to the 30-40 seconds required for manual inspection. The system's overall inspection is not simply the sum of the times for each inspection surface, but is achieved through parallel computing within a certain range, as shown in Figure 17. To avoid light source interference, the acquisition of photometric image sequences for multiple surfaces and the inference of photometric stereo models must be performed serially. However, the system software's multi-threaded operation enables parallel processing of the multi-surface image sequences. Data storage is also completed in a hidden background thread, enabling timely output of inspection results. Therefore, the system's total runtime is no longer simply the sum of the times for each individual step, improving the system's overall inspection efficiency.

[0162] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting surface defects of welds, applied to 60 steel rails, characterized in that, The weld surface defect detection method includes: Performing multi-illumination angle light source excitation on the target, collecting multiple target surface images under different illumination angles, preprocessing the multiple target surface images under different illumination angles to obtain multiple preprocessed target surface images; Introducing the L2 algorithm into the feature fusion convolutional neural network model to improve the depth non-Lambertian photometric stereo vision algorithm, obtaining an improved feature fusion convolutional neural network model; Inputting multiple preprocessed target surface images under the same illumination angle into the feature fusion convolutional neural network model to obtain a normal map of the target surface image under this illumination angle; Repeatedly performing the step of inputting multiple preprocessed target surface images under the same illumination angle into the feature fusion convolutional neural network model to obtain a normal map of the target surface image under this illumination angle until normal maps of the target surface images under all illumination angles are obtained; Selecting a normal map of the target surface image under one illumination angle; Segmenting the grinding area and the non-grinding area according to the normal map of the target surface image, obtaining the rough contour of the grinding area by outer contour search, finding the convex hull of the contour point set of the rough contour of the grinding area, outputting the convex hull as the grinding area mask image, and multiplying the mask image with the target surface image to obtain a normal map containing only the grinding area; Performing Butterworth high-pass filtering on the R channel of the normal map containing only the grinding area to obtain a defect area preliminary positioning image, extracting the minimum area circumscribed rectangle of the convex hull in the area preliminary positioning image based on the rotating calipers algorithm to obtain the rough positioning area of the transverse texture defect, and performing defect fine positioning on the rough positioning area and the edge-detected image using Sobel edge detection to obtain a normal map marked with a defect positioning frame; Obtaining the gray histogram of the normal map of the target surface image, determining the maximum segmentation threshold and the minimum segmentation threshold according to the gray histogram, positioning the height mutation defect area based on the maximum segmentation threshold and the minimum segmentation threshold, and generating a normal map marked with the height mutation defect area; Returning the normal map of the target surface image under one selected illumination angle until all normal maps of the target surface images under all illumination angles are processed.

2. The method for detecting surface defects of welds according to claim 1, characterized in that, The performing multi-illumination angle light source excitation on the target, collecting multiple target surface images under different illumination angles, and preprocessing the target surface images includes: Adjusting the illumination angle so that the illumination angle irradiates the target acquisition area; Creating a target surface image database; Collecting multiple target surface images under each illumination angle and putting the collected target surface images into the target surface image database; each target surface image has the unique illumination angle used when the image was acquired; Selecting a target surface image from the target surface image library, preprocessing the target surface image using normalization to obtain multiple preprocessed target surface images until all target surface images in the target surface image library are processed.

3. The method for detecting surface defects of welds according to claim 2, characterized in that, The introducing the L2 algorithm into the feature fusion convolutional neural network model to improve the depth non-Lambertian photometric stereo vision algorithm and obtaining an improved feature fusion convolutional neural network model includes: Select a pixel point from the target surface image; Obtain the basic imaging formula of photometric stereo, the basic imaging formula of photometric stereo is shown in formula 1; I = ρ(n, l, v) max(n T , l, 0) Formula 1; Among them, I is a target surface image collected at a certain illumination angle l, ρ(n, l, v) is the BRDF model represented by the bidirectional reflection function, max(n T , l, 0) is the attached shadow, n is the normal vector, v is the viewing direction, l is the illumination angle, and T is the transpose symbol; Convert formula 1 into formula 2; I = ρn T l Equation 2; Where I is a target surface image captured at a certain illumination angle l, ρ is a constant, n is the normal vector, l is the illumination angle, and T is the transposed sign; Calculate the preliminary normal vector according to Formula 3; Among them, is the preliminary normal vector, M is the set of collected target surface images under different illumination angles, and L is the matrix composed of all illumination angles l; According to formula 4 for the preliminary normal vector Normalize to obtain the unit normal vector: Among them, is the unit normal vector, is the initial normal vector, T is the transposition sign; The final accurate normal vector n is obtained through formula 4; n d = φ(M, n) Formula 5; where n d is the final accurate normal vector, φ is the feature fusion convolutional neural network model, M is the set of surface images of the acquisition target under different illumination angles, and n is the normal vector; Return to select a pixel point from the target surface image until all the pixels in the target surface image are selected. The normal map composed of the normal vectors of all the pixels is the normal map of the target surface image.

4. The weld surface defect detection method according to claim 3, wherein, The method of dividing the polishing area and the non-polishing area according to the normal map of the target surface image, obtaining the rough contour of the polishing area by searching the outer contour, searching the convex hull of the contour point set for the rough contour of the polishing area, outputting the convex hull as the mask map of the polishing area, and multiplying the mask map with the target surface image to obtain the normal map containing only the polishing area includes: Select a target surface image from a target surface image library; The target surface image is segmented into background area and polishing area by OTSU threshold; Obtain the rough outline of the polishing area by searching the outer contour of the polishing area; Find the convex hull of the contour point set of the rough contour of the polishing area, obtain the minimum convex polygon surrounding the contour point set of the polishing area, and output the convex hull as the mask map of the polishing area; Return to select a target surface image from the target surface image library until all target surface images in the target surface image library are selected.

5. The weld surface defect detection method according to claim 4, wherein, The R channel of the normal map of the target surface image is used to obtain the initial positioning image of the defect area based on Butterworth high-pass filtering, and the minimum area circumscribed rectangle of the convex hull in the initial positioning image of the area is extracted based on the rotating caliper algorithm to obtain the rough positioning area of ​​the transverse grain defect, and the Sobel edge detection is used to perform defect precise positioning on the rough positioning area and the image after edge detection to obtain the normal map marked with the defect positioning frame, including: The defect contour corresponding to the grayscale feature of the light and dark variation defect in the normal image R channel of the surface image of the target to be measured is separated by a second-order Butterworth high-pass filter function; Rotate the horizontal and vertical tangents of the convex hull, construct the minimum area circumscribed rectangle of the convex hull based on the horizontal and vertical tangents, and output the centroid position, length, width and rotation angle of the minimum area circumscribed rectangle Degree, output the envelope mask of the initial positioning of the defect area; The first-order Sobel operator is used to separate the thin line contour of the normal map R channel of the surface image of the target to be measured to obtain an edge detection image; Performing an AND operation on the envelope mask of the initial defect area location and the edge detection image to obtain an image intersection of the envelope mask of the initial defect area location and the edge detection image; The image intersection of the envelope mask and the edge detection image for the initial positioning of the defect area is fitted with a rotated rectangle, the outline of the defect area is extracted and screened, the outline of the defect area is fitted with an external rectangle, and the normal map marked with a defect positioning frame is output. The defect positioning frame is the position of the transverse grain defect.

6. The weld surface defect detection method according to claim 5, wherein, Separating the defect contour corresponding to the gray-scale feature of the brightness change in the R channel of the normal map of the surface image of the target to be measured through a second-order Butterworth high-pass filtering function, including: Roughly locate the defect area on the R channel of the normal map of the surface image of the target to be measured according to Formula 7; where u and v are the coordinates of discrete points in the frequency domain, D0 is the set cut-off frequency, and D (u,v) is the distance between the point (u, v) in the frequency domain and the center of the rectangle in the frequency domain, and n is the filter order; Removing image noise through Gaussian filtering.

7. The weld surface defect detection method according to claim 6, wherein, Rotating the horizontal tangent and the vertical tangent of the rotated convex hull, constructing the minimum-area circumscribed rectangle of the convex hull according to the horizontal tangent and the vertical tangent, and outputting the centroid position, length, width and rotation angle of the minimum-area circumscribed rectangle, including: Obtaining the four corner points of the convex hull convex polygon contour that are the uppermost, lowermost, leftmost and rightmost; Constructing two horizontal tangents and two vertical tangents of the convex polygon through the four corner points, and using the rectangle formed by the two horizontal tangents and the two vertical tangents as the initial circumscribed rectangle; Simultaneously rotating the two horizontal tangents and the two vertical tangents clockwise until one of the tangents coincides with one side of the convex polygon, reconstructing the circumscribed rectangle according to the coincident tangent and the remaining corner points, and calculating the area of the circumscribed rectangle; Returning to simultaneously rotate the two horizontal tangents and the two vertical tangents clockwise until the rotation angle of the tangent is greater than 90 degrees for the first time and then stopping the rotation; Comparing the areas of the multiple circumscribed rectangles formed during the rotation process, and outputting the centroid position, length, width and rotation angle of the minimum-area circumscribed rectangle.

8. The weld surface defect detection method according to claim 7, wherein, Obtaining the gray-scale histogram of the normal map of the target surface image, determining the maximum segmentation threshold and the minimum segmentation threshold according to the gray-scale histogram, locating the height mutation defect area based on the maximum segmentation threshold and the minimum segmentation threshold, and generating a normal map marked with the height mutation defect area, including: Adaptively obtaining the large threshold and the small threshold based on the gray-scale histogram of the normal map channel of the surface image of the target to be measured; Performing binaryzation on the R channel and the G channel of the normal map using the large threshold and the small threshold; Based on the R channel and the G channel of the large threshold and the small threshold, merging the preliminarily separated defect results, and eliminating the dark spots and isolated points in the image through morphological processing; Extracting all the contours in the surface image of the target to be measured, screening and deleting the small-area regions, removing the noise interference, obtaining the height mutation defect, extracting the external rectangular frame as the location of the height mutation defect area, and generating a normal map marked with the height mutation defect area.

9. The weld surface defect detection method according to claim 8, wherein, The adaptively obtaining the large threshold and the small threshold based on the gray-scale histogram of the normal map channel of the surface image of the target to be measured includes: Obtaining the gray-scale histogram of the R channel or the G channel of the normal map of the surface image of the target to be measured, and removing the small extreme value burrs in the gray-scale histogram through the mean value Smoothing operation and excluding the outlier scattered points on the left and right sides of the gray-scale histogram; Determining the left endpoint l and the right endpoint r of the gray-scale histogram; Finding all the peak points in the gray-scale histogram, and determining the leftmost peak point fl and the rightmost peak point fr; Set a distance threshold, and use the abscissa reached by moving the distance threshold from one end point to the peak point on the same side as the segmentation threshold, and determine the left threshold and the right threshold according to Formula 8; Among them, F L is the left threshold, F R is the right threshold, l is the left endpoint, r is the right endpoint, fl is the leftmost peak point, and fr is the rightmost peak point; Compare the left threshold F L with the right threshold F R to determine their magnitudes. The larger threshold is the large threshold range, and the smaller threshold is the small threshold range.

10. A weld surface defect detection system, characterized in that, The weld surface defect detection system includes: An acquisition component for acquiring the surface image of the target; A processing component communicatively connected to the acquisition device, and the processing component is used to execute the weld surface defect detection method according to any one of claims 1-9.

Images