A computer vision-based steel structure fatigue crack detection method and device

Abstract

This invention relates to a computer vision-based method and apparatus for detecting fatigue cracks in steel structures, comprising: acquiring a steel structure image; performing fatigue crack identification on the steel structure image to determine crack regions; extracting crack lengths based on the crack regions; during crack extraction, connecting crack endpoints based on morphological dilation and corrosion operations, with the constraint that no new connected components can be added to the image before and after the corrosion operation; extracting crack widths based on the crack regions; and using the crack identification result, crack length extraction result, and crack width extraction result as the steel structure fatigue crack detection result. Compared with existing technologies, this invention can integrate steel structure fatigue crack identification, length extraction, and width extraction, improving the effectiveness, efficiency, safety, and intelligence level of steel structure fatigue crack detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method and apparatus for detecting fatigue cracks in steel structures based on computer vision. Background Technology

[0002] Steel structures are widely used in industrial plants due to their advantages such as light weight and strong seismic resistance. However, under high-frequency, continuous, and repetitive dynamic loads, steel beams in industrial plants are prone to fatigue cracks. These fatigue cracks gradually propagate with increasing load cycles, affecting the normal use of the structure and even causing component tearing and failure, resulting in serious economic losses and casualties. Therefore, fatigue crack detection in steel structure industrial plants is crucial.

[0003] Currently, fatigue crack detection in steel structure workshops mainly relies on manual inspection. Manual inspection uses equipment such as vernier calipers and crack measuring instruments, relying on workers to inspect each fatigue-load-bearing part of the steel structure on-site. This method has several problems. First, manual inspection requires a significant investment of manpower and resources, especially for large or complex structures, resulting in high costs and low efficiency. Second, manual inspection is inherently delayed, often failing to detect fatigue cracks in a timely manner, and is difficult to detect minute cracks or internal structural damage. Furthermore, manual inspection is also influenced to some extent by the skill level and experience of the inspectors, introducing subjectivity. Therefore, a more efficient, accurate, and objective method for fatigue crack detection in steel structure workshops is needed.

[0004] Computer vision-based target detection methods offer insights for intelligent detection of fatigue cracks in steel structure workshops. For images containing fatigue cracks, these methods can identify the cracks using various algorithms. However, these methods often only output the existence of the cracks and cannot perform in-depth feature extraction to provide detailed information about the identified cracks. Therefore, there is an urgent need to develop an intelligent detection method and device for fatigue cracks in steel structure workshops. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a computer vision-based method and device for detecting fatigue cracks in steel structures, which performs in-depth feature extraction on the identified cracks and provides detailed information on the fatigue cracks.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A computer vision-based method for detecting fatigue cracks in steel structures, comprising:

[0008] Acquire images of the steel structure;

[0009] Fatigue crack identification is performed on the steel structure image to determine the crack region;

[0010] Crack length is extracted based on the crack region; during crack extraction, crack endpoints are connected based on morphological dilation and erosion operations, with the constraint that no new connected components can be added to the images before and after the erosion operation.

[0011] Crack width is extracted based on the crack region;

[0012] The crack identification results, crack length extraction results, and crack width extraction results are used as the fatigue crack detection results of steel structures.

[0013] Furthermore, the crack length extraction process includes:

[0014] The steel structure image is cropped based on the identified crack region to obtain a crack image containing the crack and part of the background region; the crack region and the background region are initially separated from the crack image to obtain a crack morphology image.

[0015] The crack morphology image is connected by connecting the crack endpoints based on morphological dilation and erosion operations, and then crack skeletonization is performed to obtain the crack skeleton.

[0016] Based on the crack skeleton, a skeleton pixel search is performed to obtain a set of skeleton pixel points, thereby calculating the crack length.

[0017] Furthermore, the step of connecting the crack endpoints of the crack morphology image based on morphological dilation and erosion operations includes:

[0018] The process involves iterating through each pixel in the crack morphology image and expanding each crack pixel at the edge to the surrounding background area until the number of expansions reaches the preset first number, thus completing the morphological dilation operation.

[0019] After the morphological dilation operation is completed, each pixel in the crack morphology image is continuously traversed, and the crack pixel at each edge position is eroded to expand into the background area until the erosion count reaches the first count, thus completing the erosion operation. The image before and after the erosion operation is restricted from adding new connected regions, so as to obtain the crack morphology image after the crack endpoints are connected.

[0020] Furthermore, the crack skeletonization process includes:

[0021] The crack morphology image after connecting the crack endpoints is subjected to iterative morphological operations until the shape no longer changes; during each iteration, the points at the crack edge are removed from the crack morphology image, while keeping the number of end pixels and connected regions unchanged.

[0022] Furthermore, the skeleton pixel search process includes:

[0023] The image of the crack skeleton is first traversed vertically from left to right. For each crack pixel, the number of other crack pixels in the eight neighborhood of the crack pixel is searched. If there is only one crack pixel connected to it, the current point is determined to be the endpoint of the crack skeleton; otherwise, the search direction is rotated 90 degrees clockwise to change to the horizontal direction from top to bottom, and the endpoint of the crack skeleton is searched.

[0024] After determining the skeleton endpoints, starting from the crack skeleton starting point, search for new pixels in the eight neighboring areas around the target point and place them into the skeleton set point. Then, take the newly placed crack pixel as the starting point and search for the next skeleton pixel that is not in the skeleton pixel set in its eight neighboring areas. Continue in this way until all skeleton pixels are arranged into the set in the order of front and back, and the skeleton pixel set is obtained.

[0025] Furthermore, the calculation process for the crack length is as follows:

[0026] Let set A set of skeleton pixels arranged in order, containing a total of There are 1 point, respectively Among them, point The coordinates are The expression for calculating the crack length is:

[0027]

[0028]

[0029]

[0030] In the formula, For the first i The pixel length of the crack segment. The pixel length of the crack. The actual length corresponding to a single pixel. This represents the actual crack length.

[0031] Furthermore, the crack width extraction process includes the following steps:

[0032] S401: Extract the crack morphology of the cracked region, determine the crack skeleton line and edge contour line, obtain the upper and lower edge contour point sets A and B of the crack, and determine the corresponding centerline set. ;

[0033] S402: Set the centerline Centerline pixels As the target benchmark point for crack width detection, and search The remaining centerline pixels within the neighborhood and ,in, ;

[0034] S403: Based on the target reference point and its neighboring centerline pixels, determine the corresponding tangent slope, thereby locating the centerline normal equation of the target reference point and obtaining the normal.

[0035] S404: Starting from the target reference point, find the intersection point with the edge contour line along the corresponding normal direction, and use it as the edge target point;

[0036] S405: Starting from the target edge point, sequentially search for pixels within the multiple neighborhoods of the upper edge contour point set A along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the upper edge contour point with the shortest distance. ;

[0037] S406: Starting from the target edge point, sequentially search for pixels within the multiple neighborhood of the lower edge contour point set B along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the upper edge contour point with the shortest distance. ;

[0038] S407: Calculations show that... The crack width measured in the neighborhood is calculated using the following expression:

[0039]

[0040] In the formula, For point and points The distance between them For point and points The distance between them The crack width is measured in the 1-neighborhood.

[0041] S408: The steps in S402... Set the value to 2, and repeat steps S402-S407 to obtain the crack width measured in the 2-neighborhood. ;

[0042] S409: The steps in S402... Set the value to 3, and repeat steps S402-S407 to obtain the crack width measured in the 3-neighborhood. ;

[0043] S410: Crack width measured in 1-neighborhood, crack width measured in 2-neighborhood, and crack width measured in 3-neighborhood. Find the minimum value in the middle and use it as the center line pixel. The measured crack width;

[0044] S411: Select the next centerline pixel. Repeat steps S402-S410 until the centerline set has been traversed. All centerline pixels.

[0045] Furthermore, in step S404, the process of finding the edge target point includes:

[0046] Based on each edge contour point in the crack upper and lower edge contour point sets A and B, the cosine value of the angle between the line connecting the edge contour point and the target reference point and the normal direction is calculated sequentially, and the edge contour point corresponding to the smallest cosine value is taken as the edge target point.

[0047] Furthermore, the fatigue crack identification process includes:

[0048] The steel structure image is input into a pre-trained fatigue crack recognition model to determine the crack region;

[0049] The training process of the fatigue crack identification model includes:

[0050] A fatigue crack dataset for steel structures under multiple scenarios was constructed, and data augmentation and image annotation were performed.

[0051] The fatigue crack recognition model was trained based on a multi-scenario fatigue crack dataset of steel structures.

[0052] The present invention also provides a steel structure fatigue crack detection device for implementing the computer vision-based steel structure fatigue crack detection method described above, comprising an image acquisition module, a status self-inspection module, an intelligent detection module and a network transmission module;

[0053] The image acquisition module is used to perform the acquisition and storage of steel structure images at the detection location according to the program instructions sent by the intelligent detection module.

[0054] The status self-check module is used to obtain the operating status of the steel structure fatigue crack detection device. The operating status includes whether the system is powered on normally, whether the program is running normally, and whether there are any operation error messages.

[0055] The intelligent detection module is used to identify fatigue cracks in the steel structure image and determine the crack region; extract the crack length based on the crack region; during the crack extraction process, crack endpoints are connected based on morphological dilation and erosion operations, with the constraint that no new connected components can be added to the image before and after the erosion operation; extract the crack width based on the crack region; and use the crack identification result, crack length extraction result, and crack width extraction result as the fatigue crack detection result of the steel structure.

[0056] The network transmission module is used to upload the detection results output by the intelligent detection module and the operating status obtained by the status self-inspection module to the database.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] (1) The steel structure fatigue crack detection method proposed in this invention can realize the integration of steel structure fatigue crack identification, length extraction and width extraction, thereby improving the effect, efficiency, safety and intelligence level of steel structure fatigue crack detection.

[0059] (2) In the process of crack length extraction, this invention considers that crack interruption may occur due to low image resolution and poor lighting conditions. In order to ensure the integrity of the crack, this application proposes a method for connecting crack endpoints based on morphological dilation and erosion operations. Considering the connectivity requirements in the crack erosion process, the method limits the addition of new connected regions to the entire image before and after the target pixel is eroded, thereby preserving the connectivity of the crack break point region.

[0060] (3) In the process of crack width extraction, the present invention proposes a crack width extraction method based on the improved multi-neighborhood centerline tangent method, which has the following advantages: 1) By calculating the crack centerline tangent and normal direction, it ensures that the normal and the crack edge intersect on different sides, avoiding the possible opposite-side positioning error of the edge tangent method; 2) By transforming the problem of locating the intersection point of the normal direction and the crack edge contour into the problem of finding the minimum angle between the normal direction and the line connecting the edge point and the target point, it avoids the problem of intersection point offset that may be caused by continuous coordinate function calculation in discontinuous crack pixel map; 3) By searching the minimum line segment width value of the normal and the surrounding pixel points in the 5-neighborhood, the influence of crack morphology on width measurement is considered, ensuring that the protruding or concave parts that the local edge of the crack may have are also considered in the scope of width calculation; 4) By measuring the width value of multiple neighborhoods together, it avoids the problem that when using a single neighborhood value to calculate the tangent and normal vectors, it is easily affected by the degree of change of the centerline pixel and is sensitive to its trend. Attached Figure Description

[0061] Figure 1This is a flowchart illustrating a computer vision-based method for detecting fatigue cracks in steel structures, as provided in an embodiment of the present invention.

[0062] Figure 2 This is a flowchart of a method for identifying fatigue cracks in steel structures based on a target detection algorithm, provided in an embodiment of the present invention.

[0063] Figure 3 This is a flowchart of a crack length extraction method based on image processing and breakpoint reconnection technology provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram illustrating the pixel changes within the four neighboring regions surrounding a crack pixel in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of a crack reconnection process provided in an embodiment of the present invention, wherein (a) is a magnified pixel image of the crack interruption point, (b) is a schematic diagram of the first expansion result, (c) is a schematic diagram of the second expansion result, (d) is a schematic diagram of the third expansion result, and (e) is a schematic diagram of the corrosion result.

[0066] Figure 6 This is a flowchart of crack length calculation provided in an embodiment of the present invention;

[0067] Figure 7 This is a schematic diagram of a crack endpoint search principle provided in an embodiment of the present invention, wherein (a) is a schematic diagram of the crack endpoint search principle corresponding to k=1, and (b) is a schematic diagram of the crack endpoint search principle corresponding to k=2.

[0068] Figure 8 This is a schematic diagram illustrating the sorting of new pixels in the eight neighboring regions of a target pixel during length calculation, provided in an embodiment of the present invention.

[0069] Figure 9 This is the workflow of a crack width extraction method based on an improved multi-neighborhood centerline tangent method provided in this embodiment of the invention;

[0070] Figure 10 This is a simplified schematic diagram of a crack edge pixel search provided in an embodiment of the present invention;

[0071] Figure 11 This is a schematic diagram of the width calculation method of the center line tangent method for one-neighborhood, two-neighborhood, and three-neighborhood provided in an embodiment of the present invention. (a) is the original case image of the center line tangent method, (b) is a schematic diagram of the width calculation method of the center line tangent method for one-neighborhood, (c) is a schematic diagram of the width calculation method of the center line tangent method for two-neighborhood, and (d) is a schematic diagram of the width calculation method of the center line tangent method for three-neighborhood.

[0072] Figure 12 This is a diagram illustrating the architecture of a fatigue crack detection device for steel structures provided in an embodiment of the present invention.

[0073] Figure 13 This is an example image of a fatigue crack length extraction method based on image processing and breakpoint reconnection technology provided in an embodiment of the present invention, wherein (1) is the original image, (2) is the crack recognition result, (3) is the crack region cropped according to the calibration box, (4) is the adaptive threshold segmentation result, (5) is the morphological denoising result, and (6) is the breakpoint connection and skeletonization result.

[0074] Figure 14 This is an example diagram of a fatigue crack width extraction method based on an improved multi-neighborhood centerline tangent method provided in an embodiment of the present invention, wherein (1) is the original crack image with skeleton, (2) is the crack edge contour extraction result, (3) is the multi-neighborhood centerline tangent matching result, and (4) is the multi-neighborhood centerline tangent width calculation result.

[0075] Figure 15 This is an architectural diagram of a computer vision-based method and device for detecting fatigue cracks in steel structures, provided in an embodiment of the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0077] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0078] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0079] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed during use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0080] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0081] Furthermore, terms such as "horizontal" and "vertical" do not imply that components must be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0082] Example 1

[0083] like Figure 1 As shown, this embodiment provides a computer vision-based method for detecting fatigue cracks in steel structures, including:

[0084] S1: Obtain image of the steel structure;

[0085] S2: Perform fatigue crack identification on steel structure images and determine the crack area;

[0086] S3: Extract the crack length based on the crack region; during the crack extraction process, connect the crack endpoints based on morphological dilation and erosion operations, and use the constraint that no new connected components can be added to the image before and after the erosion operation.

[0087] S4: Extract the crack width based on the crack region;

[0088] S5: The crack identification results, crack length extraction results, and crack width extraction results are used as the fatigue crack detection results of steel structures.

[0089] Specifically, this solution proposes the following technologies:

[0090] A fatigue crack identification method for steel structures based on target detection algorithm is used as a crack detection method in step S2.

[0091] A crack length extraction method based on image processing and breakpoint reconnection technology is used as the length extraction scheme after crack detection in step S3.

[0092] A crack width extraction method based on the improved multi-neighborhood centerline tangent method is used as the crack width extraction scheme after crack detection in step S4.

[0093] The following section, with reference to the accompanying drawings, details the principles and operational procedures of the fatigue crack detection method for steel structures.

[0094] Figure 2 This paper demonstrates the crack identification method in the fatigue crack detection approach for steel structures, which consists of two steps: fatigue crack dataset creation and fatigue crack identification model training and testing. The fatigue crack dataset creation includes: original image acquisition, data augmentation, preprocessing, and image annotation. The fatigue crack identification model training and testing first involves dividing the dataset, then feeding the data into a target detection algorithm for training. Next, algorithm validation and model performance testing are performed. If the model's crack identification performance is good, the crack identification model is output; if the model's crack identification performance is poor, the training parameters are adjusted, and the model is trained again, followed by validation and testing, until a model with good recognition performance is obtained.

[0095] Figure 3 The workflow of a fatigue crack length extraction method based on image processing and breakpoint reconnection technology is demonstrated, including:

[0096] The steel structure image is cropped based on the identified crack region to obtain a crack image containing the crack and part of the background region; the crack region and background region are initially separated from the crack image to obtain a crack morphology image.

[0097] The crack morphology image is connected by connecting the crack endpoints based on morphological dilation and erosion operations, and then crack skeletonization is performed to obtain the crack skeleton.

[0098] Specifically, this includes: continuously traversing each pixel in the crack morphology image, expanding each crack pixel at the edge position to the surrounding background area until the number of expansions reaches the preset first number, thus completing the morphological dilation operation; after the morphological dilation operation is completed, continuously traversing each pixel in the crack morphology image, performing an erosion operation on each crack pixel at the edge position to expand it into the background area until the number of erosions reaches the first number, thus completing the erosion operation; and using the constraint that no new connected components can be added to the image before and after the erosion operation, obtaining the crack morphology image after connecting the crack endpoints;

[0099] The crack skeletonization process specifically includes: performing iterative morphological operations on the crack morphology image after connecting the crack endpoints until the shape no longer changes; during each iteration, removing the points at the crack edge of the crack morphology image while keeping the number of end pixels and connected regions unchanged.

[0100] The crack length is calculated by searching the skeleton pixels based on the crack skeleton.

[0101] The skeleton pixel search process specifically includes:

[0102] The image of the crack skeleton is first traversed vertically from left to right. For each crack pixel, the number of other crack pixels in the eight neighborhood of the crack pixel is searched. If there is only one crack pixel connected to it, the current point is determined to be the endpoint of the crack skeleton; otherwise, the search direction is rotated 90 degrees clockwise to change to the horizontal direction from top to bottom, and the endpoint of the crack skeleton is searched.

[0103] After determining the skeleton endpoints, starting from the crack skeleton starting point, search for new pixels in the eight neighboring areas around the target point and place them into the skeleton set point. Then, take the newly placed crack pixel as the starting point and search for the next skeleton pixel that is not in the skeleton pixel set in its eight neighboring areas. Continue in this way until all skeleton pixels are arranged into the set in the order of front and back, and the skeleton pixel set is obtained.

[0104] This embodiment specifically includes the following steps:

[0105] S301: Image initialization. This step includes the following two operations: (1) cropping the image based on the crack identification bounding box to obtain an image containing only the crack and the surrounding small background area; (2) initially separating the crack and background areas by image grayscale conversion and Gaussian blur, adaptive threshold segmentation, median blur and morphological denoising operations.

[0106] S302: Crack Breakpoint Connection and Skeletonization. Crack breakpoint connection relies on morphological dilation and erosion operations. After morphological extraction and denoising in S1, a complete crack morphology can be output. However, due to low image resolution and poor lighting conditions, crack interruptions may occur. To ensure crack integrity, the crack breakpoints need to be reconnected. The following details the method for connecting crack endpoints based on morphological dilation and erosion operations.

[0107] For each crack pixel in an image, there are a total of 15 possible variations in the pixels within its four neighboring areas. Figure 4These cases are presented in order of the number of surrounding crack pixels. The red pixel at the center of the cross is the target pixel. In the four-neighborhood of the target pixel, white represents background pixels, and black represents crack pixels. Considering the connectivity of the crack region, each crack pixel must have at least one surrounding crack pixel; otherwise, it will become background noise and be removed in the previous median blurring and morphological denoising operation. Therefore, removing… Figure 4 The form (1) in the middle will not appear except for Figure 4 The remaining (2) to (16) may all appear in the actual crack image. The breakpoint connection mainly considers the pixel distribution at the crack edge, except... Figure 4 Image (2) shows the inside of the crack. Figure 4 The remaining cases (3) to (16) in the text all indicate situations that may occur at the edge of the crack.

[0108] Morphological dilation is an operation performed when the target pixel is surrounded by a distribution of... Figure 4 In schemes (3) to (16), the target pixel is set to black to expand the crack area; the morphological erosion operation is when the target pixel is surrounded by the same... Figure 4 In schemes (3) to (16), the target pixel is set to white to expand the background area.

[0109] by Figure 5 Detailed explanation of the process, Figure 5 Figure (a) shows a magnified pixel-level simulation of a crack break. Black squares represent pixels in the crack area, white squares represent weld seams in the background area, and red lines on the left and right indicate omitted subsequent pixels. This section will use morphological dilation and erosion operations to connect the crack break points.

[0110] First, continuously traverse the image pixels, expanding each edge crack pixel to the surrounding background area. Figure 5 In the diagram, (b), (c), and (d) represent the image processing results after one, two, and multiple dilation operations, respectively. Each color represents the area of ​​outward expansion of the crack during one image dilation operation. The expansion areas of different colors combine to form the pixel area occupied by the crack in the image at the current state. Because the length of the crack interruption region is usually much smaller than the overall length of the crack, after a certain number of dilation operations (assumed to be n), the two sides of the crack interruption point will coincide, as shown below. Figure 5 As shown in d. At this point, the tip of the break has reconnected, and the crack has become an expanded whole.

[0111] After the dilation operation is completed, the erosion operation is performed, which is the inverse operation of the dilation method. The image pixels are traversed n times, and the erosion operation is performed on each point on the crack edge. It is important to note that erosion and dilation are completely inverse operations. An image that has undergone the same number of dilation and erosion operations will normally return to its original state and cannot connect the crack breakpoints. Therefore, during crack erosion, the connectivity requirement must also be considered; that is, no new connected components should be added to the image before and after the erosion of the target pixel.

[0112] Therefore, the dilation results of normal regions in the image cancel each other out with the erosion results. Figure 5 (e) The underlined part in the middle); In the crack fracture region, because the connection of this region is severed, the connected domain at this point is divided into two independent connected domains, which are preserved due to connectivity protection ( Figure 5 (in the gray pixel part of (e) in the image), the crack was reconnected at the break point.

[0113] Crack skeletonization means simplifying the crack to a single pixel width through iterative morphological operations until its shape no longer changes. In each iteration, points at the crack edges are removed, but the end pixels and the number of connected components remain unchanged. At this point, the crack appears as a single entity in the image and is clearly separated from the background. Continuing the erosion operation from the previous step, the crack body is continuously eroded without increasing the number of connected components in the image, thus obtaining the skeletonized crack.

[0114] S3: Length calculation and output, the process is as follows: Figure 6 As shown. Figure 6 In this context, k represents the number of crack pixels within the eight-neighborhood of the target point. Considering the uncertain path of the crack in the image, the image pixels are first traversed vertically from left to right, and the first crack pixel encountered is marked as a suspected crack endpoint. The original crack is extracted as a single-pixel crack skeleton, ensuring that each pixel in the skeleton is surrounded by only one or two crack pixels, and that each endpoint has exactly one crack pixel connected to it in its eight-neighborhood, while each midpoint has exactly two crack pixels connected to it. Therefore, whether a point is a crack endpoint can be determined by searching the number of other crack pixels within the eight-neighborhood of the suspected endpoint.

[0115] like Figure 7 As shown, Figure 7 In (a), the crack direction is approximately horizontal. Since the target point (red pixel) has exactly one other pixel (green pixel) within its eight neighborhoods, this point can be identified as the crack skeleton initiation point. Figure 7In (b) of the image, the crack direction is approximately vertical. When traversing the image endpoints left and right, a protruding section in the middle of the crack may be found. If the number of neighboring cracks is 2, then this point is not an endpoint. The search direction is rotated 90 degrees clockwise to a horizontal direction, searching for the endpoint from top to bottom. If the number of neighboring cracks at the target point is still 2, the search for the endpoint continues vertically from right to left, rotating 90 degrees clockwise. This process is repeated. When there is only one other labeled skeleton pixel around the target point, and it is impossible to continue searching for unlabeled pixels, this point is determined to be the crack endpoint, and the search stops.

[0116] After determining the skeleton endpoints, start the search from those endpoints and proceed as follows: Figure 8 The new pixels in the eight neighborhoods surrounding the target point are added to the skeleton set. Then, using the newly added crack pixel as the starting point, the next skeleton pixel not in the skeleton pixel set is searched in its eight neighborhoods. This process is repeated until all skeleton pixels are sequentially added to the set.

[0117] Let set The set is a collection of skeleton pixels arranged in order, containing a total of There are 1 point, respectively Among them, points The coordinates are Then, for the pixel length of the skeleton... The calculation formula is as follows:

[0118]

[0119]

[0120]

[0121] The actual crack length can be approximated by the crack skeleton length, and there is a certain conversion relationship between image pixels and actual distances. This proportional length can be obtained through camera calibration. If in a certain image, a single pixel corresponds to an actual length of... Then the true length of the crack can be obtained. for:

[0122]

[0123] Therefore, the crack length can be obtained and output from the image.

[0124] Figure 9 The workflow of a crack width extraction method based on an improved multi-neighborhood centerline tangent method is demonstrated, comprising the following 11 steps:

[0125] S401: Crack Preprocessing. This step mainly involves preprocessing operations such as crack morphology extraction, exporting crack skeleton lines and edge contour lines, which serve as the basis for subsequent pixel search. Assume the extracted upper and lower edge contour point sets of the crack are... and Assuming its centerline contains N crack pixels, denoted as... .

[0126] S402: Traverse the crack centerline and extract each pixel. This serves as the target reference point for crack width detection. A search is performed on each pixel. The remaining centerline pixels within the neighborhood and .

[0127] S403: Solve for the tangent slope within the neighborhood of the neighboring pixels, and use the tangent slope and the reference point to locate the equation of the center line normal at the point.

[0128] S404: Starting from the target reference point, find the intersection point with the edge contour line along the normal direction. In this step, since it is difficult to solve for the intersection point with continuous lines in an image where each pixel is independent, the solution for the shortest distance between the target point and the contour point is transformed into the solution for calculating the minimum angle between the directions of the lines connecting the nearest edge contour point and the reference point.

[0129] Figure 10 This is a simplified diagram of edge pixel search. The edge outline and center line are represented by solid black curves and dashed black curves, respectively. The white circle in the center represents the target reference point being solved for. The red line indicates the tangent and normal directions at that point. Due to the discontinuity of crack pixels, the normal direction may not point exactly to a complete crack edge pixel. In this case, it is necessary to connect the edge pixels and the target point, and then calculate the cosine of the angle between the connecting line and the normal direction.

[0130]

[0131] In the above formula, The normal direction vector is... and These are the coordinates of the edge pixel and the centerline target point, respectively. After calculating the cosine of the direction angle, its minimum value represents the direction closest to the normal direction. The intersection of this normal direction and the edge contour line on one side is then used. As the target point.

[0132] S405: Starting from the target edge point, sequentially search for pixels within the multiple neighborhood of the upper edge contour point set A along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the upper edge contour point with the shortest distance. ;

[0133] In this embodiment, starting from the edge target point, pixels within the top and bottom 5 neighborhoods are sequentially searched along the edge contour line, and the distance between the pixel and the center line target point is calculated. The edge pixel that minimizes this distance is then found. Finding points between , so that:

[0134]

[0135] S406: Starting from the target edge point, sequentially search for pixels within the multiple neighborhood of the lower edge contour point set B along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the upper edge contour point with the shortest distance. ;

[0136] In this embodiment, the point closest to the normal direction is found in the set of crack edge points on the other side. And then Finding points between This makes it in Within 5 neighborhoods, relative to the target point on the center line The length of the connecting line segment is the smallest.

[0137] S407: Calculations show that... The crack width measured in the neighborhood is calculated using the following expression:

[0138]

[0139] In the formula, For point and points The distance between them For point and points The distance between them The crack width is measured in the 1-neighborhood.

[0140] S408: The steps in S402... Set the value to 2, and repeat steps S402-S407 to obtain the crack width measured in the 2-neighborhood. ;

[0141] S409: The steps in S402... Set the value to 3, and repeat steps S402-S407 to obtain the crack width measured in the 3-neighborhood. ;

[0142] S410: Crack width measured in 1-neighborhood, crack width measured in 2-neighborhood, and crack width measured in 3-neighborhood. Find the minimum value in the middle and use it as the center line pixel. The measured crack width is:

[0143]

[0144] S411: Select the next centerline pixel. Repeat steps S402-S410 until the centerline set has been traversed. All centerline pixels.

[0145] Through the above steps, the crack width can be calculated using the crack centerline tangent method. This crack width extraction method has the following advantages: (1) By calculating the crack centerline tangent and normal directions, it ensures that the normal and the crack edge intersect on different sides, avoiding the possible opposite-side positioning error that may occur with the edge tangent method; (2) By transforming the problem of locating the intersection point of the normal direction and the crack edge contour into the problem of finding the minimum angle between the normal direction and the line connecting the edge point and the target point, it avoids the problem of intersection point offset that may occur when performing continuous coordinate function calculations in discontinuous crack pixel maps; (3) By searching for the minimum line segment width value of the connection between the normal and the surrounding pixel points in the 5-neighborhood, it considers the influence of crack morphology on width measurement, ensuring that the protruding or concave parts that may exist on the local edge of the crack are also considered in the scope of width calculation; (4) By measuring the width values ​​of multiple neighborhoods together, it avoids the problem that when using a single neighborhood value to calculate the tangent and normal vectors, it is easily affected by the degree of change of the centerline pixels and is more sensitive to its trend.

[0146] Figure 11 The diagrams show the solution of the width using the tangent method for the centerline of the one-neighborhood, two-neighborhood, and three-neighborhood areas, respectively. Figure 11 (a) shows a simplified example of weld crack extraction, where black squares represent the centerline and white squares represent the edge contour. Figure 11 In step (b), the centerline pixels are traversed, and each selected point is used as the target reference point for width calculation (red pixel square area in the figure). The pixels in the 1-neighborhood of the centerline (green square area) are searched, and the slope of the tangent line in its neighborhood is approximated by the slope of the line connecting the pixels (green dashed line). A line is constructed using the tangent slope and the coordinates of the target point (blue dashed line) and intersects the two sides of the edge at the two nearest points (blue square area). Starting from the intersection point of the pixels on both sides of the edge, pixels are searched along the upper and lower 5-neighborhoods of the edge contour (purple square area), and it is constantly determined whether the length of the line segment connecting the pixel to the target reference point is the minimum value among all line lengths within this neighborhood. The edge pixels at the upper and lower edges that minimize the line length are selected (orange square area) and connected to the target reference point (red solid line). The width value of the crack is its line segment length value.

[0147] Similarly, in Figure 11 (c) and Figure 11 In step (d), by changing the initial search span of the neighborhood of the center line pixels, three different width values ​​can be obtained under different neighborhood conditions. Finally, the minimum of the three width values ​​is taken as the representative width value of the point obtained by the multi-neighborhood tangent method, and the process continues to traverse to the next pixel in sequence until all center line pixels have been traversed.

[0148] The following details the composition and usage logic of fatigue crack detection equipment for steel structures.

[0149] Figure 12 The architecture of the fatigue crack detection equipment for steel structures was demonstrated. The equipment consists of the following modules: a condition self-inspection module, an image acquisition module, an intelligent detection module, and a network transmission module.

[0150] The status self-check module can detect the working status of the equipment and accurately obtain fault information by sending status information. The image acquisition module is used to acquire crack images and can complete the acquisition and storage of images of the detection location according to the program instructions sent by the remote host to the embedded development board. The intelligent detection module deploys an algorithm developed by the steel structure fatigue crack detection method and is the core part of the steel structure fatigue crack detection equipment. This part performs crack identification, length extraction, and width extraction on the detection images acquired and stored by the image acquisition module, and stores the detection results in the embedded development board. The network transmission module includes an IoT module and a router, which is used to upload the detection results of the intelligent detection module in the embedded development board and the equipment operating status parameters of the status self-check module to the database, so as to facilitate subsequent calling and display.

[0151] The following examples demonstrate the effectiveness of the fatigue crack detection method proposed in this invention.

[0152] Figure 13 An embodiment of a fatigue crack length extraction method based on image processing and breakpoint reconnection technology is presented. As shown in the figure, based on an image with a crack, a steel structure fatigue crack identification method using a target detection algorithm successfully identifies the crack. Then, the crack region is cropped according to a calibration box. Following this, adaptive threshold segmentation, morphological extraction, and breakpoint reconnection and skeletonization techniques are used to obtain the skeleton of the fatigue crack. Finally, the calculated length is obtained using a length calculation method, and the error between the calculated length and the actual measured length is within 4%.

[0153] Figure 14 An embodiment of a fatigue crack width extraction method based on an improved multi-neighborhood centerline tangent method is presented. As shown in the figure, from an original image of a crack with a skeleton, its width distribution along the skeleton direction can be calculated through edge contour extraction and multi-neighborhood centerline tangent matching.

[0154] Example 2

[0155] like Figure 15 As shown, this embodiment provides a steel structure fatigue crack detection device that implements the computer vision-based steel structure fatigue crack detection method as described in Embodiment 1, including an image acquisition module, a status self-inspection module, an intelligent detection module, and a network transmission module;

[0156] The image acquisition module is used to acquire and store images of the steel structure at the detection location according to the program instructions sent by the intelligent detection module.

[0157] The status self-check module is used to obtain the operating status of the steel structure fatigue crack detection device. The operating status includes whether the system is powered on normally, whether the program is running normally, and whether there are any operation error messages.

[0158] The intelligent detection module is used to identify fatigue cracks in steel structure images and determine crack regions; extract crack lengths based on crack regions; during crack extraction, crack endpoints are connected based on morphological dilation and corrosion operations, with the constraint that no new connected components can be added to the images before and after the corrosion operation; crack widths are extracted based on crack regions; and the crack identification results, crack length extraction results, and crack width extraction results are used as the fatigue crack detection results for the steel structure.

[0159] That is, the fatigue crack detection method for steel structures serves the intelligent detection module of the fatigue crack detection device for steel structures, and performs fatigue crack identification, length extraction and width extraction.

[0160] The network transmission module is used to upload the detection results output by the intelligent detection module and the operating status obtained by the status self-inspection module to the database.

[0161] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A computer vision-based method for detecting fatigue cracks in steel structures, characterized by, include: Acquire images of the steel structure; Fatigue crack identification is performed on the steel structure image to determine the crack region; Crack length is extracted based on the crack region; During crack extraction, crack endpoints are connected based on morphological dilation and erosion operations, with the constraint that no new connected components can be added to the images before and after the erosion operation. The crack width is extracted based on the crack region. The crack identification results, crack length extraction results, and crack width extraction results are used as the fatigue crack detection results of steel structures; The process of extracting the crack length includes: The steel structure image is cropped based on the identified crack region to obtain a crack image containing the crack and part of the background region; the crack region and the background region are initially separated from the crack image to obtain a crack morphology image. The crack morphology image is connected by connecting the crack endpoints based on morphological dilation and erosion operations, and the image before and after the erosion operation cannot add new connected components as a constraint. Then, crack skeletonization is performed to obtain the crack skeleton. Based on the crack skeleton, a skeleton pixel search is performed to obtain a set of skeleton pixel points, thereby calculating the crack length; The skeleton pixel search process includes: The image of the crack skeleton is first traversed vertically from left to right. For each crack pixel, the number of other crack pixels in the eight neighborhood of the crack pixel is searched. If there is only one crack pixel connected to it, the current point is determined to be the endpoint of the crack skeleton; otherwise, the search direction is rotated 90 degrees clockwise to change to the horizontal direction from top to bottom, and the endpoint of the crack skeleton is searched. After determining the skeleton endpoints, starting from the crack skeleton starting point, search for new pixels in the eight neighboring areas around the target point and place them into the skeleton set point. Then, take the newly placed crack pixel as the starting point and search for the next skeleton pixel that is not in the skeleton pixel set in its eight neighboring areas. Continue in this way until all skeleton pixels are arranged into the set in the order of front and back to obtain the skeleton pixel set. The process of extracting the crack width includes the following steps: S401: Extract the crack morphology of the cracked region, determine the crack skeleton line and edge contour line, obtain the upper and lower edge contour point sets A and B of the crack, and determine the corresponding centerline set. ; S402: Set the centerline Centerline pixels As the target benchmark point for crack width detection, and search The remaining centerline pixels within the neighborhood and ,in, ; S403: Based on the target reference point and its neighboring centerline pixels, determine the corresponding tangent slope, thereby locating the centerline normal equation of the target reference point and obtaining the normal. S404: Starting from the target reference point, find the intersection point with the edge contour line along the corresponding normal direction, and use it as the edge target point; S405: Starting from the target edge point, sequentially search for pixels within the multiple neighborhood of the upper edge contour point set A along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the upper edge contour point with the shortest distance. ; S406: Starting from the target edge point, sequentially search for pixels within the multiple neighborhood of the lower edge contour point set B along the edge contour line, calculate the distance between the searched pixels and the target reference point, and select the lower edge contour point with the shortest distance. ; S407: Calculations show that... The crack width measured in the neighborhood is calculated using the following expression: In the formula, For point and points The distance between them For point and points The distance between them The crack width is measured in the 1-neighborhood. S408: The steps in S402... Set the value to 2, and repeat steps S402-S407 to obtain the crack width measured in the 2-neighborhood. ; S409: The steps in S402... Set the value to 3, and repeat steps S402-S407 to obtain the crack width measured in the 3-neighborhood. ; S410: Crack width measured in 1-neighborhood, crack width measured in 2-neighborhood, and crack width measured in 3-neighborhood. Find the minimum value in the middle and use it as the center line pixel. The measured crack width; S411: Select the next centerline pixel. Repeat steps S402-S410 until the centerline set has been traversed. All centerline pixels.

2. The computer vision-based method for detecting fatigue cracks in steel structures according to claim 1, characterized in that, The method of connecting crack endpoints in the crack morphology image based on morphological dilation and erosion operations includes: The process involves iterating through each pixel in the crack morphology image and expanding each crack pixel at the edge to the surrounding background area until the number of expansions reaches the preset first number, thus completing the morphological dilation operation. After the morphological dilation operation is completed, each pixel in the crack morphology image is continuously traversed, and each crack pixel at the edge position is eroded to expand into the background area until the first number of erosions is reached, and the erosion operation is completed to obtain the crack morphology image after the crack endpoints are connected.

3. The method for detecting fatigue cracks in steel structures based on computer vision according to claim 1, characterized in that, The crack skeletonization process includes: The crack morphology image after connecting the crack endpoints is subjected to iterative morphological operations until the shape no longer changes; during each iteration, the points at the crack edge are removed from the crack morphology image, while keeping the number of end pixels and connected regions unchanged.

4. The computer vision-based method for detecting fatigue cracks in steel structures according to claim 1, characterized in that, The calculation process for the crack length is as follows: Let set A set of skeleton pixels arranged in order, containing a total of There are 1 point, respectively Among them, point The coordinates are The expression for calculating the crack length is: In the formula, For the first i The pixel length of the crack segment. The pixel length of the crack. The actual length corresponding to a single pixel. This represents the actual crack length.

5. The method for detecting fatigue cracks in steel structures based on computer vision according to claim 1, characterized in that, In step S404, the process of finding the edge target point includes: Based on each edge contour point in the set of upper and lower edge contour points A and B of the crack, the cosine value of the angle between the line connecting the edge contour point and the target reference point and the normal direction is calculated in sequence, and the edge contour point corresponding to the smallest cosine value is taken as the edge target point.

6. The computer vision-based method for detecting fatigue cracks in steel structures according to claim 1, characterized in that, The fatigue crack identification process includes: The steel structure image is input into a pre-trained fatigue crack recognition model to determine the crack region; The training process of the fatigue crack identification model includes: A fatigue crack dataset for steel structures under multiple scenarios was constructed, and data augmentation and image annotation were performed. The fatigue crack recognition model was trained based on a multi-scenario fatigue crack dataset of steel structures.

7. A steel structure fatigue crack detection device for implementing the computer vision-based steel structure fatigue crack detection method as described in any one of claims 1-6, characterized in that, It includes an image acquisition module, a status self-check module, an intelligent detection module, and a network transmission module; The image acquisition module is used to perform the acquisition and storage of steel structure images at the detection location according to the program instructions sent by the intelligent detection module. The status self-check module is used to obtain the operating status of the steel structure fatigue crack detection device. The operating status includes whether the system is powered on normally, whether the program is running normally, and whether there are any operation error messages. The intelligent detection module is used to identify fatigue cracks in the steel structure image and determine the crack area; Crack length is extracted based on the crack region; during crack extraction, crack endpoints are connected based on morphological dilation and erosion operations, with the constraint that no new connected components can be added to the images before and after the erosion operation; crack width is extracted based on the crack region; the crack identification result, crack length extraction result, and crack width extraction result are used as the steel structure fatigue crack detection result; The network transmission module is used to upload the detection results output by the intelligent detection module and the operating status obtained by the status self-inspection module to the database.

Images