Intelligent detection system for surface damage of steel structure based on machine vision

Abstract

The application discloses a steel structure surface damage intelligent detection system based on machine vision and relates to the technical field of steel structure surface detection, and comprises the following modules: a welding bead identification module, which obtains a shooting picture by shooting the steel structure through a high-definition camera, obtains a gray picture through gray processing of the shooting picture, marks an abnormal area through the brightness of the gray picture, judges whether the abnormal area is a welding bead or a crack through the gradual change of the brightness in the abnormal area, and locks the welding bead area according to the judgment result; in the welding bead one-time segmentation module, the welding bead area is segmented into multiple transition areas and multiple boundary areas, so that whether there is a crack in the transition area can be analyzed through the shooting picture in the subsequent process, and the existence of the height difference in the boundary area causes light shadow, which affects the accuracy of the picture analysis result; whether there is a crack in the boundary area is judged through the subsequent laser scanning height measurement mode, so that the accuracy of the detection result is improved.

Description

Technical Field

[0001] This invention relates to the field of steel structure surface inspection technology, specifically to an intelligent detection system for steel structure surface damage based on machine vision. Background Technology

[0002] Steel structure surface inspection is an intelligent method that uses optical imaging, image processing, and intelligent analysis technologies to automatically identify defects such as cracks, corrosion, and deformation on the surface of steel. Compared with traditional manual inspection, machine vision has the advantages of high efficiency, stable accuracy, and full coverage, and has been widely used in the health monitoring and maintenance of steel structures in many fields.

[0003] The inspection of steel structure surface quality generally involves three steps: scanning the steel structure surface, comparing the scanned image with images in the database, and analyzing the comparison results to determine if the steel structure surface is qualified. However, for steel structures with welds, this inspection method has limitations because it is difficult to make the weld thickness and molten pool area exactly the same during the welding process. This is especially true for detecting cracks on the weld. While cracks on the weld molten pool surface can be detected through image brightness anomaly analysis, the connection area between the weld molten pool surfaces has a height difference. Therefore, under the influence of shadows and light, it is difficult to determine whether cracks exist in the connection area through brightness analysis. This often leads to false alarms in the inspection results. To address this, we propose an intelligent detection system for steel structure surface damage based on machine vision. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a machine vision-based intelligent detection system for surface damage in steel structures, thereby solving the aforementioned problems in the prior art.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: an intelligent detection system for surface damage of steel structures based on machine vision, comprising the following modules:

[0008] The weld recognition module takes pictures of the steel structure with a high-definition camera, processes the pictures into grayscale to obtain grayscale images, marks abnormal areas by the brightness of the grayscale images, determines whether the abnormal area is a weld or a crack by the gradual change of brightness within the abnormal area, and locks the weld area based on the judgment result.

[0009] The weld bead segmentation module divides the weld bead area into multiple transition zones and multiple boundary zones. The weld bead area is then laser-scanned to obtain height information within the weld bead area. The height difference between the transition zones and the boundary zones are obtained from the height information.

[0010] The weld bead secondary segmentation module performs secondary segmentation on the weld bead area to obtain multiple smooth areas and multiple connecting areas. It uses grayscale images to determine whether there are cracks in the smooth areas and uses the height difference between the transition area and the boundary area to determine whether there are cracks in the connecting areas.

[0011] The detection result reporting module acquires and marks the location information of all cracked areas, and activates an alarm to remind managers to identify the cracked areas.

[0012] Preferably, in the weld bead recognition module, the captured image is processed to obtain a grayscale image, and abnormal areas are marked by the brightness of the grayscale image. Specifically:

[0013] Step 1: Acquire the captured image and process it into grayscale to obtain a grayscale image;

[0014] Step 2: Divide the grayscale image into 16 equal 4×4 grids. Randomly capture 10 brightness values ​​in each grid, sum and average the 10 brightness values ​​to obtain the average brightness of a single grid, and then sum and average the 16 average brightness values ​​of a single grid to obtain the average brightness of the grayscale image.

[0015] Step 3: Set a preset threshold for brightness value fluctuation. Obtain a reference value for the brightness of the grayscale image by summing the average brightness of the grayscale image with the preset threshold for brightness value fluctuation. Capture locations in the grayscale image where the brightness is higher than the reference value for brightness and mark them as abnormal areas.

[0016] Preferably, in the weld bead recognition module, the abnormal area is determined to be either a weld bead or a crack by observing the gradual change in brightness within the abnormal area, and the weld bead area is located based on the determination result. Specifically:

[0017] Step 1: Obtain the brightness value of the abnormal area and determine whether the brightness value of the abnormal area increases or decreases gradually in a certain direction. If the result is yes, mark the abnormal area as a weld bead; if the result is no, mark the abnormal area as a crack.

[0018] Preferably, in the weld bead segmentation module, the weld bead area is segmented once to obtain multiple transition zones and multiple boundary zones, specifically as follows:

[0019] Step 1: Acquire images. Capture molten pools with two adjacent molten pools on the left and right sides by taking images, and mark them as target molten pools.

[0020] Step 2: Obtain the distance from the leftmost end of each target molten pool to the rightmost end of its adjacent left molten pool, and mark it as the left width. Set a preset threshold for the width. Obtain the boundary position between the target molten pool and the adjacent left molten pool, and mark it as the left boundary line. Shift each point on the left boundary line to the left by a preset threshold position and extend both ends to the edge of the molten pool to obtain the left target line of the left boundary line. Shift each point on the boundary line to the right by a preset threshold position and extend both ends to the edge of the molten pool to obtain the right target line of the left boundary line. Mark the weld area between the left and right target lines of the left boundary line as the non-measurement area. Obtain all weld areas within each left width other than the non-measurement area, and mark them as the left boundary area.

[0021] The distance from the rightmost end of each target molten pool to the leftmost end of its adjacent right molten pool is obtained and marked as the right width. The boundary position between the target molten pool and the adjacent right molten pool is obtained and marked as the right boundary line. The left target line of the right boundary line is obtained by shifting each point on the right boundary line to the left by a preset width threshold position and extending both ends to the edge of the molten pool. The right target line of the right boundary line is obtained by shifting each point on the boundary line to the right by a preset width threshold position and extending both ends to the edge of the molten pool. The weld area between the left and right target lines of the right boundary line is marked as the non-measurement area. All weld areas within each right width other than the non-measurement area are obtained and marked as the right boundary area. All left boundary areas and right boundary areas are combined and duplicate weld areas are deleted to obtain the boundary area.

[0022] Step 3: Obtain the weld bead area, remove all boundary areas within the weld bead area to obtain the non-boundary area, and mark the non-boundary area of ​​each molten pool on the weld bead as the transition area.

[0023] Preferably, in the weld bead segmentation module, laser scanning is performed on the weld bead area to obtain height information within the weld bead area. The height difference of the transition zone and the height difference of the boundary zone are then obtained from this height information. Specifically:

[0024] Step 1: Set the target distance. Install a laser rangefinder at the target distance position directly above the steel structure inspection surface. Use the laser rangefinder to scan the weld area of ​​the steel structure inspection surface to obtain the distance of each point on the weld from the laser rangefinder's emitting plane. Mark this distance as the measurement distance. Obtain all the height information within the weld area by subtracting the target distance from each measurement distance.

[0025] Step 2: Obtain the maximum and minimum values ​​of all height information within the boundary zone of each target molten pool. Calculate the height difference for each boundary zone by subtracting the maximum and minimum values ​​of the boundary zone height information. Obtain the maximum and minimum values ​​of all height information within the transition zone of each target molten pool. Calculate the height difference for each transition zone by subtracting the maximum and minimum values ​​of the transition zone height information.

[0026] Preferably, in the weld bead secondary segmentation module, the weld bead area is secondary segmented to obtain multiple smooth areas and multiple transition areas, specifically as follows:

[0027] Step 1: Set a preset threshold distance, obtain the boundary positions of all molten pools and adjacent molten pools, and mark them as boundary lines. Shift each point on the boundary line to the left by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the left reference line of the boundary line. Shift each point on the boundary line to the right by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the right reference line of the boundary line. Mark the weld area between the left and right reference lines of the left boundary line as the connection area.

[0028] Step 2: Obtain the weld bead area, remove all connecting areas within the weld bead area to obtain the non-connecting area, and mark the non-connecting area of ​​each molten pool on the weld bead as a smooth area.

[0029] Preferably, in the secondary segmentation module for weld seams, the presence of cracks in the smooth area is determined by using grayscale images, specifically as follows:

[0030] Step 1: Obtain a grayscale image, mark all smooth areas in the grayscale image, obtain the brightness value of the center point of each melt pool in the smooth area, and mark it as the center brightness value;

[0031] Step 2: Set a preset threshold for the gradient distance. Shift the center point of the molten pool to the left by the preset threshold to obtain a left point. Obtain the brightness value of the left point and mark it as the left-first brightness value. Use the brightness value of the center point and the left-first brightness value to obtain the range of left-first gradient brightness values. Mark the areas that are not within the range of left-first gradient brightness values ​​among all brightness values ​​between the left point and the center point, and mark them as left-first crack areas. Repeat the above filtering with the left point as the center point to obtain all left crack areas.

[0032] Step 3: Obtain a right point by shifting a gradually increasing distance to the right from the center point of the molten pool by a preset threshold. Obtain the brightness value of the right point and mark it as the right brightness value. Obtain the right gradually increasing brightness value range by using the brightness value of the center point and the right brightness value. Mark the area that is not within the right gradually increasing brightness value range among all brightness values ​​between the right point and the center point and mark it as the right crack area. Repeat the above filtering with the right point as the center point to obtain all right crack areas.

[0033] Step 4: Count the number of cracks in all left and right crack areas, and determine if the number of cracks is greater than 0. If the number of cracks is greater than 0, the result is that there are cracks and all crack locations are marked. If the number of cracks is equal to 0, the result is that there are no cracks.

[0034] Preferably, in the secondary segmentation module for weld seams, the presence of cracks in the joint area is determined by the height difference between the transition zone and the boundary zone, specifically as follows:

[0035] Step 1: Obtain the height difference of the boundary zone of each molten pool, obtain the height difference of the transition zone of each molten pool, and obtain the boundary point height by subtracting the height difference of the boundary zone of the molten pool from the height difference of the transition zone of the molten pool. Set a preset threshold for width tolerance height, and obtain the reference height by subtracting the boundary point height from the preset threshold for width tolerance height.

[0036] Step 2: Obtain the laser rangefinder distance at the boundary between each molten pool and the adjacent left molten pool, and mark it as the judgment distance. Obtain the target distance, and obtain the actual height by subtracting the target distance from the judgment distance. Determine whether the actual height is less than the reference height. If the actual height is less than the reference height, the judgment result is that there is a crack in the connection area, and all crack locations are marked. If the actual height is greater than or equal to the reference height, the judgment result is that there is no crack in the connection area.

[0037] (III) Beneficial Effects

[0038] This invention provides an intelligent detection system for surface damage of steel structures based on machine vision, which has the following advantages:

[0039] This solution divides the weld bead area into multiple transition zones and multiple boundary zones in the weld bead segmentation module. This facilitates subsequent image analysis to check for cracks in the transition zones. However, the boundary zones, due to height differences, can cause shadows that affect the accuracy of image analysis. Therefore, the presence of cracks in the boundary zones is determined by measuring the height using laser scanning. This allows for step-by-step inspection of the weld bead by dividing it into regions, enabling targeted crack inspection based on the characteristics of each region. This improves the accuracy of the inspection results and helps avoid false alarms.

[0040] This solution divides the weld bead area into multiple smooth zones and multiple transition zones in the secondary segmentation module. This facilitates the calculation of the true straight-line distance from the highest point of the weld pool to the boundary point of the adjacent right-side weld pool based on the height difference between the transition zone and the boundary zone, taking advantage of the uniform tilt of the molten pool. This allows for subsequent determination of whether there are cracks at the boundary based on the measurement results of the boundary point using a laser rangefinder. This enables targeted crack inspection based on the regional characteristics of the weld bead, thereby improving the accuracy of the detection results and avoiding false alarms. Attached Figure Description

[0041] Figure 1 This is a flowchart of the intelligent detection system for surface damage of steel structures based on machine vision, according to the present invention.

[0042] Figure 2 This is a schematic diagram of the modular structure of the intelligent detection system for steel structure surface damage based on machine vision according to the present invention.

[0043] Figure 3 This is a top view of the weld bead segmentation area of ​​the intelligent detection system for steel structure surface damage based on machine vision according to the present invention.

[0044] Figure 4 This is the main image of the total width of the weld bead in the intelligent detection system for surface damage of steel structures based on machine vision, as described in this invention.

[0045] In the diagram: 1. Smooth area; 2. Connecting area; 3. Transition area; 4. Boundary area; 5. Non-measuring area; 6. Height difference in the transition area; 7. Height difference in the boundary area; 8. Reference height; 9. Actual height. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Please see Figures 1-4 This invention provides a machine vision-based intelligent detection system for surface damage in steel structures, comprising the following modules:

[0048] The weld recognition module takes pictures of the steel structure with a high-definition camera, processes the pictures into grayscale to obtain grayscale images, marks abnormal areas by the brightness of the grayscale images, determines whether the abnormal area is a weld or a crack by the gradual change of brightness within the abnormal area, and locks the weld area based on the judgment result.

[0049] The weld bead segmentation module divides the weld bead area into multiple transition zones and multiple boundary zones. The weld bead area is then laser-scanned to obtain height information within the weld bead area. The height difference between the transition zones and the boundary zones are obtained from the height information.

[0050] The weld bead secondary segmentation module performs secondary segmentation on the weld bead area to obtain multiple smooth areas and multiple connecting areas. It uses grayscale images to determine whether there are cracks in the smooth areas and uses the height difference between the transition area and the boundary area to determine whether there are cracks in the connecting areas.

[0051] The detection result reporting module acquires and marks the location information of all cracked areas, and activates an alarm to remind managers to identify the cracked areas.

[0052] In this embodiment, the steel structure is photographed by a high-definition camera in the weld identification module, which facilitates subsequent analysis of whether there are cracks on the surface of the steel structure. At the same time, abnormal areas are marked by the brightness difference after grayscale processing. Since each weld pool is inclined with one end lower and the other end higher, the abnormal area is judged as a crack or a weld by the brightness gradient of the abnormal area, which facilitates the identification and locking of the weld position, and further facilitates the subsequent judgment of whether there are cracks on the weld.

[0053] This solution divides the weld bead area into multiple transition zones and multiple boundary zones in the weld bead segmentation module. This facilitates subsequent image analysis to check for cracks in the transition zones. However, the boundary zones are affected by shadows caused by height differences, which can affect the accuracy of image analysis results. The presence of cracks in the boundary zones is determined by measuring the height using laser scanning. This allows for step-by-step inspection of the weld bead by dividing it into regions, enabling targeted crack inspection based on the characteristics of each region. This improves the accuracy of the inspection results and helps avoid false alarms.

[0054] This solution divides the weld area into multiple smooth zones and multiple transition zones in the secondary segmentation module of the weld bead. This facilitates the calculation of the true straight-line distance from the highest point of the weld pool to the boundary point of the adjacent right-side weld pool based on the height difference between the transition zone and the boundary zone, taking advantage of the uniform tilt of the molten pool. This allows for subsequent determination of whether there are cracks at the boundary based on the measurement results of the boundary point using a laser rangefinder. This enables targeted crack inspection based on the regional characteristics of the weld bead, thereby improving the accuracy of the detection results and avoiding false alarms.

[0055] This solution summarizes all crack location and quantity information in the detection result reporting module, and then reports the summarized information to the management personnel, thereby facilitating the feedback of detection results to the management personnel.

[0056] It is worth mentioning that the preset threshold value in this scheme can be obtained through weight analysis, which will not be elaborated on here.

[0057] In the weld bead recognition module, the captured image is processed to obtain a grayscale image. Abnormal areas are marked by the brightness of the grayscale image. Specifically:

[0058] Step 1: Acquire the captured image and process it into grayscale to obtain a grayscale image;

[0059] Step 2: Divide the grayscale image into 16 equal 4×4 grids. Randomly capture 10 brightness values ​​in each grid, sum and average the 10 brightness values ​​to obtain the average brightness of a single grid, and then sum and average the 16 average brightness values ​​of a single grid to obtain the average brightness of the grayscale image.

[0060] Step 3: Set a preset threshold for brightness value fluctuation. Obtain a reference value for the brightness of the grayscale image by summing the average brightness of the grayscale image with the preset threshold for brightness value fluctuation. Capture locations in the grayscale image where the brightness is higher than the reference value for brightness and mark them as abnormal areas.

[0061] In the weld bead recognition module, the gradient change in brightness within the abnormal area determines whether the abnormal area is a weld bead or a crack, and the weld bead area is located based on the judgment result. Specifically:

[0062] Step 1: Obtain the brightness value of the abnormal area and determine whether the brightness value of the abnormal area gradually increases or decreases in a regular direction. If the result is yes, mark the abnormal area as a weld bead; if the result is no, mark the abnormal area as a crack.

[0063] Step 2: Obtain the location information of all abnormal areas marked as weld beads, and mark them as weld bead areas.

[0064] In this embodiment, the brightness gradient of the abnormal area is used to determine whether the abnormal area is a crack or a weld bead, which makes it easier to identify and locate the weld bead, and thus facilitates subsequent determination of whether there is a crack on the weld bead.

[0065] In the weld bead segmentation module, the weld bead area is segmented once to obtain multiple transition zones and multiple boundary zones, specifically:

[0066] Step 1: Acquire images. Capture molten pools with two adjacent molten pools on the left and right sides by taking images, and mark them as target molten pools.

[0067] Step 2: Obtain the distance from the leftmost end of each target molten pool to the rightmost end of its adjacent left molten pool, and mark it as the left width. Set a preset threshold for the width. Obtain the boundary position between the target molten pool and the adjacent left molten pool, and mark it as the left boundary line. Shift each point on the left boundary line to the left by a preset threshold position and extend both ends to the edge of the molten pool to obtain the left target line of the left boundary line. Shift each point on the boundary line to the right by a preset threshold position and extend both ends to the edge of the molten pool to obtain the right target line of the left boundary line. Mark the weld area between the left and right target lines of the left boundary line as the non-measurement area. Obtain all weld areas within each left width other than the non-measurement area, and mark them as the left boundary area.

[0068] The distance from the rightmost end of each target molten pool to the leftmost end of its adjacent right molten pool is obtained and marked as the right width. The boundary position between the target molten pool and the adjacent right molten pool is obtained and marked as the right boundary line. The left target line of the right boundary line is obtained by shifting each point on the right boundary line to the left by a preset width threshold position and extending both ends to the edge of the molten pool. The right target line of the right boundary line is obtained by shifting each point on the boundary line to the right by a preset width threshold position and extending both ends to the edge of the molten pool. The weld area between the left and right target lines of the right boundary line is marked as the non-measurement area. All weld areas within each right width other than the non-measurement area are obtained and marked as the right boundary area. All left boundary areas and right boundary areas are combined and duplicate weld areas are deleted to obtain the boundary area.

[0069] Step 3: Obtain the weld bead area, remove all boundary areas within the weld bead area to obtain the non-boundary area, and mark the non-boundary area of ​​each molten pool on the weld bead as the transition area.

[0070] In this embodiment, by setting a preset threshold for the width, the minimum value of the transition zone is slightly less than the distance from the highest point of the actual boundary to the laser rangefinder. This helps to avoid measuring the distance from the bottom of the crack at the boundary to the laser rangefinder, thus facilitating the subsequent accurate measurement of the transition zone height difference, and further facilitating the subsequent calculation and judgment of whether there is a crack in the connection zone.

[0071] In the weld bead segmentation module, laser scanning is performed on the weld bead area to obtain height information within the weld bead area. This height information is then used to determine the height difference in the transition zone and the height difference in the boundary zone. Specifically:

[0072] Step 1: Set the target distance. Install a laser rangefinder at the target distance position directly above the steel structure inspection surface. Use the laser rangefinder to scan the weld area of ​​the steel structure inspection surface to obtain the distance of each point on the weld from the laser rangefinder's emitting plane. Mark this distance as the measurement distance. Obtain all the height information within the weld area by subtracting the target distance from each measurement distance.

[0073] Step 2: Obtain the maximum and minimum values ​​of all height information within the boundary zone of each target molten pool. Calculate the height difference for each boundary zone by subtracting the maximum and minimum values ​​of the boundary zone height information. Obtain the maximum and minimum values ​​of all height information within the transition zone of each target molten pool. Calculate the height difference for each transition zone by subtracting the maximum and minimum values ​​of the transition zone height information.

[0074] In this embodiment, the target distance is the distance from the non-welded plane area of ​​the steel structure to the emitting plane of the laser rangefinder.

[0075] In the weld bead secondary segmentation module, the weld bead area is secondary segmented to obtain multiple smooth areas and multiple transition areas, specifically:

[0076] Step 1: Set a preset threshold distance, obtain the boundary positions of all molten pools and adjacent molten pools, and mark them as boundary lines. Shift each point on the boundary line to the left by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the left reference line of the boundary line. Shift each point on the boundary line to the right by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the right reference line of the boundary line. Mark the weld area between the left and right reference lines of the left boundary line as the connection area.

[0077] Step 2: Obtain the weld bead area, remove all connecting areas within the weld bead area to obtain the non-connecting area, and mark the non-connecting area of ​​each molten pool on the weld bead as a smooth area.

[0078] In this embodiment, the weld area is divided into multiple smooth zones and multiple transition zones. This facilitates the calculation of the true straight-line distance from the highest point of the weld pool to the boundary point of the adjacent right-side weld pool based on the height difference between the transition zone and the boundary zone, taking advantage of the uniform tilt of the weld pool. This allows for subsequent determination of whether there are cracks at the boundary based on the measurement results of the boundary point using a laser rangefinder. This enables targeted crack inspection based on the regional characteristics of the weld, thereby improving the accuracy of the detection results and avoiding false alarms.

[0079] In the secondary segmentation module for weld seams, the presence of cracks within smooth areas is determined using grayscale images. Specifically:

[0080] Step 1: Obtain a grayscale image, mark all smooth areas in the grayscale image, obtain the brightness value of the center point of each melt pool in the smooth area, and mark it as the center brightness value;

[0081] Step 2: Set a preset threshold for the gradient distance. Shift the center point of the molten pool to the left by the preset threshold to obtain a left point. Obtain the brightness value of the left point and mark it as the left-first brightness value. Use the brightness value of the center point and the left-first brightness value to obtain the range of left-first gradient brightness values. Mark the areas that are not within the range of left-first gradient brightness values ​​among all brightness values ​​between the left point and the center point, and mark them as left-first crack areas. Repeat the above filtering with the left point as the center point to obtain all left crack areas.

[0082] Step 3: Obtain a right point by shifting a gradually increasing distance to the right from the center point of the molten pool by a preset threshold. Obtain the brightness value of the right point and mark it as the right brightness value. Obtain the right gradually increasing brightness value range by using the brightness value of the center point and the right brightness value. Mark the area that is not within the right gradually increasing brightness value range among all brightness values ​​between the right point and the center point and mark it as the right crack area. Repeat the above filtering with the right point as the center point to obtain all right crack areas.

[0083] Step 4: Count the number of cracks in all left and right crack areas, and determine if the number of cracks is greater than 0. If the number of cracks is greater than 0, the result is that there are cracks and all crack locations are marked. If the number of cracks is equal to 0, the result is that there are no cracks.

[0084] In this embodiment, based on the gradual brightness change characteristics of the molten pool, the system gradually judges whether there are areas with abnormal brightness in each small area, thereby identifying and capturing cracks on the inclined surface of the gradually brightening molten pool.

[0085] In the secondary segmentation module for weld seams, the height difference between the transition zone and the boundary zone is used to determine whether there are cracks in the joint area. Specifically:

[0086] Step 1: Obtain the height difference of the boundary zone of each molten pool, obtain the height difference of the transition zone of each molten pool, and obtain the boundary point height by subtracting the height difference of the boundary zone of the molten pool from the height difference of the transition zone of the molten pool. Set a preset threshold for width tolerance height, and obtain the reference height by subtracting the boundary point height from the preset threshold for width tolerance height.

[0087] Step 2: Obtain the laser rangefinder distance at the boundary between each molten pool and the adjacent left molten pool, and mark it as the judgment distance. Obtain the target distance, and obtain the actual height by subtracting the target distance from the judgment distance. Determine whether the actual height is less than the reference height. If the actual height is less than the reference height, the judgment result is that there is a crack in the connection area, and all crack locations are marked. If the actual height is greater than or equal to the reference height, the judgment result is that there is no crack in the connection area.

[0088] In this embodiment, the height of the boundary point is obtained by subtracting the height difference between the boundary zone and the transition zone of the weld pool. Since a width preset threshold is set, the minimum value of the transition zone is slightly less than the distance from the highest point of the actual boundary to the laser rangefinder. Therefore, the height of the boundary point is slightly greater than the distance from the highest point of the actual boundary to the laser rangefinder. By setting a width tolerance height preset threshold, the boundary point height caused by the width preset threshold being slightly greater than the distance from the highest point of the actual boundary to the laser rangefinder is avoided. Thus, the obtained reference height is the distance from the highest point of the actual boundary to the laser rangefinder. The actual height is obtained by subtracting the target distance from the judgment distance. By comparing the actual height with the reference height, it is determined whether there is a crack. This avoids the defect that the shadow light caused by the height difference between adjacent weld pools affects the visual recognition of cracks in the image. This facilitates targeted crack inspection based on the characteristics of each area of ​​the weld, thereby improving the accuracy of the detection results and avoiding false alarms.

[0089] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0090] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A machine vision-based intelligent detection system for surface damage in steel structures, characterized in that, Includes the following modules: The weld recognition module takes pictures of the steel structure with a high-definition camera, processes the pictures into grayscale to obtain grayscale images, marks abnormal areas by the brightness of the grayscale images, determines whether the abnormal area is a weld or a crack by the gradual change of brightness within the abnormal area, and locks the weld area based on the judgment result. The weld bead segmentation module divides the weld bead area into multiple transition zones and multiple boundary zones. The weld bead area is then laser-scanned to obtain height information within the weld bead area. The height difference between the transition zones and the boundary zones are obtained from the height information. The weld bead secondary segmentation module performs secondary segmentation on the weld bead area to obtain multiple smooth areas and multiple connecting areas. It uses grayscale images to determine whether there are cracks in the smooth areas and uses the height difference between the transition area and the boundary area to determine whether there are cracks in the connecting areas. In the secondary segmentation module for weld seams, the height difference between the transition zone and the boundary zone is used to determine whether there are cracks in the joint area. Specifically: Step 1: Obtain the height difference of the boundary zone of each molten pool, obtain the height difference of the transition zone of each molten pool, and obtain the boundary point height by subtracting the height difference of the boundary zone of the molten pool from the height difference of the transition zone of the molten pool. Set a preset threshold for width tolerance height, and obtain the reference height by subtracting the boundary point height from the preset threshold for width tolerance height. Step 2: Obtain the laser rangefinder distance at the boundary between each molten pool and the adjacent left molten pool, and mark it as the judgment distance. Obtain the target distance. Calculate the actual height by subtracting the target distance from the judgment distance. Determine whether the actual height is less than the reference height. If the actual height is less than the reference height, the judgment result is that there is a crack in the connection area, and all crack locations are marked. If the actual height is greater than or equal to the reference height, the judgment result is that there is no crack in the connection area. The detection result reporting module acquires and marks the location information of all cracked areas, and activates an alarm to remind managers to identify the cracked areas.

2. The intelligent detection system for steel structure surface damage based on machine vision according to claim 1, characterized in that: In the weld bead recognition module, the captured image is processed to obtain a grayscale image. Abnormal areas are marked by the brightness of the grayscale image. Specifically: Step 1: Acquire the captured image and process it into grayscale to obtain a grayscale image; Step 2: Divide the grayscale image into 16 equal 4×4 grids. Randomly capture 10 brightness values ​​in each grid, sum and average the 10 brightness values ​​to obtain the average brightness of a single grid, and then sum and average the 16 average brightness values ​​of a single grid to obtain the average brightness of the grayscale image. Step 3: Set a preset threshold for brightness value fluctuation. Obtain a reference value for the brightness of the grayscale image by summing the average brightness of the grayscale image with the preset threshold for brightness value fluctuation. Capture locations in the grayscale image where the brightness is higher than the reference value for brightness and mark them as abnormal areas.

3. The intelligent detection system for steel structure surface damage based on machine vision according to claim 1, characterized in that: In the weld bead recognition module, the gradient change in brightness within the abnormal area determines whether the abnormal area is a weld bead or a crack, and the weld bead area is located based on the judgment result. Specifically: Step 1: Obtain the brightness value of the abnormal area and determine whether the brightness value of the abnormal area gradually increases or decreases in a regular direction. If the result is yes, mark the abnormal area as a weld bead; if the result is no, mark the abnormal area as a crack. Step 2: Obtain the location information of all abnormal areas marked as weld beads, and mark them as weld bead areas.

4. The intelligent detection system for surface damage of steel structures based on machine vision according to claim 1, characterized in that: In the weld bead segmentation module, the weld bead area is segmented once to obtain multiple transition zones and multiple boundary zones, specifically: Step 1: Acquire images. Capture molten pools with two adjacent molten pools on the left and right sides by taking images, and mark them as target molten pools. Step 2: Obtain the distance from the leftmost end of each target molten pool to the rightmost end of its adjacent left molten pool, and mark it as the left width. Set a preset threshold for the width. Obtain the boundary position between the target molten pool and the adjacent left molten pool, and mark it as the left boundary line. Shift each point on the left boundary line to the left by a preset threshold position and extend both ends to the edge of the molten pool to obtain the left target line of the left boundary line. Shift each point on the boundary line to the right by a preset threshold position and extend both ends to the edge of the molten pool to obtain the right target line of the left boundary line. Mark the weld area between the left and right target lines of the left boundary line as the non-measurement area. Obtain all weld areas within each left width other than the non-measurement area, and mark them as the left boundary area. The distance from the rightmost end of each target molten pool to the leftmost end of its adjacent right molten pool is obtained and marked as the right width. The boundary position between the target molten pool and the adjacent right molten pool is obtained and marked as the right boundary line. The left target line of the right boundary line is obtained by shifting each point on the right boundary line to the left by a preset width threshold position and extending both ends to the edge of the molten pool. The right target line of the right boundary line is obtained by shifting each point on the boundary line to the right by a preset width threshold position and extending both ends to the edge of the molten pool. The weld area between the left and right target lines of the right boundary line is marked as the non-measurement area. All weld areas within each right width other than the non-measurement area are obtained and marked as the right boundary area. All left boundary areas and right boundary areas are combined and duplicate weld areas are deleted to obtain the boundary area. Step 3: Obtain the weld bead area, remove all boundary areas within the weld bead area to obtain the non-boundary area, and mark the non-boundary area of ​​each molten pool on the weld bead as the transition area.

5. The intelligent detection system for steel structure surface damage based on machine vision according to claim 4, characterized in that: In the weld bead segmentation module, laser scanning is performed on the weld bead area to obtain height information within the weld bead area. This height information is then used to determine the height difference in the transition zone and the height difference in the boundary zone. Specifically: Step 1: Set the target distance. Install a laser rangefinder at the target distance position directly above the steel structure inspection surface. Use the laser rangefinder to scan the weld area of ​​the steel structure inspection surface to obtain the distance of each point on the weld from the laser rangefinder's emitting plane. Mark this distance as the measurement distance. Obtain all the height information within the weld area by subtracting the target distance from each measurement distance. Step 2: Obtain the maximum and minimum values ​​of all height information within the boundary zone of each target molten pool. Calculate the height difference for each boundary zone by subtracting the maximum and minimum values ​​of the boundary zone height information. Obtain the maximum and minimum values ​​of all height information within the transition zone of each target molten pool. Calculate the height difference for each transition zone by subtracting the maximum and minimum values ​​of the transition zone height information.

6. The intelligent detection system for surface damage of steel structures based on machine vision according to claim 1, characterized in that: In the weld bead secondary segmentation module, the weld bead area is secondary segmented to obtain multiple smooth areas and multiple transition areas, specifically: Step 1: Set a preset threshold distance, obtain the boundary positions of all molten pools and adjacent molten pools, and mark them as boundary lines. Shift each point on the boundary line to the left by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the left reference line of the boundary line. Shift each point on the boundary line to the right by a preset threshold distance and extend both ends to the edge of the molten pool to obtain the right reference line of the boundary line. Mark the weld area between the left and right reference lines of the left boundary line as the connection area. Step 2: Obtain the weld bead area, remove all connecting areas within the weld bead area to obtain the non-connecting area, and mark the non-connecting area of ​​each molten pool on the weld bead as a smooth area.

7. The intelligent detection system for surface damage of steel structures based on machine vision according to claim 2, characterized in that: In the secondary segmentation module for weld seams, the presence of cracks within smooth areas is determined using grayscale images. Specifically: Step 1: Obtain a grayscale image, mark all smooth areas in the grayscale image, obtain the brightness value of the center point of each melt pool in the smooth area, and mark it as the center brightness value; Step 2: Set a preset threshold for the gradient distance. Shift the center point of the molten pool to the left by the preset threshold to obtain a left point. Obtain the brightness value of the left point and mark it as the left-first brightness value. Use the brightness value of the center point and the left-first brightness value to obtain the range of left-first gradient brightness values. Mark the areas that are not within the range of left-first gradient brightness values ​​among all brightness values ​​between the left point and the center point, and mark them as left-first crack areas. Repeat the above filtering with the left point as the center point to obtain all left crack areas. Step 3: Obtain a right point by shifting a gradually increasing distance to the right from the center point of the molten pool by a preset threshold. Obtain the brightness value of the right point and mark it as the right brightness value. Obtain the right gradually increasing brightness value range by using the brightness value of the center point and the right brightness value. Mark the area that is not within the right gradually increasing brightness value range among all brightness values ​​between the right point and the center point and mark it as the right crack area. Repeat the above filtering with the right point as the center point to obtain all right crack areas. Step 4: Count the number of cracks in all left and right crack areas, and determine if the number of cracks is greater than 0. If the number of cracks is greater than 0, the result is that there are cracks and all crack locations are marked. If the number of cracks is equal to 0, the result is that there are no cracks.

Images