Wave-shaped beam steel guardrail apparent disease detection method and device

Abstract

The present application provides a kind of wave beam steel guardrail apparent disease detection device, including 3 line scan cameras, 1 area array camera, rotary encoder, satellite navigation system locator, industrial computer, camera mounting bracket and detection vehicle;Also provides a kind of wave beam steel guardrail apparent disease detection method using the detection device, comprising: obtaining the line scan image data and area array image data of wave beam steel guardrail, after image data pre-processing, analysis is carried out, determines guardrail plate type and column type;From line scan image and pre-processed line scan image, the region of interest of wave bottom surface subgraph, upper wave surface subgraph and lower wave surface subgraph is separated;According to guardrail plate type, the bolt missing condition and deformation damage condition of guardrail plate wave bottom surface, guardrail plate upper wave surface and guardrail plate lower wave surface are detected from the region of interest;According to column type, the missing condition of column bolt is detected;Can solve the problem of low efficiency of manual inspection for plate surface deformation damage and bolt missing.

Description

Technical Field

[0001] This invention relates to the field of road traffic defect detection technology, specifically to a method and device for detecting apparent defects in corrugated beam steel guardrails. Background Technology

[0002] Corrugated beam steel guardrails are widely used in road traffic. Corrugated beam steel guardrails are mainly composed of guardrail panels, posts, anti-blocking blocks and bolts. Among them, guardrail panels are divided into two types: double-wave guardrail panels and triple-wave guardrail panels. Posts are divided into two types: round tube posts and square tube posts. The guardrail panels, posts and anti-blocking blocks are all connected by bolts.

[0003] To ensure road traffic safety, highway operation and management units require regular safety hazard inspections and identification of typical defects in in-service corrugated beam steel guardrails. Visual defects in corrugated beam steel guardrails include irregular alignment, panel deformation and damage (including deformation and damage at the top and bottom edges and internal surfaces), minor dents / protrusions, scratches, corrosion, misaligned posts, missing post caps, and missing or loose bolts. Among these, panel deformation and damage, and missing bolts, have the greatest impact on the mechanical safety performance of the guardrails.

[0004] However, with current technology, the main method for detecting panel deformation and damage as well as missing bolts is manual inspection, which relies on visual inspection on-site and is extremely inefficient. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method and device for detecting surface defects in corrugated beam steel guardrails, thereby solving the technical problems of low efficiency caused by manual inspection of panel deformation and bolt loss in existing technologies.

[0006] The technical solution adopted in this invention is as follows:

[0007] Firstly, a method for detecting apparent defects in corrugated beam steel guardrails is provided, including the following steps:

[0008] The line scan image data and area array image data of the corrugated beam steel guardrail are acquired. After preprocessing the image data, they are analyzed to determine the guardrail plate type and post type.

[0009] Regions of interest for wavefront sub-maps, upper wavefront sub-maps, and lower wavefront sub-maps are separated from the line scan image and the preprocessed line scan image.

[0010] Based on the type of guardrail, detect the bolt missingness and deformation / damage on the bottom surface, upper surface, and lower surface of the guardrail wave from the region of interest;

[0011] Inspect the column for missing bolts and column caps based on the column type.

[0012] Furthermore, line scan image data and area array image data of the corrugated beam steel guardrail are acquired. After preprocessing, the image data is analyzed to determine the guardrail panel type and post type, including:

[0013] Using an image acquisition-based detection device, the distance between the detection device and the inspected guardrail is always kept within a given range to acquire line scan images and area array images of the corrugated beam steel guardrail.

[0014] Preprocessing is performed on the line scan image and the area scan image to obtain the preprocessed line scan image and the preprocessed area scan image;

[0015] The guardrail type is determined based on the pixel spacing between the topmost and bottommost straight line segments in the preprocessed line scan image; the post type is determined based on whether there is a circular or elliptical outline in the area above the two vertical lines in the preprocessed area array image.

[0016] Furthermore, the line scan image is preprocessed, including: rotating the line scan image so that the length direction of the guardrail in the rotated line scan image is consistent with the horizontal direction, and the top edge of the guardrail is above the bottom edge, to obtain the rotated line scan image; and performing grayscale conversion, edge detection, binarization, and dilation / erosion operations on the rotated line scan image in sequence.

[0017] Preprocessing of the array image includes: performing grayscale conversion, edge detection, binarization, and dilation / erosion operations on the array image in sequence.

[0018] Furthermore, based on the guardrail type, the bolt loss status is detected from the region of interest on the bottom surface, upper surface, and lower surface of the guardrail wave, including:

[0019] Step 1: Identify all circular and elliptical outlines in the second region of interest;

[0020] Step 2: For any circular outline, check if there exists a rectangular outline of a specific size that is centered on the circle and contains the circular outline; if so, proceed to step 3; otherwise, go to step 4.

[0021] Step 3: In the first region of interest, determine whether the color inside the circular outline is the same as the color inside the rectangular outline outside the circular outline; if so, determine that the bolt at the circular outline is not missing; otherwise, determine that the bolt at the circular outline is missing and record the position information; then proceed to step 5.

[0022] Step 4: Check if there is a complete elliptical outline in the area where the circular outline is located, so that the circular outline is contained in the elliptical outline; if so, determine that the bolt is missing at the circular outline and record the position information; otherwise, determine that the bolt is not missing at the circular outline.

[0023] Step 5: Repeat steps 2 to 4 for the next circular outline until all circular outlines have been processed.

[0024] Furthermore, based on the type of guardrail, the deformation and damage of the bottom surface, upper surface, and lower surface of the guardrail wave are detected in the region of interest, including detecting the deformation and damage of the top and bottom edges of the guardrail using the following methods:

[0025] Step 1: In the second region of interest, determine whether the topmost or bottommost straight line segment horizontally penetrates the image. If so, determine that the top or bottom edge of the guardrail is normal and end the process. Otherwise, proceed to Step 2.

[0026] Step 2: Extend the topmost or bottommost line segment horizontally across the image, and connect the topmost or bottommost line segment with the remaining contours in a certain area above and below it to obtain several closed contours with the required size.

[0027] Step 3: For any closed contour above the top line segment or below the bottom line segment, determine in the first region of interest whether the color inside the contour is the same as the color of the area outside the contour below the top line segment or above the bottom line segment; if so, determine that the top or bottom edge of the guardrail is deformed and damaged and record the location information; otherwise, determine that there is no abnormality at the top or bottom edge of the guardrail at the contour and proceed to step 5.

[0028] Step 4: For any closed contour below the topmost straight line segment or above the bottommost straight line segment, determine in the first region of interest whether the color inside the contour is consistent with the color of the left and right sides outside the contour; if so, determine that there is no abnormality at the top or bottom edge of the guardrail at that contour; otherwise, determine that the top or bottom edge of the guardrail is deformed or damaged and record the position information.

[0029] Step 5: Repeat steps 3 and 4 for the next closed contour until all closed contours have been processed.

[0030] Furthermore, based on the type of guardrail, the deformation and damage of the bottom surface, upper surface, and lower surface of the guardrail wave are detected in the region of interest, including detecting the deformation and damage inside the guardrail surface using the following methods:

[0031] Step 1: For all straight lines between the top and bottom lines, determine whether all straight lines horizontally penetrate the image; if so, determine that there are no abnormalities inside the guardrail panel and end the process; otherwise, proceed to Step 2.

[0032] Step 2: For any straight line segment L between the topmost and bottommost straight line segments... k , will L kAfter connecting the endpoints with the nearby contours, check if there is a closed contour with a size greater than the set threshold; if so, determine that there is deformation or damage inside the guardrail panel and record the location information, then proceed to step 5; otherwise, execute step 3.

[0033] Step 3, in the first region of interest L k Extract texture information from the region near the endpoint;

[0034] Step 4: Determine if the texture contains an element that matches L. k The straight line segment where the endpoints meet; if so, it is determined that there is no abnormality inside the guardrail panel at the straight line; otherwise, it is determined that there is deformation or damage inside the guardrail panel and the location information is recorded.

[0035] Step 5: Repeat steps 2 to 4 for the next line segment until all line segments have been processed.

[0036] Furthermore, the missing bolts and caps of the columns are inspected according to the column type, including the missing bolts of square tube columns using the following methods:

[0037] Step 1: Select the top of the region between the left and right vertical lines in the preprocessed area image as the key region;

[0038] Step 2: Identify hexagonal contours within the key area that meet specific size requirements, and count the number of hexagonal contours, denoted as k1;

[0039] Step 3: Identify concentric circular and concentric elliptical contours within the key area that meet specific size requirements, count the number of concentric circular and concentric elliptical contours, and denot the sum of the two as k2.

[0040] Step 4: Determine whether k1+k2=2 is true; if yes, determine that the column bolt is not missing; otherwise, determine that the column bolt is missing and record the position information.

[0041] Furthermore, the missing bolts and caps of the columns are inspected according to the column type, including the missing bolts of round tube columns, using the following methods:

[0042] Step 1: If the area array image captures the image of the guardrail post on the left side of the driving direction, then take the left vertical line as the reference line; otherwise, take the right vertical line as the reference line.

[0043] Step 2: Check if there is a closed contour connected to the left or right vertical line in the area to the left of the left vertical line or the right of the right vertical line; if so, proceed to step 3; otherwise, determine that the column bolt is missing and record the location information.

[0044] Step 3: Find whether there exists a closed contour that simultaneously satisfies the following two conditions: the perimeter of the closed contour meets a specific requirement; and in the area image, the color within the closed contour is consistent with the color of the area between the left vertical line and the right vertical line.

[0045] If yes, then the column bolts are not missing; otherwise, the column bolts are missing and their location information is recorded.

[0046] Furthermore, the missing bolts and caps of the columns are detected according to the column type. This includes detecting the missing caps of round pipe columns using the following method: It is determined whether the pre-processed area image simultaneously meets the following two conditions: There is no contour connected to the left or right vertical line in the top region to the left of the left vertical line or to the right of the right vertical line; There are two concentric circular contours or two concentric elliptical contours in the key area, and the size ratio of the two concentric contours meets specific requirements; if so, the cap is determined to be missing and its position information is recorded; otherwise, the cap is determined not to be missing.

[0047] Secondly, a device for detecting apparent defects in corrugated beam steel guardrails is provided, used in the method for detecting apparent defects in corrugated beam steel guardrails described in the first aspect, comprising:

[0048] Three line scan cameras, one area scan camera, a rotary encoder, a satellite navigation system locator, an industrial computer, camera mounting brackets, and a testing vehicle;

[0049] The line scan camera and the area scan camera are connected to the side and top of the inspection vehicle via camera mounting brackets, and are used to acquire images of guardrails and pillars.

[0050] The rotary encoder is installed at the center of the rear wheel hub of the inspection vehicle and is used to sense changes in the position of the inspection vehicle and trigger the line scan camera and area scan camera to acquire images.

[0051] The satellite navigation system locator is used to obtain the real-time location information of the inspection vehicle, and the industrial computer is used to control image and location acquisition as well as the identification and analysis of surface defects in the guardrail.

[0052] Furthermore, among the three line scan cameras, the first line scan camera is installed at the same height as the center of the roadside guardrail, and after installation, the scan line is perpendicular to the road surface, used to acquire images of the guardrail and posts from a frontal view; the second line scan camera is installed at a height higher than the top of the guardrail, and the third line scan camera is installed at a height lower than the bottom of the guardrail. After installation, the projections of the scan lines of the second and third line scan cameras on the road surface are both perpendicular to the direction of travel, used to acquire images of the guardrail from top and bottom views, respectively.

[0053] The area array camera is installed at a height higher than the top of the guardrail and its projection on the ground is located outside the guardrail. After installation, the central optical axis is parallel to the driving direction and intersects the ground at an acute angle, and is used to collect images of the guardrail's outer posts and their bolts.

[0054] As can be seen from the above technical solution, the beneficial technical effects of the present invention are as follows:

[0055] 1. The surface defect detection method for corrugated beam steel guardrails can automatically collect and analyze the deformation, damage and missing bolts on the surface of the corrugated beam steel guardrails, with high detection efficiency.

[0056] 2. The surface defect detection device for corrugated beam steel guardrails adopts vehicle-mounted detection, which eliminates the need for traffic control measures. Compared with manual detection, it can significantly improve efficiency and avoid traffic safety risks, thus having considerable social benefits. Attached Figure Description

[0057] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0058] Figure 1 This is a front view of the mounting surface of the guardrail surface defect detection device according to an embodiment of the present invention;

[0059] Figure 2 This is a side view of the mounting surface of the detection device according to an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of the double-wave guardrail structure according to an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the three-wave guardrail structure according to an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the process for detecting apparent defects in guardrails according to an embodiment of the present invention.

[0063] Figure label:

[0064] 1-1# Line Scan Camera; 2-2# Line Scan Camera; 3-3# Line Scan Camera; 4-Area Scan Camera; 5-Camera Mounting Bracket; 6-Rotary Encoder; 7-BeiDou System Positioner; 8-Industrial Computer; 9-Inspection Vehicle; 10-Guardrail; 11-Post; 12-Anti-block; 13-Panel Bolt; 14-Post Bolt; 15-Round Tube Post Cap; 16-Wave Bottom Surface; 17-Upper Wave Surface; 18-Lower Wave Surface; 19-Rectangular Gasket. Detailed Implementation

[0065] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0066] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0067] Example

[0068] This embodiment provides a method and device for detecting apparent defects in corrugated beam steel guardrails. The specific detection device is as follows:

[0069] like Figure 1 , Figure 2 As shown, the testing device consists of 3 line scan cameras, 1 area scan camera, a rotary encoder, a satellite navigation system locator, an industrial computer, a camera mounting bracket, and a testing vehicle.

[0070] In some embodiments, the line scan camera and the area scan camera are both full-color cameras, which are connected to the side and top of the inspection vehicle via camera mounting brackets.

[0071] In a specific implementation, the installation height of line scan camera #1 is consistent with the center height of the roadside guardrail (approximately 650mm). After installation, the scan line is perpendicular to the road surface and is used to acquire images of the guardrail and posts from a frontal view. The installation height of line scan camera #2 is higher than the top of the guardrail, and the installation height of line scan camera #3 is lower than the bottom of the guardrail. After installation, the projections of the scan lines of line scan cameras #2 and #3 onto the road surface are both perpendicular to the direction of traffic and are used to acquire images of the guardrail from top and bottom views, respectively.

[0072] In a specific implementation, the area array camera is installed at a height higher than the top of the guardrail and its projection on the ground is located outside the guardrail. After installation, the central optical axis is parallel to the driving direction and intersects the ground at an acute angle, which is used to collect images of the guardrail's outer posts and their bolts.

[0073] In some embodiments, a rotary encoder is installed at the center of the rear wheel hub of the inspection vehicle to sense changes in the vehicle's position and trigger the line scan camera and area scan camera to acquire images.

[0074] In some embodiments, the satellite navigation system locator is selected from the Beidou system locator and installed inside the inspection vehicle to obtain the real-time location information of the inspection vehicle; the industrial computer is installed inside the inspection vehicle to control image and location acquisition and to identify and analyze the surface defects of the guardrail.

[0075] In some embodiments, the testing vehicle is a commonly used four-wheeled car.

[0076] The method for detecting apparent defects in corrugated beam steel guardrails, using the aforementioned testing device, includes the following steps:

[0077] S1. Use an image acquisition-based detection device to acquire line scan images and area array images of the corrugated beam steel guardrail. After preprocessing the image data, analyze it to determine the guardrail panel type and post type.

[0078] Figure 3 , Figure 4 The diagram shows the structure of two types of corrugated beam steel guardrails. It can be seen from the diagram that the guardrail panels of the corrugated beam steel guardrails include double-wave guardrail panels and triple-wave guardrail panels, the posts include square tube posts and round tube posts, and the bolts include plate bolts and post bolts.

[0079] The specific implementation method for this step is as follows:

[0080] S11. Using an image acquisition-based detection device, acquire line scan images and area array images of the corrugated beam steel guardrail.

[0081] The inspection vehicle is controlled to maintain the distance between the vehicle body inspection device and the inspected guardrail within a given range. Line scan and area scan cameras continuously acquire images, resulting in line scan images Img1, Img2, and Img3 acquired by the line scan cameras and area scan image ImgM acquired by the area scan cameras. During the acquisition process, the BeiDou system locator continuously collects the inspection vehicle's position information.

[0082] S12. Preprocess the line scan image and the area array image to obtain the preprocessed line scan image and the preprocessed area array image.

[0083] The line scan images Img1, Img2, and Img3 are rotated so that the length direction of the guardrail in the rotated line scan image is aligned with the horizontal direction, and the top edge of the guardrail is above the bottom edge, thus obtaining the rotated line scan image. Then, the rotated line scan image is sequentially subjected to grayscale conversion, edge detection, binarization, and dilation / erosion operations to obtain the preprocessed line scan image Img1. edge 1mg2 edge 1mg3 edge ;

[0084] The area array image ImgM is sequentially subjected to grayscale conversion, edge detection, binarization, and dilation / erosion operations to obtain the preprocessed area array image ImgM. edge .

[0085] S13. Determine the guardrail type based on the pixel spacing between the topmost and bottommost straight line segments in the preprocessed line scan image.

[0086] Preprocessed line scan image Img1 edgeFind all line segments whose length is greater than a set value, and denote the uppermost and lowermost line segments as L and L, respectively. top and L bot By L top L bot In Img1 edge The pixel spacing determines whether the guardrail is a double-wave or triple-wave guardrail. In some embodiments, the setting value is typically 0.5 times the image pixel width. For example, if the image pixel size is 3000 (width) × 1000 (height), then the setting value here is 1500 pixels. For an image with a pixel size of 3000 (width) × 1000 (height), a pixel spacing of less than or equal to 1800 pixels indicates a double-wave guardrail, and a pixel spacing greater than 1800 pixels indicates a triple-wave guardrail.

[0087] S14. Determine the column type based on whether there is a circular or elliptical outline in the area above the two vertical lines in the preprocessed area image.

[0088] Preprocessed area array image ImgM edge Find two parallel vertical lines L in the given information, whose lengths and spacing meet specific requirements. left L right In some embodiments, specific requirements are illustrated, such as for a 4000 (width) × 3000 (height) area image, where the length is in the range of 500 to 1500 pixels and the spacing is in the range of 100 to 200 pixels. In the following, L... left Defined as the left vertical line, L right Define it as a right vertical line.

[0089] In L left L right The system checks if a circular or elliptical outline with the required size exists in the upper region; if so, the column is determined to be a circular tube column; otherwise, it is determined to be a square tube column. In some embodiments, the requirements for determining the column type are illustrated as follows: for a 4000 (width) × 3000 (height) area array image, a circle with a radius in the range of 100 to 200 pixels, or an ellipse with a major axis in the range of 100 to 200 pixels.

[0090] In specific implementations, the execution order of steps S13 and S14 is not limited; they can be performed in any order or simultaneously.

[0091] S2. Based on the guardrail type, separate the regions of interest (ROIs) from the line scan image and the preprocessed line scan image to obtain the wavefront base sub-image, upper wavefront sub-image, and lower wavefront sub-image.

[0092] In a specific implementation, the Region of Interest (ROI) for double-wave and triple-wave guardrails are obtained as follows:

[0093] (1) Double wave guardrail

[0094] ROI bd =Img1(r top +0.4h:r top +0.6h; :)

[0095] ROI edge—bd =Img1 edge (r top +0.4h:r top +0.6h; :)

[0096] ROI sb =[Img2(r top +p 2s :r top +p 2s +0.25h; :); Img2(r top +p 2s +0.45h:r top +p 2s +0.75h; :)]

[0097] ROI edge—sb =[Img2 edge (r top +p 2s :r top +p 2s +0.25h; :); Img2 edge (r top +p 2s +0.45h:r top +p 2s +0.75h; :)]

[0098] ROI xb =[Img3(r top +p 2x :r top +p 2x +0.3h; :); Img3(r top +p 2x +0.5h:r top +p 2x +0.75h; :)]

[0099] ROI edge—xb =[Img3 edge (r top +p 2x :r top +p 2x +0.3h; :); Img3 edge (r top +p2x +0.5h:r top +p 2x +0.75h; :)]

[0100] (2) Three-wave guardrail:

[0101] ROI bd =[Img1(r top +0.2h:r top +0.4h; :); Img1(r top +0.6h:r top +0.8h; :)]

[0102] ROI edge—bd =[Img1 edge (r top +0.2h:r top +0.4h; :); Img1 edge (r top +0.6h:r top +0.8h; :)]

[0103] ROI sb =[Img2(r top +p 3s :r top +p 3s +0.30h;:);Img2(r top +p 3s +0.55h:r top +p 3s +0.75h; :)

[0104] Img2(r top +p 3s +0.9h:r top +p 3s +h;:)]

[0105] ROI edge—sb =[Img2 edge (r top +p 3s :r top +p 3s +0.30h; :); Img2 edge (r top +p 3s +0.55h:

[0106] r top +p 3s +0.75h; :); Img2 edge (r top +p 3s +0.9h:rtop +p 3s +h;:)]

[0107] ROI xb =[Img3(r top +p 3x :r top +p 3x +0.10h;:);Img3(r top +p 3x +0.25h:r top +p 3x +0.45h; :)

[0108] Img3(r top +p 3x +0.7h:r top +p 3x +h;:)]

[0109] ROIedge —xb =[Img3edge(r top +p 3x :r top +p 3x +0.10h;:);Img3edge(r top +p 3x +0.25h:

[0110] r top +p 3x +0.45h; :); Img3edge(r top +p 3x +0.7h:r top +p 3x +h;:)]

[0111] Where, the subscript bd corresponds to the wavefront sub-image, the subscript sb corresponds to the upper wavefront sub-image, the subscript xb corresponds to the lower wavefront sub-image, and the subscript edge represents the preprocessed image; h = r bot -r top r top and r bot Img1 edge L in the image top L bot line number; p 2s p 2x p 3s p 3x This is the row number offset value, determined by adjusting based on the spatial relationship of the imaging fields of line scan cameras #1, #2, and #3.

[0112] When using regions of interest (ROIs) later, the edges without subscripts will be designated as the first ROI. bd The region with the subscripted edge is divided into the second region of interest, such as ROI. edge—bd .

[0113] S3. Inspect the area of ​​interest for bolt loss and deformation / damage on the bottom surface, upper surface, and lower surface of the guardrail.

[0114] The surface color of corrugated beam steel guardrails is usually a uniform pure silver, pure green, or pure blue.

[0115] S31. Inspect the missing bolts on the bottom, upper, and lower wave surfaces of the guardrail panel.

[0116] Follow these steps to extract the Region of Interest (ROI) from the wave base subplot. bd ROI edge—bd Inspect the missing bolts on the bottom surface of the guardrail panel:

[0117] (1) Find the second region of interest (ROI) edge—bd All circular and elliptical outlines;

[0118] (2) For any circular outline, check if there exists a rectangular outline of a specific size that is centered on the circle and contains the circular outline; if so, proceed to step (3), otherwise proceed to step (4); For example, the “characteristic size” in this step is: for an image with a pixel size of 3000 (width) × 1000 (height), the “characteristic size” refers to the rectangle’s long side being in the range of 100 to 150 pixels and the ratio of the short side to the long side being in the range of 0.65 to 0.85.

[0119] (3) Region of Interest (ROI) bd The color inside the circular outline is determined to be consistent with the color inside the rectangular outline outside the circular outline; if so, it is determined that the bolt at the circular outline is not missing; otherwise, it is determined that the bolt at the circular outline is missing and the position information is recorded; then proceed to step (5).

[0120] (4) Check if there is a complete elliptical outline in the area where the circular outline is located, so that the circular outline is contained in the elliptical outline; if so, determine that the bolt is missing at the circular outline and record the position information; otherwise, determine that the bolt is not missing at the circular outline.

[0121] (5) Repeat steps (2) to (4) for the next circular contour until all circular contours have been processed.

[0122] Using the same method as sub-steps (1)-(5) in step S31, the region of interest (ROI) of the upper wavefront submap is obtained. sb ROIedge—sb The detection method identifies missing bolts on the corrugated surface of guardrail panels and identifies the Region of Interest (ROI) from the lower corrugated surface submap. xb ROI edge—xb The inspection focused on the missing bolts on the lower corrugated surface of the guardrail.

[0123] S32. Inspect the deformation and damage of the top and bottom edges of the guardrail.

[0124] Follow these steps to select the Region of Interest (ROI) bd ROI edge—bd The inspection focused on the deformation and damage of the top edge of the guardrail panel.

[0125] (1) The second region of interest (ROI) edge—bd In the judgment of L top or L bot If the image is horizontally traversed, determine if there is no abnormality at the top or bottom edge of the guardrail and end the process; otherwise, proceed to step (2).

[0126] (2) Extend L top or L bot Make it span horizontally across the image, and make L top or L bot Connect the closed contours within a certain area above and below the image to obtain several closed contours with the required dimensions. For example, for an image with pixel dimensions of 3000 (width) × 1000 (height), the perimeter of the closed contours must be greater than 60 pixels. For example, the "certain area" in this step refers to a 200-pixel height range above and below the image for a 300-pixel image with pixel dimensions of 3000 (width) × 1000 (height).

[0127] (3) For L top Above or L bot Any closed contour below, within the first region of interest (ROI) bd Determine the color inside the outline and outside the outline L. top Below or L bot Check if the colors in the upper area are consistent; if so, determine that the top or bottom edge of the guardrail is deformed or damaged and record the location information; otherwise, determine that the top or bottom edge of the guardrail at this outline is normal; then proceed to step (5).

[0128] (4) For L top Below or L bot Any closed contour above, within the first region of interest (ROI) bd The system determines whether the color inside the outline is consistent with the color of the left and right sides outside the outline; if so, it determines that there is no abnormality at the top or bottom edge of the guardrail at the outline; otherwise, it determines that the top or bottom edge of the guardrail is deformed or damaged and records the location information.

[0129] (5) Repeat steps (3) to (4) for the next closed contour until all closed contours have been processed.

[0130] Using the same method as sub-steps (1)-(5) in S32, from the ROI bd ROI edge—bd The inspection focused on the deformation and damage of the bottom edge of the guardrail.

[0131] S33. Inspect the internal deformation and damage of the guardrail panels.

[0132] Follow these steps to obtain ROI bd ROI edge—bd Inspect the internal deformation and damage of the guardrail panels.

[0133] (1) For L top L bot Check all straight line segments between them to see if they all horizontally penetrate the image; if so, determine that there is no abnormality inside the guardrail panel and end the process; otherwise, proceed to step (2).

[0134] (2) For L top L bot Any straight line segment L between k , will L k After connecting the endpoints of the guardrail with their nearby contours, check if there is a closed contour with a size greater than the set threshold. If so, determine that there is deformation or damage inside the guardrail panel and record the location information, then proceed to step (5). Otherwise, execute step (3). For example, for an image with a pixel size of 3000 (width) × 1000 (height), "nearby" refers to a row and column interval of less than 200 pixels. The set threshold is determined according to the internal size of the guardrail panel.

[0135] (3) In ROI bd L k Texture information is extracted from the region near the endpoints. In specific implementations, the method for extracting texture information is not limited and can be implemented in any feasible manner in the prior art, such as gray-level co-occurrence matrix. For example, the term "nearby" in this step refers to an image with a pixel size of 3000 (width) × 1000 (height) where the row and column spacing of "nearby" is within 100 pixels.

[0136] (4) Determine if there is a texture that matches L. k The straight line segment where the endpoints meet; if so, it is determined that there is no abnormality inside the guardrail panel at the straight line; otherwise, it is determined that there is deformation or damage inside the guardrail panel and the location information is recorded.

[0137] (5) Repeat steps (2) to (4) for the next line segment until all line segments have been processed.

[0138] In specific implementations, the execution order of steps S32 and S33 is not limited; they can be performed in any order or simultaneously.

[0139] S4. Inspect the missing bolts and caps of the columns according to their type.

[0140] S41. Inspect the missing bolts on the square tube column.

[0141] (1) From the preprocessed area array image ImgM edge Select vertical straight line L left L right The top of the area between them is the critical area;

[0142] (2) Identify hexagonal contours within the key area whose dimensions meet specific requirements, count the number of hexagonal contours, and record them as k1; In some embodiments, the determination requirements are illustrated as follows: For a 4000 (width) × 3000 (height) pixel area array image, the specific requirement is that the side length is in the range of 30 to 60 pixels.

[0143] (3) Identify concentric circular contours and concentric elliptical contours within the key area whose dimensions meet specific requirements, count the number of concentric circular contours and concentric elliptical contours, and denot the sum of the two as k2; In some embodiments, the determination requirements are illustrated as follows: For a 4000 (width) × 3000 (height) pixel area array image, the outer circle pixel radius is between 15 and 30, and the ratio of the inner circle pixel radius to the outer circle pixel radius is between 0.7 and 0.95.

[0144] The major axis of the outer ellipse has a pixel length between 15 and 30, and the ratio of the radius of the major axis of the inner ellipse to the radius of the major axis of the outer ellipse is between 0.7 and 0.95.

[0145] (4) Determine whether k1+k2=2 is true; if yes, determine that the column bolt is not missing; otherwise, determine that the column bolt is missing and record the position information.

[0146] S42. Inspect the missing bolts and column caps of the round pipe column.

[0147] (1) If the area array image ImgM captures the image of the guardrail post on the left side of the driving direction, then take the vertical straight line L left Use the vertical line as a reference line; otherwise, use the vertical straight line L. right Used as a reference line;

[0148] (2) In L left Left side or L right Check if there exists a match for L in the area on the right. left or L right Connected closed contours; if so, proceed to step (3); otherwise, determine that the column bolts are missing and record the position information.

[0149] (3) Check if there exists a closed contour that simultaneously satisfies the following two conditions. If yes, determine that the column bolts are not missing; otherwise, determine that the column bolts are missing and record their location information;

[0150] ① The size of the closed contour meets specific requirements; in some embodiments, the determination requirements are illustrated as follows: for a 4000 (width) × 3000 (height) pixel area array image, the specific requirement is that the perimeter of the closed contour is in the range of 100 to 250.

[0151] ② Corresponding to ImgM, the color within the closed contour is the same as L. left L right The colors of the areas are consistent.

[0152] In some embodiments, for circular tube columns, it is also necessary to detect whether the column cap is missing, specifically as follows: Determine the preprocessed area array image ImgM. edge Does the following two conditions meet simultaneously? If so, determine that the column cap is missing and record its location information; otherwise, determine that the column cap is not missing.

[0153] (1) In L left Left side or L right The top area on the right does not have an L. left or L right Connected outlines;

[0154] (2) The critical region contains two concentric circular contours or two concentric elliptical contours, and the size ratio of the two concentric contours meets specific requirements. The critical region here is defined as: from the preprocessed area image ImgM edge Select vertical straight line L left L right The top of the area between the columns is the critical area, and specific requirements are determined based on the size of the column cap.

[0155] In specific implementations, the execution order of steps S41 and S42 is not limited; they can be performed in any order or simultaneously.

[0156] The surface defect detection method for corrugated beam steel guardrails described in this embodiment can automatically collect and analyze data on surface deformation, damage, and missing bolts in the corrugated beam steel guardrail panels, resulting in high detection efficiency. The surface defect detection device for corrugated beam steel guardrails is vehicle-mounted, eliminating the need for traffic control measures. Compared to manual detection, it significantly improves efficiency and avoids traffic safety risks, thus offering considerable social benefits.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for detecting apparent defects in corrugated beam steel guardrails, characterized in that, Includes the following steps: The line scan image data and area array image data of the corrugated beam steel guardrail are acquired. After preprocessing the image data, they are analyzed to determine the guardrail plate type and post type. Regions of interest (ROIs) for wavefront sub-maps, upper wavefront sub-maps, and lower wavefront sub-maps are separated from the line scan image and the preprocessed line scan image. The ROIs include a first ROI and a second ROI, wherein the first ROI is separated from the line scan image and the second ROI is separated from the preprocessed line scan image. Based on the guardrail type, the detection of missing bolts on the bottom surface, upper surface, and lower surface of the guardrail wave is performed within the region of interest. This includes: Step 1: Identify all circular and elliptical contours within the second region of interest; Step 2: For any circular contour, check if a rectangular contour of a specific size exists centered on the circle and encompassing it; if so, proceed to Step 3; otherwise, proceed to Step 4; Step 3: In the first region of interest, determine if the color within the circular contour matches the color within the rectangular contour outside the circular contour; if so, determine that the bolts at that circular contour are not missing; otherwise, determine that the bolts at that circular contour are missing and record the location information; then proceed to Step 5; Step 4: Check if a complete elliptical contour exists within the area containing the circular contour; if so, determine that the bolts at that circular contour are missing and record the location information; otherwise, determine that the bolts at that circular contour are not missing; Step 5: Repeat steps 2-4 for the next circular contour until all circular contours have been processed. Based on the type of guardrail, detect the deformation and damage of the bottom, upper, and lower wave surfaces of the guardrail within the region of interest. This includes detecting the deformation and damage of the top and bottom edges of the guardrail using the following method: Step 1: In the second region of interest, determine whether the uppermost or lowermost straight line segment horizontally penetrates the image. If so, determine that the top or bottom edge of the guardrail is normal and end the process; otherwise, proceed to Step 2. Step 2: Extend the uppermost or lowermost straight line segment horizontally through the image, connecting it to the remaining contours within a certain area above and below it to obtain several closed contours with the required dimensions. Step 3: For any closed contour above or below the uppermost or lowermost straight line segment... Step 4: For any closed contour below the uppermost line segment or above the lowermost line segment, determine in the first region of interest whether the color inside the contour is consistent with the color of the area below the uppermost line segment or above the lowermost line segment outside the contour. If so, determine that the top or bottom edge of the guardrail is deformed or damaged and record the position information. Otherwise, determine that the top or bottom edge of the guardrail at the contour is not abnormal and proceed to step 5. Step 5: Repeat steps 3 to 4 for the next closed contour until all closed contours are processed. Based on the guardrail type, detect the deformation and damage of the bottom surface, upper surface, and lower surface of the guardrail in the region of interest. This includes detecting the deformation and damage inside the guardrail surface using the following method: Step 1: For all straight segments between the top and bottom lines, determine whether all straight segments horizontally penetrate the image; if so, determine that there are no abnormalities inside the guardrail surface and end the process; otherwise, proceed to Step 2; Step 2: For any straight segment L between the top and bottom lines... k , will L k After connecting the endpoints of the guardrail with its nearby contours, check if there is a closed contour with a size greater than a set threshold; if so, determine that there is deformation or damage inside the guardrail panel and record the location information, then proceed to step 5; otherwise, execute step 3; Step 3, in the first region of interest L k Extract texture information from the region near the endpoint; Step 4: Determine if the texture contains information related to L. k The straight line segments where the endpoints meet; if so, it is determined that there is no abnormality inside the guardrail panel at the straight line; otherwise, it is determined that there is deformation or damage inside the guardrail panel and the location information is recorded; Step 5: Repeat steps 2 to 4 for the next straight line segment until all straight line segments are processed. The detection of missing bolts and caps for column pillars is based on the column type, including the detection of missing bolts for square tube columns using the following method: Step 1: Select the top of the area between the left and right vertical lines in the preprocessed area array image as the key area; Step 2: Identify hexagonal contours within the key area that meet specific size requirements, and count the number of hexagonal contours, denoted as k1; Step 3: Identify concentric circular contours and concentric elliptical contours within the key area that meet specific size requirements, and count the number of concentric circular contours and concentric elliptical contours, denoted as k2; Step 4: Determine whether k1 + k2 = 2 is true; if yes, the column bolts are not missing; otherwise, the column bolts are missing, and their location information is recorded. The detection of missing bolts and caps for column pillars is based on the column type, including the detection of missing bolts for round pipe columns using the following method: Step 1: If the area array image captures an image of the guardrail column on the left side of the driving direction, then take the left vertical line as the reference line; otherwise, take the right vertical line as the reference line; Step 2: Search within the area to the left of the left vertical line or to the right of the right vertical line for a closed contour that connects to either the left or right vertical line; if yes, proceed to Step 3; otherwise, determine that the column bolt is missing and record its location information; Step 3: Search within the closed contour for a closed contour that simultaneously satisfies the following two conditions: the perimeter of the closed contour meets a specific requirement; corresponding to the area array image, the color within the closed contour is consistent with the color of the area between the left and right vertical lines; if yes, determine that the column bolt is not missing; otherwise, determine that the column bolt is missing and record its location information. The detection of missing bolts and caps of columns is based on column type, including the detection of missing caps of round pipe columns using the following method: Determine whether the pre-processed area image simultaneously meets the following two conditions: There is no contour connected to the left vertical line in the top area to the left of the left vertical line, or there is no contour connected to the right vertical line in the top area to the right of the right vertical line; There are two concentric circular contours or two concentric elliptical contours in the key area, and the size ratio of the two concentric contours meets specific requirements; if so, the cap is determined to be missing and its position information is recorded; otherwise, the cap is determined not to be missing.

2. The method for detecting apparent defects in corrugated beam steel guardrails according to claim 1, characterized in that, Obtain line scan and area array image data of the corrugated beam steel guardrail. After preprocessing the image data, analyze it to determine the guardrail panel type and post type, including: Using an image acquisition-based detection device, the distance between the detection device and the inspected guardrail is always kept within a given range to acquire line scan images and area array images of the corrugated beam steel guardrail. Preprocessing is performed on the line scan image and the area scan image to obtain the preprocessed line scan image and the preprocessed area scan image; The guardrail type is determined based on the pixel spacing between the topmost and bottommost straight line segments in the preprocessed line scan image; the post type is determined based on whether there is a circular or elliptical outline in the area above the two vertical lines in the preprocessed area array image.

3. The method for detecting apparent defects in corrugated beam steel guardrails according to claim 2, characterized in that, The line scan image is preprocessed, including: rotating the line scan image so that the length direction of the guardrail in the rotated line scan image is consistent with the horizontal direction, and the top edge of the guardrail is above the bottom edge, thus obtaining the rotated line scan image; and performing grayscale conversion, edge detection, binarization, and dilation / erosion operations on the rotated line scan image in sequence. Preprocessing of the array image includes: performing grayscale conversion, edge detection, binarization, and dilation / erosion operations on the array image in sequence.

4. A device for detecting apparent defects in corrugated beam steel guardrails, characterized in that, The method for detecting apparent defects in corrugated beam steel guardrails according to any one of claims 1-3 includes: Three line scan cameras, one area scan camera, a rotary encoder, a satellite navigation system locator, an industrial computer, camera mounting brackets, and a testing vehicle; The line scan camera and the area scan camera are connected to the side and top of the inspection vehicle via camera mounting brackets, and are used to acquire images of guardrails and pillars. The rotary encoder is installed at the center of the rear wheel hub of the inspection vehicle and is used to sense changes in the position of the inspection vehicle and trigger the line scan camera and area scan camera to acquire images. The satellite navigation system locator is used to obtain the real-time location information of the inspection vehicle, and the industrial computer is used to control image and location acquisition as well as the identification and analysis of surface defects in the guardrail.

5. The device for detecting apparent defects in corrugated beam steel guardrails according to claim 4, characterized in that, Of the three line scan cameras, the first line scan camera is installed at the same height as the center of the roadside guardrail, and its scan line is perpendicular to the road surface after installation. It is used to acquire images of the guardrail and posts from a frontal view. The second line scan camera is installed at a height higher than the top of the guardrail, and the third line scan camera is installed at a height lower than the bottom of the guardrail. After installation, the projections of the scan lines of the second and third line scan cameras onto the road surface are perpendicular to the direction of travel, and they are used to acquire images of the guardrail from top and bottom views, respectively. The area scan camera is installed at a height higher than the top of the guardrail, and its projection on the ground is located outside the guardrail. After installation, its central optical axis is parallel to the direction of travel and intersects the ground at an acute angle. It is used to acquire images of the posts and their bolts on the outer side of the guardrail.

Images