Corrugated beam steel guardrail apparent disease detection method and device

Abstract

The invention provides an apparent disease detection device for a corrugated beam steel guardrail. The apparent disease detection device comprises three line scanning cameras, one area array camera, a rotary encoder, a satellite navigation system positioner, an industrial computer, a camera mounting bracket and a detection vehicle, the invention also provides a waveform beam steel guardrail apparent disease detection method using the detection device, and the method comprises the steps: obtaining line scanning image data and area array image data of a waveform beam steel guardrail, preprocessing the image data, analyzing the image data, and determining the type of a guardrail plate and the type of a stand column; separating from the line scanning image and the preprocessed line scanning image to obtain regions of interest of a wave bottom surface sub-image, an upper wave surface sub-image and a lower wave surface sub-image; according to the type of the guardrail plate, bolt missing conditions and deformation damage conditions of the guardrail plate wave bottom surface, the guardrail plate upper wave surface and the guardrail plate lower wave surface are detected from the region of interest; detecting the missing condition of the column bolt according to the column type; the problem of low efficiency of manual inspection for plate surface deformation and damage and bolt loss can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of road traffic disease detection, and particularly relates to a method and device for detecting apparent diseases of corrugated beam steel guardrails. Background Art

[0002] In road traffic, corrugated beam steel guardrails are widely used. The corrugated beam steel guardrail mainly consists of guardrail plates, columns, anti-collision blocks and bolts. Among them: the guardrail plates are divided into two categories: double-wave guardrail plates and triple-wave guardrail plates, the columns are divided into two categories: round pipe columns and square pipe columns, and bolts are used to connect the guardrail plates, columns and anti-collision blocks.

[0003] To ensure road traffic safety, highway operation and management units require regular safety hazard inspections and identification of typical diseases for in-service corrugated beam steel guardrail facilities. The apparent defects of corrugated beam steel guardrails include uncomfortable alignment, deformation and damage of the plate surface (including deformation and damage of the top edge, bottom edge and inside of the plate surface), minor depressions / protrusions, scratches, rust, column skew, column cap loss, bolt loss, looseness, etc. Among them, the deformation and damage of the plate surface and the loss of bolts are the apparent defects that have the greatest impact on the mechanical safety performance of the guardrail.

[0004] However, in current technologies, for the deformation and damage of the plate surface and the loss of bolts, manual inspection is mainly used, and judgments are made through on-site visual inspection, with very low efficiency. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the present invention proposes a method and device for detecting apparent diseases of corrugated beam steel guardrails to solve the technical problems of manual inspection and low efficiency for the deformation and damage of the plate surface and the loss of bolts in the existing technology.

[0006] The technical solution adopted by the present invention is as follows:

[0007] In the first aspect, a method for detecting apparent diseases of corrugated beam steel guardrails is provided, including the following steps:

[0008] Obtain the line scan image data and area array image data of the corrugated beam steel guardrail, preprocess the image data and then analyze it to determine the guardrail plate type and column type;

[0009] Separate the regions of interest of the wave bottom sub-image, upper wave sub-image and lower wave sub-image from the line scan image and the preprocessed line scan image;

[0010] Detect the bolt loss situation and deformation and damage situation of the wave bottom surface, upper wave surface and lower wave surface of the guardrail plate from the regions of interest according to the guardrail plate type;

[0011] Detect the loss situation of the column bolts and column caps according to the column type.

[0012] Further, obtain the line-scan image data and area-array image data of the corrugated beam steel guardrail, preprocess the image data and then analyze it to determine the guardrail plate type and column type, including:

[0013] Use a detection device based on image acquisition to keep the distance between the detection device and the guardrail to be inspected within a given range all the time, and obtain the line-scan image and area-array image of the corrugated beam steel guardrail;

[0014] Preprocess the line-scan image and area-array image to obtain the preprocessed line-scan image and preprocessed area-array image;

[0015] Determine the guardrail plate type according to the pixel spacing between the uppermost and lowermost straight line segments in the preprocessed line-scan image; determine the column type according to whether there are circular or elliptical contours in the area above the two vertical straight lines in the preprocessed area-array image.

[0016] Further, preprocess the line-scan image, including: rotate the line-scan image so that the length direction of the guardrail plate in the rotated line-scan image is consistent with the horizontal direction, and the top edge of the guardrail plate is above the bottom edge to obtain the rotated line-scan image; sequentially perform graying, edge detection, binarization, and dilation / erosion operations on the rotated line-scan image;

[0017] Preprocess the area-array image, including: sequentially perform graying, edge detection, binarization, and dilation / erosion operations on the area-array image.

[0018] Further, detect the bolt missing conditions of the bottom wave surface, upper wave surface, and lower wave surface of the guardrail plate from the region of interest according to the guardrail plate type, including:

[0019] Step 1: Find all circular and elliptical contours in the second region of interest;

[0020] Step 2: For any circular contour, check whether there is a specific-sized rectangular contour centered on the center of the circle and containing the circular contour; if so, execute Step 3, otherwise go to Step 4;

[0021] Step 3: In the first region of interest, judge whether the color inside the circular contour is the same as the color inside the rectangular contour outside the circular contour; if so, determine that the bolt at the circular contour is not missing, otherwise determine that the bolt at the circular contour is missing and record the position information; then go to Step 5;

[0022] Step 4: Check whether there is a complete elliptical contour in the region where the circular contour is located, with the circular contour contained in the elliptical contour; if so, determine that the bolt at the circular contour is missing and record the position information, otherwise determine that the bolt at the circular contour is not missing;

[0023] Step 5: Repeat Steps 2-4 for the next circular contour until all circular contours are processed.

[0024] Furthermore, detect the deformation and damage conditions of the bottom wave surface, upper wave surface, and lower wave surface of the guardrail plate from the region of interest according to the guardrail plate type, including detecting the deformation and damage conditions of the top edge and bottom edge of the guardrail plate by the following method:

[0025] Step 1: In the second region of interest, determine whether the uppermost straight line segment or the lowermost straight line segment horizontally penetrates the image. If so, it is determined that the top edge or bottom edge of the guardrail plate is normal and the process ends. If not, execute Step 2;

[0026] Step 2: Extend the uppermost straight line segment or the lowermost straight line segment to make it horizontally penetrate the image, and connect the uppermost straight line segment or the lowermost straight line segment with the remaining contours within a certain area above and below it to obtain several closed contours that meet the requirements;

[0027] Step 3: For any closed contour above the uppermost straight line segment or below the lowermost straight line segment, in the first region of interest, determine whether the color inside the contour is the same as the color of the region above the lowermost straight line segment or below the uppermost straight line segment outside the contour. If so, it is determined that the top edge or bottom edge of the guardrail plate is deformed and damaged and the position information is recorded. If not, it is determined that the top edge or bottom edge of the guardrail plate at this contour is normal, and go to Step 5;

[0028] Step 4: For any closed contour below the uppermost straight line segment or above the lowermost straight line segment, in the first region of interest, determine whether the color inside the contour is the same as the colors of the regions on the left and right sides outside the contour. If so, it is determined that the top edge or bottom edge of the guardrail plate at this contour is normal. If not, it is determined that the top edge or bottom edge of the guardrail plate is deformed and damaged and the position information is recorded;

[0029] Step 5: Repeat Steps 3-4 for the next closed contour until all closed contours are processed.

[0030] Furthermore, detect the deformation and damage conditions of the bottom wave surface, upper wave surface, and lower wave surface of the guardrail plate from the region of interest according to the guardrail plate type, including detecting the deformation and damage conditions inside the guardrail plate surface by the following method:

[0031] Step 1: For all straight line segments between the uppermost and lowermost straight line segments, determine whether all the straight line segments horizontally penetrate the image. If so, it is determined that there is no abnormality inside the guardrail plate surface and the process ends. If not, execute Step 2;

[0032] Step 2: For any straight line segment L k , of L kAfter the endpoint is connected to the nearby contour, check if there is a closed contour with a size larger than the set threshold; if so, it is determined that there is deformation damage inside the guardrail panel surface and the position information is recorded, then go to step 5, if not, execute step 3;

[0033] Step 3. In the first region of interest L k Extract texture information from the area near the endpoint;

[0034] Step 4. Determine whether there is a straight line segment in the texture that is connected to the endpoint L k If so, it is determined that there is no abnormality inside the guardrail panel surface at this straight line, if not, it is determined that there is deformation damage inside the guardrail panel surface and the position information is recorded;

[0035] Step 5. Repeat steps 2 to 4 for the next straight line segment until all straight line segments are processed.

[0036] Furthermore, detect the missing situation of column bolts and column caps according to the column type, including detecting the missing situation of square tube column bolts by the following method:

[0037] Step 1. Select the top of the area between the left and right vertical lines from the preprocessed area array image as the key area;

[0038] Step 2. Identify hexagonal contours in the key area that meet specific size requirements, and count the number of hexagonal contours, denoted as k1;

[0039] Step 3. Identify concentric circular contours and concentric elliptical contours in the key area that meet specific size requirements, and count the number of concentric circular contours and concentric elliptical contours. The sum of the two is denoted as k2;

[0040] Step 4. Determine whether k1 + k2 = 2 holds; if so, it is determined that the column bolts are not missing, if not, it is determined that the column bolts are missing and the position information is recorded.

[0041] Furthermore, detect the missing situation of column bolts and column caps according to the column type, including detecting the missing situation of round tube column bolts by the following method:

[0042] Step 1. If the area array image is of the guardrail column on the left side of the driving direction, take the left vertical line as the reference line; if not, take the right vertical line as the reference line;

[0043] Step 2. Check whether there is a closed contour connected to the left vertical line or the right vertical line in the area on the left side of the left vertical line or on the right side of the right vertical line; if so, execute step 3, if not, it is determined that the column bolts are missing and the position information is recorded;

[0044] Step 3: Check if there is a closed contour in the closed contours that simultaneously satisfies the following two conditions. The conditions include: the perimeter of the closed contour meets specific requirements; corresponding to the area array image, the color inside the closed contour is the same as the color in the area between the left vertical line and the right vertical line.

[0045] If so, it is determined that the column bolts are not missing; if not, it is determined that the column bolts are missing and the position information is recorded.

[0046] Furthermore, according to the column type, check for the missing column bolts and column caps, including checking for the missing column caps of circular tube columns by the following method: determine whether the pre-processed area array image simultaneously satisfies the following two conditions; there is no contour connected to the left vertical line or the right vertical line in the top area on the left side of the left vertical line or on the right side of the right vertical line; there are two concentric circular contours or two concentric elliptical contours in the key area, and the size ratio of the two concentric contours meets specific requirements; if so, it is determined that the column cap is missing and the position information is recorded, otherwise it is determined that the column cap is not missing.

[0047] In a second aspect, there is provided an apparent disease detection device for a corrugated beam steel guardrail, which is used for the apparent disease detection method of the corrugated beam steel guardrail described in the first aspect, including:

[0048] 3 line scan cameras, 1 area array camera, a rotary encoder, a satellite navigation system locator, an industrial computer, a camera mounting bracket, and a detection vehicle;

[0049] The line scan cameras and the area array camera are connected to the side and top of the detection vehicle body through the camera mounting bracket, and are used to collect images of the guardrail plates and columns;

[0050] The rotary encoder is installed at the center of the rear wheel hub of the detection vehicle and is used to sense the position change of the detection vehicle and trigger the line scan cameras and the area array camera to collect images;

[0051] The satellite navigation system locator is used to obtain the real-time position information of the detection vehicle, and the industrial computer is used to control image and position acquisition and the identification and analysis of the apparent diseases of the guardrail.

[0052] Furthermore, among the 3 line scan cameras, the installation height of the first line scan camera is the same as the center height of the roadside guardrail plate, and the scanning line is perpendicular to the road surface after installation, and is used to collect images of the guardrail plate and the column from the front view angle; the installation height of the second line scan camera is higher than the top of the guardrail plate, and the installation height of the third line scan camera is lower than the bottom of the guardrail plate. The projections of the scanning lines of the second line scan camera and the third line scan camera on the road surface are both perpendicular to the driving direction, and are respectively used to collect images of the guardrail plate from the top view and bottom view angles.

[0053] The installation height of the area array camera is higher than the top of the guardrail plate, and its projection on the ground is located outside the guardrail. After installation, the central optical axis is parallel to the driving direction and intersects the ground at an acute angle, and is used to collect images of the outer columns of the guardrail and their bolts.

[0054] As can be seen from the above technical solutions, the beneficial technical effects of the present invention are as follows:

[0055] 1. The detection method for the apparent diseases of the corrugated beam steel guardrail can automatically collect and analyze the deformation and damage of the corrugated beam steel guardrail plate surface and the missing bolts, and has high detection efficiency.

[0056] 2. The detection device for the apparent diseases of the corrugated beam steel guardrail uses vehicle-mounted detection, without the need to take traffic control measures. Compared with manual detection, it can greatly improve the efficiency and avoid the safety risks of traffic organization, and has considerable social benefits. Brief Description of the Drawings

[0057] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to actual scale.

[0058] Figure 1 It is a front view of the installation surface of the guardrail apparent disease detection device according to an embodiment of the present invention;

[0059] Figure 2 It is a side view of the installation surface of the detection device according to an embodiment of the present invention;

[0060] Figure 3 It is a schematic structural diagram of a double-wave guardrail according to an embodiment of the present invention;

[0061] Figure 4 It is a schematic structural diagram of a three-wave guardrail according to an embodiment of the present invention;

[0062] Figure 5 It is a schematic flow diagram of the guardrail apparent disease detection method according to an embodiment of the present invention;

[0063] Reference Numerals:

[0064] 1 - Line scan camera 1#; 2 - Line scan camera 2#; 3 - Line scan camera 3#; 4 - Area array camera; 5 - Camera mounting bracket; 6 - Rotary encoder; 7 - Beidou system locator; 8 - Industrial computer; 9 - Detection vehicle; 10 - Guardrail plate; 11 - Column; 12 - Anti-collision block; 13 - Plate surface bolt; 14 - Column bolt; 15 - Round pipe column cap; 16 - Wave bottom surface; 17 - Upper wave surface; 18 - Lower wave surface; 19 - Rectangular gasket. Detailed Embodiments

[0065] The embodiments of the technical solution of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solution of the present invention more clearly, so they are only examples and cannot be used to limit the protection scope of the present invention.

[0066] It should be noted that unless otherwise specified, the technical terms or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art to which the present invention belongs.

[0067] Embodiment

[0068] This embodiment provides a method and a detection device for detecting the apparent diseases of a corrugated beam steel guardrail. The detection device is specifically as follows:

[0069] As Figure 1 , Figure 2 shown, the detection device consists of 3 line-scan cameras, 1 area array camera, a rotary encoder, a satellite navigation system locator, an industrial computer, a camera mounting bracket, and a detection vehicle.

[0070] In some embodiments, both the line-scan cameras and the area array camera are full-color cameras, and are connected to the side and top of the detection vehicle body through the camera mounting bracket.

[0071] In a specific implementation manner, the installation height of the 1# line-scan camera is the same as the center height of the roadside guardrail plate (about 650 mm). After installation, the scanning line is perpendicular to the road surface and is used to collect images of the guardrail plate and the column from the front view angle; the installation height of the 2# line-scan camera is higher than the top of the guardrail plate, and the installation height of the 3# line-scan camera is lower than the bottom of the guardrail plate; after the 2# and 3# line-scan cameras are installed, the projections of the scanning lines on the road surface are both perpendicular to the driving direction, and are respectively used to collect images of the guardrail plate from the top view and bottom view angles.

[0072] In a specific implementation manner, the installation height of the area array camera is higher than the top of the guardrail plate and its projection on the ground is located outside the guardrail. After installation, the central optical axis is parallel to the driving direction and intersects the ground at an acute angle, and is used to collect images of the outer column of the guardrail and its bolts.

[0073] In some embodiments, the rotary encoder is installed at the center of the rear wheel hub of the detection vehicle and is used to sense the position change of the detection vehicle and trigger the line-scan cameras and the area array camera to collect images.

[0074] In some embodiments, the satellite navigation system locator selects a Beidou system locator and is installed inside the detection vehicle to obtain the real-time position information of the detection vehicle; the industrial computer is installed inside the detection vehicle and is used to control the image and position acquisition and the identification and analysis of the apparent diseases of the guardrail.

[0075] In some embodiments, the detection vehicle is a common 4-wheel car.

[0076] Combined with the above detection device, the apparent disease detection method for corrugated beam steel guardrails includes the following steps:

[0077] S1. Use the detection device based on image acquisition to obtain the line scan image and area array image data of the corrugated beam steel guardrail. After preprocessing the image data, analyze it to determine the guardrail plate type and column type.

[0078] Figure 3 , Figure 4 Figures 1 and 2 are structural schematic diagrams of two types of corrugated beam steel guardrails. It can be seen from the figures that the guardrail plates of the corrugated beam steel guardrail include double-wave guardrail plates or triple-wave guardrail plates, the columns include square pipe columns and round pipe columns, and the bolts include plate surface bolts and column bolts.

[0079] The specific implementation method of this step is as follows:

[0080] S11. Use the detection device based on image acquisition to obtain the line scan image and area array image of the corrugated beam steel guardrail.

[0081] Control the detection vehicle to drive so that the distance between the vehicle body detection device and the guardrail to be detected is always within a given range. The line scan camera and the area array camera continuously collect images respectively, and obtain the line scan images Img1, Img2, Img3 collected by the line scan camera and the area array image ImgM collected by the area array camera. During the collection process, the Beidou system locator continuously collects the position information of the detection vehicle.

[0082] S12. Preprocess the line scan image and the area array image to obtain the preprocessed line scan image and the preprocessed area array image.

[0083] Rotate the line scan images Img1, Img2, Img3 so that the length direction of the guardrail plate in the rotated line scan image is consistent with the horizontal direction, and the top edge of the guardrail plate is above the bottom edge, obtaining the rotated line scan image; then sequentially perform graying, edge detection, binarization, and dilation / erosion operations on the rotated line scan image respectively to obtain the preprocessed line scan images Img1 edge , Img2 edge , Img3 edge ;

[0084] Sequentially perform graying, edge detection, binarization, and dilation / erosion operations on the area array image ImgM to obtain the preprocessed area array image ImgM. edge .

[0085] S13. Determine the guardrail plate type according to the pixel distance between the uppermost and lowermost straight line segments in the preprocessed line scan image.

[0086] In the preprocessed line scan image Img1 edgeFind all line segments in [[]] with lengths greater than the set value, and denote the uppermost and lowermost line segments as L top and L bot . Determine whether the guardrail plate is a double-wave guardrail plate or a triple-wave guardrail plate based on the pixel spacing between L top and L bot in Img1 edge . In some embodiments, the set value is usually 0.5 times the width of the image pixels. For example, if the image pixel size is 3000 (width) × 1000 (height), then the set value here is 1500 pixels; for an image with a pixel size of 3000 (width) × 1000 (height), a pixel spacing less than or equal to 1800 pixels is a double-wave guardrail, and greater than 1800 pixels is a triple-wave guardrail.

[0087] S14. Determine the column type based on whether there are circular or elliptical contours in the area above the two vertical lines in the preprocessed area array image

[0088] Find two vertical lines L edge that are parallel to each other and whose lengths and spacings meet specific requirements in the preprocessed area array image ImgM left and L right . In some embodiments, an example of the specific requirements is given. For example, for an area array image of 4000 (width) × 3000 (height), the specific requirements are that the length is in the range of 500 to 1500 pixels and the spacing is in the range of 100 to 200 pixels. In the following text, L left is defined as the left vertical line, and L right is defined as the right vertical line.

[0089] Search for circular or elliptical contours with sizes that meet the requirements in the area above L left and L right ; if so, determine that the column is a circular tube column, and if not, determine that the column is a square tube column. In some embodiments, an example of the determination requirements for the column type is given: for an area array image of 4000 (width) × 3000 (height), a circle with a radius in the range of 100 to 200 pixels, or an ellipse with a major axis in the range of 100 to 200 pixels.

[0090] In a specific implementation manner, the execution order of steps S13 and S14 is not limited and can be in any order or synchronized.

[0091] S2. Separate the regions of interest of the wave bottom sub-image, the upper wave sub-image, and the lower wave sub-image from the line scan image and the preprocessed line scan image according to the guardrail plate type

[0092] In a specific implementation manner, the regions of interest ROI of the double-wave guardrail plate and the triple-wave guardrail plate are obtained respectively in the following manner:

[0093] (1) Double-wave guardrail panel

[0094] ROI bd = Img1(r top + 0.4h:r top + 0.6h;:)

[0095] ROI edge—bd = Img1 edge (r top + 0.4h:r top + 0.6h;:)

[0096] ROI sb = [Img2(r top + p 2s :r top + p 2s + 0.25h;:);Img2(r top + p 2s + 0.45h:r top + p 2s + 0.75h;:)]

[0097] ROI edge—sb = [Img2 edge (r top + p 2s :r top + p 2s + 0.25h;:);Img2 edge (r top + p 2s + 0.45h:r top + p 2s + 0.75h;:)]

[0098] ROI xb = [Img3(r top + p 2x :r top + p 2x + 0.3h;:);Img3(r top + p 2x + 0.5h:r top + p 2x + 0.75h;:)]

[0099] ROI edge—xb = [Img3 edge (r top + p 2x :r top + p 2x + 0.3h;:);Img3 edge (r top + p2x +0.5h:r top +p 2x +0.75h;:)]]

[0100] (2) Three-wave guardrail plate:

[0101] ROI bd =[Img1(r top +0.2h:r top +0.4h;:); Img1(r top +0.6h:r top +0.8h;:)]]

[0102] ROI edge—bd =[Img1 edge (r top +0.2h:r top +0.4h;:); Img1 edge (r top +0.6h:r top +0.8h;:)]]

[0103] ROI sb =[Img2(r top +p 3s :r top +p 3s +0.30h;:); Img2(r top +p 3s +0.55h:r top +p 3s +0.75h;:);

[0104] Img2(r top +p 3s +0.9h:r top +p 3s +h;:)]]

[0105] ROI edge—sb =[Img2 edge (r top +p 3s :r top +p 3s +0.30h;:); Img2 edge (r top +p 3s +0.55h:

[0106] r top +p 3s +0.75h;:); Img2 edge (r top +p 3s +0.9h:rtop +p 3s +h; :)]

[0107] ROI xb =[Img3(r top +p 3x :r top +p 3x +0.10h; :) ; Img3(r top +p 3x +0.25h:r top +p 3x +0.45h; :) ;

[0108] Img3(r top +p 3x +0.7h:r top +p 3x +h; :)]

[0109] ROIedge —xb =[Img3edge(r top +p 3x :r top +p 3x +0.10h; :) ; Img3edge(r top +p 3x +0.25h:

[0110] r top +p 3x +0.45h; :) ; Img3edge(r top +p 3x +0.7h:r top +p 3x +h; :)]

[0111] Wherein, the subscript bd corresponds to the bottom wavelet sub - figure, the subscript sb corresponds to the upper wavelet sub - figure, the subscript xb corresponds to the lower wavelet sub - figure, and the subscript edge represents the pre - processed image; h = r bot −r top , r top and r bot are respectively the row numbers of L edge in the Img1 top , L bot image; p 2s , p 2x , p 3s , p 3x are row number offset values, which are adjusted and determined according to the spatial relationship of the imaging fields of the 1#, 2#, and 3# line - scanning cameras.

[0112] When using the region of interest subsequently, the one without the subscript "edge" is classified as the first region of interest, such as ROI bd , and the one with the subscript "edge" is classified as the second region of interest, such as ROI edge—bd .

[0113] S3. Detect the missing bolts and deformation damage of the bottom wave surface, upper wave surface and lower wave surface of the guardrail plate from the region of interest

[0114] The surface color of the corrugated beam steel guardrail is usually uniformly pure silver, pure green or pure blue.

[0115] S31. Detect the missing bolts of the bottom wave surface, upper wave surface and lower wave surface of the guardrail plate

[0116] Follow the following steps to detect the missing bolts of the bottom wave surface of the guardrail plate from the region of interest ROI bd 、ROI edge—bd :

[0117] (1) Find all circular contours and elliptical contours in the second region of interest ROI edge—bd ;

[0118] (2) For any circular contour, check whether there is a rectangular contour of a specific size centered at the center of the circle and containing the circular contour; if so, execute step (3), if not, go to step (4); An example of the "specific size" in this step: for an image with a pixel size of 3000 (width) × 1000 (height), the "specific size" means that the long side of the rectangle is in the range of 100 - 150 pixels, and the ratio of the short side to the long side is in the range of 0.65 - 0.85.

[0119] (3) In the first region of interest ROI bd , judge whether the color inside the circular contour is the same as the color inside the rectangular contour outside the circular contour; if so, it is determined that the bolt at the circular contour is not missing, if not, it is determined that the bolt at the circular contour is missing and the position information is recorded; then go to step (5);

[0120] (4) Check whether there is a complete elliptical contour in the region where the circular contour is located, so that the circular contour is included in the elliptical contour; if so, it is determined that the bolt at the circular contour is missing and the position information is recorded, if not, it is determined that the bolt at the circular contour is not missing;

[0121] (5) Repeat steps (2) - (4) for the next circular contour until all circular contours are processed.

[0122] Use the same method as sub - steps (1) - (5) in step S31 to detect from the region of interest ROI sb 、ROIedge—sb Detect the absence of the upper-wave surface bolts on the guardrail plate, and the region of interest ROI from the lower-wave subfigure xb 、ROI edge—xb Detect the absence of the lower-wave surface bolts on the guardrail plate.

[0123] S32. Detect the deformation and damage conditions of the top and bottom edges of the guardrail plate

[0124] Follow the following steps to detect the deformation and damage conditions of the top edge of the guardrail plate from the region of interest ROI bd 、ROI edge—bd :

[0125] (1) In the second region of interest ROI edge—bd Judge whether L top or L bot horizontally penetrates the image. If so, it is determined that there is no abnormality in the top or bottom edge of the guardrail plate and the process ends. If not, go to step (2);

[0126] (2) Extend L top or L bot to make it horizontally penetrate the image, and connect L top or L bot with the remaining contours within a certain area above and below it to obtain several closed contours that meet the requirements; For example: for an image with a pixel size of 3000 (width) × 1000 (height), it is required that the perimeter of the closed contour is greater than 60 pixels. For the "certain area" in this step, for example: for an image with a pixel size of 3000 (width) × 1000 (height), the "certain area" refers to the range of 200 pixels in height above and below it.

[0127] (3) For any closed contour above L top or below L bot , judge whether the color inside the contour is the same as the color of the area below L bd or above L top in the first region of interest ROI bot ; If so, it is determined that the top or bottom edge of the guardrail plate is deformed and damaged and the position information is recorded. If not, it is determined that there is no abnormality in the top or bottom edge of the guardrail plate at this contour; Then go to step (5);

[0128] (4) For any closed contour below L top or above L bot , judge whether the color inside the contour is the same as the color of the areas on the left and right sides outside the contour in the first region of interest ROI bd ; If so, it is determined that there is no abnormality in the top or bottom edge of the guardrail plate at this contour. If not, it is determined that the top or bottom edge of the guardrail plate is deformed and damaged and the position information is recorded;

[0129] (5) Repeat steps (3) to (4) for the next closed contour until all closed contours are processed.

[0130] Using the same method as sub-steps (1)-(5) in S32, from ROI bd ROI edge—bd Detect deformation and damage of the bottom edge of the guardrail.

[0131] S33. Detect deformation and damage inside the guardrail panel

[0132] Follow these steps to extract the ROI bd ROI edge—bd Detect deformation and damage inside the guardrail panel

[0133] (1) For L top , L bot All straight line segments between them are judged whether they all penetrate the image horizontally; if so, it is determined that there is no abnormality inside the guardrail board surface and the process ends; if not, step (2) is executed;

[0134] (2) For L top , L bot Any straight line segment L between k , L k After the endpoints of the image are connected to the contours near it, find out whether there is a closed contour whose size is greater than the set threshold; if so, it is determined that there is deformation and damage inside the guardrail board surface and the position information is recorded, and then go to step (5); if not, go to step (3); for example, the "nearby" in this step is explained: for an image with a pixel size of 3000 (width) × 1000 (height), "nearby" refers to the interval of 200 pixels in each row and column. The threshold is determined according to the size of the inner surface of the guardrail board.

[0135] (3) ROI bd Medium k Extract texture information from the area near the endpoint; in a specific implementation, the method for extracting texture information is not limited, and can be implemented in any achievable manner in the prior art, such as a gray-level co-occurrence matrix. An example of "nearby" in this step is given: for an image with a pixel size of 3000 (width) × 1000 (height), the "nearby" rows and columns are within 100 pixels of each other.

[0136] (4) Determine whether there is any content in the texture that matches L k A straight line segment with connected endpoints; if yes, it is determined that there is no abnormality inside the guardrail board surface at the straight line; if no, it is determined that there is deformation and damage inside the guardrail board surface and the position information is recorded;

[0137] (5) Repeat steps (2) to (4) for the next straight line segment until all straight line segments are processed.

[0138] In a specific embodiment, the execution order of steps S32 and S33 is not limited, and they can be executed in any order or simultaneously.

[0139] S4. Detect the missing situation of column bolts and column caps according to the column type

[0140] S41. Detect the missing situation of square tube column bolts

[0141] (1) Select the vertical line L edge from the preprocessed area array image ImgM left The top of the area between L right is the key area;

[0142] (2) Identify the hexagonal contours in the key area whose dimensions meet specific requirements, and count the number of hexagonal contours, denoted as k1; In some embodiments, an example of the determination requirements is given: for a 4000 (width) × 3000 (height) pixel area array image, the specific requirements refer to the side length within the range of 30 to 60 pixels.

[0143] (3) Identify the concentric circular contours and concentric elliptical contours in the key area whose dimensions meet specific requirements, and count the number of concentric circular contours and concentric elliptical contours. The sum of the two is denoted as k2; In some embodiments, an example of the determination requirements is given: for a 4000 (width) × 3000 (height) pixel area array image, the outer circle pixel radius is between 15 and 30, and the ratio of the inner circle pixel radius to the outer circle pixel radius is within the range of 0.7 to 0.95.

[0144] The outer ellipse major axis pixel length is between 15 and 30, and the ratio of the inner ellipse major axis radius to the outer ellipse major axis radius is within the range of 0.7 to 0.95.

[0145] (4) Determine whether k1 + k2 = 2 holds; if so, it is determined that the column bolts are not missing, and if not, it is determined that the column bolts are missing and the position information is recorded.

[0146] S42. Detect the missing situation of round tube column bolts and column caps

[0147] (1) If the area array image ImgM captures an image of the left - hand side guardrail column in the driving direction, then take the vertical line L left as the reference line; otherwise, take the vertical line L right as the reference line;

[0148] (2) Search for whether there is a closed contour connected to L left on the left side of L right or on the right side of L left or L right ; if so, execute step (3), and if not, determine that the column bolts are missing and record the position information;

[0149] (3) Check whether there is a closed contour in the closed contour that satisfies the following two conditions at the same time. If so, it is determined that the column bolts are not missing; if not, it is determined that the column bolts are missing and the position information is recorded;

[0150] ① The size of the closed contour meets specific requirements; in some embodiments, an example of the determination requirements is given: for a planar array image of 4000 (width) × 3000 (height) pixels, the specific requirement means that the perimeter of the closed contour is in the range of 100 to 250.

[0151] ② Corresponding to ImgM, the color inside the closed contour is the same as the color in the area between L left and L right .

[0152] In some embodiments, for circular tube columns, it is also necessary to detect whether the column caps are missing. Specifically as follows: Determine whether the preprocessed planar array image ImgM edge simultaneously satisfies the following two conditions; if so, it is determined that the column cap is missing and the position information is recorded, otherwise it is determined that the column cap is not missing;

[0153] (1) There is no contour connected to L left on the left side of L right or on the right side of L left or L right in the top area;

[0154] (2) There are two concentric circular contours or two concentric elliptical contours in the key area, and the size ratio of the two concentric contours meets specific requirements. The key area here is defined as: select the vertical lines L edge and L left from the preprocessed planar array image ImgM right . The top area between them is the key area, and the specific requirements are determined according to the size of the column cap.

[0155] In a specific implementation manner, the execution order of steps S41 and S42 is not limited, and can be in any order, or synchronized.

[0156] By adopting the method for detecting the apparent diseases of the corrugated beam steel guardrail in this embodiment, it is possible to automatically collect and analyze the deformation and damage of the corrugated beam steel guardrail plate surface and the missing bolts, and the detection efficiency is high. The device for detecting the apparent diseases of the corrugated beam steel guardrail adopts vehicle-mounted detection, without the need to take traffic control measures, which can greatly improve the efficiency compared with manual detection and avoid the safety risks of traffic organization, and has considerable social benefits.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and they should all be covered within the scope of the claims and the description of the present invention.

Claims

1. A method for detecting apparent defects of corrugated steel guardrails, characterized in that: The following steps are involved: Obtain line scan image data and array image data of the corrugated steel guardrail, analyze the image data after preprocessing, and determine the type of guardrail plate and column; Separating the regions of interest of the wave bottom surface sub-image, the upper wave surface sub-image and the lower wave surface sub-image from the line scan image and the pre-processed line scan image; According to the type of guardrail, detect the bolt missing and deformation damage of the guardrail wave bottom surface, the wave surface on the guardrail and the wave surface under the guardrail from the area of ​​interest; Check for missing column bolts and column caps based on column type.

2. The method for detecting apparent defects of corrugated steel guardrail according to claim 1 is characterized in that: Obtain line scan image data and array image data of the corrugated steel guardrail, pre-process the image data and analyze it to determine the guardrail plate type and column type, including: Using an image acquisition-based detection device, the distance between the detection device and the guardrail being inspected is always within a given interval, and a line scan image and a surface array image of the corrugated steel guardrail are obtained; Preprocessing the line scan image and the area array image to obtain a preprocessed line scan image and a preprocessed area array image; The type of guardrail is determined according to the pixel spacing between the top and bottom straight line segments in the preprocessed line scan image; the type of column is determined according to whether there is a circular or elliptical outline in the area above the two vertical straight lines in the preprocessed array image.

3. The method for detecting apparent defects of corrugated steel guardrail according to claim 2 is characterized in that: Preprocessing the line scan image includes: rotating the line scan image so that the length direction of the guardrail in the rotated line scan image is consistent with the horizontal direction and the top edge of the guardrail is located above the bottom edge, thereby obtaining a rotated line scan image; performing grayscale conversion, edge detection, binarization, and dilation / erosion operations on the rotated line scan image in sequence; The array image is preprocessed, including: graying, edge detection, binarization, and dilation / erosion operations are performed on the array image in sequence.

4. The method for detecting apparent defects of corrugated steel guardrail according to claim 1 is characterized in that: According to the guardrail type, detect the bolt missing on the guardrail bottom surface, guardrail upper surface and guardrail lower surface in the area of ​​interest, including: Step 1, find all circular contours and elliptical contours in the second region of interest; Step 2: For any circular contour, find out whether there is a rectangular contour of a certain size centered on the center of the circle and containing the circular contour; if so, go to step 3; if not, go to step 4; Step 3, in the first region of interest, determine whether the color inside the circular outline is consistent with the color inside the rectangular outline outside the circular outline; if so, determine that the bolt at the circular outline is not missing; if not, determine that the bolt at the circular outline is missing and record the position information; then go to step 5; Step 4, find out whether there is a complete elliptical outline in the area where the circular outline is located, so that the circular outline is included in the elliptical outline; if so, it is determined that the bolt is missing at the circular outline and the position information is recorded; if not, it is determined that the bolt is not missing at the circular outline; Step 5: Repeat steps 2 to 4 for the next circular contour until all circular contours are processed.

5. The method for detecting apparent defects of corrugated steel guardrail according to claim 1 is characterized in that: According to the type of guardrail, the deformation and damage of the bottom surface of the guardrail, the upper wave surface of the guardrail and the lower wave surface of the guardrail are detected in the area of ​​interest, including the deformation and damage of the top and bottom edges of the guardrail by the following methods: Step 1: In the second region of interest, determine whether the topmost straight line segment or the bottommost straight line segment horizontally penetrates the image. If so, determine that there is no abnormality on the top edge or the bottom edge of the guardrail and end. If not, execute step 2. Step 2: Extend the topmost straight line segment or the bottommost straight line segment to make it run through the image horizontally, connect the topmost straight line segment or the bottommost straight line segment with other contours in a certain area above and below it, and obtain several closed contours with sizes that meet the requirements; Step 3: For any closed contour above the top straight line segment or below the bottom straight line segment, determine in the first region of interest whether the color inside the contour is consistent with the color of the area below the top straight line segment or above the bottom straight line segment outside the contour; if so, determine that the top edge or bottom edge of the guardrail is deformed and damaged and record the position information; if not, determine that there is no abnormality in the top edge or bottom edge of the guardrail at the contour, and go to step 5; Step 4: For any closed contour below the top straight line segment or above the bottom straight line segment, determine in the first region of interest whether the color inside the contour is consistent with the colors of the left and right areas outside the contour; if so, determine that there is no abnormality on the top or bottom edge of the guardrail at the contour; if not, determine that the top or bottom edge of the guardrail is deformed and damaged and record the position information; Step 5: Repeat steps 3 to 4 for the next closed contour until all closed contours are processed.

6. The method for detecting apparent defects of corrugated steel guardrail according to claim 1, characterized in that: According to the type of guardrail board, the deformation and damage of the guardrail board wave bottom surface, the guardrail board upper wave surface and the guardrail board lower wave surface are detected from the area of ​​interest, including the deformation and damage of the guardrail board surface inside by the following methods: Step 1: for all the straight line segments between the top and bottom straight line segments, determine whether all the straight line segments horizontally penetrate the image; if so, determine that there is no abnormality inside the guardrail board surface and end; if not, execute step 2; Step 2: For any straight line segment L between the top and bottom straight lines k , L k After the endpoints are connected to the contours nearby, find out whether there is a closed contour with a size greater than the set threshold; if so, determine that there is deformation and damage inside the guardrail board surface and record the position information, then go to step 5, if not, go to step 3; Step 3: In the first region of interest, L k Extract texture information from the area near the endpoint; Step 4: Determine whether there is any L in the texture k A straight line segment with connected endpoints; if yes, it is determined that there is no abnormality inside the guardrail board surface at the straight line; if no, it is determined that there is deformation and damage inside the guardrail board surface and the position information is recorded; Step 5: Repeat steps 2 to 4 for the next straight line segment until all straight line segments are processed.

7. The method for detecting apparent defects of corrugated steel guardrail according to claim 1 is characterized in that: Check the missing of column bolts and column caps according to the column type, including checking the missing of square tube column bolts in the following ways: Step 1: Select the top of the area between the left and right vertical lines from the pre-processed area array image as the key area; Step 2, identify the hexagonal outlines whose sizes meet specific requirements in the key area, and count the number of hexagonal outlines, recorded as k1; Step 3, identifying the concentric circular contours and concentric elliptical contours whose sizes meet specific requirements in the key area, and counting the number of concentric circular contours and concentric elliptical contours, and the sum of the two is recorded as k2; Step 4, determine whether k1+k2=2 is established; if so, determine that the column bolt is not missing, if not, determine that the column bolt is missing and record the position information.

8. The method for detecting apparent defects of corrugated steel guardrail according to claim 1 is characterized in that: Check the missing of column bolts and column caps according to the column type, including checking the missing of round tube column bolts in the following ways: Step 1: If the area array image captures an image of the guardrail column on the left side of the driving direction, the left vertical straight line is taken as the reference line; otherwise, the right vertical straight line is taken as the reference line; Step 2: Search in the area to the left of the left vertical line or to the right of the right vertical line whether there is a closed contour connected to the left vertical line or the right vertical line; if yes, execute step 3; if no, determine that the column bolt is missing and record the position information; Step 3: Find whether there is a closed contour in the closed contour that satisfies the following two conditions at the same time, wherein the conditions include: the perimeter of the closed contour meets specific requirements; and corresponding to the area array image, the color inside the closed contour is consistent with the color of the area between the left vertical straight line and the right vertical straight line; If yes, it is determined that the column bolt is not missing; if no, it is determined that the column bolt is missing and the position information is recorded.

9. The method for detecting apparent defects of corrugated steel guardrail according to claim 1, characterized in that: Check the missing of column bolts and column caps according to the column type, including checking the missing of column caps of round tube columns by the following methods: Determine whether the preprocessed array image satisfies the following two conditions at the same time; There is no contour connected to the left vertical straight line or the right vertical straight line in the top area on the left side of the left vertical straight line or on the right side of the right vertical straight line; There are two concentric circular contours or two concentric elliptical contours in the critical area, and the size ratio of the two concentric contours meets specific requirements; If so, it is determined that the column cap is missing and the position information is recorded; otherwise, it is determined that the column cap is not missing.

10. A device for detecting apparent defects of corrugated steel guardrails, characterized in that: The method for detecting apparent defects of a corrugated steel guardrail according to any one of claims 1 to 9 comprises: 3 line scan cameras, 1 area array camera, rotary encoder, satellite navigation system locator, industrial computer, camera mounting bracket and inspection vehicle; The line scan camera and the area array camera are connected to the side and top of the inspection vehicle body through a camera mounting bracket, and are used to collect images of guardrail panels and pillars; The rotary encoder is installed at the center of the rear wheel hub of the inspection vehicle and is used to sense the position change of the inspection vehicle and trigger the line scan camera and the area array camera to collect images; The satellite navigation system locator is used to obtain the real-time position information of the inspection vehicle, and the industrial computer is used to control image and position acquisition and guardrail surface disease recognition and analysis.

11. The device for detecting apparent defects of corrugated steel guardrail according to claim 10, characterized in that: Among the three line scan cameras, the installation height of the first line scan camera is consistent with the center height of the roadside guardrail board. After installation, the scanning line is perpendicular to the road surface and is used to collect images of the guardrail board and the column from a front view angle; the installation height of the second line scan camera is higher than the top of the guardrail board, and the installation height of the third line scan camera is lower than the bottom of the guardrail board. After the second and third line scan cameras are installed, the projections of the scanning lines on the road surface are perpendicular to the driving direction, and are used to collect images of the guardrail board from a top view and a bottom view, respectively; The installation height of the area array camera is higher than the top of the guardrail board and its projection on the ground is located outside the guardrail. After installation, the central optical axis is parallel to the driving direction and intersects the ground at an acute angle, so as to collect images of the outer columns and bolts of the guardrail.

Images