A detection method for the inclination fault of urban road signal poles

Abstract

The present invention discloses a detection method for the inclination fault of urban road signal poles. Using the YOLOv3 detection framework, it completes the output from the input of the original image to the position and category of the object. By training on manually annotated images, a detection model of YOLOv3 is obtained. This model is used to perform object detection on the signal poles and crosswalk lines of the traffic lights. The target object areas of the detected signal poles and crosswalk lines are subjected to gray-scale transformation to obtain a gray-scale image. Canny edge detection is performed on the gray-scale image, and regional connectivity is carried out through morphological processing. The Hough transform lines of the crosswalk lines and signal poles of the target objects are extracted, and the polar coordinate angles of the lines are calculated to obtain the included angle between the lines. It is judged whether the included angle between the crosswalk lines and the signal poles exceeds the reasonable threshold range, and based on this, it is judged whether there is an inclination phenomenon. The present invention can monitor the inclination status of the signal poles in real time, and give an alarm in time after the monitored inclination angle exceeds the threshold, and has the advantages of low cost and simple operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent transportation, and relates to the application of video monitoring and image processing technologies. Specifically, it relates to a method for detecting the inclination fault of urban road signal poles. Background Technique

[0002] In order to facilitate the passage of people and vehicles, traffic lights are installed at most intersections of urban roads and crosswalk lines are set, and the traffic lights are erected on lamp posts. In some intersections with a large spatial span, in order to facilitate drivers to observe the status of traffic lights, the lamp posts need to span multiple lanes at this time. Affected by external forces (strong winds, large vehicles, road construction, etc.), there is a possibility that the lamp post will tilt or even fall, so it is necessary to monitor whether the lamp post is tilted.

[0003] Currently, there are mainly three monitoring methods:

[0004] (1) Manual inspection, that is, directly going to the road intersection for inspection by personnel or through video monitoring for inspection. This method relies on the observation and experience of operators to make judgments, which is time-consuming and laborious, and cannot effectively control the actual effect. At the same time, problems cannot be discovered in time, which is likely to cause major traffic safety hazards;

[0005] (2) Analyzing and judging after collecting data through an angle sensor. In practice, an inclination sensor is installed to sense the inclination deviation angle, combined with a microprocessor and a communication module, and the inclination angle is detected by the inclination sensor. When the inclination value exceeds the angle threshold, an alarm is generated. However, this method has the following problems:

[0006] 1. The construction is complex and physical devices need to be installed on the signal lamp post;

[0007] 2. The detection effect is poor. The inclination sensor is prone to shift on the lamp post, affecting the detection effect;

[0008] 3. The detection range is narrow. When the lamp post is long, only the vicinity of the installation position can be detected, and the remote position cannot be detected;

[0009] (3) Automatic image analysis and judgment. Currently, the mainstream solution in the industry is to adopt the third image analysis and detection method. By collecting pictures of electronic police or monitoring video images, analyzing and identifying the offset of the position of the signal light panel to detect the inclination problem of the lamp post. First, record the position of the signal light panel when it is normal, regularly detect the current position of the signal light panel, and then calculate the offset pixels according to the center coordinates. When the offset threshold is exceeded, an alarm is generated. However, this method also has the following problems:

[0010] 1. Poor anti-interference ability. When the image acquisition device shakes or the image acquisition device is offset, the position of the signal light panel cannot be accurately located, affecting the detection;

[0011] 2. Complex operation. Detection requires prior calibration and recording of normal position values, and then real-time comparison of the detected values with the calibrated normal values.

[0012] 3. Poor adaptability. The detection methods for the inclination of the horizontal and vertical poles of the signal lights are not universal. Summary of the Invention

[0013] In view of the problems existing in the above-mentioned existing detection technical methods for the inclination of signal light poles, the present invention proposes a detection method for the inclination fault of urban road signal light poles. Through the real-time images or pictures of the cameras at signal light intersections, target detection of signal light poles and crosswalk lines is carried out. Hough lines are taken for the detected targets, and the numerical calculation of the line included angles is performed. When the detected angle value changes beyond the alarm threshold, an inclination alarm is generated, achieving the effect of real-time monitoring, improving work efficiency, timely discovering problems, so as to straighten the signal light poles and avoid affecting the safe travel of traffic and pedestrians.

[0014] To achieve the above object, the technical solution proposed by the present invention is a detection method for the inclination fault of urban road signal light poles, specifically including the following steps:

[0015] (1) Obtain the captured sample images through the intelligent transportation violation monitoring and management system (i.e., electronic police) set at urban road intersections. The images include signal light poles and crosswalk lines.

[0016] (2) Traverse each of the above sample images, perform smoothing processing on the images, mark the signal light poles and crosswalk lines with rectangular frames to obtain the corresponding label files. The label files and sample pictures form an image dataset, and the image dataset is divided into a training set and a validation set according to a certain ratio.

[0017] (3) Scale the sample images in the image dataset, uniformly adjust the image size, and scale the corresponding label files of the images in the same proportion. Then, use the target detection algorithm (YOLOv3) model to train the data in the training set, and use the validation set to verify the model generated by the training to obtain the final target detection model.

[0018] (4) Scale the real-time collected images according to the same image smoothing processing method, image size, and scaling method as in step 3, input them into the trained target detection algorithm model to output the position coordinates of the signal light poles and crosswalk lines, and obtain the image regions of the signal light poles and crosswalk lines from the original detected images input according to the coordinates.

[0019] (5) Perform gray-scale transformation, edge detection, and morphological processing on the smoothed images to obtain clear edges of the target objects of the signal light poles and crosswalk lines.

[0020] (6) Perform a line detection on the target object area using the Hough transform to obtain line A of the signal lamp pole and line B in the middle area of the crosswalk line;

[0021] (7) Calculate the polar coordinate angles of line A and line B. If the calculated included angle value exceeds the reasonable angle threshold, it is determined as suspected tilt, and the tilt anomaly count is incremented;

[0022] (8) Repeat the above steps for multiple frames of images to calculate the included angle and determine the tilt result. When the tilt anomaly count exceeds the count threshold, a tilt alarm is reported.

[0023] Preferably, in step 2, the image data set is divided into a training set and a validation set according to a ratio of approximately 4:1.

[0024] Preferably, in step 3, the image size is uniformly adjusted to 416×416.

[0025] Preferably, in step 3, the image scaling uses bilinear interpolation, which performs linear interpolation once in each of the two directions and then obtains the pixel to be calculated through interpolation of four adjacent pixels.

[0026] Preferably, in step 2, the image smoothing process uses a Gaussian filter to smooth the image and filter out noise.

[0027] In step 3, the object detection algorithm uses the YOLOv3 detection framework, and the network structure of the YOLOv3 algorithm is modified so that the modified YOLOv3 algorithm network structure only performs two types of detections to obtain outputs of three scales.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1. It can monitor the tilt condition of the signal lamp pole in real time. After the monitored tilt angle exceeds the threshold, an alarm is given in time to remind the operation and maintenance personnel to solve it in time. Using the original traffic camera, there is no need to calibrate the position of the signal lamp panel, and no external device needs to be installed, with low cost and simple operation;

[0030] 2. It has strong anti-interference ability, and the detection effect is not affected by external force shaking or the offset of the image acquisition device;

[0031] 3. Quantify the tilt angle value to facilitate manual review of the image recognition result;

[0032] 4. It has strong adaptability and is suitable for tilt detection of signal lamp crossbars and vertical bars. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic diagram of a typical signal lamp intersection;

[0034] Figure 2 is the flowchart of the detection scheme of the present invention;

[0035] Figure 3 This is the flowchart of the object detection of the present invention;

[0036] Figure 4 This is the flowchart of the embodiment of the present invention. Specific embodiments

[0037] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0038] Figure 1 Shown is a spatial schematic diagram of a typical traffic signal intersection. In image object detection, deep learning-based methods have been proven to be superior to traditional detection methods. As the best implementation mode, the object detection algorithm of the present invention adopts the YOLOv3 detection framework to complete the output from the input of the original image to the object position and category. By training on manually annotated images, the detection model of YOLOv3 is obtained. The YOLOv3 detection model is used to detect the traffic signal pole and the crosswalk line. Then, the gray-scale transformation is performed on the detected traffic signal pole and crosswalk line object regions to obtain a gray-scale image. The canny edge detection is performed on the gray-scale image, and then the region connectivity is performed through morphological processing. The Hough transform lines of the crosswalk line and the traffic signal pole are extracted. The polar coordinate angle of the line is calculated to obtain the line included angle. It is judged whether the included angle between the crosswalk line and the traffic signal pole exceeds the reasonable threshold range, and it is judged whether there is an inclination phenomenon according to the result.

[0039] The detection process is as shown in Figure 3 and Figure 4 shown, and specifically includes the following steps:

[0040] (1) Obtain the captured sample images through the existing electronic police at the urban road intersection. The images include traffic signal poles and crosswalk lines;

[0041] (2) Traverse each sample image, perform smoothing processing on the image, mark the traffic signal pole and the crosswalk line with a rectangular box to obtain the corresponding label file. The label file and the sample image constitute an image data set. The image data set is divided into a training set and a validation set according to a ratio of approximately 4:1;

[0042] (3) Scale the sample images in the image data set. The image size is uniformly adjusted to 416×416. The corresponding label file of the image is scaled in the same proportion. Then, the training set data is trained with the YOLOv3 model, and the generated model is verified with the validation set to obtain the final object detection model;

[0043] (4) Scale the images collected in real time according to the same image smoothing method, image size, and scaling method as in step (3), input them into the trained YOLOv3 object detection model to output the position coordinates of the signal lamp poles and crosswalk lines, and obtain the image regions of the signal lamp poles and crosswalk lines from the original input detection images according to the coordinates, see Figure 3 ;

[0044] (5) Perform grayscale transformation, edge detection, and morphological processing on the smoothed images to obtain clear edges of the signal lamp poles and crosswalk line objects;

[0045] (6) Use the Hough transform to detect lines in the object regions to obtain line A of the signal lamp pole and line B in the middle region of the crosswalk line;

[0046] (7) Calculate the polar coordinate angles of line A and line B. If the calculated included angle value exceeds the reasonable angle threshold, it is determined as suspected tilt, and the tilt anomaly count is incremented;

[0047] (8) Repeat the above steps for multiple frames of images to calculate the included angle and determine the tilt result. When the tilt anomaly count exceeds the count threshold, report a tilt alarm, see Figure 4 .

[0048] Now, the individual algorithms involved in the above detections are described.

[0049] 1. In the technical solution, in steps (3) and (4), bilinear interpolation is used for image scaling. Linear interpolation is performed separately in two directions, and the pixel to be obtained is interpolated through four adjacent pixels. Given that Q11, Q12, Q21, and Q22 are the four neighboring pixels in the original image, and P is the pixel to be obtained, the steps of bilinear interpolation are as follows:

[0050] (1) Obtain R2 by linearly interpolating Q12 and Q22, and obtain R1 by linearly interpolating Q11 and Q21;

[0051]

[0052]

[0053] Among them, x1 represents the horizontal coordinate value of Q11, x2 represents the horizontal coordinate value of Q21, x represents the horizontal coordinate value of the interpolation point P, f(Q11) represents the pixel value of pixel point Q11, f(Q21) represents the pixel value of pixel point Q21, f(Q12) represents the pixel value of pixel point Q12, f(Q22) represents the pixel value of pixel point Q22, f(R1) represents the pixel value of point R1, and f(R2) represents the pixel value of point R2;

[0054] (2) Obtain P by linearly interpolating R1 and R2.

[0055]

[0056] Among them, y1 represents the longitudinal coordinate value of Q11, y2 represents the longitudinal coordinate value of Q21, and y represents the longitudinal coordinate value of the interpolation point P;

[0057] The final result f(P) is as follows:

[0058]

[0059] 2. In the technical solution, in steps (3) and (4), a Gaussian filter is used to smooth the image and filter out noise. Gaussian smoothing uses a Gaussian filter to convolve with the image to reduce the obvious noise effect. The generation equation of the Gaussian filter kernel with a size of (2k + 1) x (2k + 1) is as follows:

[0060]

[0061] 3. In the technical solution, the object detection in step (3) adopts the YOLOv3 detection framework. YOLOv3 improves the detection speed, reduces the background misdetection rate, and meets the requirements of general object detection by adjusting the network structure, using multi-scale features, and replacing softmax with Logistic for object classification. The original YOLOv3 algorithm obtains the detection results at three scales of 13×13×75, 26×26×75, and 52×52×75 through downsampling; among them, 13, 26, and 52 represent the three scales of downsampling; 75 is split into 3×(4 + 1 + 20), where 3 represents that there are 3 detection boxes at each scale, 4 represents the offset information of each detection box, 1 represents the recognition rate of each class of detection, and 20 represents detecting 20 classes of targets; the network structure of the YOLOv3 algorithm is modified so that the modified YOLOv3 algorithm network structure only performs two-class detection, obtaining outputs at three scales of 13×13×21, 26×26×21, and 52×52×21.

[0062] 4. In the technical solution, the gray-scale transformation of the image in step (5) is also called the point operation of the image (for a certain pixel point of the image) and is a basic technique in all image processing technologies. Its transformation form is as follows:

[0063] s = T(r)

[0064] Among them, T is the gray-scale transformation function; r is the gray scale before transformation; s is the pixel after transformation.

[0065] 5. In the technical solution, the edge detection of the image in step (5) adopts the Canny edge detection. The Canny edge detection algorithm is a multi-level edge detection algorithm. The algorithm implementation steps:

[0066] (1) Gaussian smooth the input image to reduce the error rate.

[0067] (2) Calculate the gradient magnitude and direction to estimate the edge intensity and direction at each point.

[0068] The Canny algorithm uses four gradient operators to detect horizontal, vertical, and diagonal edges in the image. The edge detection operator returns the first derivative values in the horizontal Gx and vertical Gy directions, from which the gradient G and direction θ of the pixel can be determined.

[0069]

[0070] θ = arctan(G y / G x )

[0071] where G is the gradient intensity, θ represents the gradient direction, and arctan is the arctangent function.

[0072] (3) According to the gradient direction, perform non-maximum suppression on the gradient magnitude.

[0073] Non-maximum suppression is an edge thinning technique. Non-maximum suppression can suppress all gradient values outside the local maximum to 0. The algorithm for performing non-maximum suppression on each pixel in the gradient image is as follows:

[0074] (a) Compare the gradient intensity of the current pixel with two pixels along the positive and negative gradient directions.

[0075] (b) If the gradient intensity of the current pixel is the largest compared to the other two pixels, then this pixel is retained as an edge point,

[0076] otherwise this pixel will be suppressed.

[0077] (4) Use double-threshold processing and connect the edges.

[0078] 6. In the technical solution, the image morphological processing included in step (5) is one of the most widely used techniques in image processing, mainly used to extract image components that are meaningful for expressing and depicting the shape of regions, such as boundaries and connected regions, etc. The basic morphological operations of binary images include erosion, dilation, opening, and closing operations.

[0079] Erosion is an operation to find the local minimum, which eliminates the boundary points of the object. The expression of erosion:

[0080] dst(x,y) = min{src(Sx,Sy)}

[0081] Dilation is an operation to find the local maximum, which eliminates small black holes in the highlighted area and smooths the edges of the highlighted area. The expression of dilation:

[0082] dst(x,y) = max{src(Sx,Sy)}, src(Sx,Sy) != 0

[0083] The opening operation is a process of erosion followed by dilation, removing isolated small dots and burrs. Its expression is as follows:

[0084] dist = open(src, element) = dilate(erode(src, element))

[0085] The closing operation is a process of dilation followed by erosion, filling small holes and bridging small cracks. Its expression is as follows:

[0086] dist = close(src, element) = erode(dilate(src, element))

[0087] 7. In the technical solution, the Hough transform in step (6) is one of the basic methods for identifying geometric shapes from an image in image processing. It is mainly used to separate geometric shapes with certain identical features from the image. The basic Hough transform is to detect straight lines (line segments) from a black-and-white image.

[0088] When performing Hough line detection, for a certain point (x0, y0) in the rectangular coordinate space of the image, all the line parameters passing through this point must satisfy:

[0089] Y0 = mX0 + b

[0090] where m is the slope and b is the intercept.

[0091] When the edge of the image is a straight line perpendicular to the x-axis in the rectangular coordinate space, since the slope of the straight line is positive infinity and cannot be represented by m in the rectangular Hough space, we introduce the polar coordinate Hough space. A straight line can be represented in the polar coordinate system as:

[0092] ρ = xcosθ + ysinθ

[0093] where ρ is the distance from the origin to the straight line.

[0094] 8. In the technical solution, step (6) polar coordinates mean taking a fixed point O in the plane, called the pole, drawing a ray OX, called the polar axis, and then selecting a unit of length and the positive direction of the angle (usually the counterclockwise direction). For any point M in the plane, use ρ to represent the length of the line segment OM, θ to represent the angle from OX to OM. ρ is called the polar radius of point M, θ is called the polar angle of point M, and the ordered pair (ρ, θ) is called the polar coordinates of point M. Usually, the unit of the polar radius coordinate of M is 1 (unit of length), and the unit of the polar angle coordinate is rad.

[0095] Figure 4 It is a flowchart of an optimal embodiment of the present invention, including the following steps:

[0096] 1. Collect a frame of image from the camera, convert it into RGB format, and perform smoothing processing on the image to reduce noise;

[0097] 2. Scale the processed image to a size of 416×416, input it into the trained YOLOv3 object detection model, output the position coordinates of the signal light pole and the crosswalk line, and obtain the image regions of the signal light pole and the crosswalk line from the original image according to the coordinate and the proportional relationship between the scaled size and the original size;

[0098] 3. Perform gray-scale transformation, canny edge detection and morphological processing on the image after smoothing in step 1 to complete region connectivity, and combine the object regions of the signal light pole and the crosswalk line obtained in step 2 to obtain clear edges of the two objects;

[0099] 4. Use the Hough transform to detect straight lines in the object region, and obtain straight line A of the signal light pole; the crosswalk line is a group of parallel straight lines, and select straight line B at the center position of the crosswalk line, as Figure 1 shown;

[0100] 5. Calculate the polar coordinate angles of straight line A and straight line B. If the calculated included angle value exceeds the reasonable angle threshold, it is determined as suspected tilt, and the tilt anomaly count is incremented; otherwise, the tilt anomaly count is cleared;

[0101] 6. Repeat steps 1-5 for multiple frames of images, calculate the included angle and determine the tilt result. When the tilt anomaly count exceeds the count threshold, report a tilt alarm;

[0102] The method for detecting the tilt of the signal light pole proposed by the present invention has many advantages. It does not depend on the signal light disc, is not affected by the offset and shaking of the acquisition device, and has strong anti-interference ability; the images used for detection are directly obtained from the original traffic cameras, and there is no need to deploy separate image acquisition devices; the detection configuration is simple, and there is no need to separately calibrate the position of the signal light.

[0103] It should be noted that the above embodiments provided by the present invention are only illustrative and do not have the effect of limiting the specific implementation scope of the present invention. The protection scope of the present invention should include those transformations or alternative solutions that are obvious to those of ordinary skill in the art.

Claims

1. A detection method for the inclination fault of urban road signal poles, characterized in that, Specifically, it includes the following steps: (1) Obtain the captured sample images through the intelligent traffic violation monitoring and management system set at the urban road intersection. The images include signal lamp poles and crosswalk lines; (2) Traverse the above sample images, perform smoothing processing on the images, mark the signal lamp poles and crosswalk lines with rectangular frames to obtain the corresponding label files. The label files and sample pictures constitute the image dataset, and the image dataset is divided into a training set and a validation set according to a certain ratio; (3) Scale the sample images in the image dataset, uniformly adjust the image size, and scale the corresponding label files of the images in the same proportion. Then, use the object detection algorithm model to train the data in the training set, and use the validation set to verify the model generated by the training to obtain the final object detection model. Modify the YOLOv3 algorithm network structure so that the modified YOLOv3 algorithm network structure only performs two types of detections, and obtains outputs of three scales: 13×13×21, 26×26×21, and 52×52×21; (4) Scale the images collected in real time according to the same image smoothing processing method in step 2 and the image size scaling method in step 3, input them into the trained object detection algorithm model to output the position coordinates of the signal lamp poles and crosswalk lines, and obtain the image regions of the signal lamp poles and crosswalk lines from the original detection images input according to the coordinates; (5) Perform gray-scale transformation, edge detection, and morphological processing on the smoothed images to obtain clear edges of the signal lamp poles and crosswalk line objects; (6) Use the Hough transform to detect straight lines in the object region to obtain straight line A of the signal lamp pole and straight line B in the middle region of the crosswalk line; (7) Calculate the polar coordinate angles of straight line A and straight line B. If the calculated included angle value exceeds the reasonable angle threshold, it is determined as suspected tilt, and the tilt anomaly count is incremented; (8) Repeat the above steps for multiple frames of images to calculate the included angle and determine the tilt result. When the tilt anomaly count exceeds the count threshold, report a tilt alarm.

2. The detection method for the inclination fault of an urban road signal pole according to claim 1, characterized in that, In step 2, the image dataset is divided into a training set and a validation set according to a ratio of 4:

1.

3. The detection method for the inclination fault of an urban road signal pole according to claim 1, wherein, In step 3, the image size is uniformly adjusted to 416×416.

4. The detection method for the inclination fault of an urban road signal lamp pole according to claim 1, characterized in that, In step 3, the image scaling adopts bilinear interpolation, which performs linear interpolation once in each of the two directions, and then obtains the pixel to be calculated through interpolation of four adjacent pixels.

5. The detection method for the inclination fault of an urban road signal pole according to claim 1, wherein, The image smoothing processing uses a Gaussian filter to smooth the image and filter out noise.

Images