A method for estimating the angle of a boarding bridge by adaptive Hough transform combined with RANSAC algorithm

Abstract

The application discloses a boarding bridge angle estimation method combining adaptive Hough transformation and RANSAC algorithm, and comprises the following steps: step 1) a boarding bridge detection model is constructed by using YOLOv5, a to-be-detected image is detected, and length and width data of the boarding bridge are obtained; step 2) a detected local image of the boarding bridge is cut, and edge detection is performed by using a Canny operator; step 3) according to the length and width data of the boarding bridge, a dynamic straight line length threshold is set, an adaptive Hough transformation model is constructed, and straight line detection is performed on the boarding bridge edge image; and step 4) according to the straight line inspection result of the boarding bridge image, a RANSAC algorithm is used to estimate the included angle between the boarding bridge and the horizontal direction. The application can effectively adaptively adjust the threshold value, accurately detect the straight line, accurately estimate the angle between the boarding bridge and the horizontal direction in a complex edge noise environment, and significantly enhance the accuracy and robustness of the boarding angle detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer image graphics, and particularly relates to a boarding bridge angle estimation method combining adaptive Hough transformation and RANSAC algorithm. TECHNICAL BACKGROUND

[0002] The boarding bridge is a pedestrian passage connecting the berthed ship and the wharf, and facilitates the tourists to safely get on the ship from the ship. If the angle between the boarding bridge and the horizontal direction exceeds 30 degrees, there will be a risk of falling of the tourists. Therefore, automatically detecting the angle between the boarding bridge and the horizontal direction can guarantee the safety of the pedestrians getting on and off the bridge. The accuracy and robustness of the boarding bridge angle estimation are affected by the following reasons based on the image detection method. (1) The distance between the boarding bridge and the camera is different, so that the sizes of the detected images are different, resulting in the uncertainty of the minimum length required to be set for the straight line detection. (2) The common boarding bridge is wrapped with a protective net, resulting in a large amount of noise in the edge and straight line detection. SUMMARY

[0003] The application aims at the above problems, and provides a boarding bridge angle estimation method combining adaptive Hough transformation and RANSAC algorithm. In view of the deficiencies of the existing method, the dynamic straight line length threshold is set by using the proportional relationship of the length of the boarding bridge, the adaptive Hough transformation model is constructed, and the number and quality of the straight line detection are guaranteed. The angle of the boarding bridge is accurately estimated in the noise of the protective net by using the RANSAC algorithm, and the accuracy and robustness of the angle estimation are enhanced.

[0004] TECHNICAL SCHEME A boarding bridge angle estimation method combining adaptive Hough transformation and RANSAC algorithm comprises the following steps:

[0005] Step 1) A boarding bridge detection model is constructed by using YOLOv5, the length and width data of the boarding bridge are obtained by detecting the to-be-detected image;

[0006] Step 2) The detected local image of the boarding bridge is cut, and the edge detection is performed by using a Canny operator;

[0007] Step 3) The dynamic straight line length threshold is set according to the length and width data of the boarding bridge, the adaptive Hough transformation model is constructed, and the straight line detection is performed for the boarding bridge edge image;

[0008] Step 4) The angle between the boarding bridge and the horizontal direction is estimated by using the RANSAC algorithm for the straight line inspection result of the boarding bridge image.

[0009] Further, the specific content of step 1) is as follows:

[0010] 1-1 The boarding bridge image under the test side view is collected by using the wharf camera, and the boarding bridge detection data set is prepared;

[0011] 1-2 Train the YOLOv5 network using the ship boarding bridge detection dataset to obtain a ship boarding bridge detection model;

[0012] 1-3 Use the trained ship boarding bridge detection model to face the image to be detected, and detect the ship boarding bridge body; and obtain the length L and width W data of the detection frame.

[0013] Further, the specific steps of step 2) are as follows:

[0014] 2-1 Use a Gaussian smoothing filter to convolve and denoise;

[0015] 2-2 Use different gradient operators to calculate the gradient amplitude and direction of the smoothed image; apply non-maximum suppression to the gradient amplitude to find the local maximum value points in the image gradient, and set other non-local maximum value points to zero;

[0016] 2-3 Use double-threshold detection and connected edges: a high threshold is used to find each line segment, and a low threshold is used to determine the points on the line segment;

[0017] Further, the specific content of step 3) is as follows:

[0018] 3-1 Face the edge binary image I edge of the ship boarding bridge local image, establish a parameter space accumulation array P[ρ,θ] and initialize it to zero; set the straight line parameter set S L for detection and empty set.

[0019] 3-2 Traverse each edge pixel point (x,y) in the edge binary image I edge , calculate its corresponding straight line polar coordinates (ρ,θ) to satisfy the following formulas (1), (2), (3):

[0020]

[0021]

[0022] ρ=xcosθ+ysinθ (3)

[0023] 3-3 For each straight line segment, use the parameter space accumulation array P[ρ,θ] to measure the length of the straight line, as shown in formula (4),

[0024] P[ρ,θ]=P[ρ,θ]+1 (4)

[0025] 3-4 According to the length L value of the ship boarding bridge detection frame, set an adaptive straight line length threshold, as shown in formula (5)

[0026]

[0027] Tρ is a straight line length threshold value; a is a proportionality coefficient belonging to [0, 1], and the present application takes 0.4, that is, the straight line length threshold value is adaptively set as a times of the boarding bridge.

[0028] 3-4 Keep all straight lines with length greater than T in the parameter space accumulated array P[ρ, θ] and remove the rest of the short straight line noise. Obtain the detected straight line set S of the boarding bridge image. ρ L .

[0029] (ρ, θ) ∈ S L , if P[ρ, θ] > T ρ (6)

[0030] 3-5 Calculate whether the number of detected straight lines exceeds the predetermined straight line number threshold value T N : if the number of detected straight lines is greater than or equal to T N , store the straight line parameter set S L and the corresponding accumulated data P[ρ, θ]; if the number of detected straight lines is less than T N , correct the straight line length threshold value T ρ in the following formula and return to step 3-4) to reselect straight lines with length greater than the corrected straight line length threshold value T ρ .

[0031] T ρ = 0.9 × T ρ (7).

[0032] Further, the specific content of step 4) is as follows:

[0033] 4-1 Establish a rectangular coordinate system for the set of all edge straight line segments {(x i , y i ), i ∈ N} detected and obtained in step 3). Randomly select two points (x1, y1) and (x2, y2) in the set.

[0034] 4-2 Determine the straight line equation using (x1, y1) and (x2, y2).

[0035]

[0036] 4-3 Calculate the distance of the remaining points to the straight line, and then calculate the number of points with distance less than the threshold value t as the inliers according to the set threshold value, and count the number of inliers.

[0037] 4-4 Set the maximum number of iterations, repeat the above iteration, record the number of inliers obtained after drawing each iteration of selected points, and determine the straight line model corresponding to the maximum number of inliers as the optimal straight line model. The slope k * ​The corresponding angle is the estimated value of the angle between the boarding bridge and the horizontal direction;

[0038] θ=actank * (9).

[0039] Beneficial effects: The method of this invention uses a dynamic straight-line length threshold set by the proportion of the boarding bridge detection length to construct an adaptive Hough transform model, thereby eliminating the influence of the size of the boarding bridge detection results on angle estimation. The RANSAC algorithm is used to estimate angle values ​​in noisy data, reducing the impact of straight-line noise generated by the protective netting on angle estimation. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the steps of an adaptive Hough transform combined with RANSAC algorithm for estimating the boarding bridge angle according to the present invention.

[0041] Figure 2 This is a schematic diagram of the edge detection results of the boarding bridge according to the present invention. Detailed Implementation

[0042] The technical method of the present invention will now be described in further detail with reference to the accompanying drawings.

[0043] like Figure 1 As shown, an adaptive Hough transform combined with the RANSAC algorithm for estimating the boarding bridge angle includes the following steps:

[0044] Step 1) Use the dock camera to collect images of the boarding bridge from the test side view and create a boarding bridge detection dataset; use the boarding bridge detection dataset to train a YOLOv5 network and obtain a boarding bridge detection model; use the trained boarding bridge detection model to detect the boarding bridge body in the image to be tested; and obtain the length L and width W data of the detection frame.

[0045] Step 2) Use a Gaussian smoothing filter for convolutional noise reduction; employ different gradient operators to calculate the gradient magnitude and direction of the smoothed image; apply non-maximum suppression to the gradient magnitude to find local maxima in the image gradient, and set other non-local maxima to zero; use dual thresholding for edge detection and connection: a high threshold is used to find each line segment, and a low threshold is used to determine points on the line segment; obtain as shown... Figure 2 The edge image shown.

[0046] Step 3), 3-1 Binary edge image I of the local image facing the boarding bridge edge Create a parameter space accumulation array P[ρ,θ] and initialize it to zero; set the set of line parameters S for detection. L , and set the empty set.

[0047] 3-2 Traversing the Binary Image of Edges Iedge , each edge pixel point (x, y) calculates its corresponding straight line polar coordinates (p, q), which satisfies the following formula (1), (2), (3):

[0048]

[0049]

[0050] p = x cos q + y sin q (3)

[0051] 3-3 For each straight line segment, use the parameter space accumulation array P[p, q] to measure the length of the straight line, as shown in formula (4),

[0052] PP[p, q] = P[p, q] + 1 (4)

[0053] 3-4 According to the length L value of the boarding bridge detection frame, set the adaptive straight line length threshold, as shown in formula (5)

[0054] T ρ = aL (5)

[0055] In the formula, T ρ is the straight line length threshold; a is the proportional coefficient belonging to [0, 1], and the present application takes 0.4, that is, the straight line length threshold is adaptively set as a times of the boarding bridge.

[0056] 3-4 Keep the straight lines with length greater than T ρ in all straight line parameter space accumulation arrays P[p, q], and remove the rest of the short straight line noise to obtain the intersection point set in the parameter space. Further, obtain the detection straight line set S L of the boarding bridge image.

[0057] (p, q) e S L , if P[p, q] > T ρ (6)

[0058] 3-5 Calculate whether the number of detection straight lines exceeds the predetermined straight line number threshold T N : if the number of detection straight lines is greater than or equal to T N , store the straight line parameter set S L and the corresponding accumulation data P[p, q]; if the number of detection straight lines is less than T N , correct the straight line length threshold T ρ according to the following formula, and return to step 3-4) to reselect the straight line with length greater than the corrected straight line length threshold T ρ ;

[0059] T ρ = 0.9 x T ρ (7).

[0060] Step 4) Target the set of all edge line segments obtained in Step 3) {(x i ,y i Let N ∈ N, and establish a rectangular coordinate system. Randomly select two points (x1, y1) and (x2, y2) from the set. Use (x1, y1) and (x2, y2) to determine the equation of the line.

[0061]

[0062] Calculate the distance from the remaining points to the line, then calculate the number of points whose distance is less than the threshold based on the set threshold t, and count the number of internal points.

[0063] Set a maximum number of iterations, repeat the iteration process in step 4, and record the number of interior points drawn after selecting points in each iteration. The line model corresponding to the maximum number of interior points is determined as the optimal line model, and the slope k of the optimal line model is determined. * The corresponding angle is the estimated value of the angle between the boarding bridge and the horizontal direction;

[0064] θ=actank * (9).

[0065] The technical content and features of the present invention have been disclosed above. The scope of protection of the present invention is not limited to the above-described embodiments. Any equivalent modifications and other alterations made by those skilled in the art based on the content disclosed in the present invention should be included in the scope of protection set forth in the claims.

Claims

1. A method for estimating the boarding bridge angle using adaptive Hough transform combined with the RANSAC algorithm, characterized in that: Includes the following steps: Step 1) Use YOLOv5 to build a boarding bridge detection model, detect the image to be tested, and obtain the length and width data of the boarding bridge; Step 2) Crop the detected local image of the boarding bridge and perform edge detection using the Canny operator; Step 3) Based on the length and width data of the boarding bridge, set a dynamic straight line length threshold and construct an adaptive Hough transform model; perform straight line detection on the edge image of the boarding bridge; Step 4) Based on the straight line inspection results of the boarding bridge image, use the RANSAC algorithm to estimate the angle between the boarding bridge and the horizontal direction; In step 1), a boarding bridge detection model is constructed using YOLOv5 to detect the image to be tested and obtain the length and width data of the boarding bridge. The specific steps are as follows: 1-1 Use dock cameras to collect images of the boarding bridge from the test side view and create a boarding bridge detection dataset; 1-2 Train the YOLOv5 network using the boarding bridge detection dataset to obtain the boarding bridge detection model; 1-3 Using the trained boarding bridge detection model, we perform boarding bridge detection on the image to be tested and obtain the length L and width W of the detection frame. In step 2), the detected local image of the boarding bridge is cropped, and edge detection is performed using the Canny operator; the specific steps are as follows: 2-1 Noise reduction using Gaussian smoothing filter convolution; 2-2 Using different gradient operators, the gradient magnitude and direction of the smoothed image are calculated; non-maximum suppression is applied to the gradient magnitude to find local maxima in the image gradient, and other non-local maxima are set to zero; 2-3 Use dual thresholds to detect and connect edges: a high threshold is used to find each line segment, and a low threshold is used to determine the points on the line segment; In step 3), based on the length and width data of the boarding bridge, a dynamic straight line length threshold is set, an adaptive Hough transform model is constructed, and straight line detection is performed on the edge image of the boarding bridge; the specific steps are as follows: 3-1 Binary edge image of a local image facing the boarding bridge I edge Create a parameter space accumulation array P[ρ,θ] and initialize it to zero; set the set of line parameters S for detection. L , and set the empty set; 3-2 Traversing the Binary Image of Edges I edge For each edge pixel (x, y), calculate its corresponding linear polar coordinates (ρ, θ) that satisfy the following formulas (1), (2), and (3): ρ=xcosθ+ysinθ (3) 3-3 For each line segment, the length of the line is measured using the parameter space accumulation array P[ρ,θ], as shown in equation (4). P[ρ,θ]=P[ρ,θ]+1 (4) 3-4 Based on the length L of the boarding bridge detection frame, set an adaptive straight line length threshold, as shown in equation (5). T ρ =αL (5) In the formula, T ρ α is the threshold for the straight line length; α is the proportional coefficient belonging to [0,1], which is set to 0.4, that is, the threshold for the straight line length is adaptively set to α times the boarding bridge; 3-4 All lines whose lengths are greater than T in the cumulative array P[ρ,θ] of the line parameter space are summed. ρ The straight lines are preserved, while the remaining short straight line noise is removed; Obtain the set of detected lines S from the boarding bridge image. L ; (ρ,θ)∈S L ,if P[ρ, θ]> T ρ (6) 3-5 Calculate whether the number of detected straight lines exceeds the predetermined threshold T. N If the number of lines detected is greater than or equal to T N Then store the set of line parameters S L and the corresponding accumulated data P[ρ,θ]; if the number of detected lines is less than T N Then, the line length threshold T is corrected according to the following formula. ρ And return to steps 3-4) to reselect a line with a length greater than the corrected line length threshold T. ρ A straight line; T ρ =0.9×T ρ (7); In step 4), based on the straightness check results of the boarding bridge image obtained in step 3), the RANSAC algorithm is used to estimate the angle between the boarding bridge and the horizontal direction; the specific steps are as follows: 4-1 Targeting the set of all edge line segments detected in step 3) {(x i ,y i Establish a rectangular coordinate system with i∈N}; randomly select two points (x1,y1) and (x2,y2) from the set; 4-2 Using (x1, y1) and (x2, y2) to determine the equation of a straight line 4-3 Calculate the distance from the remaining points to the line, and then calculate the number of points with a distance less than the threshold as interior points based on the set threshold t, and count the number of interior points. 4-4 Set the maximum number of iterations, repeat the above iterations, and record the number of interior points drawn after selecting points in each iteration. The line model corresponding to the maximum number of interior points is determined as the optimal line model, and the slope k of the optimal line model is determined. * The corresponding angle is the estimated value of the angle between the boarding bridge and the horizontal direction; θ=actank * (9)。

Images