A dynamic object removal method based on image edge adaptive closure

Abstract

The present invention proposes a method for removing dynamic objects based on adaptive closure of image edges. First, the input image is preprocessed, including noise reduction, grayscale conversion and contrast enhancement. An adaptive edge extraction operation is used to calculate the threshold according to the local texture characteristics for edge detection. Then, edge connection and regional closure are performed, and the connection is performed by analyzing the eigenvalues ​​of its 8-neighborhood structure and the adaptive threshold. For edges that are not completely closed, the edges are completely closed by endpoint feature extraction, curve fitting and adaptive extension. Finally, based on the feature point density and random sampling consistency algorithm, the dynamic area is accurately detected and removed, and the camera posture is estimated. The present invention utilizes an improved edge detection and connection algorithm, combined with an adaptive threshold and gradient information, to achieve accurate extraction and reliable connection of image edges, overcome the defects of edge breakage and non-closure in traditional edge detection methods, and improve the accuracy of dynamic object area division in the scene.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a dynamic object removal method based on image edge adaptive closure. Background Art

[0002] The main difficulty of current visual SLAM is how to accurately identify and remove the interference caused by dynamic objects while ensuring real-time performance. Traditional methods usually assume that the scene remains static when calculating the camera pose. However, dynamic objects are common in actual application environments, and the feature points on these objects will seriously affect the accuracy of pose estimation, causing system instability or even failure.

[0003] At present, the solutions for dynamic SLAM are mainly divided into two categories: deep learning-based methods and geometric constraint-based methods. Among them, deep learning-based methods, such as DS-SLAM and DynaSLAM, identify possible dynamic areas through semantic segmentation or target detection networks. Although such methods have high accuracy in dynamic object recognition, they have the following problems: network model training relies on a large amount of labeled data, and can only identify object categories that have appeared in the training set, and has poor adaptability to unknown dynamic targets; deep neural networks have high computational overhead, which seriously affects the real-time performance of the system.

[0004] Comparative document CN115588133A discloses a visual SLAM method suitable for dynamic environments, including: extracting feature points from an input image and dividing the input image into an area inside a frame and an area outside a frame; filtering out moving points outside the frame through geometric constraints to obtain static points outside the frame; calculating the velocity of the feature points in the area inside the frame to divide them into moving points inside the frame and static points inside the frame; and incorporating the static points inside the frame and outside the frame into pose solution to obtain the optimal pose.

[0005] Although this method solves the positioning problem in a dynamic environment to a certain extent, its simple area division method is difficult to accurately reflect the boundary information of dynamic objects in the actual scene, and the method that only relies on speed calculation to distinguish between dynamic and static feature points is prone to misjudgment when dealing with dynamic objects that move at low speed or with uneven speed. At the same time, since the rich local texture features and edge information in the image are not fully utilized, more complex semantic information is used, which increases the processing time of the algorithm. The dynamic object removal method based on image edge adaptive closure proposed by the present invention can effectively solve the above problems. Summary of the invention

[0006] In order to solve at least one of the problems mentioned in the above background technology, the present invention provides a method for removing dynamic objects based on image edge adaptive closure.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for removing dynamic objects based on image edge adaptive closure, comprising the following steps:

[0008] S1: performing preprocessing operations on the image, including performing noise reduction, grayscale conversion and contrast enhancement processing on the input RGB color image;

[0009] S2: Perform an adaptive edge extraction operation, calculate an adaptive threshold according to the local texture characteristics of the image, implement edge detection, and obtain an initial edge image;

[0010] S3: Perform edge connection and region closure operations, perform feature point analysis, connection cost calculation and edge closure processing on the initial edge, and obtain a completely closed region marking result;

[0011] S4: The unclosed edges after S3 are closed based on contour fitting, and the edges are completely closed through endpoint feature extraction, curve fitting, adaptive step extension and smooth connection operations;

[0012] S5: Perform dynamic area detection and removal, perform preliminary screening through feature point density evaluation, use random sampling consistency algorithm to obtain rough camera pose, perform dynamic feature point detection based on closed area characteristics, and finally complete dynamic area removal and accurate camera pose estimation.

[0013] Preferably, the S1 comprises the following steps:

[0014] S11: A 5×5 Gaussian filter kernel is used to perform noise reduction on the input RGB color image. The standard deviation of the Gaussian kernel is 1.4. A denoised image is obtained by a two-dimensional convolution operation.

[0015] S12: The three RGB color channels are synthesized into a single grayscale channel using the weighted average method, where the weight coefficients of the green, red, and blue channels are 0.587, 0.299, and 0.114, respectively;

[0016] S13: performing adaptive histogram equalization processing on the grayscale image, dividing the image into sub-blocks of 16×16 pixels, calculating the grayscale histogram and cumulative distribution function for each sub-block, and achieving smooth transition between adjacent sub-blocks through bilinear interpolation technology.

[0017] Preferably, S2 comprises the following steps:

[0018] S21: Use a 15×15 pixel sliding window to calculate the local mean and variance of the image, and obtain a variance map reflecting the local texture complexity through the normalization formula σnorm²(x,y) = σ²(x,y) / (μ(x,y) + c), where c is a stabilization factor with a value of 10;

[0019] S22: Calculate the adaptive threshold based on the variance map, and determine the basic threshold and adjustment coefficient in combination with the OTSU method;

[0020] S23: Using the Sobel operator to calculate the gradients in the horizontal and vertical directions, and obtaining the normalized gradient magnitude and gradient direction, wherein the gradient magnitude is normalized to the interval [0, 1] by the ratio to the maximum gradient value of the image;

[0021] S24: performing non-maximum suppression processing on each pixel point according to its gradient direction to suppress non-local maximum points;

[0022] S25: using an adaptive threshold to perform binarization processing on the image after non-maximum suppression, using a higher threshold in a region with complex textures and a lower threshold in a region with simple textures, to obtain an initial edge image.

[0023] Preferably, S3 comprises the following steps:

[0024] S31: calculating the 8-neighborhood structural feature value for each edge point in the edge image, and classifying the edge point into endpoints, continuous points, intersection points, and invalid edge points according to the 8-neighborhood structural feature values;

[0025] S32: Design a comprehensive cost function including distance term, direction difference term and gradient support to evaluate the rationality of the connection of endpoint pairs. The distance term considers the influence of Euclidean distance and texture, the direction difference term considers the difference between the main direction of the endpoint and the direction of the connection line, and the gradient support considers the gradient characteristics on the connection path.

[0026] S33: An iterative optimization strategy based on priority is used to achieve edge connection. The priority is determined by the cost function and reliability index of the endpoint, and an adaptive threshold is used to control the connection of the endpoint pair.

[0027] S34: determining whether the contour is closed by calculating the ratio of the distance between the starting point and the end point of the contour to the total length of the contour, and determining that the contour is not closed when the ratio exceeds a preset threshold.

[0028] Preferably, S4 comprises the following steps:

[0029] S41: Perform local analysis on the endpoints of the unclosed contour, extract the last N contour points in the neighborhood of each endpoint, N=10, use the least squares method to perform quadratic curve fitting, and introduce the tangent vector at the endpoint as a constraint condition;

[0030] S42: Use an adaptive step size inversely proportional to the curvature to extend the fitting curve for sampling. A smaller step size is used in places with larger curvature to ensure accuracy, and a larger step size is used in places with smaller curvature to improve efficiency. The maximum extension length is set to 10% of the image diagonal length.

[0031] S43: The edges that still have gaps after extension are smoothly connected using Bezier curves, and the newly generated edge segments are made to better conform to the image gradient characteristics through small range adjustments;

[0032] S44: Fusing the optimized edge segment with the original contour, completing the edge closing process, and assigning a unique identifier to each closed area.

[0033] Preferably, S5 comprises the following steps:

[0034] S51: Perform preliminary screening by comparing the feature point density evaluation value and the number of regional pixel points;

[0035] S52: Use the RANSAC algorithm to roughly estimate the camera pose and calculate the reprojection error of the feature points;

[0036] S53: Counting the proportion of feature points in each closed area whose reprojection error exceeds a preset threshold, and marking them as dynamic areas and removing them when the proportion exceeds 30%-40%;

[0037] S54: Use the Levenberg-Marquardt algorithm to perform nonlinear optimization on the retained static feature points to obtain the optimized camera pose.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention utilizes an improved edge detection and connection algorithm, combined with adaptive threshold and gradient information, to achieve accurate extraction and reliable connection of image edges, overcomes the defects of edge breakage and non-closure in traditional edge detection methods, and improves the accuracy of dynamic object area division in the scene;

[0040] 2. The present invention avoids the high computational overhead of using deep learning networks for target detection and semantic segmentation by designing a region closure processing method based on edge features, including steps such as feature point analysis, connection cost calculation and contour fitting, and improves the real-time performance of the dynamic SLAM system;

[0041] 3. The present invention effectively combines the local texture features and edge information of the image, designs an edge connection evaluation mechanism that comprehensively considers distance, direction difference and gradient support, and improves the accuracy of edge detection and the reliability of edge connection in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 It is a flow chart of a method for removing dynamic objects based on image edge adaptive closure;

[0044] Figure 2 Flowchart for edge connection and area closure operations. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention.

[0046] First, refer to Figure 1 to Figure 2 A method for removing dynamic objects based on image edge adaptive closure according to an embodiment of the present invention is described in detail.

[0047] A method for removing dynamic objects based on image edge adaptive closure includes the following steps:

[0048] S1: Perform image preprocessing operations to perform noise reduction, grayscale conversion and contrast enhancement on the input RGB color image to provide better quality input data for subsequent edge detection;

[0049] S2: Perform an adaptive edge extraction operation, calculate an adaptive threshold according to the local texture characteristics of the image, implement edge detection, and obtain an initial edge image;

[0050] S3: Perform edge connection and region closure operations, perform feature point analysis, connection cost calculation and edge closure processing on the initial edge, and finally obtain a completely closed region marking result.

[0051] S4: For the edges that are still not closed after S3 processing, close them based on contour fitting. First, extract the local features at the endpoints of the unclosed contour, select the last N contour points in the neighborhood of each endpoint for quadratic curve fitting, and determine the extension trend of the contour. Then use a curvature-adaptive step size strategy to extend and sample the curve, and detect the intersection with other edges in real time during the extension process. For the gaps that still exist after extension, use smooth curves to connect them and locally optimize the newly generated edge segments to make them better conform to the image features, and finally achieve complete closure of the edges;

[0052] S5: Detect and remove dynamic areas. Based on the image area marking results, perform preliminary screening through feature point density evaluation, then use the random sampling consistency algorithm to obtain the rough camera pose, and accurately detect dynamic feature points based on the overall characteristics of the closed area. Finally, remove the dynamic area, and use the nonlinear optimization method to accurately estimate the camera pose for the remaining static area.

[0053] In order to provide better quality data input for subsequent edge detection, the image preprocessing operation in S1 includes the following specific steps:

[0054] S11: Perform Gaussian filtering denoising on the input RGB color image, and use a 5×5 Gaussian filter kernel to smooth the image to eliminate random noise in the image; the standard deviation σ of the Gaussian kernel is selected as 1.4, and the selection of this parameter value can well preserve the image details while ensuring the filtering effect; a denoised image is obtained by a two-dimensional convolution operation of the Gaussian kernel and the image, which retains the main structural features of the original image and significantly reduces the impact of noise on subsequent processing.

[0055] S12: Considering the differences in the sensitivity of the human visual system to different colors, the denoised RGB color image is converted to grayscale. The weighted average method is used to combine the three color channels into a single grayscale channel, where the weight coefficients of the green channel, the red channel, and the blue channel are 0.587, 0.299, and 0.114, respectively. This weight configuration fully considers the physiological characteristics of the human eye and ensures that the converted grayscale image can better maintain the layering and detail characteristics of the original image in human eye perception.

[0056] S13: In order to improve the local contrast of the image, the grayscale image is subjected to adaptive histogram equalization processing; first, the image is divided into sub-blocks of 16×16 pixels, the grayscale histogram and cumulative distribution function are calculated for each sub-block, and a contrast limit threshold is set to prevent excessive noise enhancement. Then, the bilinear interpolation method is used to make a smooth transition between adjacent sub-blocks, and a contrast-enhanced image C is obtained, which provides a better quality input data source for subsequent edge detection and other processing.

[0057] Further, S2 comprises the following steps:

[0058] S21: Calculate the local variance of the image, use a sliding window of 15×15 pixels to calculate the local mean and variance of each position, and perform normalization to obtain a variance map reflecting the local texture complexity; the normalized calculation formula for the local variance is:

[0059] σnorm²( x,y ) = σ²( x,y ) / ( μ( x,y ) + c) ;

[0060] Where (x,y) refers to the horizontal and vertical coordinates of the pixel in the image, σ²(x,y) is the local variance in the sliding window, μ(x,y) is the local mean in the sliding window, and c is a stabilization factor to prevent the denominator from being zero. The value is 10, which is an empirical value determined based on the image grayscale value range [0,255].

[0061] S22: Perform adaptive threshold calculation based on the variance map, calculate the global mean M and standard deviation S of the variance map, and design the threshold mapping function T(x,y) as follows

[0062] T(x,y) = Tbase + α × ( σnorm²( x,y ) - M ) / S,

[0063] Tbase is the basic threshold, which is automatically calculated by the OTSU method, and the adjustment coefficient α is adaptively determined according to the overall variance level of the image. The specific formula is:

[0064] α= k × ( Smax - Smin ) / ( Smax + Smin ) ;

[0065] Among them, Smax and Smin are the maximum and minimum values ​​of the variance map, and k is the empirical coefficient, which is 0.5.

[0066] S23: Use the Sobel operator to traverse each pixel point, calculate the horizontal and vertical gradients of the image C, and obtain the normalized gradient amplitude Gnorm and direction θgradient corresponding to the pixel point, which indicates the direction in which the grayscale changes most dramatically at the pixel point. The gradient calculation uses the Sobel operator Gx for the horizontal direction and the Sobel operator Gy for the vertical direction, and the weights of the uplink and downlink are set as:

[0067] Gx = [-1 0 1; -2 0 2; -1 0 1];

[0068] Gy = [-1 -2 -1; 0 0 0; 1 2 1];

[0069] The gradient amplitude |G| is calculated using operators Gx and Gy and normalized to obtain Gnorm, which has a value range of [0,1].

[0070] The calculation formula of the local gradient direction θgradient is:

[0071] θgradient(x,y)= arctan(Gy / Gx);

[0072] S24: Perform non-maximum suppression processing on each pixel point; first, perform non-maximum suppression processing, for each pixel point, determine the two closest adjacent pixel points according to its gradient direction θgradient, if the gradient amplitude of the current pixel point is not the maximum value among these three points, it is suppressed to 0.

[0073] S25: Binarize each pixel; use the adaptive threshold mapping function T(x,y) calculated above to binarize the image after non-maximum suppression; for each pixel (x,y) in the image, compare its normalized gradient amplitude Gnorm(x,y) with the threshold T(x,y) corresponding to the point; when Gnorm(x,y) is greater than T(x,y), mark the point as an edge point and assign a value of 1; otherwise, mark it as a non-edge point and assign a value of 0. This adaptive threshold method takes into account the local texture characteristics of the image, uses a higher threshold in complex texture areas to suppress excessive response, and uses a lower threshold in simple texture areas to maintain sensitivity, thereby obtaining a more reasonable initial edge image F.

[0074] S3: Perform edge connection and region closure operations, perform feature point analysis, connection cost calculation and edge closure processing on the initial edge, and finally obtain a completely closed region marking result.

[0075] Further, S3 includes the following steps:

[0076] For the convenience of subsequent explanation, the following definition is made: a pixel point with a value of 1 in the edge image F is defined as an edge point, indicating that there is an image edge at that position; for any pixel point p(x,y) in the image, its 8-neighborhood is defined as: the coordinates of the 4 adjacent pixels in the horizontal and vertical directions are (x±1,y) and (x,y±1), and the coordinates of the 4 adjacent pixels in the diagonal direction are (x±1,y±1). The pixels at these 8 positions together constitute the 8-neighborhood of the pixel point p, and are numbered 1 to 8 in a clockwise direction.

[0077] S31: Analyze the 8-neighborhood structural eigenvalue ES(p) of each edge point in the edge image F, classify the edge points into endpoints, continuous points, intersection points and invalid edge points according to the value of ES(p), and calculate the main direction θp and reliability index Rp of the endpoints; where θp represents the most likely edge extension direction starting from point p, and its value range is [0,2π].

[0078] The calculation formula of the structural eigenvalue of edge points is:

[0079] ;

[0080] Where: Bi is the edge state at the i-th position in the 8-neighborhood, and wi is the position weight: 1.0 for the orthogonal direction and 0.707 for the diagonal direction. The value of ES(p) corresponds to the edge point classification as follows:

[0081] When 0.9 ≤ ES(p) ≤ 1.1, it is determined as an endpoint, indicating that the point is only connected to one edge pixel and is the starting or ending point of an edge segment, and it needs to participate in subsequent edge connection processing;

[0082] When 1.9 ≤ ES(p) ≤ 2.1, it is determined as a continuous point, indicating that the point is exactly connected to two edge pixels and is a part of the edge segment, and no connection processing is required;

[0083] When ES(p) > 2.1, it is determined as an intersection point, indicating that the point is connected to three or more edge pixels and is the intersection of multiple edge segments, and it needs to be used as a boundary condition in subsequent connection processing;

[0084] For other value ranges of ES(p), that is, ES(p) < 0.9 or 1.1 < ES(p) < 1.9, considering that these values usually represent unstable or noise-affected edge points in practical applications, the present invention determines them as invalid edge points and eliminates them in subsequent processing.

[0085] The calculation formula for the main direction is:

[0086] θp = max(H(k)) × (2π / 36);

[0087] Where the calculation formula for the direction histogram H(k) is:

[0088] ;

[0089] Where k is the direction interval index, and its value range is an integer in [0, 35]; Gnorm(x, y) is the normalized gradient amplitude, w(r) = exp(-r² / 2σ²) is the distance weight, r is the distance from the current pixel point coordinate (x, y) to the endpoint, σ is 1 / 3 of the window size, bin(θ) is the function that maps the direction θ to the corresponding interval, and δ(...) is the indicator function, which takes the value 1 when the independent variable is 0 and otherwise takes the value 0.

[0090] The calculation formula for the main direction reliability index is:

[0091] Rp = Hpeak / Hmean;

[0092] Where Hpeak is the peak value of the direction histogram and Hmean is the average value of the direction histogram.

[0093] S32: Design a comprehensive cost function Cost(p1,p2) to evaluate the rationality of the connection of endpoint pairs. An endpoint pair is a combination of any two endpoints obtained in step S31. A path is all the points along the continuous points or intersections from one endpoint to the other endpoint. The smaller the value of the comprehensive cost function, the smaller the connection cost of the two endpoints, the more reasonable the connection between the two endpoints. The function contains three components: distance term D(p1,p2), direction difference term A(p1,p2) and gradient support G(p1,p2); the specific calculation method of the comprehensive cost function is:

[0094] Cost(p1,p2) = w1×D(p1,p2) + w2×A(p1,p2) + w3×G(p1,p2);

[0095] Among them: the weight coefficients are set as: w1 = 0.4, w2 = 0.3, w3 = 0.3;

[0096] The distance term is:

[0097] D(p1,p2) = deuclidean(p1,p2) × (1 + λ×Ctexture);

[0098] Among them, deuclidean is the Euclidean distance, λ is the texture influence factor, which takes a value of 0.3, and Ctexture is the average normalized variance on the path. For each pixel point on the connection path, the corresponding local variance value is extracted and normalized to eliminate the dimension effect, and then the average value of these normalized variance values ​​is calculated as the texture complexity index of the path. Ctexture is used to measure the texture complexity of the connection path between two endpoints. The larger its value, the more complex the texture of the area passed by the path. It is obtained by calculating the normalized variance on the path.

[0099] The directional difference term is:

[0100] A(p1,p2)=(1-cos(θp1-θline))×Rp1+(1-cos(θp2-θline))×Rp2;

[0101] Among them, θline is the direction of the connecting line segment, θp1 and θp2 are the main directions of the endpoints p1 and p2 respectively. This main direction is the main peak direction of the gradient direction histogram in the 15×15 window in step S31, and Rp1 and Rp2 are the direction reliability indicators of the endpoints;

[0102] The gradient support is:

[0103] G( p1,p2 ) = 1 - ( 1 / L )∑( path )( Gnorm( x,y ) × cos²(θdiff))

[0104] Among them, L is the path length of the connection path between p1 and p2, path is the shortest path between p1 and p2, Gnorm is the normalized gradient amplitude; θdiff is the difference between the local gradient direction and the path direction, and the calculation method is:

[0105] θdiff = min(|θgradient(x,y) - θpath( x,y )|, 2π - | θgradient( x,y )- θpath( x,y )|) ;

[0106] Where θpath(x,y) is the path direction of each point (x,y) on the path:

[0107] θpath(x,y) = arctan2(ynext - yprev, xnext - xprev);

[0108] Among them, (xprev, yprev) and (xnext, ynext) are the adjacent points before and after the point on the path respectively;

[0109] For the first and last points on the path, the path direction is calculated using their adjacent points; for the middle point on the path, the path direction is calculated using its preceding and following points, thus ensuring the continuity of direction calculation;

[0110] S33: Edge connection is realized by using priority-based iterative optimization strategy, and the priority is calculated by the formula:

[0111] Priority(p) = min(Cost( p,q ))Rp;

[0112] Calculate and use the adaptive threshold formula:

[0113] Tadaptive = Tbase(1 + k×Rp);

[0114] Control connection establishment, where: Tbase is 0.8 times the mean of the comprehensive cost function, k is the reliability adjustment coefficient, and its value is 0.2.

[0115] Since the smaller the Cost(p,q) value is, the more reasonable the connection is, only endpoint pairs with priority values ​​lower than the adaptive threshold are connected, and endpoint pairs exceeding the threshold are not connected.

[0116] S34: determining whether the contour is closed by calculating the ratio of the distance between the starting point and the end point of the contour to the total length of the contour, and determining that the contour is not closed when the ratio exceeds a preset threshold.

[0117] Further, S4 includes the following detailed steps:

[0118] S41: Perform curve fitting on the endpoints of the unclosed edge and their neighborhoods; perform local analysis on the endpoints of the unclosed contour, extract the last N contour points in the neighborhood of each endpoint, N=10, and use the least squares method to perform quadratic curve fitting on these points to obtain the local direction of the contour at the endpoint.

[0119] The fitting curve takes the form of a parametric equation: P(t) = at² + bt + c;

[0120] Where t is the parameter variable, a, b, c are the coefficient vectors to be determined, and the optimal fitting parameters are obtained by minimizing the sum of squared errors.

[0121] In order to ensure the rationality of the fitting, the tangent vector at the endpoint is introduced as a constraint condition to make the direction of the fitting curve at the endpoint transition smoothly with the original contour.

[0122] S42: After obtaining the fitting curve, the curve is sampled and extended by an adaptive step size. The sampling step size is inversely proportional to the curvature, that is, a smaller step size is used at a larger curvature to ensure accuracy, and a larger step size is used at a smaller curvature to improve efficiency;

[0123] During the extension process, it is detected in real time whether it intersects with other edges or extension segments. If an intersection point is detected, the current extension ends with this point as the end point. At the same time, in order to prevent excessive extension, a maximum extension length threshold Lmax is set. This threshold is adaptively determined according to the image size and is usually set to 10% of the image diagonal length.

[0124] S43: smoothly connect the edges that are still not closed; after the two endpoints of each unclosed contour are extended as described above, if there is still a gap between the two extended segments, the Bezier curve is used to smoothly connect the gap while considering the direction of the extended segment. After the connection is completed, the newly generated edge segment is locally optimized, and the edge position is adjusted in a small range to make it better conform to the gradient characteristics of the image.

[0125] S44: The optimized edge segment is merged with the original contour to complete the edge closing process; the closing method based on contour fitting can generate reasonable completion results while maintaining edge continuity, which is particularly suitable for processing situations where the edge direction is clear but there are local breaks; and the completely closed edge image is region marked, and a unique identifier is assigned to each closed area to obtain the final region marking result.

[0126] Further, S5 comprises the following steps:

[0127] S51: After obtaining the region marking result of the image, density evaluation is performed on the feature points in each closed region. Specifically, the density evaluation value of the feature points is calculated:

[0128] D = len( Ni ) / sum(len( N )) - σ × (Ni / sum( N ));

[0129] Where len(Ni) represents the number of feature points in the i-th closed area, sum(len(N)) represents the total number of feature points in all closed areas, Ni represents the number of pixels in the i-th closed area, sum(N) represents the number of all pixels in the image, and σ is the density control parameter.

[0130] When the calculated D value is greater than 0, it indicates that the density of feature points in the closed area is too high. At this time, the feature points in the area need to be randomly downsampled. This density-based preliminary screening mechanism can effectively avoid excessive concentration of feature points in certain areas and provide a more uniform distribution of feature points for subsequent dynamic detection.

[0131] S52: Perform a rough estimate of the camera pose; use the preliminarily screened feature point set and the random sampling consistency RANSAC algorithm to perform pose estimation. The specific implementation process is: first randomly select the minimum number of feature point pairs to calculate the basic matrix, and then solve the camera motion parameters; then calculate the reprojection error e of all feature point pairs:

[0132] e = ||p2 - K[R|t] P1 || ;

[0133] Among them, p2 is the coordinate of the feature point in the current frame, K is the camera internal parameter matrix, R is the rotation matrix, t is the translation vector, and P1 is the homogeneous coordinate of the feature point in the reference frame; finally, the pose estimation result with the most internal point support is selected through multiple iterations. The purpose of this step is to obtain a relatively reliable initial pose estimation to provide a basis for subsequent dynamic feature point detection.

[0134] S53: Accurate detection and removal of dynamic regions;

[0135] First, based on the camera posture obtained in step S52, the reprojection error of each feature point is calculated; then, the number of feature points in each closed area whose reprojection error exceeds the preset threshold and their proportion in the closed area is counted; when the proportion of abnormal feature points in a closed area exceeds the preset threshold, and the proportion is 30%-40%, the closed area is marked as a dynamic area, and all feature points in it are marked as dynamic feature points and removed; for closed areas that are not marked as dynamic areas, the feature points with smaller reprojection errors are retained for subsequent processing. The batch processing method based on closed areas can effectively reduce the instability that may be caused by independent feature point judgment.

[0136] S54: Camera pose optimization: After completing the detection and removal of dynamic feature points, the retained static feature points are used to construct an optimization problem to minimize the reprojection error, and the Levenberg-Marquardt algorithm is used for nonlinear optimization to finally obtain the optimized camera pose. This step not only improves the accuracy of pose estimation, but also significantly improves the robustness in dynamic scenes by removing the interference of dynamic feature points.

[0137] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for removing dynamic objects based on image edge adaptive closure, characterized in that: The following steps are involved: S1: performing preprocessing operations on the image, including performing noise reduction, grayscale conversion and contrast enhancement processing on the input RGB color image; S2: Perform an adaptive edge extraction operation, calculate an adaptive threshold according to the local texture characteristics of the image, implement edge detection, and obtain an initial edge image; S3: Perform edge connection and region closure operations, perform feature point analysis, connection cost calculation and edge closure processing on the initial edge, and obtain a completely closed region marking result; S4: The unclosed edges after S3 are closed based on contour fitting, and the edges are completely closed through endpoint feature extraction, curve fitting, adaptive step extension and smooth connection operations; S5: Detect and remove dynamic areas, perform preliminary screening through feature point density evaluation, use random sampling consistency algorithm to obtain rough camera pose, detect dynamic feature points based on closed area characteristics, and finally complete dynamic area removal and accurate camera pose estimation; The S3 comprises the following steps: S31: calculating the 8-neighborhood structural feature value for each edge point in the edge image, and classifying the edge point into endpoints, continuous points, intersection points, and invalid edge points according to the 8-neighborhood structural feature values; S32: Design a comprehensive cost function including distance term, direction difference term and gradient support; evaluate the rationality of the connection of endpoint pairs, where the distance term considers the Euclidean distance and texture influence, the direction difference term considers the difference between the main direction of the endpoint and the direction of the connection line, and the gradient support considers the gradient characteristics on the connection path; S33: An iterative optimization strategy based on priority is used to achieve edge connection. The priority is determined by the cost function and reliability index of the endpoint, and the connection establishment is controlled by an adaptive threshold. S34: determining whether the contour is closed by calculating the ratio of the distance between the starting point and the end point of the contour to the total length of the contour, and determining that the contour is not closed when the ratio exceeds a preset threshold.

2. The method for removing dynamic objects based on image edge adaptive closure according to claim 1, characterized in that: The S1 comprises the following steps: S11: A 5×5 Gaussian filter kernel is used to perform noise reduction on the input RGB color image. The standard deviation of the Gaussian kernel is 1.

4. A denoised image is obtained by a two-dimensional convolution operation. S12: The three RGB color channels are synthesized into a single grayscale channel using the weighted average method, where the weight coefficients of the green, red, and blue channels are 0.587, 0.299, and 0.114, respectively; S13: performing adaptive histogram equalization processing on the grayscale image, dividing the image into sub-blocks of 16×16 pixels, calculating the grayscale histogram and cumulative distribution function for each sub-block, and achieving smooth transition between adjacent sub-blocks through bilinear interpolation technology.

3. The method for removing dynamic objects based on image edge adaptive closure according to claim 1, characterized in that: The S2 comprises the following steps: S21: Use a 15×15 pixel sliding window to calculate the local mean and variance of the image, and obtain a variance map reflecting the local texture complexity through the normalization formula σnorm²(x,y) = σ²(x,y) / (μ(x,y) + c), where c is a stabilization factor with a value of 10; where (x,y) refers to the horizontal and vertical coordinates of the pixel in the image, σ²(x,y) is the local variance in the sliding window, μ(x,y) is the local mean in the sliding window, and c is a stabilization factor to prevent the denominator from being 0, with a value of 10, which is an empirical value determined based on the image grayscale value range [0,255]; S22: Calculate the adaptive threshold based on the variance map, and determine the basic threshold and adjustment coefficient in combination with the OTSU method; S23: Using the Sobel operator to calculate the gradients in the horizontal and vertical directions, and obtaining the normalized gradient magnitude and gradient direction, wherein the gradient magnitude is normalized to the interval [0, 1] by the ratio to the maximum gradient value of the image; S24: performing non-maximum suppression processing on each pixel point according to its gradient direction to suppress non-local maximum points; S25: using an adaptive threshold to perform binarization processing on the image after non-maximum suppression, using a higher threshold in a region with complex textures and a lower threshold in a region with simple textures, to obtain an initial edge image.

4. The method for removing dynamic objects based on image edge adaptive closure according to claim 1, characterized in that: The S4 comprises the following steps: S41: Perform local analysis on the endpoints of the unclosed contour, extract the last N contour points in the neighborhood of each endpoint, N=10, use the least squares method to perform quadratic curve fitting, and introduce the tangent vector at the endpoint as a constraint condition; S42: Use an adaptive step size inversely proportional to the curvature to extend the fitting curve for sampling. A smaller step size is used in places with larger curvature to ensure accuracy, and a larger step size is used in places with smaller curvature to improve efficiency. The maximum extension length is set to 10% of the image diagonal length. S43: The edges that still have gaps after extension are smoothly connected using Bezier curves, and the newly generated edge segments are made to better conform to the image gradient characteristics through small range adjustments; S44: Fusing the optimized edge segment with the original contour, completing the edge closing process, and assigning a unique identifier to each closed area.

5. The method for removing dynamic objects based on image edge adaptive closure according to claim 1, characterized in that: The S5 comprises the following steps: S51: Perform preliminary screening by comparing the feature point density evaluation value and the number of regional pixel points; S52: Use the RANSAC algorithm to roughly estimate the camera pose and calculate the reprojection error of the feature points; S53: Counting the proportion of feature points in each closed area whose reprojection error exceeds a preset threshold, and marking them as dynamic areas and removing them when the proportion exceeds 30%-40%; S54: Use the Levenberg-Marquardt algorithm to perform nonlinear optimization on the retained static feature points to obtain the optimized camera pose.

Images