Robust berth mark point tracking method under abnormal light interference

Abstract

The invention discloses a berth mark point robust tracking method under abnormal light interference, and the method comprises the steps: dynamically adjusting the weight of a classifier through analyzing the confidence coefficient of a berth mark point, enhancing the similarity between a mark point feature in a prediction window and a historical feature, effectively eliminating the accumulated interference caused by abnormal light in the historical feature, and improving the tracking precision. The position of the mark point is accurately determined, the problem of wrong tracking caused by abnormal light interference near the berth mark point is solved, and the tracking accuracy and stability of the berth mark point under the abnormal light condition are improved. Meanwhile, the parking mark points can be effectively, continuously and accurately tracked under the condition of light interference, compared with a KCF algorithm, the method is remarkably improved, the method can be applied to the automatic parking technology in unmanned driving, the accuracy of automatic parking is improved, the better autonomous valet parking effect is achieved, and the method has the better practical application value.

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to a method for robustly tracking berth markers under abnormal light interference. Background Art

[0002] With the continuous breakthroughs in computer vision and artificial intelligence technologies, target tracking technology has also ushered in rapid development. In particular, target tracking algorithms based on deep learning have become the current mainstream trend. Through the training of large-scale data, these algorithms have significantly improved the accuracy and robustness of target detection, making target tracking technology widely and deeply applied in many fields such as video surveillance, autonomous driving, and robot navigation.

[0003] A common target tracking algorithm is the KCF (Kernelized Correlation Filter) tracking algorithm, which uses a kernel correlation filter to match the target model with the candidate target in the current frame to achieve accurate target tracking. The core idea of ​​the KCF tracking principle is to use the position and feature information of the target in the previous frame to track the target through a kernel correlation filter in the current frame.

[0004] However, in harsh environments, the features of the tracked markers are deteriorated due to the interference of abnormal light near the berth markers. The existing KCF-based tracking method cannot continuously and accurately track the berth markers disturbed by abnormal light, which ultimately leads to erroneous tracking and deteriorates the berth tracking performance. Summary of the invention

[0005] The object of the present invention is to provide a method for robustly tracking berth markers under abnormal light interference.

[0006] The present invention provides a method for robustly tracking a berth marker under abnormal light interference, which comprises the following steps: Step 1: Set multiple target points of the berths in the AVM image at the initial moment as initial marking points and construct initial classifier parameters.

[0007] Step 2: Obtain the target feature matrix at the current moment k based on the mark point at the previous moment k-1 ; Using the target feature matrix Get the historical confidence set of the i-th marker track and the position of the mark point at the current time k; where, is the confidence level of k at the current moment; ; n is the number of berths.

[0008] Step 3: Update the marked point track according to the marked point position at the current time k.

[0009] Step 4: Train the classifier and perform weighted fusion of the classifier parameters at the current time k and the previous time k-1; according to the historical confidence set Get the confidence mean square error of k at the current moment If the confidence mean square error and Both are less than the set threshold , then reset the weight coefficient To reduce the weight of the classifier parameters at the previous moment; otherwise, the preset weight coefficient is used Update classifier parameters.

[0010] Step 5: loop steps 2 to 4, obtain the positions of the marking points at different times, complete the tracking of the marking points, and realize the trajectory tracking of the berth.

[0011] Preferably, the classifier parameters are a classifier weight matrix and a historical target feature matrix.

[0012] Preferably, in step 4, the method of fusing the classifier parameters at the current time k and the previous time k-1 is as follows: If the confidence mean square error and Both are less than the set threshold , then the classifier weight matrix of the fused i-th track at the current time k is and the historical target feature matrix The expression is: If the confidence mean square error and Are greater than or equal to the set threshold , then the classifier weight matrix of the fused i-th track at the current time k is and the historical target feature matrix The expression is: Reset weight coefficient Greater than the preset weight coefficient .

[0013] Preferably, in step 1, the target point is the coordinates of two corner points of the plurality of berths at an initial moment.

[0014] Preferably, in step 2, the confidence The method to obtain is as follows: The target feature matrix at the current moment The historical target feature matrix of the previous moment Perform similarity calculation and obtain Gaussian response matrix ; Use Gaussian response matrix Point multiplication of the classifier weight matrix of the previous moment k-1 , get the output response matrix , take the output response matrix The maximum element in is used as the confidence of k at the current moment .

[0015] As a preferred method, the method for obtaining the position of the marking point at the current moment k is: by inverse Fourier transforming the output response matrix F k Convert to the image coordinate system and take the position in the AVM image corresponding to the largest element in the output response matrix as the marker point position at the current moment k.

[0016] Preferably, the target feature matrix The method to obtain is as follows: The target frame is established by the position of the mark point at the previous moment k-1, and the prediction window at the current moment k is obtained after the target frame is expanded. After feature extraction of the prediction window at the current moment k, the target feature matrix at the current moment k is obtained. .

[0017] Preferably, the classifier weight matrix and the historical target feature matrix The expression is: in, is the target feature matrix of the marking point of the ith track at the current time k; is the target feature matrix The corresponding feature similarity matrix; and λ are the preset Gaussian labels and regularization parameters respectively.

[0018] Preferably, in step 3, the method for updating the marked point track is as follows: a) If there is a detection mark point obtained by the detection network at the current time k, establish a set of all detection mark points at the current time k ;in, is the position of the i-th detection mark point in the image coordinate system at the current moment k; and are respectively the horizontal and vertical coordinates of the marking point of the i-th track at the current time k.

[0019] Use the current marker position k obtained in step 2 as the tracking marker position to establish a set of all tracking markers ;in, is the position of the j-th tracking marker point in the image coordinate system at the current time k; j=1,2,...,2n.

[0020] like , then use the tracking mark point to update the track information, and the number of track association failures is t i Add 1; where, is the set confidence threshold. , calculate the detection mark point and track markers The Euclidean distance dis between , If the threshold is set, the detection mark point is used to update the track information; otherwise, the tracking mark point is used to update the track information and the number of track association failures is t i Add 1. If there is a detection mark point that cannot be successfully associated with any existing mark point track, the detection mark point is used as the initial mark point to generate a new track.

[0021] b) If there is no detection mark point obtained by the detection network at the current time k, the track is updated using the tracking mark point; , and If any of the conditions is met, the number of track association failures is t i Add 1; where δ e is the set drift threshold.

[0022] According to the number of association failures t i Judge the track of all updated mark points at the current time k; if t i > , the corresponding track is deleted; otherwise, the track is retained; It is the preset association failure threshold.

[0023] As a preference, in step 4, the confidence mean square error of the current moment k is The expression is: in, is the confidence set mean.

[0024] The present invention has the following beneficial effects: 1. The present invention dynamically adjusts the weight of the classifier by analyzing the confidence of the berth marking point, enhances the similarity between the marking point features and the historical features in the prediction window, effectively eliminates the cumulative interference caused by abnormal light in the historical features, accurately determines the position of the marking point, solves the problem of erroneous tracking caused by abnormal light interference near the berth marking point, and improves the tracking accuracy and stability of the berth marking point under abnormal light conditions.

[0025] 2. The present invention can effectively and continuously track parking space marking points accurately under the condition of light interference, which is significantly improved compared with the KCF algorithm. It can be applied to automatic parking technology in unmanned driving, improve the accuracy of automatic parking, and achieve better autonomous valet parking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the overall flow chart of the present invention.

[0027] Figure 2 Schematic diagram of different data segments containing jump scenes in the secondary construction of the PSV data set in the present invention.

[0028] Figure 3 It is a comparison diagram of the marking point tracking effect in the present invention; wherein, (a) is a schematic diagram of the KCF marking point tracking effect; (b) is a schematic diagram of the marking point tracking effect in the present invention.

[0029] Figure 4 It is a schematic diagram of the result of marker point tracking for data segment 1 in the present invention; wherein, (a) is a schematic diagram of the marker point tracking trajectory of KCF and the present invention; and (b) is a schematic diagram of the marker point tracking position error of KCF and the present invention.

[0030] Figure 5 It is a schematic diagram of the result of marking point tracking for data segment 2 in the present invention; wherein, (a) is a schematic diagram of the marking point tracking trajectory of KCF and the present invention; and (b) is a schematic diagram of the marking point tracking position error of KCF and the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings.

[0032] like Figure 1 As shown, a method for robustly tracking berth markers under abnormal light interference includes the following steps: Step 1: Set the coordinate system of the image, define the upper left corner of the image as the origin of the coordinates, the positive direction of the horizontal axis is the right direction of the origin, and the positive direction of the vertical axis is the downward direction of the origin. Use the detection network to detect the parking information in the AVM (panoramic imaging system) image at the initial moment. The parking information includes the coordinates of the two corner points of n parking spaces (initial marking points), where the corner points are the upper and lower points on the outside of the parking space in the side parking.

[0033] Step 2: Create a target frame based on the position of the mark point at the previous moment k-1, and its size is [s 1 s 2 ], and after expanding the target frame, obtain the prediction window of the current time k, and after extracting the features of the prediction window of the current time k, obtain the target feature matrix of the current time k . The target feature matrix The historical target feature matrix of the previous moment Perform similarity calculation and obtain Gaussian response matrix , the Gaussian response matrix The expression is:

[0034] in, is the target feature matrix With the historical target feature matrix The corresponding feature similarity matrix.

[0035] Using Gaussian response matrix Point multiplication of the classifier weight matrix of the previous moment k-1 , get the output response matrix , the output response matrix The expression is: in, is the dot product symbol.

[0036] Initial moment classifier weight matrix and the historical target feature matrix z 0 The expression is: in, is the initial target feature The corresponding feature similarity matrix; is the target feature of the marked point in the i-th track at the initial moment; and λ are the preset Gaussian labels and regularization parameters respectively; ; is the number of berths.

[0037] Get the output response matrix The maximum element in is used as the confidence of k at the current moment , since there is no confidence at the first moment, the confidence is calculated from the second moment, so at the current moment The historical confidence set of the i-th marked point track is expressed as The output response matrix F is transformed by inverse Fourier transform k Convert to the image coordinate system and take the position in the AVM image corresponding to the largest element in the output response matrix as the marker point position at the current moment k.

[0038] Step 3: Update track 3-1. If there is a detection result obtained by the detection network at the current time k, use the detection result as the detection mark point and establish a set of all detection mark points ;in, is the position of the i-th detection mark point in the image coordinate system at the current moment k; and are the horizontal and vertical coordinates of the i-th marking point k at the current moment respectively.

[0039] Use the current marker position k obtained in step 2 as the tracking marker position to establish a set of all tracking markers ;in, is the position of the j-th tracking marker point in the image coordinate system at the current time k; j=1,2,...,2n.

[0040] like , it means that the track association of the mark point fails, and the track information is updated using the tracking mark point. The number of track association failures is t i Add 1; where, is the set confidence threshold. , calculate the detection mark point and track markers The Euclidean distance dis between:

[0041] If the Euclidean distance dis is less than or equal to the set threshold , it is considered that the detection mark point is successfully associated with the mark point track, and the detection mark point is used to update the track information; otherwise, it is considered that the association fails, and the tracking mark point is used to update the track information and the track association fails times t i Add 1. If a detected marker point is found to be unsuccessfully associated with any existing marker track during the detection process, the marker point is considered as a new initial point and a new track is created for it.

[0042] If there is no detection result obtained by the detection network at the current time k, the track is updated using the tracking mark point; the tracking result is judged by the confidence and historical track information. , and If any of the following conditions is met, the tracking result is considered inaccurate, and the number of track association failures is t iAdd 1; where δ e is the set drift threshold.

[0043] 3-2. According to the number of association failures t i Judge the track of all updated mark points at the current time k; if t i >ξ t , the corresponding track is deleted; otherwise, the track is retained; where ξ t It is the preset association failure threshold.

[0044] Step 4: Due to the historical confidence set With the target feature matrix and the historical target feature matrix Related, through the historical confidence set To evaluate the target feature matrix at k-2 time Is it disturbed by abnormal light? If the target feature matrix If the abnormal light briefly disturbs the target, the target feature matrix The confidence level obtained If the confidence level of the previous moment k-1 Lower, the target feature matrix in the previous moment k-1 It may still represent the true target. If the confidence of k at the current moment is Still not high, then it is considered that the target feature matrix It is not accurate, and its similarity with the real target feature matrix is ​​low, which affects the update of the historical target feature matrix in subsequent moments. In this case, even if there is no problem in the target tracking process, the confidence is always low when calculating the similarity with the real target features because the similarity between the historical target feature matrix and the real feature matrix is ​​not high. Although this low confidence situation will not affect the tracking accuracy temporarily, once a non-target marker point similar to the disturbed historical target feature matrix appears in the subsequent moment, it may cause tracking errors. Therefore, it is necessary to analyze the confidence change trend based on the statistical marker point confidence set.

[0045] Get the confidence mean square error of k at the current moment , confidence mean square error The expression is: in, is the confidence set mean, and its expression is: By confidence mean square error Represents the historical confidence set The degree of discreteness.

[0046] Step 5: Use the target features of the current moment k to train a new set of classifiers, and the classifier weight matrix and the historical target feature matrix The expression is: in, is the target feature of the marking point of the i-th track at the current time k.

[0047] If the mean square error of the confidence at the current time k and the previous time k-1 is greater than or equal to the set threshold , it is considered that the target is not disturbed by abnormal light in k-1, and the preset fixed classifier weight is used , the classifier parameters at time k and time k-1 are fixedly weighted and fused, and the updated classifier weight matrix of the i-th track at the current time k is and the historical target feature matrix The expression is: If the mean square error of the confidence at the current time k and the previous time k-1 is less than the set threshold , it means that the track of the marked point has maintained a low confidence level for two consecutive times, and it is believed that there was a short-term abnormal light interference around the target at time k-2. The abnormal light is eliminated by reducing the weight of the historical classifier parameter, so the weight parameter is reset ( ) Update the classifier parameters at the current moment k. If the weight parameter When it is 1, it means that the historical classifier parameters are completely discarded and only the current newly trained classifier weights are used. The updated classifier weight matrix of the ith track at the current time k is and the historical target feature matrix The expression is:

[0048] Step 5: loop steps 2 to 4, obtain the position of the marker point at different times in the AVM image, and complete the tracking of the marker point.

[0049] Step 6: In order to verify the feasibility and effectiveness of the method proposed in the present invention, an experiment of tracking marker points under abnormal light interference in AVM images was carried out.

[0050] The experimental parameter settings in this embodiment are shown in Table 1. The experimental data are different data segments containing jump scenes based on the secondary construction of the PSV data set. Some data segments are as follows Figure 2 shown.

[0051] Table 1 Parameter settings of the proposed method parameter value parameter value <![CDATA[s 1 ]]> 40 / pixel <![CDATA[ξ t ]]> 3 / frame <![CDATA[s 2 ]]> 40 / pixel <![CDATA[δ c ]]> 0.1 #timg# 0.05 <![CDATA[δ d ]]> 9 / pixel <![CDATA[β new ]]> 1 <![CDATA[δ e ]]> 30 / pixel In the comparative experiments based on the secondary constructed data segments, the proposed method shows significant differences in target tracking performance compared with the existing KCF (Kernelized Correlation Filters) tracking algorithm. Figure 3 As shown in the figure, the KCF tracking algorithm encountered a tracking error at the fifth moment of the sequence. The root cause of this problem can be traced back to the beginning of the sequence, that is, the first moment, when abnormal light interference appeared above the target marker in the image. This interference was mistakenly included in the historical target feature matrix during the classifier initialization process and was retained as part of the classifier parameters. As the sequence progresses, starting from the second moment, the abnormal light interference gradually disappears, and the KCF algorithm can continue to track the target marker relatively stably. However, at the end of the sequence, a new interference similar to the abnormal light at the first moment appears near the non-target marker above the marker. Since the feature differences between different target markers are subtle and the historical features already contain the "memory" of previous interference, the KCF algorithm mistakenly regards the features of non-target markers as more similar to the historical target feature matrix when extracting features, resulting in tracking errors. In contrast, the method proposed in the present invention effectively solves this problem by introducing a classifier weight adaptive adjustment mechanism for abnormal light interference. At the third moment of the sequence, the method intelligently identifies and adjusts the historical target feature matrix, successfully eliminating the feature components that have deteriorated due to the initial light interference. This adjustment not only optimizes the internal state of the classifier, but also significantly enhances its robustness in the face of similar interference. Therefore, in the subsequent tracking process, even if similar interference factors are encountered again, the method proposed by the present invention can still maintain accurate tracking of the target markers, showing higher stability and reliability.

[0052] In order to ensure the comprehensiveness and effectiveness of the experimental results, the present invention adopts a more rigorous experimental design, conducts independent experiments on multiple segments of secondary structured data, and calculates the average value of multiple segments of data for comprehensive explanation. Figure 4 and Figure 5As shown in the figure, these two examples respectively show the tracking trajectory diagram and the corresponding position error diagram of two specific data segments in the experimental data segment after being disturbed by abnormal light in the target marker initialization stage. These charts intuitively reflect the performance differences of the tracking algorithm in dealing with interference under different data segments. In order to more systematically evaluate the performance of the proposed method and the KCF tracking algorithm, the present invention conducts a comprehensive statistical analysis of the experimental results of all data segments of the secondary construction. The statistical results are summarized in Table 2, which lists in detail various key performance indicators, such as the average error in pixels and the continuous successful tracking rate, and quantitatively compares the performance of the two algorithms on different data segments.

[0053] Table 2 Performance comparison of image tracking algorithms algorithm KCF Proposed method Continuous tracking success rate 84.29% 91.43% Average error / pixel 8.9474 1.7532 Through the data analysis in Table 2, it can be clearly seen that the comprehensive performance of the proposed method on multiple segments of data is significantly better than that of the KCF tracking algorithm. Especially when dealing with complex scenes such as abnormal light interference, the proposed method effectively reduces the impact of interference on tracking performance through its unique classifier weight adaptive adjustment mechanism, and achieves a more stable and accurate tracking effect. This conclusion not only verifies the rationality of the experimental design, but also fully proves the effectiveness and superiority of the proposed method in practical applications.

Claims

1. A method for robustly tracking berth markers under abnormal light interference, characterized in that: The following steps are involved: Step 1: setting target points of multiple berths in the AVM image at the initial moment as initial marking points, and constructing initial classifier parameters; Step 2: Obtain the target feature matrix at the current moment k based on the mark point at the previous moment k-1 ; Using the target feature matrix Get the historical confidence set of the i-th marker track and the position of the mark point at the current time k; where, is the confidence level of k at the current moment; ; n is the number of berths; Step 3: Update the marked point track according to the marked point position at the current time k; Step 4: Train the classifier and perform weighted fusion of the classifier parameters at the current time k and the previous time k-1; according to the historical confidence set Get the confidence mean square error of k at the current moment ; If the confidence mean square error and Both are less than the set threshold , then reset the weight coefficient To reduce the weight of the classifier parameters at the previous moment; otherwise, the preset weight coefficient is used Update classifier parameters; Step 5: loop steps 2 to 4, obtain the positions of the marking points at different times, complete the tracking of the marking points, and realize the trajectory tracking of the berth.

2. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: The classifier parameters are the classifier weight matrix and the historical target feature matrix.

3. The method for robustly tracking berth markers under abnormal light interference according to claim 2 is characterized in that: In step 4, the method of fusing the classifier parameters at the current time k and the previous time k-1 is as follows: If the confidence mean square error and Both are less than the set threshold , then the expression of the classifier parameter of the fused i-th track at the current time k is: in, is the classifier weight matrix; is the historical target feature matrix; If the confidence mean square error and Are greater than or equal to the set threshold , then the classifier weight matrix of the fused i-th track at the current time k is and the historical target feature matrix The expression is: Reset weight factor Greater than the preset weight coefficient .

4. The method for robustly tracking berth markers under abnormal light interference according to claim 3 is characterized in that: In step 4, the classifier weight matrix before fusion and the historical target feature matrix The expression is: in, is the target feature matrix of the marking point of the ith track at the current time k; is the target feature matrix The corresponding feature similarity matrix; and λ are the preset Gaussian labels and regularization parameters respectively.

5. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: In the step 1, the target point is the coordinates of two corner points of the plurality of berths at the initial moment.

6. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: In the step 2, the confidence The method to obtain is as follows: The target feature matrix at the current moment and the historical target feature matrix of the previous moment Perform similarity calculation and obtain Gaussian response matrix ; Using Gaussian response matrix Point multiplication of the classifier weight matrix of the previous moment k-1 , get the output response matrix , take the output response matrix The maximum element in is used as the confidence of k at the current moment .

7. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: In the step 2, the method for obtaining the position of the mark point at the current time k is: by inverse Fourier transforming the output response matrix F k Convert to the image coordinate system and take the position in the AVM image corresponding to the largest element in the output response matrix as the marker point position at the current moment k.

8. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: The target feature matrix The method to obtain is as follows: The target frame is established by the position of the mark point at the previous moment k-1, and the prediction window at the current moment k is obtained after the target frame is expanded. After feature extraction of the prediction window at the current moment k, the target feature matrix at the current moment k is obtained. .

9. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: In step 3, the method for updating the marked point track is as follows: a) If there is a detection mark point obtained by the detection network at the current time k, establish a set of all detection mark points at the current time k ;in, is the position of the i-th detection mark point in the image coordinate system at the current moment k; and are the horizontal and vertical coordinates of the mark point of the ith track at the current time k; Use the current marker position k obtained in step 2 as the tracking marker position to establish a set of all tracking markers ;in, is the position of the jth tracking marker point in the image coordinate system at the current time k; j=1,2,...,2n; like , then use the tracking mark point to update the track information, and the number of track association failures is t i Add 1; where, is the set confidence threshold; if , calculate the detection mark point and track markers The Euclidean distance dis between , If the threshold is set, the detection mark point is used to update the track information; otherwise, the tracking mark point is used to update the track information and the number of track association failures is t i Add 1; if there is a detection mark point that fails to be successfully associated with any existing mark point track, the detection mark point is used as the initial mark point to generate a new track; b) If there is no detection mark point obtained by the detection network at the current time k, the track is updated using the tracking mark point; , and If any of the conditions is met, the number of track association failures is t i Add 1; where δ e is the set drift threshold; According to the number of association failures t i Judge the track of all updated mark points at the current time k; if t i > , the corresponding track is deleted; otherwise, the track is retained; It is the preset association failure threshold.

10. The method for robustly tracking berth markers under abnormal light interference according to claim 1 is characterized in that: In step 4, the confidence mean square error of k at the current moment is The expression is: in, is the confidence set mean.

Images