High-precision tracking method for patrol cars based on echo state network

By adding salt and pepper noise to the image acquisition system of the patrol car and using the Canny edge detection algorithm and echo state network, the accuracy and stability of patrol car tracking in harsh environments are solved, and high-precision tracking in noise environments is achieved .

CN112419345BActive Publication Date: 2024-11-26YANCHENG ZHONGKE HIGH THROUGHPUT COMPUTING RES INST CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202011405929.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-04
Publication Date
2024-11-26
Estimated Expiration
2040-12-04

AI Technical Summary

Technical Problem

In harsh environments, it is difficult for patrol vehicles to achieve high accuracy and stability, especially in the case of large noise interference, and it is difficult for the prior art to effectively detect lane lines and establish an accurate tracking model.

Method used

Using an echo state network-based method, road surface images are collected through a real-time image acquisition system, and salt and pepper noise is added to the image to simulate harsh environments. The Canny edge detection algorithm is used to detect lane lines, and the detected lane line data is input into the echo state network. To establish a nonlinear tracing model, enhance the robustness of the model.

Benefits of technology

It realizes high-precision tracking of patrol vehicles in noise environments, enhances the stability and robustness of tracking, effectively detects lane lines and solves the problem of nonlinear roads, and provides an important technical means to improve The tracking accuracy of the patrol vehicle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112419345B_ABST
    Figure CN112419345B_ABST
Patent Text Reader

Abstract

A high-precision tracking method for patrol cars based on echo state networks. The method comprises the following steps: step 1, collecting patrol car driving image data; step 2, adding salt and pepper noise to the collected image; step 3, detecting lane lines in the image; step 4, establishing a tracking model for patrol cars and lane lines; step 5, collecting abnormal conditions on the patrol route; step 6, turning on the patrol car driving interruption. The present invention simulates the harsh environment during driving, and realizes the patrol car tracking function under the interference of the noisy environment, thereby enhancing the stability, reliability and robustness of the patrol car tracking. The edge line of the lane line is detected by the Canny algorithm, and the collected road surface image can be smoothed and denoised, and the edge line of the lane line is effectively detected, providing important lane line data for the tracking model, and the echo state network solves the nonlinear tracking model of the patrol car, effectively solving the problem of nonlinear road lines.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of patrol car tracking, and in particular to a patrol car high-precision tracking method based on an echo state network. Background Art

[0002] In recent years, with the continuous development of autonomous driving technology, many scholars around the world have spent a lot of manpower, material and financial resources to study autonomous driving technology. With the development of machine learning and deep learning in the fields of image processing, speech recognition, natural language processing, etc., autonomous driving technology can be perfectly combined with artificial intelligence.

[0003] Machine learning and depth can provide a good algorithmic foundation for environmental perception and cognitive decision-making in autonomous driving systems. At the same time, the outstanding performance of artificial intelligence technology in image recognition, lidar point cloud processing, decision planning, and intelligent control has greatly promoted the development speed of autonomous driving technology. Major technology companies and automobile OEMs such as Google, Tesla, Mercedes-Benz, and BMW have successively carried out research and testing of autonomous driving technology. As the investment, technology research and development, and experimental testing of the autonomous driving industry gradually return to rationality, everyone clearly realizes that autonomous driving technology, especially L5-level mass production of autonomous driving vehicles in public road travel scenarios, is much more difficult to achieve than expected; but in the case of specific scenarios and limited speed ranges, combined with 5G, V2X, command and dispatch technologies, a safe, reliable, and efficient solution can be found between the degree of networking and the degree of intelligence.

[0004] Security patrol is one of the best scenarios for the application of intelligent connected patrol vehicles. It has typical characteristics such as high frequency, low speed, rigid demand, and large traffic. At the same time, the technical implementation difficulty is relatively low. Summary of the invention

[0005] To solve the above problems, the real-time image acquisition system of the present invention collects road images in a simulated noise environment, and proposes a patrol car high-precision tracking method based on an echo state network. In order to reduce the impact of environmental noise on the data collected by the sensor as much as possible, the present invention uses the Canny edge detection algorithm to reduce the impact of noise, detect road lines, and enhance the robustness of the trained model. The present invention provides a patrol car high-precision tracking method based on an echo state network, the specific steps are as follows, and it is characterized by:

[0006] Step 1, collecting patrol car driving image data: collecting the road surface image on which the patrol car is driving through the real-time image acquisition system on the patrol car;

[0007] Step 2, adding salt and pepper noise to the collected image: adding salt and pepper noise to the road image obtained in step 1 to simulate the interference of the collected signal in a noisy environment, adding 20% ​​to 30% salt and pepper noise;

[0008] Step 3: Detect lane lines in the image: Use the Canny edge detection algorithm to detect lane lines in the image and segment the lane lines.

[0009] Step 4, establish a tracking model for the patrol car and the lane line: use the lane line value detected in step 3 as the input vector to train the echo state network, and use the echo state network to solve the nonlinear tracking model of the lane line;

[0010] Step 5: Collect abnormal situations on the patrol route: collect emergency situations such as people seeking help and crimes on the patrol route through the real-time image acquisition system;

[0011] Step 6, turn on the patrol car driving interruption. When a patrol abnormality occurs within a safe distance, the system starts the interruption processing mechanism, and the log module records the abnormal situation of the vehicle terminal.

[0012] Furthermore, the process of adding salt and pepper noise to the collected image in step 2 can be expressed as:

[0013] The added salt and pepper noise model is:

[0014]

[0015] Among them, I max and I min is the maximum and minimum value of the image pixel, p is the probability of noise in the image, and the value range of p is 20% to 30%. xy is the actual value of the pixel point (x, y) of the collected image, and f(x, y) is the image pixel value after adding salt and pepper noise.

[0016] Furthermore, the process of detecting lane lines in the image in step 3 can be expressed as:

[0017] Step 3.1 Use the Gaussian smoothing function to smooth the road image and suppress the salt and pepper noise in the image:

[0018] The road surface image can be expressed as f(x,y), and the formula for image smoothing filtering is:

[0019] g(x,y)=h(x,y,σ)*f(x,y) (2)

[0020] Where σ is the Gaussian standard deviation, * represents convolution, g(x,y) is the road image after smoothing filtering, and h(x,y,σ) is:

[0021]

[0022] Step 3.2 calculates the magnitude and direction of the road image gradient by calculating the finite difference of the first-order partial derivative using the Sobel operator:

[0023] d x =g(x,y)*Sobel x (x,y) (4)

[0024] d y =g(x,y)*Sobel y (x,y) (5)

[0025] Where, d x and d y They are the gradients of the image in the x and y directions, Sobel x (x,y) is the Sobel operator in the x direction. y (x, y) is the sobel operator in the y direction. The gradient M(x, y) of the road image can be expressed as:

[0026]

[0027] Gradient direction θ M for:

[0028]

[0029] Step 3.3: Perform non-maximum suppression on the amplitude along the gradient direction calculated in step 3.2;

[0030] Step 3.4 uses a dual threshold algorithm to detect and connect edges: select two thresholds to set the threshold range, set the pixel grayscale value that is less than the threshold range gradient value to 0, mark the points that are greater than the threshold range with a value of 1, connect the contour lines of the lane line, and segment to obtain the edge detection line f1(x, y) of the lane line.

[0031] Furthermore, the process of establishing the patrol car and lane line tracking model in step 4 can be expressed as:

[0032] Step 4.1 Establish a patrol car and lane line tracking model. The patrol car involves left deviation and right deviation during driving. When the patrol car is on the right side of the lane line on both sides, the patrol car needs to deviate to the left. When the patrol car is on the left side of the lane line on both sides, the patrol car needs to deviate to the right. The edge detection line of the training sample road surface image in step 3 is marked with the left and right deviation values ​​E(i);

[0033] Step 4.2 Initialize the network. The edge detection lines of the road image constitute the sample D(i) and the left and right deviations of the patrol car E(i) constitute the training sample u(i) = {D(i), E(i)}. The training sample features D(i) are connected through the input weight matrix W in Entering the reserve pool, E(i) passes through the feedback connection weight W backEnter the reserve pool and collect system status and output status according to the following sequence:

[0034] x(i+1)=f(W in D(i+1)+Wx(i)+W back E(i)) (8)

[0035] E(i+1)=f out (W out D(i+1),x(i+1),E(i)) (9)

[0036] Among them, x(i) is the system parameter with an initial value of 0, f() is the activation function of the reserve pool node, and f out () is the activation function of the output unit of the reserve pool, W represents the connection weight matrix of the neurons inside the reserve pool, and W out represents the output value matrix;

[0037] Step 4.3 Calculate the output value matrix W out :

[0038]

[0039] Among them, K is the number of neurons in the input layer, N is the number of neurons in the reserve pool, L is the number of neurons in the output layer, and λ∈ + represents the regularization factor, and |||| represents the Euclidean distance.

[0040] The echo state network trained in step 4.3 is used as the patrol car tracking route model. The left and right deviations of the patrol car are accurately calculated through the real-time image acquisition system and the trained echo state network to calibrate the patrol car's driving route.

[0041] Furthermore, the process of collecting abnormal conditions on the patrol route in step 5 can be expressed as:

[0042] First, images around the patrol route are collected through the real-time image acquisition system, and the images are identified and classified through the event classification CNN model. If there are people asking for help or illegal crimes, the patrol processing system of the patrol car will order the patrol car to stop, start video evidence, upload event pictures to the cloud, and remind staff to make further interventions.

[0043] The high-precision tracking method for patrol vehicles based on echo state network of the present invention has the following beneficial effects:

[0044] 1. The present invention simulates the harsh driving environment and realizes the patrol car tracking function under the interference of the noise environment, thereby enhancing the stability, reliability and robustness of the patrol car tracking;

[0045] 2. The Canny algorithm of the present invention detects the edge line of the lane line, can smooth and reduce noise on the collected road surface image, effectively detect the edge line of the lane line, and provide important lane line data for the tracking model;

[0046] 3. The present invention solves the nonlinear tracking model of the patrol car through the echo state network, effectively solving the problem of nonlinear road lines;

[0047] 4. The present invention provides an important technical means for patrol vehicles to track with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0050] The present invention proposes a patrol car high-precision tracking method based on echo state network, aiming to improve the patrol car tracking accuracy. At the same time, in order to improve the stability and accuracy of the patrol car tracking solution by the model, salt and pepper noise is added to the collected image to simulate the harsh environment during driving, and the patrol car tracking model is solved by combining the Canny edge detection algorithm and the echo state network. Figure 1 The steps of the present invention are described in detail below in conjunction with the flow chart.

[0051] Step 1, collecting patrol car driving image data: collecting the road surface image on which the patrol car is driving through the real-time image acquisition system on the patrol car;

[0052] Step 2, adding salt and pepper noise to the collected image: adding salt and pepper noise to the road image obtained in step 1 to simulate the interference of the collected signal in a noisy environment, adding 20% ​​to 30% salt and pepper noise;

[0053] The process of adding salt and pepper noise to the collected image in step 2 can be expressed as:

[0054] The added salt and pepper noise model is:

[0055]

[0056] Among them, I max and I min is the maximum and minimum value of the image pixel, p is the probability of noise in the image, and the value range of p is 20% to 30%. xy is the actual value of the pixel point (x, y) of the collected image, and f(x, y) is the image pixel value after adding salt and pepper noise.

[0057] Step 3: Detect lane lines in the image: Use the Canny edge detection algorithm to detect lane lines in the image and segment the lane lines.

[0058] The process of detecting lane lines in the image in step 3 can be expressed as:

[0059] Step 3.1 Use the Gaussian smoothing function to smooth the road image and suppress the salt and pepper noise in the image:

[0060] The road surface image can be expressed as f(x,y), and the formula for image smoothing filtering is:

[0061] g(x,y)=h(x,y,σ)*f(x,y) (2)

[0062] Where σ is the Gaussian standard deviation, * represents convolution, g(x,y) is the road image after smoothing filtering, and h(x,y,σ) is:

[0063]

[0064] Step 3.2 calculates the magnitude and direction of the road image gradient by calculating the finite difference of the first-order partial derivative using the Sobel operator:

[0065] d x =g(x,y)*Sobel x (x,y) (4)

[0066] d y =g(x,y)*Sobel y (x,y) (5)

[0067] Where, d x and d y They are the gradients of the image in the x and y directions, Sobel x (x,y) is the Sobel operator in the x direction. y (x, y) is the sobel operator in the y direction. The gradient M(x, y) of the road image can be expressed as:

[0068]

[0069] Gradient direction θ M for:

[0070]

[0071] Step 3.3: Perform non-maximum suppression on the amplitude along the gradient direction calculated in step 3.2;

[0072] Step 3.4 uses a dual threshold algorithm to detect and connect edges: select two thresholds to set the threshold range, set the pixel grayscale value that is less than the threshold range gradient value to 0, mark the points that are greater than the threshold range with a value of 1, connect the contour lines of the lane line, and segment to obtain the edge detection line f1(x, y) of the lane line.

[0073] Step 4, establish a tracking model for the patrol car and the lane line: use the lane line value detected in step 3 as the input vector to train the echo state network, and use the echo state network to solve the nonlinear tracking model of the lane line;

[0074] The process of establishing the patrol car and lane line tracking model in step 4 can be expressed as:

[0075] Step 4.1 Establish a patrol car and lane line tracking model. The patrol car involves left deviation and right deviation during driving. When the patrol car is on the right side of the lane line on both sides, the patrol car needs to deviate to the left. When the patrol car is on the left side of the lane line on both sides, the patrol car needs to deviate to the right. The edge detection line of the training sample road surface image in step 3 is marked with the left and right deviation values ​​E(i);

[0076] Step 4.2 Initialize the network. The edge detection lines of the road image constitute the sample D(i) and the left and right deviations of the patrol car E(i) constitute the training sample u(i) = {D(i), E(i)}. The training sample features D(i) are connected through the input weight matrix W in Entering the reserve pool, E(i) passes through the feedback connection weight W back Enter the reserve pool and collect system status and output status according to the following sequence:

[0077] x(i+1)=f(W in D(i+1)+Wx(i)+W back E(i)) (8)

[0078] E(i+1)=f out (W out D(i+1),x(i+1),E(i)) (9)

[0079] Among them, x(i) is the system parameter with an initial value of 0, f() is the activation function of the reserve pool node, and f out () is the activation function of the output unit of the reserve pool, W represents the connection weight matrix of the neurons inside the reserve pool, and W out represents the output value matrix;

[0080] Step 4.3 Calculate the output value matrix W out :

[0081]

[0082] Among them, K is the number of neurons in the input layer, N is the number of neurons in the reserve pool, L is the number of neurons in the output layer, and λ∈ + represents the regularization factor, and |||| represents the Euclidean distance.

[0083] The echo state network trained in step 4.3 is used as the patrol car tracking route model. The left and right deviations of the patrol car are accurately calculated through the real-time image acquisition system and the trained echo state network to calibrate the patrol car's driving route.

[0084] Step 5: Collect abnormal situations on the patrol route: collect emergency situations such as people seeking help and crimes on the patrol route through the real-time image acquisition system;

[0085] The process of collecting abnormal conditions on the patrol route in step 5 can be expressed as:

[0086] First, images around the patrol route are collected through the real-time image acquisition system, and the images are identified and classified through the event classification CNN model. If there are people asking for help or illegal crimes, the patrol processing system of the patrol car will order the patrol car to stop, start video evidence, upload event pictures to the cloud, and remind staff to make further interventions.

[0087] Step 6, turn on the patrol car driving interruption. When a patrol abnormality occurs within a safe distance, the system starts the interruption processing mechanism, and the log module records the abnormal situation of the vehicle terminal.

[0088] The echo state network training sample data used in the present invention is processed with salt and pepper noise to simulate the interference of environmental noise, while improving the stability and robustness of the echo state network model, which to a certain extent avoids the overfitting of the model. In addition, when the trained echo state network output produces an error, the echo state network model is fine-tuned.

[0089] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A patrol car high-precision tracking method based on an echo state network, the specific steps are as follows, and the characteristics are as follows: Step 1, collecting patrol car driving image data: collecting the road surface image on which the patrol car is driving through the real-time image acquisition system on the patrol car; Step 2, adding salt and pepper noise to the collected image: adding salt and pepper noise to the road image obtained in step 1 to simulate the interference of the collected signal in a noisy environment, adding 20% ​​to 30% of salt and pepper noise; Step 3: Detect lane lines in the image: Use the Canny edge detection algorithm to detect lane lines in the image and segment the lane lines. Step 4, establish a tracking model for the patrol car and the lane line: use the lane line value detected in step 3 as the input vector to train the echo state network, and use the echo state network to solve the nonlinear tracking model of the lane line; The process of establishing the patrol car and lane line tracking model in step 4 is expressed as: Step 4.1 Establish a patrol car and lane line tracking model. The patrol car involves left deviation and right deviation during driving. When the patrol car is on the right side of the lane line on both sides, the patrol car needs to deviate to the left. When the patrol car is on the left side of the lane line on both sides, the patrol car needs to deviate to the right. The edge detection line of the training sample road surface image in step 3 is marked with the left and right deviation values ​​E(i); Step 4.2 Initialize the network. The edge detection lines of the road image constitute the sample D(i) and the left and right deviations of the patrol car E(i) constitute the training sample u(i) = {D(i), E(i)}. The training sample features D(i) are connected through the input weight matrix W in Entering the reserve pool, E(i) passes through the feedback connection weight W back Enter the reserve pool and collect system status and output status according to the following sequence: x(i+1)=f(W in D(i+1)+Wx(i)+W back E(i)) (2) E(i+1)=f out (W out D(i+1),x(i+1),E(i)) (3) Among them, x(i) is the system parameter with an initial value of 0, f(·) is the activation function of the reserve pool node, and f out (·) is the activation function of the output unit of the reserve pool, W represents the connection weight matrix of the neurons inside the reserve pool, and W out represents the output value matrix; Step 4.3 Calculate the output value matrix W out : Among them, K is the number of neurons in the input layer, N is the number of neurons in the reserve pool, and L is the number of neurons in the output layer. represents the regularization factor, ||·|| represents the Euclidean distance; The echo state network trained in step 4.3 is used as the patrol car tracking route model. The left and right deviation values ​​of the patrol car are accurately calculated through the real-time image acquisition system and the trained echo state network to calibrate the patrol car's driving route. Step 5: Collect abnormal situations on the patrol route: collect requests for help and criminal emergencies on the patrol route through a real-time image acquisition system; Step 6, turn on the patrol car driving interruption. When patrol abnormalities occur within the safe distance, the system starts the interruption processing mechanism, and the log module records the abnormal situation of the vehicle terminal.

2. The patrol car high-precision tracking method based on echo state network according to claim 1 is characterized in that: The process of adding salt and pepper noise to the collected image in step 2 can be expressed as: The added salt and pepper noise model is: Among them, I max and I min is the maximum and minimum value of the image pixel, p is the probability of noise in the image, and the value range of p is 20% to 30%. xy is the actual value of the pixel point (x, y) of the collected image, and f(x, y) is the image pixel value after adding salt and pepper noise.

3. The patrol car high-precision tracking method based on echo state network according to claim 1 is characterized in that: The process of detecting lane lines in the image in step 3 can be expressed as: Step 3.1: Use a Gaussian smoothing function to smooth the road surface image and suppress the salt and pepper noise in the image; Step 3.2 calculates the magnitude and direction of the road surface image gradient by calculating the finite difference of the first-order partial derivative using the Sobel operator; Step 3.3: Perform non-maximum suppression on the amplitude along the gradient direction calculated in step 3.2; Step 3.4 uses a dual threshold algorithm to detect and connect edges: select two thresholds to set the threshold range, set the pixel grayscale value that is less than the threshold range gradient value to 0, mark the points that are greater than the threshold range with a value of 1, connect the contour lines of the lane line, and segment to obtain the edge detection line f1(x, y) of the lane line.

4. The patrol car high-precision tracking method based on echo state network according to claim 1 is characterized in that: The process of collecting abnormal conditions on the patrol route in step 5 can be expressed as: First, images around the patrol route are collected through the real-time image acquisition system, and the images are identified and classified through the event classification CNN model. If there are people asking for help or illegal crimes, the patrol processing system of the patrol car will order the patrol car to stop, start video evidence, upload event pictures to the cloud, and remind staff to make further interventions.

Citation Information

Patent Citations

  • Method for predicting positioning, device, storage medium, and electronic equipment

    CN109462821A

  • Lane departure alarm system

    CN111260957A

  • Lane / object detection and tracking perception system for autonomous vehicles

    US20200026282A1