Transformer substation indoor unmanned aerial vehicle accurate positioning method based on improved LK optical flow

Abstract

The invention discloses a transformer substation indoor unmanned aerial vehicle accurate positioning method based on improved LK optical flow, and the method comprises the steps: carrying out an LK optical flow method based on an optical flow constraint condition, and obtaining an equation set through linear transformation; using a Shii-Tomasi angular point detection algorithm to obtain feature points; establishing an optical flow pyramid model to zoom an indoor large-scale motion image, and approximately converting the indoor large-scale motion image into small-displacement motion; the forward estimation offset is updated through forward and backward error compensation, abnormal feature points are removed, and meanwhile feature points with the good tracking effect are extracted for LK optical flow calculation; according to the method, the feature point extraction efficiency is improved by introducing a corner detection algorithm, the optical flow pyramid model is constructed, and the positioning accuracy is enhanced by utilizing forward and backward error compensation, so that the positioning precision and stability of the unmanned aerial vehicle in a complex indoor environment are improved while the calculation efficiency is ensured, and the method is suitable for popularization and application. The positioning problem of the unmanned aerial vehicle in indoor inspection of the transformer substation is solved, and powerful support is provided for autonomous navigation of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention belongs to the technical field of application of computer vision algorithms in substations, and in particular relates to a method for accurately positioning an indoor unmanned aerial vehicle in a substation based on improved LK optical flow. Background Art

[0002] UAV inspection in substations has gradually become a new application trend. However, the indoor environment of substations is complex and closed, making it difficult to use GPS signals for effective positioning. Traditional indoor positioning methods, such as vision, UWB radio communication, and simultaneous localization and mapping (SLAM) technologies, can achieve indoor positioning of UAVs to a certain extent, but there are problems such as low accuracy, error accumulation, and poor real-time performance. Especially in large spaces such as substations, traditional positioning methods are difficult to meet the needs of UAVs to hover accurately for a long time under the influence of various environmental factors.

[0003] Optical flow is a technique commonly used to calculate motion information in images. The traditional LK optical flow method is widely used in visual positioning due to its high accuracy and simple implementation. However, the LK optical flow method usually performs well under small displacement and stable lighting conditions, but its performance is limited under large displacement or complex lighting environments, and it cannot directly meet the needs of substation indoor drone positioning. In order to improve the applicability of the LK optical flow method, it is necessary to improve it and enhance its positioning ability in complex large-scale environments. Therefore, it is necessary to design a substation indoor drone precise positioning method based on improved LK optical flow to solve the above problems. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for precise positioning of UAVs in substation rooms based on improved LK optical flow. The Shi-Tomasi corner detection algorithm is introduced to improve the efficiency of feature point extraction, an optical flow pyramid model is constructed to approximately convert large displacement motion into small displacement, and forward and backward error compensation is used to enhance the accuracy of positioning. This method improves the positioning accuracy and stability of UAVs in complex indoor environments while ensuring calculation efficiency, solves the positioning problem of UAVs in indoor inspections of substations, and provides strong support for autonomous navigation of UAVs.

[0005] In order to achieve the above technical effects, the technical solution adopted by the present invention is: A method for accurately positioning an indoor UAV in a substation based on improved LK optical flow includes the following steps: S1, execute the LK optical flow method based on the optical flow constraint condition, and obtain the equation set after linear transformation; S2, use Shi-Tomasi corner detection algorithm to obtain feature points to improve the calculation speed of LK optical flow algorithm; S3, build an optical flow pyramid model to scale the indoor large-scale motion image and convert it into a small displacement motion; S4, using forward and backward error compensation to update the forward estimation offset and remove abnormal feature points to improve the positioning accuracy of the LK optical flow algorithm, and at the same time extract feature points with good tracking effect for LK optical flow calculation.

[0006] Furthermore, in step S1, due to the complex indoor environment of the substation, indoor drones cannot be located by GPS signals like outdoor drones. The traditional LK optical flow method is usually used to locate indoor drones. Its principle is to detect the movement of light spots and dark spots in the captured indoor image of the substation to determine the moving speed of the pixel points in the image relative to the drone. If the ground is stationary, the moving speed of the drone relative to the ground can be obtained, and then the specific position of the drone can be obtained by integral positioning.

[0007] Preferably, the calculation process of performing the LK optical flow method based on the optical flow constraint condition in step S1 is: ; (1) In the formula, for x The axial optical flow value is the speed of movement in the horizontal direction; for y The axial optical flow value is the speed of movement in the vertical direction; , and Indicates that the image is x , y , t The partial derivative in the direction can be estimated by the first-order difference of the adjacent image target pixels; Set a specific window W To track the feature points, calculate the SSD value in this window as the target function and find the solution Suppose the SSD value between two frames is , and its calculation formula is as follows: ; (2) In the formula, Display Window W The center coordinates of the window are set to ; is the pixel grayscale value. Taking the derivative of both sides of equation (2) and performing the first-order Taylor expansion, we get: ; (3) make: ;

[0008] ; Substituting into equation (3) we get equation (4): ; (4) when is the minimum value, When it is 0, use formula (5) to calculate the optical flow value; ; (5) in G is the optical flow gradient equation.

[0009] Preferably, in step S2, obtaining feature points using the Shi-Tomasi corner point detection algorithm includes: Since there are a large number of corner points that need to be detected in the acquired substation indoor image, the Shi-Tomasi algorithm is usually used to detect corner points. The principle is to select the corner points by selecting the minimum eigenvalue as the response metric. The calculation process of the Shi-Tomasi corner point detection algorithm is as follows: Select a window to obtain information on the target image and set the window to move , resulting in grayscale changes , we can get formula (6): ; (6) In the formula, x and y is the pixel coordinate within the window; function I is the pixel density function: ; (7) Compare the eigenvalue of the above determinant with the threshold. If it is greater than the set threshold, the point is selected as the ST feature point. In the formula, Indicates the size of the image acquisition sliding window.

[0010] Furthermore, the traditional LK optical flow algorithm is constrained by small displacement, constant brightness and regional consistency when tracking feature points. However, large-scale motion images are collected indoors in the substation. The optical flow pyramid model is usually used to approximate large displacement motion into small displacement motion so that it satisfies the optical flow constraints. The principle is to obtain the transformation matrix and optical flow value of the top layer image through calculation, and pass the calculated value as the initial value to the next layer of image. Each layer of image will calculate a new transformation matrix and optical flow value based on the image value of the previous layer, and pass it down layer by layer until the lowest layer, that is, the original image layer, to obtain the final transformation matrix and optical flow value.

[0011] Preferably, the calculation process of establishing the optical flow pyramid model in step S3 is: ; (8) In the formula,L represents the number of pyramid layers, L The value of is usually an integer.

[0012] Preferably, in step S4, using forward and backward error compensation to update the forward estimation offset and remove abnormal feature points, while extracting feature points with good tracking effect to perform LK optical flow calculation includes: Theoretically, the positions of eigenpoints-prev and eigenpoints-prev-eva should coincide, but due to errors in the optical flow calculation process, there must be a certain deviation between the positions of eigenpoints-prev and eigenpoints-prev-eva. Therefore, error analysis and compensation are introduced to improve positioning accuracy. Error analysis is as follows: Figure 1 As shown, the principle is to estimate the position eigenpoint-tracked of the same feature point in the current frame through the feature point eigenpoints-prev of the previous frame image, and then estimate the position eigenpoints-prev-eva of the previous frame in turn through the feature point eigenpoint-tracked in the current frame.

[0013] Specifically, the calculation process of error compensation is: Assume that the image sequence t The gray value at this moment is , t +1 Moment , using the forward and backward model to establish a new objective function: ; (9) In the formula, and Indicates the offset for forward and backward tracking; For the objective function and By taking partial derivatives and performing first-order Taylor expansion respectively, we can find: ; (10) ; (11) Substituting equations (10) and (11) into equation (9), we can obtain the backward motion offset of the feature point: The values ​​are: ; (12) Finally, update the weighted value of the forward and backward directions according to the offset , the calculation formula is shown in formula (13): ; (13) In the formula, the weighted value Obtained through experiments.

[0014] The beneficial effects of the present invention are as follows: 1. The invention provides a method for accurately positioning UAVs in substations based on improved LK optical flow, which can accurately locate the position of UAVs in substations, and provides ideas for accurate positioning methods of UAVs in substations in actual projects.

[0015] 2. The present invention improves the efficiency of feature point extraction by introducing the Shi-Tomasi corner detection algorithm, constructs an optical flow pyramid model to approximately convert large displacement motion into small displacement, and uses forward and backward error compensation to enhance positioning accuracy. This method improves the positioning accuracy and stability of UAVs in complex indoor environments while ensuring calculation efficiency, solves the positioning problem of UAVs in indoor inspections of substations, and provides strong support for autonomous navigation of UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the flow chart of the improved LK optical flow algorithm of the present invention; Figure 2 is a bidirectional reversible model view of the present invention; Figure 3 It is a schematic diagram of pitch angle information when the drone of the present invention is hovering; Figure 4 It is a schematic diagram of the hovering three-dimensional position information of the UAV of the present invention; Figure 5 This is a schematic diagram of the flight time of the UAV of the present invention; Figure 6 Schematic diagram of the X-axis displacement offset of the drone of the present invention. DETAILED DESCRIPTION

[0017] Embodiment 1: like Figure 1 As shown, a method for accurately positioning a UAV in a substation indoor area based on improved LK optical flow includes the following steps: S1, execute the LK optical flow method based on the optical flow constraint condition, and obtain the equation set after linear transformation; S2, use Shi-Tomasi corner detection algorithm to obtain feature points to improve the calculation speed of LK optical flow algorithm; S3, build an optical flow pyramid model to scale the indoor large-scale motion image and convert it into a small displacement motion; S4, using forward and backward error compensation to update the forward estimation offset and remove abnormal feature points to improve the positioning accuracy of the LK optical flow algorithm, and at the same time extract feature points with good tracking effect for LK optical flow calculation.

[0018] Furthermore, in step S1, due to the complex indoor environment of the substation, indoor drones cannot be located by GPS signals like outdoor drones. The traditional LK optical flow method is usually used to locate indoor drones. Its principle is to detect the movement of light spots and dark spots in the captured indoor image of the substation to determine the moving speed of the pixel points in the image relative to the drone. If the ground is stationary, the moving speed of the drone relative to the ground can be obtained, and then the specific position of the drone can be obtained by integral positioning.

[0019] like Figure 2 As shown, preferably, the calculation process of performing the LK optical flow method based on the optical flow constraint condition in step S1 is: ; (1) In the formula, for x The axial optical flow value is the speed of movement in the horizontal direction; for y The axial optical flow value is the speed of movement in the vertical direction; , and Indicates that the image is x , y , t The partial derivative in the direction can be estimated by the first-order difference of the adjacent image target pixels; Set a specific window W To track the feature points, calculate the SSD value in this window as the target function and find the solution Let the SSD value between two frames be , and its calculation formula is as follows: ; (2) In the formula, Display Window W The center coordinates of the window are set to ; is the pixel grayscale value. Taking the derivative of both sides of equation (2) and performing the first-order Taylor expansion, we get: ; (3) make: ;

[0020] ; Substituting into equation (3) we get equation (4): ; (4) when is the minimum value, When it is 0, use formula (5) to calculate the optical flow value; ; (5) in G is the optical flow gradient equation.

[0021] Preferably, in step S2, obtaining feature points using the Shi-Tomasi corner point detection algorithm includes: Since there are a large number of corner points that need to be detected in the acquired substation indoor image, the Shi-Tomasi algorithm is usually used to detect corner points. The principle is to select the corner points by selecting the minimum eigenvalue as the response metric. The calculation process of the Shi-Tomasi corner point detection algorithm is as follows: Select a window to obtain information on the target image and set the window to move , resulting in grayscale changes , we can get formula (6): ; (6) In the formula, x and y is the pixel coordinate within the window; function I is the pixel density function: ; (7) Compare the eigenvalue of the above determinant with the threshold. If it is greater than the set threshold, the point is selected as the ST feature point. In the formula, Indicates the size of the image acquisition sliding window.

[0022] Furthermore, the traditional LK optical flow algorithm is constrained by small displacement, constant brightness and regional consistency when tracking feature points. However, large-scale motion images are collected indoors in the substation. The optical flow pyramid model is usually used to approximate large displacement motion into small displacement motion so that it satisfies the optical flow constraints. The principle is to obtain the transformation matrix and optical flow value of the top layer image through calculation, and pass the calculated value as the initial value to the next layer of image. Each layer of image will calculate a new transformation matrix and optical flow value based on the image value of the previous layer, and pass it down layer by layer until the lowest layer, that is, the original image layer, to obtain the final transformation matrix and optical flow value.

[0023] Preferably, the calculation process of establishing the optical flow pyramid model in step S3 is: ; (8) In the formula, L represents the number of pyramid layers, L The value of is usually an integer.

[0024] Preferably, in step S4, using forward and backward error compensation to update the forward estimation offset and remove abnormal feature points, while extracting feature points with good tracking effect to perform LK optical flow calculation includes: Theoretically, the positions of eigenpoints-prev and eigenpoints-prev-eva should coincide, but due to errors in the optical flow calculation process, there must be a certain deviation between the positions of eigenpoints-prev and eigenpoints-prev-eva. Therefore, error analysis and compensation are introduced to improve positioning accuracy. Error analysis is as follows: Figure 1 As shown, the principle is to estimate the position eigenpoint-tracked of the same feature point in the current frame through the feature point eigenpoints-prev of the previous frame image, and then estimate the position eigenpoints-prev-eva of the previous frame in turn through the feature point eigenpoint-tracked in the current frame.

[0025] Specifically, the calculation process of error compensation is: Assume that the image sequence t The gray value at this moment is , t +1 Moment , using the forward and backward model to establish a new objective function: ; (9) In the formula, and Indicates the offset for forward and backward tracking; For the objective function and By taking partial derivatives and performing first-order Taylor expansion respectively, we can find: ; (10) ; (11) Substituting equations (10) and (11) into equation (9), we can obtain the backward motion offset of the feature point: The values ​​are: ; (12) Finally, update the weighted value of the forward and backward directions according to the offset , the calculation formula is shown in formula (13): ; (13) In the formula, the weighted value Obtained through experiments.

[0026] Embodiment 2: In this embodiment, a self-designed UAV experimental platform is used to conduct flight experiments. Its specific performance parameters are shown in Table 1: Table 1: Important parameters of the UAV experimental platform

[0027] The feasibility and stability of the algorithm were verified based on actual flight results. The experiment was conducted under natural light and the flight time was greater than 1 minute. The pitch angle information of the drone when hovering was as follows: Figure 3 shown.

[0028] The pitch angle error of the drone is between -3.5° and 1°, which meets the pitch angle stability error of ±3.5° for drone flight attitude stability mentioned in the "China (Shenzhen) Drone Industry Alliance Standard"; at 0~20s, the pitch angle of the drone just took off and there was a large fluctuation. This is because the focal length of the optical flow sensor camera is fixed. At this time, the camera is close to the ground, the collected image is very blurry, and it is difficult to collect feature points. After 20s, the pitch angle error of the drone is between -1° and 1°, which meets the "General Standard for Public Safety Drone Systems" mentioned that the normal range of the pitch angle error during autonomous positioning of drones should be within ±1°, which shows that this method has a good effect on indoor fixed-point hovering of drones.

[0029] Figure 4 , Figure 5 , Figure 6 Indicates the three-dimensional and two-dimensional information of the drone during hovering; Figure 4 As shown in the figure, the UAV will deviate when taking off, mainly because the height is not constant during takeoff, and accurate speed information cannot be obtained. When the flight altitude reaches 1.5m, the UAV can hover stably; Figure 5 As shown in Figure 1, when the UAV's flight altitude reaches 1.5m, there is no large fluctuation in vertical flight, and its error can be stabilized within the range of ±0.35dm; Figure 6 As shown, when the drone is hovering, it can fly stably in a certain area for a period of time, and the positioning effect is good. The error in the xy direction is within the range of ±0.35dm, and its hovering accuracy is approximately a circle with a radius of 35cm.

[0030] The present invention uses a drone as the experimental object, conducts a drone flight experiment in a substation room on a designed experimental platform, and verifies the positioning effect of the improved LK optical flow algorithm in the substation room by analyzing the pitch angle, two-dimensional and three-dimensional information of the drone when it is hovering. The experimental results show that the drone can stably hover in a certain area in the substation room, with an accuracy of a circle with a radius of 35cm and the pitch angle error meets the requirements.

Claims

1. A method for accurate positioning of UAVs in substations based on improved LK optical flow, characterized in that: The following steps are involved: S1, execute the LK optical flow method based on the optical flow constraint condition, and obtain the equation set after linear transformation; S2, use Shi-Tomasi corner detection algorithm to obtain feature points; S3, build an optical flow pyramid model to scale the indoor large-scale motion image and convert it into a small displacement motion; S4, using forward and backward error compensation to update the forward estimation offset and remove abnormal feature points, while extracting feature points with good tracking effect for LK optical flow calculation.

2. According to claim 1, a method for accurately positioning indoor UAVs in substations based on improved LK optical flow is characterized in that: The calculation process of performing the LK optical flow method based on the optical flow constraint condition in step S1 is as follows: ；（1） In the formula, for x The axial optical flow value is the speed of movement in the horizontal direction; for y The axial optical flow value is the speed of movement in the vertical direction; , and Indicates that the image is x , y , t The partial derivative in the direction can be estimated by the first-order difference of the adjacent image target pixels; Set a specific window W To track the feature points, calculate the SSD value in this window as the target function and find the solution Let the SSD value between two frames be , and its calculation formula is as follows: ；（2） In the formula, Display Window W The center coordinates of the window are set to ; is the pixel grayscale value. Taking the derivative of both sides of equation (2) and performing the first-order Taylor expansion, we get: ；（3） make: ； ； Substituting into equation (3) we get equation (4): ；（4） when is the minimum value, When it is 0, use formula (5) to calculate the optical flow value; ；（5） in G is the optical flow gradient equation.

3. According to claim 2, a method for accurately positioning indoor UAVs in substations based on improved LK optical flow is characterized in that: In step S2, the Shi-Tomasi corner detection algorithm is used to obtain feature points, including: The calculation process of the Shi-Tomasi corner detection algorithm is: Select a window to obtain information on the target image and set the window to move , resulting in grayscale changes , we can get formula (6): ；（6） In the formula, x and y is the pixel coordinate within the window; function I is the pixel density function: ；（7） Compare the eigenvalue of the above determinant with the threshold. If it is greater than the set threshold, the point is selected as the ST feature point. In the formula, Indicates the size of the image acquisition sliding window.

4. According to claim 3, a method for accurately positioning indoor UAVs in substations based on improved LK optical flow is characterized in that: The calculation process of establishing the optical flow pyramid model in step S3 is: ；（8） In the formula, L represents the number of pyramid layers, L The value of is usually an integer.

5. According to claim 4, a method for accurately positioning indoor UAVs in substations based on improved LK optical flow is characterized in that: In step S4, the forward estimation offset is updated by using forward and backward error compensation and abnormal feature points are eliminated, and feature points with good tracking effect are extracted for LK optical flow calculation, including: Error compensation calculation: Assume that the image sequence t The gray value at this moment is , t +1 Moment , using the forward and backward model to establish a new objective function: ；（9） In the formula, and Indicates the offset for forward and backward tracking; For the objective function and By taking partial derivatives and performing first-order Taylor expansion respectively, we can find: ；（10） ；（11） Substituting equations (10) and (11) into equation (9), we can obtain the backward motion offset of the feature point: The values ​​are: ；（12） Finally, update the weighted value of the forward and backward directions according to the offset , the calculation formula is shown in formula (13): ；（13） In the formula, the weighted value Obtained through experiments.

Images