Method and apparatus for monitoring flight path guidance parameters in PFD images based on FPGA

Abstract

This invention provides a method and apparatus for monitoring flight path guidance parameters in a PFD (Programmable Graph Display) screen based on FPGA. Using the actual display screen as a basis, binarization is performed. First, a magenta solid circle is identified based on its features. Then, the slope of the magenta line passing through the magenta solid circle is identified to obtain the horizontal flight guidance value. Next, a green circle is identified, and the distance from the center of the green circle to the center of the magenta solid circle is calculated to obtain the vertical flight guidance value. Finally, the offset angle is obtained based on the distance from the magenta solid circle to the perpendicular bisector of the horizon. This invention is based on typical patterns and color weighting methods for identification, which can effectively resist interference from patterns of the same color, improve the recognition rate, increase the mean time between failures (MTBF), and ensure flight safety.

Description

Technical Field

[0001] This invention belongs to the field of graphic monitoring, specifically relating to a method and device for monitoring flight path guidance parameters in a PFD screen based on FPGA. Background Technology

[0002] Digital displays (DPs) are an important component of human-machine interaction in aircraft. DPs receive parameter information from various sensors via an onboard bus and render it on the display screen using a GPU. Pilots obtain various aircraft parameter information through different screens on the DPs. Among them, the PFD screen displays the most important flight parameters, such as airspeed, pressure altitude, pitch angle, roll angle, metric Mach number, and flight path guidance.

[0003] To improve the reliability of digital displays and increase the mean time between failures (MTBF), it is necessary to monitor the display output from the GPU. Furthermore, real-time monitoring of the video data stream output from the GPU can significantly improve the confidence level of the monitoring. Considering the limited processing resources of CPUs and GPUs, the real-time monitoring requirements, and the need for programming flexibility, using FPGAs is a very promising approach.

[0004] The flight path guidance parameters are displayed on a map that moves dynamically within the trackball's range. Therefore, it is necessary to find relevant methods to accurately identify these map views. Summary of the Invention

[0005] This invention provides a method and apparatus for monitoring flight path guidance parameters in a PFD (Programmable Array Display) based on FPGA, enabling monitoring of flight path guidance parameters in a PFD.

[0006] The first aspect of this invention provides a method for monitoring flight path guidance parameters in a PFD (Programmable Array Display) screen based on FPGA, comprising:

[0007] Get the PFD screen at the current moment;

[0008] The PFD screen at the current moment is binarized to obtain a binarized image; the pixels of the first preset color in the binarized image are set to the first preset value, the pixels of the second preset color are set to the second preset value, and the pixels of the remaining colors are set to 0; the first preset color is the color of the symbol of the local machine in the PFD screen; the second preset color is the color of the symbol of the target in the PFD screen.

[0009] Starting from the top left corner of the binarized image, the first preset window is used to traverse the binarized image from left to right and from top to bottom. The sum of the values ​​of all pixels within the first preset mask area of ​​the first preset window is obtained. The center position corresponding to the largest sum value that is greater than the first preset threshold is taken as the center point coordinate of the local symbol. Based on all pixels in the binarized image with values ​​of the first preset value and the center point coordinate of the local symbol, the horizontal flight path is obtained.

[0010] Starting from the top left corner of the binarized image, the second preset window is used to traverse the binarized image from left to right and from top to bottom. The sum of the number of all pixels with values ​​of the second preset value within the second preset mask area of ​​the second preset window is obtained. The center position corresponding to the maximum sum of the number of pixels with values ​​greater than the second preset threshold is taken as the center point coordinate of the target symbol. The vertical flight path is obtained based on the center point coordinates of the target symbol and the center point coordinates of the local symbol.

[0011] Based on the roll angle of the aircraft and the coordinates of the origin of the aircraft in the binarized image, the expression of the perpendicular bisector of the horizon in the binarized image is obtained; the shortest distance between the center point of the aircraft symbol and the perpendicular bisector of the horizon in the binarized image is obtained, and the flight guidance yaw angle is obtained based on the shortest distance;

[0012] Based on the received horizontal flight path, vertical flight path, and flight guidance yaw angle values, the horizontal flight path, vertical flight path, and flight guidance yaw angle in the PFD screen are monitored.

[0013] Optionally, the first preset color is magenta or reddish magenta, and the second preset color is green; the PFD image at the current moment is binarized to obtain a binarized image, including:

[0014] Iterate through the values ​​of all pixels in the PFD screen at the current moment. If a pixel's value meets the following conditions: red component greater than or equal to 0xEB, green component within the range of 0x23 to 0x41, and blue component greater than or equal to 0xEB, then set the pixel value to the first preset value. If a pixel's value meets the following conditions: red component within the range of 0x5A to 0x6E, green component greater than or equal to 0xE6, and blue component less than or equal to 0x14, then set the pixel value to the second preset value. Otherwise, set the pixel value to 0.

[0015] Optionally, the first preset value is K, and the second preset value is 1; K is the weighting multiple.

[0016] Optionally, the first preset mask is a solid circle.

[0017] Optionally, based on all pixels in the binarized image that have a value of the first preset value, and the coordinates of the center point of the local symbol, the horizontal flight path is obtained, including:

[0018] Based on all pixels in the binarized image that take the first preset value, and the coordinates of the center point of the local symbol, the least squares fitting principle is used, employing the formula... Obtain the local sign slope b;

[0019] Determine the horizontal flight path based on the slope of the aircraft symbol;

[0020] Where, x i The difference in x-coordinate between the pixel with a value of the first preset value and the center point of the local symbol, and the difference in y-coordinate between the pixel with a value of the first preset value and the center point of the local symbol. i y is the difference between the y-coordinate of a pixel with a value of the first preset value and the center point of the local symbol, and n is the number of pixels with a value of the first preset value.

[0021] Based on the slope of the local symbol, the FD horizontal angle is obtained by searching the ROLL_ATAN_VAL_ROM module, which serves as the horizontal flight path.

[0022] Optionally, the second preset mask is two symmetrical arc segments on the same ring.

[0023] Optionally, based on the center point coordinates of the target symbol and the center point coordinates of the machine symbol, the vertical flight path is obtained, including:

[0024] Obtain the distance between the center point of the target symbol and the center point of the local symbol;

[0025] Obtain the vertical angle value based on the distance and the preset distance-angle conversion coefficient;

[0026] The sign position of the vertical angle value is determined based on the positional relationship between the center point coordinates of the target symbol and the center point coordinates of the local symbol;

[0027] The vertical flight path is determined based on the sign bit and the vertical angle value.

[0028] A second aspect of the present invention provides a flight path guidance parameter monitoring device for a PFD screen implemented based on FPGA, for performing the method as described in any one of the first aspects.

[0029] This invention provides a method and apparatus for monitoring flight path guidance parameters in a PFD (Programmable Graph Display) screen based on FPGA. Using the actual display screen as a basis, binarization is performed. First, a magenta solid circle is identified based on its features. Then, the slope of the magenta line passing through the magenta solid circle is identified to obtain the horizontal flight guidance value. Next, a green circle is identified, and the distance from the center of the green circle to the center of the magenta solid circle is calculated to obtain the vertical flight guidance value. Finally, the offset angle is obtained based on the distance from the magenta solid circle to the perpendicular bisector of the horizon. This invention is based on typical patterns and color weighting methods for identification, which can effectively resist interference from patterns of the same color, improve the recognition rate, increase the mean time between failures (MTBF), and ensure flight safety. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the flight path guidance display in the PFD screen of the present invention;

[0031] Figure 2 A schematic diagram showing a solid magenta circle obscured by green.

[0032] Figure 3 Match the template for magenta;

[0033] Figure 4 The scene is displayed as a green circle;

[0034] Figure 5 Green circular template;

[0035] Figure 6 This is a diagram showing the distance between the magenta circle and the central angle. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.

[0038] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0039] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0040] In this invention, under actual operating conditions, based on the display characteristics of the flight path guidance, the angle between the short line on the solid magenta sphere and the horizontal line represents the horizontal flight path (lateral FD), the length of the line segment connecting the center of the solid magenta sphere and the center of the hollow green circle represents the vertical flight path (vertical FD), and the distance from the solid magenta sphere to the perpendicular bisector of the horizon represents the drift angle. The origin is the upper left corner, the horizontal line to the right is the positive x-axis, and the vertical line downwards is the positive y-axis. The flight path guidance in the PFD is displayed as follows: Figure 1 As shown.

[0041] The flight path guide contains two symbols:

[0042] 1) FPV symbol: green circle;

[0043] 2) FPD symbol: A solid magenta circle with a line passing through the center and the apexes of two hollow triangles connected to the endpoints of the line.

[0044] Vertical FD:

[0045] The distance from the center of the FPV symbol to the center of the FPD symbol is the vertical FD, calculated using the following formula. If there is a solid magenta circle above the green circle, the vertical FD is positive; otherwise, the vertical FD is negative.

[0046] |Vertical FD|=the distance×10 / 91

[0047] Level FD:

[0048] The angle between the magenta line of the FPD symbol and the pfd_coor_x-axis is the horizontal FD. If the angle between the positive direction of pfd_coor_x-axis and the magenta line of the FPD symbol is acute, then the horizontal FD is negative; otherwise, the horizontal FD is positive.

[0049] Yaw angle:

[0050] The distance from the center of the FPD symbol to the perpendicular bisector of the skyline is the yaw angle, which is calculated using the following formula. If the solid magenta circle is to the right of the perpendicular bisector of the skyline, the yaw angle is positive; otherwise, the yaw angle is negative.

[0051] |Drift Angle|=the distance×10 / 91

[0052] The vertical FD angle is invalid when neither the magenta circle nor the green circle is detected. The vertical FD, lateral FD, and yaw angle are all invalid when the magenta circle is not detected.

[0053] The implementation process of this invention is as follows:

[0054] In this invention, the FPGA performs flight guidance recognition based on the coordinate system, binarized values, and trigonometric function values ​​of the roll angle provided by other modules, and outputs the recognition results.

[0055] A) Binarization module

[0056] Binarization converts a 24-bit wide RGB data stream into a double-bit data stream, thereby saving resources and simplifying subsequent data extraction, template matching and other processing steps.

[0057] Image binarization is essentially a filtering algorithm that filters out useless background data and extracts the necessary feature data. There are various binarization algorithms, and in practice, different algorithms should be selected based on different application scenarios.

[0058] This article takes common threshold-based binarization and special color extraction-based binarization as examples to briefly introduce their logical implementation principles.

[0059] In the input 24-bit RGB data, the high 8 bits represent the red component, the middle 8 bits represent the green component, and the low 8 bits represent the blue component.

[0060]

[0061] Then, the binarized value is input into the flight guidance recognition module.

[0062] B) Coordinate Generation Module

[0063] The coordinate generation module calculates the position coordinates of each incoming pixel within the entire image based on the video horizontal and vertical synchronization signals and the valid data signal. Generally, the top-left corner of the image is chosen as the origin. Taking an 800×600 image as an example, the correspondence between its coordinates and pixel points is as follows: Figure 3 As shown.

[0064] The X-axis coordinate, coor_x, is generated based on the valid video data signal DE and the pixel clock CLK. The coor_x counter is started on the rising edge of DE, and while DE is high, coor_x is incremented by 1 on each rising edge of CLK and cleared on the falling edge of DE, repeating this cycle. For 800×600 video data, with 800 pixels per line, the range of coor_x is [0, 799].

[0065] The Y-coordinate, coor_y, is started and cleared to zero at the beginning of each video frame, i.e., at the rising edge of the VS signal. Then, at the falling edge of each DE signal, coor_y is incremented by 1 until it is cleared to zero at the rising edge of the VS signal of the next video frame, and this cycle repeats. For 800×600 video data, with a total of 600 lines of valid data, the range of coor_y is [0, 599].

[0066] C) Roll angle calculation

[0067] This invention directly uses the trigonometric function of the roll angle calculated by other modules.

[0068] D) Preprocessing module (PFD_RPEPROCESS)

[0069] The FPD_PREPROCESS module implements a pipeline buffer for 19 lines of video data. This allows a single clock cycle to output 20 lines of video data for computation. Simultaneously, it generates start signals for matching green and magenta circles, as well as start signals for calculating the horizontal FD angle, vertical FD angle, and yaw angle, based on the input pixel coordinates.

[0070] E) Calculation of magenta solid circles

[0071] The MAGENTA_CIRCLE_MATCH module matches magenta solid circles and outputs the coordinates of the center point of the magenta solid circle and a match completion flag. The magenta solid circle matching algorithm is a weighted matching algorithm. In this algorithm, 15 rows of image data are input in parallel and accumulated in parallel. The matching algorithm only calculates pixels within the magenta solid circle; pixels outside the magenta solid circle are not included in the calculation range. Therefore, a complete solid circle contains a total of 200 points.

[0072] The meaning of weight is that the weight coefficient of each pixel is different, as shown in the table below.

[0073]

[0074]

[0075] Therefore, a complete solid circle contains 200 points, and the maximum sum of its weighted values ​​is 400. When the magenta circle is obscured (see...), Figure 2 If the sum of the weighted values ​​of the magenta solid circles is greater than 280, then a magenta solid circle is considered to have matched successfully. By comparing the weighted values ​​of the magenta solid circles, the optimal matching result and the coordinates of the magenta solid circles can be found.

[0076] In actual design, according to Figure 3 The template performs real-time scanning and accumulation of 15 rows of parallel input data, obtaining the sum of the numerical information in each column of the 15 rows. If the sum of the numerical information of the current magenta solid circle is greater than the sum of the numerical information of the previously recorded magenta solid circle, the numerical information and center point coordinates of the current magenta solid circle are recorded. If the sum is less than or equal to the previous record, the recorded numerical information and coordinates will not be replaced or updated. (Note: The center point coordinates of the magenta solid circle are located in the 8th row and 8th column of the 15 rows of data).

[0077] This module calculates the coordinates of the center point of the magenta solid circle based on the coordinates of the bottom left pixel ram_0_dout(row_16,col_16) of the current solid circle within the cropped area. The radius of the magenta solid circle is 7, therefore the coordinates of the center point of the magenta solid circle are (row_16-7,col_16+7).

[0078] F) Green Circle Recognition

[0079] The GREEN_CIRCLE_MATCH module matches the green circle and outputs the coordinates of the center point of the green circle and a match completion flag.

[0080] The matching algorithm for the green circle is a texture shape matching algorithm. Sometimes the green circle background has green pixel interference, such as... Figure 4 As shown. To improve the matching rate, only the positions of green pixels in the 20 parallel rows of input data are accumulated. Pixels outside the green circular area are not included in the calculation. The 20 rows of pixels for a complete green circle contain a total of 120 points. If 100 points are detected, a green circle is considered to exist; the matching template is based on the FPV symbol ( Figure 5When the number of green pixels in each frame of the FPV symbol is equal to or greater than 100, a green circle is considered to have been identified. When the image data is valid, this module calculates the coordinates of the center point of the green circle based on the coordinates of the lower left pixel ram_0_dout(row_16,col_16) of the current calculation circle in the cropped area. The radius of the green circle is 11, therefore the coordinates of the center point of the magenta solid circle are (row_16-10,col_16+11).

[0081] G) Horizontal Flight Path (Lateral FD) Calculation

[0082] The coordinates (row and column counts) of valid magenta pixels are stored in a FIFO. The FIFO is 1024 bits deep and 32 bits wide, with [27:16] storing column coordinates and [11:0] storing row coordinates. Pixels that are invalid or whose color is not magenta will not be stored in the FIFO.

[0083] Once the coordinates of the center point of the magenta circle are obtained and the magenta circle is deemed valid, the module begins reading the coordinates of the magenta pixels from the FIFO. If the center point of the magenta circle is not detected, the following calculations will not be performed.

[0084] The module then determines whether the coordinates of the magenta pixel are valid pixels for line detection and calculates the slope of the magenta line based on the least squares fitting principle (linear regression model). The formula for calculating the slope of the magenta line is as follows:

[0085]

[0086] Where: x i The difference on the x-axis (row) between the magenta pixel and the magenta solid circle, (y i ) represents the difference along the y(col) axis between the magenta pixel and the solid magenta circle. n is the numerical sum of all magenta pixels.

[0087] Based on the slope of the straight line, the FD horizontal angle is obtained by searching the ROLL_ATAN_VAL_ROM module, which serves as the horizontal flight path.

[0088] H) Vertical Flight Path (Vertical FD) Calculation

[0089] When the center point of the magenta or green circle is detected, the difference between the x-axis and y-axis values ​​of the straight-line distance between the center points of the magenta solid circle and the green circle is obtained:

[0090] Δx=x magenta -x green

[0091] Δy=y magenta -y green

[0092] After obtaining the difference between the x-axis and y-axis, the squared distance d can be obtained using a multiplier. 2 ,

[0093] d 2 =Δx 2 +Δy 2

[0094] Use the SQRT_INT_FRACTION module to find the root d.

[0095]

[0096] In the hardware design, the distance d is 16 bits of unsigned data, with the last 2 bits being the decimal part.

[0097] After obtaining the distance d, the perpendicular angle value is calculated using a multiplier:

[0098] Vertical_FD = d × 10 / 91

[0099] 10 / 91 is a distance / angle conversion method. Multiplying the distance d by 10 / 91 converts it to an angle value, where 10 represents 10 degrees and 91 represents 91 pixels on the screen.

[0100] The sign of the angle value is determined by the positional relationship (row direction) between the magenta and green circles. If the magenta circle is above the green circle, the angle value is positive; otherwise, the angle value is negative.

[0101] I) Yaw angle calculation

[0102] Based on the tan value of the roll angle +90° and the coordinates of the center point, the expression for the perpendicular bisector of the horizon line is obtained.

[0103] Based on the coordinates of the center of the magenta circle, calculate the shortest distance from the center of the solid magenta circle to the perpendicular bisector of the horizon. After obtaining the shortest distance, convert it into an angle value for output, namely the FD yaw angle.

[0104] See Figure 6 The angle of the centerline is equal to the roll angle plus 90 degrees. Once the coordinates of the center point and the sin and cos values ​​of the straight-line angle are obtained, the module can calculate the coordinates of any point on the centerline. Therefore, the module can calculate the distance between each point on the centerline and the magenta solid circle, and record the shortest distance. The shortest distance between two points is the distance from the magenta circle to the centerline of the scale. Multiplying this distance by a constant gives the yaw angle.

[0105] The above description is merely a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for monitoring flight path guidance parameters in a PFD (Programmable Array Display) based on FPGA, characterized in that, include: Get the PFD screen at the current moment; The PFD image at the current moment is binarized to obtain a binarized image; In the binarized image, the pixel value of the first preset color is the first preset value, the pixel value of the second preset color is the second preset value, and the pixel value of the remaining colors is 0; the first preset color is the color of the symbol of the local device in the PFD screen; The second preset color is the color of the symbol of the target in the PFD image; Starting from the top left corner of the binarized image, the first preset window is used to traverse the binarized image from left to right and from top to bottom. The sum of the values ​​of all pixels within the first preset mask area of ​​the first preset window is obtained. The center position corresponding to the largest sum value that is greater than the first preset threshold is taken as the center point coordinate of the local symbol. Based on all pixels in the binarized image with values ​​of the first preset value and the center point coordinate of the local symbol, the horizontal flight path is obtained. Starting from the top left corner of the binarized image, the second preset window is used to traverse the binarized image from left to right and from top to bottom. The sum of the number of all pixels with values ​​of the second preset value within the second preset mask area of ​​the second preset window is obtained. The center position corresponding to the maximum sum of the number of pixels with values ​​greater than the second preset threshold is taken as the center point coordinate of the target symbol. The vertical flight path is obtained based on the center point coordinates of the target symbol and the center point coordinates of the local symbol. Based on the roll angle of the aircraft and the coordinates of the origin of the aircraft in the binarized image, the expression of the perpendicular bisector of the horizon in the binarized image is obtained; the shortest distance between the center point of the aircraft symbol and the perpendicular bisector of the horizon in the binarized image is obtained, and the flight guidance yaw angle is obtained based on the shortest distance; Based on the received horizontal flight path, vertical flight path, and flight guidance yaw angle values, the horizontal flight path, vertical flight path, and flight guidance yaw angle in the PFD screen are monitored.

2. The method for monitoring flight path guidance parameters in a PFD image based on FPGA implementation according to claim 1, characterized in that, The first preset color is magenta or crimson, and the second preset color is green; The PFD image at the current moment is binarized to obtain a binarized image, including: Iterate through the values ​​of all pixels in the PFD screen at the current moment. If a pixel's value meets the following conditions: red component greater than or equal to 0xEB, green component within the range of 0x23 to 0x41, and blue component greater than or equal to 0xEB, then set the pixel value to the first preset value. If a pixel's value meets the following conditions: red component within the range of 0x5A to 0x6E, green component greater than or equal to 0xE6, and blue component less than or equal to 0x14, then set the pixel value to the second preset value. Otherwise, set the pixel value to 0.

3. The method for monitoring flight path guidance parameters in a PFD image based on FPGA implementation according to claim 2, characterized in that, The first preset value is K, and the second preset value is 1; K is the weighting factor.

4. The method for monitoring flight path guidance parameters in a PFD image based on FPGA implementation according to claim 3, characterized in that, The first preset mask is a solid circle.

5. The method for monitoring flight path guidance parameters in a PFD image based on FPGA according to claim 4, characterized in that, Based on all pixels in the binarized image with values ​​of the first preset value, and the coordinates of the center point of the local symbol, the horizontal flight path is obtained, including: Based on all pixels in the binarized image that take the first preset value, and the coordinates of the center point of the local symbol, the least squares fitting principle is used, employing the formula... Obtain the local sign slope b; Determine the horizontal flight path based on the slope of the aircraft symbol; Where, x i The difference in x-coordinate between the pixel with a value of the first preset value and the center point of the local symbol, and the difference in y-coordinate between the pixel with a value of the first preset value and the center point of the local symbol. i The value is the difference between the y-coordinate of the pixel with the value of the first preset value and the center point of the local symbol, and n is the number of pixels with the value of the first preset value. Based on the slope of the local symbol, the FD horizontal angle is obtained by searching the ROLL_ATAN_VAL_ROM module, which serves as the horizontal flight path.

6. The method for monitoring flight path guidance parameters in a PFD image based on FPGA according to claim 3, characterized in that, The second preset mask consists of two symmetrical arc segments on the same circular ring.

7. The method for monitoring flight path guidance parameters in a PFD image based on FPGA implementation according to claim 6, characterized in that, Based on the center point coordinates of the target symbol and the center point coordinates of the machine symbol, the vertical flight path is obtained, including: Obtain the distance between the center point of the target symbol and the center point of the local symbol; Obtain the vertical angle value based on the distance and the preset distance-angle conversion coefficient; The sign position of the vertical angle value is determined based on the positional relationship between the center point coordinates of the target symbol and the center point coordinates of the local symbol; The vertical flight path is determined based on the sign bit and the vertical angle value.

8. A device for monitoring flight path guidance parameters in a PFD (Programmable FPGA) display based on FPGA, characterized in that, Used to perform the method as described in any one of claims 1-7.

Images