Aircraft instrument panel pointer reading method based on target detection and key point recognition
By constructing an instrument panel image dataset and training and optimizing it using the YOLOv11-Pose model, combined with numerical mapping algorithms, the accuracy and efficiency issues of automated instrument panel testing in existing technologies have been resolved. This ensures efficient and accurate detection of aviation instrument panels, indirectly safeguarding flight safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANCHANG HANGKONG UNIVERSITY
- Filing Date
- 2026-02-09
- Publication Date
- 2026-06-26
AI Technical Summary
In existing technologies, automated testing methods for aviation instrument panels cannot accurately identify key points and convert them into numerical values, resulting in high computational resource consumption. This makes them unsuitable for embedded or real-time testing scenarios, and manual reading is prone to errors, affecting flight safety.
An aviation instrument panel image dataset was constructed, trained using the YOLOv11-Pose model, and optimized through sparse training and Slim pruning. Combined with a numerical mapping algorithm, accurate detection and reading conversion of instrument pointers were achieved. Sparse training and Slim pruning methods were used to optimize computational resources, and a differentiated algorithm was adopted to process different types of instruments.
It enables accurate target detection and key point identification for different types of instrument panels, reduces computing resource consumption, improves detection efficiency and accuracy, and ensures the reliability of aviation equipment and flight safety.
Smart Images
Figure CN121686426B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automated testing technology for aviation instruments, and in particular to a method for reading pointers on aviation instrument panels based on target detection and key point identification. Background Technology
[0002] As the core visual terminal for aircraft flight status, the accuracy of the data displayed by the aircraft instrument panel is directly related to the crew's operational decisions and flight safety. Therefore, it is necessary to continuously verify the reliability of the instrument display through reliable testing methods.
[0003] In aviation equipment operation and maintenance scenarios, the real-world data in the background originates from raw physical quantity data (such as oil pressure, speed, and voltage) collected by core components like the aircraft flight control system and sensors. This data serves as the "data source benchmark" for instrument displays, providing objective and accurate physical quantity references rather than directly providing visual interactive information to the crew. Instrument readings are typically manual readings of values from the physical display terminals. The core purpose is to reproduce the actual display results presented to the crew and verify the consistency between the "data source benchmark" and the "terminal display." However, manual readings can introduce errors. While some automated testing solutions based on image processing exist, most are limited to simple target detection and cannot accurately identify key instrument points and convert them into numerical values. Furthermore, these methods often neglect model optimization, resulting in high computational resource consumption and making them unsuitable for embedded or real-time testing scenarios.
[0004] To address the aforementioned issues, there is an urgent need for an efficient and accurate automated testing method that can combine deep learning models to achieve key point mapping and numerical transformation, thereby improving testing accuracy and efficiency and ensuring the reliability of aviation equipment and flight safety. Summary of the Invention
[0005] The purpose of this invention is to provide a method for reading pointers on an aviation instrument panel based on target detection and key point recognition, so as to improve the problems existing in the prior art.
[0006] This invention is implemented as follows: a method for reading pointers on an aviation instrument panel based on target detection and key point recognition, comprising the following steps:
[0007] S1: Construct an image dataset of aviation instrument panels and annotate the rectangular detection boxes and key points of instrument pointers in the images. The aviation instruments include bar-shaped instruments and pointer instruments. The YOLOv11-Pose model is used for training. After training, the model is optimized using sparse training and Slim pruning methods to obtain the optimized instrument recognition model.
[0008] S2: Use the optimized instrument recognition model to recognize the image of the aircraft instrument panel under test, and output the coordinates of the rectangular detection box of the instrument and the coordinates of the key points of the instrument pointer;
[0009] S3: Based on the identified coordinates of key points of the instrument pointer, the coordinates of key points of the instrument pointer are converted into specific reading values of the instrument through a numerical mapping algorithm;
[0010] S4: Compare the mapped instrument readings with the actual data in the background to determine whether the instrument display is correct, thus achieving automated detection of the instrument display.
[0011] More preferably, the specific implementation of the Slim pruning method in step S1 includes: evaluating the importance of the γ value of the BN layer, adding L1 regularization to the loss function to make γ tend to zero; setting a sparsity parameter to reduce the computational cost by 50% and maintain or improve the model accuracy; skipping the detection head and feature fusion layer during the pruning process to avoid accuracy loss.
[0012] More preferably, in step S2, the optimized instrument recognition model outputs differentiated key point information for different types of instruments:
[0013] For pointer-type instruments, the output instrument rectangular detection frame and the coordinates of four key points of the instrument pointer are: the center point A of the dial, the tip point B of the pointer, the smallest scale point C, and the largest scale point D.
[0014] For columnar instruments, the output instrument rectangular detection frame and the coordinates of one instrument pointer key point are provided, where the key point is the pointer position point.
[0015] More preferably, in step S3, the numerical mapping algorithm uses different calculation methods according to the instrument type: the angle method is used to calculate the reading for pointer instruments, and the ratio method is used to calculate the reading for bar-type instruments.
[0016] More preferably, the calculation process of the angle method includes the following steps:
[0017] S301: Obtain the center point A of the dial, the tip point B of the pointer, the smallest scale point C, and the largest scale point D, and define a vector. , , , ,in for The opposite direction vector;
[0018] S302: Calculate vectors sum vector The cross product scalar value is used to determine the pointer direction, vector The coordinate components are ,vector The coordinate components are The formula for calculating the scalar value of the cross product is: ;
[0019] in, The cross product of vectors is represented by its sign, which is used to determine the direction:
[0020] when At that time, the pointer rotates counterclockwise;
[0021] when At that time, the pointer rotates clockwise;
[0022] when At that time, the pointer and the vector are collinear;
[0023] S303: Calculate the reading proportionality coefficient based on the cross product result. :
[0024] when hour, ,in For vectors with vector The included angle, Preset the reference angle corresponding to the range of the instrument panel;
[0025] when hour, ,in For vectors and The included angle;
[0026] when hour, ;
[0027] S304: Multiply the proportional coefficient pre by the preset range of the pointer instrument to obtain the final reading.
[0028] More preferably, the calculation process of the ratio method includes the following steps:
[0029] The height of the regression prediction box for the bar graph instrument and the relative height of the pointer feature points relative to the lower boundary of the prediction box are obtained by recognizing the model; the pointer display ratio is calculated by combining the error compensation value, and the calculation formula is as follows: The error compensation value is the average systematic error obtained by statistical analysis of multiple test samples; the ratio is multiplied by the range of the columnar instrument to obtain the final reading.
[0030] More preferably, the parameters for model training in step S1 are set as follows: 300 training epochs, batch size 42, initial learning rate 0.01, Adam optimizer, loss function combining bounding box regression loss, classification loss and key point localization loss; data augmentation strategies include Mosaic stitching, RandAugment automatic augmentation and random erasure; and NMS post-processing of prediction results with an IoU threshold of 0.7 is used.
[0031] More preferably, the criteria for determining whether the instrument display is correct in step S4 are as follows: when the difference between the reading of the pointer instrument and the actual data in the background is ≥5, the pointer instrument is determined to be displaying incorrectly; when the difference between the reading of the bar instrument and the actual data in the background is ≥5, the bar instrument is determined to be displaying incorrectly; after the error is determined, a screenshot of the erroneous instrument is taken and saved locally.
[0032] Compared with the prior art, the present invention has the following advantages:
[0033] (1) This invention achieves accurate target detection and key point recognition of different types of aircraft instrument panels through the pruned instrument recognition model. The output position information and key point coordinates provide reliable data support for subsequent reading calculations, avoiding the subjectivity and error of manual recognition.
[0034] (2) Different numerical mapping algorithms are adopted for different types of instruments to ensure the accuracy of reading conversion. Combined with the comparison and judgment of real data in the background, the automatic detection of instrument panel display is realized, which significantly improves the detection efficiency, provides efficient technical means for aviation equipment maintenance, and indirectly ensures flight safety. Attached Figure Description
[0035] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but does not exclude other elements or objects.
[0037] Example 1: This example provides a method for reading aircraft instrument panel pointers based on target detection and key point recognition, such as... Figure 1 As shown, it includes the following steps:
[0038] Step S1: Data Acquisition and Model Optimization;
[0039] Data Acquisition: Prepare video data of aviation instrument panels, extract video frame images as a dataset, and annotate the images. The annotation scope includes the rectangular detection boxes of instrument objects and key points of instrument pointers. Aviation instrument panels include bar-type instruments and pointer-type instruments. For pointer-type instruments, annotate the rectangular detection boxes and mark four key points: pointer start point (center point A), pointer end point (tip B), minimum scale point C, and maximum scale point D. For bar-type instruments, annotate the rectangular detection boxes and mark one key point: pointer position point. This annotation ensures that the key points accurately correspond to the positions of instrument components to support subsequent numerical mapping calculations. The annotated dataset is divided into a training set and a validation set in an 8:2 ratio. The training set is used for the model to learn the localization rules of instrument detection boxes and key points, while the validation set is used to independently evaluate the model's generalization ability and avoid overfitting.
[0040] Model Training: YOLOv11-Pose was selected as the single-stage object detection and keypoint recognition model. This model inherits the real-time advantages of the YOLO series and improves robustness to small targets and complex backgrounds through cross-layer feature fusion and attention mechanisms. Before training, the resolution of the input images was uniformly scaled to 640×640 pixels, and then the model was trained using the training set data. The training parameters were set as follows: 300 training epochs, 42 batch size, initial learning rate of 0.01, Adam optimizer, and loss function combining bounding box regression loss, classification loss, and keypoint localization. Loss; Data augmentation strategies include Mosaic (randomly stitching images to enhance diversity), RandAugment (automatic random augmentation, such as rotation, scaling, and brightness adjustment), and random erasure (simulating occlusion); In addition, NMS (non-maximum suppression) is used to post-process the prediction results, and redundant detection boxes are filtered by NMS post-processing with an IoU threshold of 0.7. During training, the model iteratively optimizes the loss function based on the training set. After each training round, the model performance is evaluated using the validation set. By monitoring the trend of the loss curves of the training set and the validation set, it is ensured that the model converges without overfitting, and the validation set mAP index reaches 0.997.
[0041] Model optimization: Sparse training and Slim pruning methods are introduced to optimize the computational resource requirements of the YOLOv11-Pose model and enable deployment on edge devices. Sparse training is a technique that induces parameter sparsity during model training. By adding a regularization term to the loss function, some weights or channels in the network are made to tend to zero, thereby reducing redundant parameters and improving model efficiency. The advantage of sparse training is that it improves the model's generalization ability and provides a basis for subsequent pruning, avoiding the accuracy loss caused by direct pruning.
[0042] The Slim pruning method is a channel-level structured pruning algorithm specifically designed for convolutional neural networks. It utilizes the scaling factor γ of the BatchNormalization (BN) layer as a channel importance index to achieve model compression. The transformation formula for the BN layer is as follows:
[0043] ;
[0044] in The output data of the Batch Normalization (BN) layer consists of feature values that have been normalized, scaled, and shifted. These features are used to optimize the stability of network training. The input features are those of the BN layer. This represents the average value of each channel in a batch. Let Variance be the variance of each channel in a batch. It is a small constant to prevent the denominator from being zero. and These are learnable parameters. The value directly reflects the channel contribution;
[0045] The core of the Slim pruning method lies in combining sparse training and the pruning process, including: evaluating the importance of the γ value of the BN layer, adding L1 regularization to the loss function to make γ tend to zero; setting a sparsity parameter to reduce the computation by 50% while maintaining or improving model accuracy; and skipping the detector head and feature fusion layer during the pruning process to avoid accuracy loss.
[0046] Specifically, this includes: first, performing sparse training, and then adding an L1 regularization term to the loss function, as shown in the formula:
[0047] ;
[0048] in, Represents the loss function. The input training image samples, For the corresponding real tags, The original classification loss function is... This is the predicted output of the YOLOv11-Pose model, where λ is the regularization strength. For all L1 regularization term, Indicates to The penalty function, regularization, causes the γ value of unimportant channels to tend to zero, forming a sparse structure;
[0049] After sparse training, channel pruning is performed: the |γ| of each channel is calculated as an importance score, all layers of γ are globally sorted, and low-importance channels are removed by selecting a threshold or proportion. In the specific implementation, key modules are skipped to avoid accuracy loss: the PSABlock module of the Detect head and the C2PSA feature fusion layer is skipped.
[0050] Step S2: Target detection and key point recognition;
[0051] The optimized instrument recognition model is used to process the image of the aircraft instrument panel under test (target detection and key point recognition), and outputs the position information of the instruments (coordinates of the rectangular detection box) and the coordinates of the key points of the instrument pointers:
[0052] The pointer-type instrument outputs the coordinates of the rectangular detection frame and the center point A of the dial, the pointer tip B, the smallest scale point C, and the largest scale point D.
[0053] The columnar instrument outputs the coordinates of the rectangular detection frame and pointer position points.
[0054] Step S3: Use a numerical mapping algorithm to map the results into readings;
[0055] Based on the instrument type, a differentiated numerical mapping algorithm is used (angle method for reading calculation for pointer instruments; proportional method for reading calculation for bar-type instruments) to convert the coordinates of key points of the instrument pointer into specific reading values of the instrument:
[0056] 1. Pointer-type instruments – angle method;
[0057] ① Identify the dial center point A, pointer tip point B, smallest scale point C, and largest scale point D through key point detection, and define a vector. (From A to B) , , (From A to E, where E is a point on the line extending AC in the opposite direction, i.e., AE is the vector in the opposite direction of AC), where for The opposite direction vector;
[0058] ② Calculate vectors sum vector The cross product scalar value determines the pointer direction, vector sum vector At the same point Starting from the vector The coordinate components are ,vector The coordinate components are The formula for calculating the scalar value of the cross product is as follows:
[0059] :in, The cross product of vectors is indicated by its sign: when crossProduct > 0, the pointer points counterclockwise; when crossProduct < 0, the pointer points clockwise; when crossProduct = 0, the pointer points to the vector. Collinear;
[0060] ③ Based on vectors with vector The proportionality coefficient is calculated from the cross product result, where: if the cross product is greater than 0, then the proportionality coefficient is... and The ratio; if the cross product is less than 0, then the proportionality coefficient is and The ratio; if the cross product is equal to 0, then the proportionality coefficient is and The ratio of .
[0061] Based on the orientation judgment conclusions corresponding to the above cross product results, a differentiated angle combination calculation method is used to determine the proportional coefficient of the interpretation. The specific calculation rules are as follows:
[0062] When the cross product result is greater than 0, calculate the vector. with vector The included angle ,Will relative to the reference angle The ratio of the two values is used as the proportionality coefficient of the reading. The calculation formula is as follows: ;
[0063] When the cross product result is less than 0, calculate the vector. with vector The included angle ,Will and Angles and reference angles The ratio of the two values is used as the reading ratio coefficient. The calculation formula is as follows: ;
[0064] When the cross product result is equal to 0, relative to the reference angle The ratio of the two values is used as the reading ratio coefficient. The calculation formula is as follows: ;
[0065] Among them, the reference angle The preset range corresponding to the instrument panel, that is, the angle range corresponding to the pointer when the instrument is at full scale;
[0066] ④ The proportionality coefficient of the readings obtained above. Multiply the reading by the preset range of the pointer instrument to obtain the instrument's reading.
[0067] 2. Column-type instrument – proportional method;
[0068] ① Detect and obtain the height of the regression prediction box of the bar gauge and the relative height of the current pointer feature point relative to the lower boundary of the regression prediction box through the model;
[0069] ② The pointer display scale is calculated based on the height and relative height, combined with the error compensation value. The calculation formula is as follows: The error compensation value is the average systematic error obtained by statistical analysis of multiple test samples.
[0070] ③ Multiply the ratio by the range of the columnar instrument to obtain the instrument reading.
[0071] Step S4: Automated detection and judgment;
[0072] The instrument readings obtained in step S3 are compared with the actual data from the aircraft flight control system backend. Three types of errors may occur: pointer error in pointer-type instruments, pointer error in bar-type instruments, and errors in both pointer-type and bar-type instrument pointers. A pointer error report will appear when the difference between the pointer-type instrument reading and the actual backend data is greater than 5. A bar-type instrument error report will appear when the difference is greater than 5. If both types of instruments are erroneous, both reports will appear simultaneously. When the system generates a pointer indication anomaly report for a pointer-type display instrument, the administrator needs to verify whether the actual indication state of the instrument pointer is abnormal. For bar-type display instrument pointer indication anomaly reports, the administrator needs to verify whether the actual indication state of the pointer is abnormal. The report will take a screenshot of the erroneous instrument and save it locally. The administrator needs to retrieve the screenshot for verification to determine whether the anomaly stems from a model recognition error or a genuine anomaly in the actual indication state of the instrument pointer.
[0073] Example 2: Based on Example 1, to make the technical solution of the present invention clearer, the following detailed description is provided in conjunction with the specific implementation process:
[0074] 1. Dataset Construction: Collect video data of the aircraft instrument panel of a certain type of aircraft, and extract 10,000 frames of images as the dataset, including 6,000 frames of pointer instrument images and 4,000 frames of bar-type instrument images; complete the labeling of instrument detection boxes and key points according to the labeling specifications, and divide 80% into training set and 20% into validation set.
[0075] 2. Model training and pruning:
[0076] Training environment: GPU is NVIDIA A100, framework is PyTorch;
[0077] Training process: Train for 300 rounds according to the parameter settings, monitor the loss curve to ensure no overfitting, and achieve a validation set mAP of 0.997;
[0078] Pruning optimization: Adding L1 regularization (λ=0.04) for sparse training reduces the computational cost of the model by 50% while maintaining an accuracy of over 0.995.
[0079] 3. Detection of the image to be tested: Input the image of the aircraft instrument panel to be tested into the pruned model, and output the instrument detection frame and the coordinates of key points—pointer-type instruments output the coordinates of four points A, B, C, and D, and bar-type instruments output the coordinates of the pointer position point.
[0080] 4. Example of reading calculation:
[0081] Pointer instrument: range 0-100, reference angle ∠CAD=180°; the cross product of vectors AB and AE is identified as 3.2 (>0), ∠CAB=90°, then pre=90° / 180°=0.5, final reading=0.5×100=50;
[0082] For a columnar instrument with a range of 0-200, a prediction frame height H=200 pixels, a relative height h=150 pixels, and an error compensation value ε=2 pixels, then pre=(150+2) / 200=0.76, and the final reading=0.76×200=152.
[0083] 5. Error Judgment: If the actual data in the background of the pointer instrument is 52, and the reading of 50 is 2 (<5) away from the actual data, it is judged as normal; if the actual data in the background of the bar instrument is 160, and the reading of 152 is 8 (≥5) away from the actual data, it is judged as a display error, and a screenshot is automatically saved.
[0084] Through the above implementation process, efficient and accurate automated detection of aviation instrument panels can be achieved, solving existing technical pain points and providing a guarantee for the safe operation of aviation equipment.
[0085] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as set forth in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways.
Claims
1. A method for reading pointers on an aviation instrument panel based on target detection and key point recognition, characterized in that, Includes the following steps: S1: Construct an image dataset of aviation instrument panels and annotate the rectangular detection boxes and key points of instrument pointers in the images. The aviation instruments include bar-shaped instruments and pointer instruments. The YOLOv11-Pose model is used for training. After training, the model is optimized using sparse training and Slim pruning methods to obtain the optimized instrument recognition model. S2: Use the optimized instrument recognition model to recognize the image of the aircraft instrument panel under test, and output the coordinates of the rectangular detection box of the instrument and the coordinates of the key points of the instrument pointer; S3: Based on the identified coordinates of key points of the instrument pointer, the coordinates of key points of the instrument pointer are converted into specific reading values of the instrument through a numerical mapping algorithm; S4: Compare the specific reading values of the mapped instrument with the real data in the background to determine whether the instrument display is correct, and realize the automated detection of the instrument display; The specific implementation of the Slim pruning method in step S1 includes: evaluating the importance of the γ value of the BN layer, adding L1 regularization to the loss function to make γ tend to zero; setting a sparsity parameter to reduce the computational cost by 50% while maintaining or improving model accuracy; skipping the detection head and feature fusion layer during pruning to avoid accuracy loss; in step S2, the optimized instrument recognition model outputs differentiated key point information for different types of instruments: For pointer-type instruments, the output is a rectangular detection frame of the instrument and the coordinates of four key pointer points, namely, the center point A of the dial, the tip point B of the pointer, the minimum scale point C, and the maximum scale point D; for bar-type instruments, the output is a rectangular detection frame of the instrument and the coordinates of one key pointer point, wherein the key point is the pointer position point; in step S3, the numerical mapping algorithm adopts different calculation methods according to the instrument type: the angle method is used to calculate the reading for pointer-type instruments, and the proportional method is used to calculate the reading for bar-type instruments; the calculation process of the angle method includes the following steps: S301: Obtain the dial center point A, pointer tip point B, minimum scale point C, and maximum scale point D, and define a vector. , , , ,in for The opposite direction vector; S302: Calculate vectors sum vector The cross product scalar value is used to determine the pointer direction, vector The coordinate components are ,vector The coordinate components are The formula for calculating the scalar value of the cross product is: ; in, The cross product of vectors is represented by its sign, which is used to determine the direction: when At that time, the pointer rotates counterclockwise; when At that time, the pointer rotates clockwise; when At that time, pointers and vectors Collinear; S303: Calculate the reading proportionality coefficient based on the cross product result. : when hour, ,in For vectors with vector The included angle, Preset the reference angle corresponding to the range of the instrument panel; when hour, ,in For vectors and The included angle; when hour, ; S304: Multiply the proportional coefficient pre by the preset range of the pointer instrument to obtain the final reading.
2. The method according to claim 1, characterized in that, The calculation process of the ratio method includes the following steps: The height of the rectangular detection frame of the columnar instrument and the relative height of the pointer position point with respect to the lower boundary of the rectangular detection frame are obtained by recognizing the model; the pointer display ratio is calculated by combining the error compensation value, and the calculation formula is as follows: The error compensation value is the average systematic error obtained by statistical analysis of multiple test samples; the ratio is multiplied by the range of the columnar instrument to obtain the final reading.
3. The method according to claim 1, characterized in that, In step S1, the parameters for model training are set as follows: 300 training epochs, batch size 42, initial learning rate 0.01, Adam optimizer, loss function combining rectangular detection box regression loss, classification loss and key point localization loss; data augmentation strategies include Mosaic stitching, RandAugment automatic augmentation and random erasure; and NMS post-processing of prediction results with an IoU threshold of 0.7 is used.
4. The method according to claim 1, characterized in that, The standard for judging whether the instrument display is correct in step S4 is: when the difference between the reading of the pointer instrument and the actual data in the background is ≥5, the pointer instrument is judged to be displaying incorrectly. When the difference between the bar chart reading and the actual data in the background is ≥5, the bar chart is judged to be displaying an error. After an error is identified, a screenshot of the erroneous instrument is taken and saved locally.