A method and system for monitoring the fuel level of an aircraft fuel tank

Abstract

The present application provides a method and a system for monitoring the fuel level of an aircraft fuel tank. The method for monitoring the fuel level of an aircraft fuel tank provided by the present application is applied to a fuel level monitoring system. The method collects a liquid level image through a vision sensor, and at the same time fuses the real-time vibration signal and two-dimensional inclination information of the fuel tank to be measured. Then, based on a pre-trained fuel level prediction model, adaptive feature extraction and fusion are performed on the collected liquid level photo, real-time vibration signal, and two-dimensional inclination information to more comprehensively capture the state changes of the fuel tank to be measured, and finally, real-time measurement of the remaining fuel level of the fuel tank to be measured under vibration interference and non-steady flight conditions is realized, which can effectively achieve high-precision measurement of the remaining fuel level of the fuel tank to be measured in a flight jitter state. In addition, multi-information fusion can also provide the anti-interference ability of the model, enabling it to adapt to different flight postures and working conditions, improving adaptability, having strong robustness, and ensuring the accuracy of real-time prediction.

Description

Technical Field

[0001] This application relates to the technical field of fuel tank fuel quantity monitoring, and particularly to a method and a system for monitoring the fuel quantity of an aircraft fuel tank. Background Art

[0002] Aircraft fuel measurement is one of the core functions of the aircraft fuel system. Its main function is to obtain the aircraft's fuel load and its distribution in the fuel tank in real time and accurately, which directly affects the aircraft's long-range attack ability, agility, maneuverability, and intelligent fuel management ability. The fuel quantity is the largest variable weight of the aircraft. Real-time and accurate measurement of the remaining fuel quantity in the fuel tank is of great significance for reasonably adjusting the distribution of fuel in each fuel tank, effectively implementing the optimal fuel usage sequence control, accurately calculating the aircraft's endurance time, controlling the aircraft's center of gravity, improving the aircraft's handling and stability, and also helps to maximize the use of the aircraft's effective payload and meet the requirements of real-time scientific planning of flight missions.

[0003] Currently, domestic aircraft fuel level measurement still uses capacitive sensors. However, capacitive sensors have poor anti-electromagnetic interference capabilities. In recent years, in order to meet the strategic requirements of airspace integration and offensive and defensive capabilities, aiming at the development needs of airborne electromechanical systems in the future air combat environment, and accelerating the improvement of the long-range and high-intensity air combat capabilities in complex electromagnetic environments, it is imperative to develop a fuel tank fuel quantity monitoring system based on new principles. Summary of the Invention

[0004] In view of this, this application provides a method and a system for monitoring the fuel quantity of an aircraft fuel tank, so as to achieve accurate prediction of the remaining fuel quantity in the aircraft fuel tank under non-steady flight states.

[0005] Specifically, this application is implemented through the following technical solutions:

[0006] In the first aspect of this application, a method for monitoring the fuel quantity of an aircraft fuel tank is provided. The method is applied to a fuel quantity monitoring system, which includes a vision sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor. The vision sensor is arranged obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be measured. The method includes:

[0007] The vision sensor collects the liquid surface image of the fuel tank to be measured, the vibration sensor collects the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor collects the two-dimensional inclination information of the fuel tank to be measured.

[0008] The processor inputs the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel quantity prediction model, so as to use the fuel quantity prediction model to extract features from the liquid level image to obtain a first high-dimensional feature, and use the fuel quantity prediction model to extract features from the vibration signal to obtain a second high-dimensional feature; wherein, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface.

[0009] The processor uses the fuel quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information to obtain a global feature.

[0010] The processor predicts the current fuel quantity of the fuel tank to be measured based on the global feature by using the fuel quantity prediction model.

[0011] A second aspect of the present application provides a fuel quantity monitoring system, which includes a vision sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor; the vision sensor is arranged obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be measured; wherein, the vision sensor is used to collect the liquid level image of the fuel tank to be measured, the vibration sensor is used to collect the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor is used to collect the two-dimensional inclination information of the fuel tank to be measured.

[0012] The processor is used to input the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel quantity prediction model, so as to use the fuel quantity prediction model to extract features from the liquid level image to obtain a first high-dimensional feature, and use the fuel quantity prediction model to extract features from the vibration signal to obtain a second high-dimensional feature; wherein, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface.

[0013] The processor is further used to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information by using the fuel quantity prediction model to obtain a global feature.

[0014] The processor is further used to predict the current fuel quantity of the fuel tank to be measured based on the global feature by using the fuel quantity prediction model.

[0015] The fuel quantity monitoring method and fuel quantity monitoring system provided by this application collect liquid level images through a vision sensor, and at the same time fuse the real-time vibration signal and two-dimensional inclination information of the fuel tank to be measured. Then, based on a pre-trained fuel quantity prediction model, adaptive feature extraction and fusion are performed on the collected liquid level photos, real-time vibration signals, and two-dimensional inclination information to more comprehensively capture the state changes of the fuel tank to be measured, and finally realize the real-time measurement of the remaining fuel quantity of the fuel tank to be measured under vibration interference and non-steady flight conditions, and can effectively realize the high-precision measurement of the remaining fuel quantity of the fuel tank to be measured under flight jitter conditions. In addition, multi-information fusion can also provide the anti-interference ability of the model, enabling it to adapt to different flight postures and working conditions, improving adaptability (strong robustness), and ensuring the accuracy of real-time prediction. Brief Description of the Drawings

[0016] Figure 1 It is a flowchart of the first embodiment of the fuel quantity monitoring method for an aircraft fuel tank provided by this application;

[0017] Figure 2 It is a schematic diagram of the fuel quantity monitoring system shown in an exemplary embodiment of this application;

[0018] Figure 3 It is a schematic diagram of the fuel quantity prediction model shown in an exemplary embodiment of this application;

[0019] Figure 4 It is a flowchart of the second embodiment of the fuel quantity monitoring method for an aircraft fuel tank provided by this application;

[0020] Figure 5 It is a flowchart of the third embodiment of the fuel quantity monitoring method for an aircraft fuel tank provided by this application. Detailed Description of the Embodiments

[0021] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0022] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0024] Specific embodiments are given below to introduce the technical solutions of this application in detail.

[0025] Figure 1 The flowchart of the first embodiment of the method for monitoring the fuel quantity of an aircraft fuel tank provided by this application is shown. Please refer to Figure 1 The method for monitoring the fuel quantity of an aircraft fuel tank provided in this embodiment is applied to a fuel quantity monitoring system. The fuel quantity monitoring system includes a vision sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor. The vision sensor is disposed obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are disposed at the top or bottom of the fuel tank to be measured. The method includes:

[0026] S101. The vision sensor collects the liquid level image of the fuel tank to be measured, the vibration sensor collects the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor collects the two-dimensional inclination information of the fuel tank to be measured.

[0027] The method for monitoring the fuel quantity of an aircraft fuel tank provided by this application is applied to a fuel quantity monitoring system. Figure 2 The schematic diagram of the fuel quantity monitoring system shown in an exemplary embodiment of this application is shown. Please refer to Figure 2 The fuel quantity monitoring system is installed on the fuel tank to be measured. The fuel quantity monitoring system includes a vision sensor (in one embodiment, the vision sensor may be a camera), a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor. The vision sensor is disposed obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are disposed at the top or bottom of the fuel tank to be measured. Among them, the front end of the vision sensor faces the inside of the fuel tank to be measured and is used to collect the liquid level image of the fuel tank to be measured, and it can fully collect the liquid level images from the full fuel quantity to the fuel shortage state. The vibration sensor is used to detect the vibration signal of the fuel tank to be measured to measure the vibration interference intensity. The two-dimensional inclination sensor can collect the two-dimensional inclination information of the fuel tank to be measured. It should be noted that the vibration signal is a time series signal, which can record the process of the vibration amplitude changing with time. The two-dimensional inclination information refers to the inclination angles of the fuel tank to be measured around the pitch axis and the roll axis.

[0028] Specifically, the specific installation positions of the two-dimensional inclination sensor and the vibration sensor are set according to actual needs and are not limited in this embodiment. For example, in this embodiment, both the two-dimensional inclination sensor and the vibration sensor are arranged at the bottom of the fuel tank to be measured; for another example, in another embodiment, the two-dimensional inclination sensor and the vibration sensor can also be arranged at the top of the fuel tank to be measured.

[0029] It should be noted that the vibration sensor works by detecting the acceleration or displacement of the vibration of the fuel tank to be measured. Usually, it is an acceleration sensor made of piezoelectric materials (such as piezoelectric ceramics). When the fuel tank to be measured vibrates, the piezoelectric material generates charges proportional to the vibration amplitude, which are converted into electrical signals and output through a circuit. In addition, the vibration sensor is usually installed at the top or bottom of the fuel tank to be measured, as close as possible to the area that may be most affected by vibration. In addition, a part with relatively strong rigidity of the fuel tank to be measured is selected to reduce the influence of environmental noise and ensure that the vibration sensor can accurately collect vibration signals.

[0030] Furthermore, there are usually two mutually perpendicular measurement axes (pitch axis and roll axis) inside the two-dimensional inclination sensor. When the fuel tank to be measured is tilted, gravity acts on the internal measurement element, causing a change in its output electrical signal. By calculating the gravity components in the two axial directions, the inclination angles of the fuel tank to be measured around the pitch axis and the roll axis are obtained. In addition, the two-dimensional inclination sensor needs to be installed on the top or bottom plane of the fuel tank to be measured to ensure that it can accurately sense the pitch and roll angle changes of the fuel tank to be measured relative to the ground plane.

[0031] Please continue to refer to Figure 2 , the vision sensor, the vibration sensor, and the two-dimensional inclination sensor are all electrically connected to the processor. The processor is used to receive the liquid level image collected by the vision sensor, the vibration signal collected by the vibration sensor, and the two-dimensional inclination information collected by the two-dimensional inclination sensor, and predict the fuel quantity at the current moment based on the liquid level image, the vibration signal, and the two-dimensional inclination information.

[0032] In specific implementation, the vision sensor collects the liquid level image in real time, the vibration sensor collects the vibration signal in real time, and the two-dimensional inclination sensor collects the two-dimensional inclination information in real time, realizing real-time online measurement of the dynamic fuel quantity of the fuel tank to be measured.

[0033] S102. The processor inputs the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel quantity prediction model to extract features from the liquid level image using the fuel quantity prediction model to obtain a first high-dimensional feature, and extract features from the vibration signal using the fuel quantity prediction model to obtain a second high-dimensional feature; wherein, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface.

[0034] S103. The processor uses the fuel quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information to obtain a global feature.

[0035] S104. The processor predicts the current fuel quantity of the fuel tank to be measured based on the global feature by using the fuel quantity prediction model.

[0036] It should be noted that a pre-trained fuel quantity prediction model is deployed on the processor. This fuel quantity prediction model is used to predict the fuel quantity at a certain moment according to the liquid surface image, vibration signal, and two-dimensional inclination information at that moment. The input of this fuel quantity prediction model is the liquid surface image, vibration signal, and two-dimensional inclination information, and the output is the predicted fuel quantity. After inputting the liquid surface image, vibration signal, and two-dimensional inclination information into this fuel quantity prediction model, it can extract features from the liquid surface image to obtain the first high-dimensional feature, extract features from the vibration signal to obtain the second high-dimensional feature, and then fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information to obtain a global feature, and predict the current fuel quantity of the fuel tank to be measured based on the global feature.

[0037] Specifically, Figure 3 is a schematic diagram of a fuel quantity prediction model exemplarily shown in this application. Please refer to Figure 3 , in a possible implementation manner, the fuel quantity prediction model may include a first feature extraction network, a second feature extraction network, and a feature fusion network. It can use the first feature extraction network to extract features from the liquid surface image, use the second feature extraction network to extract features from the vibration signal, and use the feature fusion network to fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature network, and the input two-dimensional inclination information, and then use the feature fusion network to predict the fuel quantity based on the global feature.

[0038] Specific embodiments will be given below to introduce the training process of this fuel quantity prediction model in detail, which will not be elaborated here.

[0039] It should be noted that in this embodiment, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface to improve the accuracy.

[0040] The method for monitoring the fuel quantity of the fuel tank of an aircraft provided in this embodiment collects the liquid surface image through a visual sensor, and at the same time fuses the real-time vibration signal and two-dimensional inclination information of the fuel tank to be measured. Then, based on the pre-trained fuel quantity prediction model, adaptive feature extraction and fusion are performed on the collected liquid surface photos, real-time vibration signals, and two-dimensional inclination information to more comprehensively capture the state changes of the fuel tank to be measured. Finally, the real-time measurement of the remaining fuel quantity of the fuel tank to be measured under vibration interference and non-steady flight conditions is realized, and the high-precision measurement of the remaining fuel quantity of the fuel tank to be measured in the flight jitter state can be effectively achieved. In addition, multi-information fusion can also provide the anti-interference ability of the model, enabling it to adapt to different flight postures and working conditions, improving adaptability (strong robustness), and ensuring the accuracy of real-time prediction.

[0041] Figure 4 The flowchart of the second embodiment of the method for monitoring the fuel quantity of the fuel tank of an aircraft provided in this application is shown in Figure 4 On the basis of the above embodiment, for the method provided in this embodiment, the training process of the fuel quantity prediction model may include:

[0042] S401. Collect multiple groups of data of the fuel tank to be measured under typical flight states of the aircraft.

[0043] It should be noted that the multiple groups of data are multiple groups of data of the fuel tank to be measured under different fuel weights. Among them, the sample data of each group of data includes the real-time liquid surface image, real-time vibration signal, and real-time two-dimensional inclination information, and the label of this group of data is the fuel weight corresponding to the sample data.

[0044] It can be understood that under the typical flight states of the aircraft (it should be noted that the typical flight states of the aircraft include flight states such as takeoff, cruise, and landing), as the aircraft flies, the weight of the fuel in the fuel tank to be measured is constantly decreasing, and along with different flight states, the distribution and flow conditions of the fuel in the fuel tank to be measured are also different. In this way, it will cause certain difficulties in the measurement of the fuel quantity. By collecting multiple groups of data under different fuel weights, according to the real-time liquid surface image, real-time vibration signal, and real-time two-dimensional inclination information in each group of data, it can help the model better learn the relationship between the fuel weight and various characteristic information, and improve the accuracy of the model's prediction of the fuel quantity.

[0045] Optionally, in a possible implementation, the collecting multiple groups of data of the fuel tank to be measured under typical flight states of the aircraft includes:

[0046] (1)Construct a simulation system; wherein, the simulation system includes a flight state simulation device, a fuel tank to be tested placed on the flight state simulation device, a weight testing device for testing the fuel weight, a vibration table for simulating the vibration state of the fuel tank to be tested under the actual service state, and the fuel quantity monitoring system.

[0047] (2)Use the simulation system to collect multiple groups of data of the fuel tank to be tested under typical flight states of an aircraft; wherein, during the collection process, use the vibration table to simulate the vibration state, and during the process of gradually draining the fuel, collect the liquid level image, vibration signal, two-dimensional inclination information, and the corresponding fuel weight in real time, and then use the collected liquid level image, vibration signal, and two-dimensional inclination information as sample data, and the corresponding fuel weight as labels to form multiple groups of data.

[0048] Specifically, in order to collect multiple groups of data of the fuel tank to be tested under typical flight states of an aircraft, a simulation system is constructed to simulate multiple typical flight states of the aircraft and collect multiple groups of data of the fuel tank to be tested.

[0049] It should be noted that the simulation system includes a flight state simulation device, a fuel tank to be tested placed on the flight state simulation device, a weight testing device for testing the fuel weight, a vibration table for simulating the vibration state of the fuel tank to be tested under the actual service state, and a fuel quantity monitoring system. Specifically, in implementation, the fuel tank to be tested, the fuel quantity monitoring system arranged on the fuel tank to be tested, the weight measuring device, and the vibration table can be placed on the flight state simulation device to simulate the vibration state of the fuel tank to be tested under typical flight states of an aircraft.

[0050] Furthermore, after the simulation system is constructed, the simulation system can be used to collect multiple groups of data of the fuel tank to be tested under typical flight states of an aircraft.

[0051] It should be noted that please refer to Figure 2 , the measured fuel quantity also has a fuel inlet valve and a fuel outlet valve. During the data collection process, the vibration table, the fuel quantity monitoring system, the fuel outlet valve, and the weight measuring device can be opened, and the fuel can be drained under the vibration state. At the same time, the liquid level image, vibration signal, and two-dimensional inclination signal are collected in real time through the fuel quantity monitoring system, and the corresponding fuel weight is collected synchronously through the weight measuring device; furthermore, using the real-time liquid level image, real-time vibration signal, and real-time two-dimensional inclination information as sample data, and the corresponding fuel weight as labels, a "real-time liquid level image, real-time vibration signal, real-time two-dimensional inclination information and real-time fuel weight" data set is formed.

[0052] The method provided in this embodiment builds a simulation system and uses the simulation system to collect multiple groups of data of the fuel tank to be measured under typical flight states of an aircraft. In this way, various flight states of the aircraft can be simulated in a ground environment, and data similar to the actual flight state can be collected, providing rich and accurate training data for the fuel quantity prediction model, thereby improving the accuracy of the trained fuel quantity prediction model in predicting the fuel quantity of the fuel tank to be measured.

[0053] S402. Build an initial model. The initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network.

[0054] Specifically, please continue to refer to Figure 3 In a possible implementation, the initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network.

[0055] Among them, the first feature extraction network is used to extract features from the input liquid level image. It can be understood that the liquid level image is a two-dimensional image, and the first feature extraction network is a two-dimensional convolutional neural network to effectively extract the spatial features in the liquid level image.

[0056] Optionally, in a possible implementation, the two-dimensional convolutional neural network includes 3 to 5 convolutional layers to focus on the texture features of the target area where the oil liquid contacts the wall of the fuel tank to be measured.

[0057] It should be noted that each convolutional layer can extract features at different levels. By setting 3 to 5 convolutional layers, the texture features can be focused on. In specific implementation, the number of convolutional layers can be set according to actual needs, and it is not limited here. For example, in one embodiment, the two-dimensional convolutional neural network can include 3 convolutional layers; for another example, in another embodiment, the two-dimensional convolutional neural network can include 4 convolutional layers.

[0058] It can be understood that each convolutional layer can include a convolutional layer, an activation function layer, and a pooling layer.

[0059] Furthermore, the second feature extraction network is used to extract features from the input vibration signal. It can be understood that the vibration signal is time series data, and the second feature extraction network can be a one-dimensional convolutional neural network to process the vibration signal with the one-dimensional convolutional neural network and effectively capture the energy information in the vibration signal.

[0060] Optionally, in a possible implementation, the dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network should be greater than 7*7 to increase the perception area of the vibration signal and make the neural network pay more attention to the energy information of the vibration signal. It should be noted that the specific dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network is set according to actual needs and is not limited herein. For example, in one embodiment, the dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network is 32*32.

[0061] It should be noted that the number of convolutional layers included in the one-dimensional convolutional neural network is also set according to actual needs and is not limited in this embodiment.

[0062] Furthermore, the feature fusion network is used to fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the fuel quantity of the fuel tank to be measured based on the fused features. Optionally, in a possible implementation, the feature fusion network is a fully connected neural network, and the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature extraction network, and the input two-dimensional inclination information can be effectively fused by using the fully connected neural network, thereby improving the accuracy of fuel quantity prediction.

[0063] The method provided in this embodiment is based on a deep neural network to realize the adaptive extraction of dynamic liquid surface images and vibration signal features, and uses a fully connected neural network to fuse multi-source data, avoiding manual screening and extraction of fuel quantity sensitive features, introducing condition monitoring data, enhancing the robustness of the algorithm, and improving the algorithm accuracy.

[0064] S403. Use the multiple sets of data to train the initial model to obtain a trained fuel quantity prediction model.

[0065] Specifically, after obtaining multiple sets of data of the fuel tank to be measured in the typical flight state of the aircraft and constructing the initial model, the initial model can be trained with the multiple sets of data to obtain a trained fuel quantity prediction model.

[0066] In specific implementation, the multiple sets of data can be divided into a training set and a test set according to a certain ratio. Further, parameter training is performed on the initial model based on the training set to obtain a trained fuel quantity prediction model. Further, the trained fuel quantity prediction model is saved; and the measurement accuracy of the trained fuel quantity prediction model is tested based on the test set to evaluate the performance of the trained fuel quantity prediction model.

[0067] The method for monitoring the fuel quantity of the fuel tank to be measured of the aircraft provided in this embodiment collects multiple groups of data of the measured fuel quantity under typical flight states of the aircraft, then constructs an initial model, and uses the multiple groups of data to train the initial model to obtain a trained fuel quantity prediction model. In this way, the trained fuel quantity prediction model can combine the liquid level image, vibration signal and two-dimensional inclination information to predict the fuel quantity, so as to improve the accuracy of the model in predicting the fuel quantity. In addition, the trained fuel quantity prediction model can be adaptively adjusted according to the characteristics of the fuel tank to be measured under different flight states to accurately predict the fuel quantity in real time.

[0068] Figure 5 It is the flowchart of the third embodiment of the method for monitoring the fuel quantity of the fuel tank of the aircraft provided in this application. Please refer to Figure 5 , on the basis of the above embodiment, the method provided in this embodiment further includes:

[0069] S501. After training the initial model, use the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area, and obtain an evaluation result.

[0070] It should be noted that the effectiveness of each convolutional layer in the first feature extraction network in extracting the texture features of the target area is not the same. In this step, after training the initial model, the correlation between the features extracted by each convolutional layer and the texture features of the target area can be evaluated, and the evaluation result can be used to determine which layers extract the features that can best represent the texture features of the contact area between the oil and the wall of the fuel tank to be measured.

[0071] Specifically, the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool can be selected to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area.

[0072] It should be noted that the working principle of the Grad-CAM visualization tool is as follows:

[0073] Step 1. First, the Grad-CAM visualization tool calculates the gradient of the output of the model for a specific category with respect to the output feature map of a specific convolutional layer. These gradients reflect the importance of the features extracted by the model in this convolutional layer for the final output.

[0074] Step 2. Use these gradients to weight the feature map of the convolutional layer. Specifically, the gradient of each feature map is obtained through a global average pooling operation to get a weight value, which reflects the contribution of this feature map to the category.

[0075] Step 3: ReLU activate the weighted feature map to generate a heat map, which indicates which regions in the image contribute more to the prediction of a specific category. Among them, the highlighted regions represent the parts that have the greatest impact on the model's decision-making.

[0076] Step 4: Overlay the heat map on the original image, and it can be intuitively seen which regions are concerned by the model, especially the texture regions where the oil contacts the wall surface of the fuel tank to be measured.

[0077] It should be noted that in this step, the evaluation result is the result after overlaying the heat map on the original image (the original liquid level image).

[0078] S502: Dynamically adjust the trained fuel quantity prediction model according to the evaluation result to retain the convolutional layers that are effective for extracting the texture features of the target region and delete the convolutional layers that are unnecessary for extracting the texture features of the target region.

[0079] Specifically, according to the evaluation result, it can be decided which effective convolutional layers to retain and which layers that are not very helpful for texture feature extraction to delete, so as to simplify the model, improve the efficiency, and ensure that the model focuses more on important features.

[0080] In specific implementation, for example, if the correlation between a certain convolutional layer and the texture features of the target region is low, it can be deleted, while the convolutional layers with high correlation with the texture features of the target region can be retained, so that the retained convolutional layers can accurately extract the texture features of the target region and improve the accuracy of fuel quantity prediction.

[0081] S503: Retrain the adjusted model and execute again the step of using the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target region until the trained model can effectively extract the texture features of the target region.

[0082] Specifically, after adjusting the fuel quantity prediction model, retrain it so that it can adapt to the structural changes brought about by deleting the convolutional layers with low correlation. Subsequently, execute again the step of using the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target region, and then continuously loop this process to gradually optimize the fuel quantity prediction model until the model can effectively extract the texture features of the target region.

[0083] The method for monitoring the fuel level of an aircraft fuel tank provided in this embodiment, after training the initial model, uses the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area, obtaining an evaluation result. Then, according to the evaluation result, the trained fuel level prediction model is dynamically adjusted to retain the convolutional layers that are effective in extracting the texture features of the target area and delete the convolutional layers that are unnecessary for extracting the texture features of the target area. Furthermore, the adjusted model is retrained, and the step of using the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area is executed again until the trained model can effectively extract the texture features of the target area. In this way, not only can the finally obtained model be more focused on extracting relevant features, thereby improving its ability to capture the texture features of the target area and ultimately enhancing the accuracy of fuel level prediction and the overall performance of the model, but also unnecessary convolutional layer calculations can be avoided, reducing the complexity of the model.

[0084] Please continue to refer to Figure 2 , this application also provides an oil quantity monitoring system. The oil quantity monitoring system provided by this application will be introduced below:

[0085] The oil quantity monitoring system provided by this application includes a vision sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor; the vision sensor is arranged obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be measured;

[0086] Among them, the vision sensor is used to collect the liquid level image of the fuel tank to be measured, the vibration sensor is used to collect the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor is used to collect the two-dimensional inclination information of the fuel tank to be measured;

[0087] The processor is used to input the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel level prediction model, so as to use the fuel level prediction model to extract features from the liquid level image to obtain a first high-dimensional feature, and use the fuel level prediction model to extract features from the vibration signal to obtain a second high-dimensional feature; among them, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface;

[0088] The processor is also used to use the fuel level prediction model to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information to obtain a global feature;

[0089] The processor is further configured to predict the current fuel level of the fuel tank to be measured based on the global feature by using the fuel level prediction model.

[0090] Optionally, the training process of the fuel level prediction model includes:

[0091] Collect multiple groups of data of the fuel tank to be measured in typical flight states of the aircraft; wherein, the multiple groups of data are multiple groups of data of the fuel tank to be measured under different fuel weights; the sample data of each group of data includes a real-time liquid level image, a real-time vibration signal, and real-time two-dimensional inclination information, and the label of this group of data is the fuel weight corresponding to the sample data;

[0092] Construct an initial model; wherein, the initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network; the first feature extraction network is configured to extract features from the input liquid level image; the second feature extraction network is configured to extract features from the input vibration signal; the feature fusion network is configured to fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature network, and the input two-dimensional inclination information, and predict the fuel level of the fuel tank to be measured based on the fused features;

[0093] Train the initial model by using the multiple groups of data to obtain a trained fuel level prediction model.

[0094] Optionally, the first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers;

[0095] And / or, the second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolutional kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of the vibration signal.

[0096] Referring to the previous description, the fuel level monitoring system provided by the present application has been introduced in the previous description, and will not be elaborated here.

[0097] The above are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for monitoring the fuel quantity in the fuel tank of an aircraft, characterized in that, The method for monitoring the fuel level of an aircraft fuel tank is applied to a fuel level monitoring system. The fuel level monitoring system includes a vision sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the vision sensor, the vibration sensor, and the two-dimensional inclination sensor. The vision sensor is arranged obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be measured. The method includes: The vision sensor collects the liquid level image of the fuel tank to be measured, the vibration sensor collects the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor collects the two-dimensional inclination information of the fuel tank to be measured. Among them, the vibration signal is a time series signal, which can record the process of the vibration amplitude changing with time. The two-dimensional inclination information refers to the inclination angles of the fuel tank to be measured around the pitch axis and the roll axis. The processor inputs the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel level prediction model, so as to use the fuel level prediction model to extract features from the liquid level image to obtain a first high-dimensional feature, and use the fuel level prediction model to extract features from the vibration signal to obtain a second high-dimensional feature. Among them, the first high-dimensional feature focuses on the texture features of the target area where the oil liquid contacts the wall surface. The processor uses the fuel level prediction model to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information to obtain a global feature. The processor predicts the current fuel level of the fuel tank to be measured based on the global feature by using the fuel level prediction model. Among them, the training process of the fuel level prediction model includes: Collect multiple groups of data of the fuel tank to be measured under typical flight states of the aircraft. Construct an initial model. Among them, the initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network. Use the multiple groups of data to train the initial model to obtain a trained fuel level prediction model. After training the initial model, use the Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area to obtain an evaluation result. Dynamically adjust the trained fuel level prediction model according to the evaluation result to retain the convolutional layers effective for extracting the texture features of the target area and delete the convolutional layers unnecessary for extracting the texture features of the target area. Retrain the adjusted model and execute the step of using the Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area again until the trained model can effectively extract the texture features of the target area.

2. The method according to claim 1, wherein Among them, The multiple groups of data are multiple groups of data of the fuel tank to be measured under different fuel weights. The sample data of each group of data includes a real-time liquid level image, a real-time vibration signal, and real-time two-dimensional inclination information, and the label of this group of data is the fuel weight corresponding to this sample data. The first feature extraction network is used to extract features from the input liquid level image; the second feature extraction network is used to extract features from the input vibration signal; the feature fusion network is used to fuse the first high-dimensional features extracted by the first feature extraction network, the second high-dimensional features extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the fuel quantity of the fuel tank to be measured based on the fused features.

3. The method according to claim 1, characterized in that, The first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers.

4. The method according to claim 2, characterized in that, The second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolutional kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of the vibration signal.

5. The method according to claim 2, wherein The feature fusion network is a fully connected neural network.

6. The method according to claim 1, characterized in that Collecting multiple groups of data of the fuel tank to be measured under typical flight states of an aircraft, including: Constructing a simulation system; wherein, the simulation system includes a flight state simulation device, a fuel tank to be measured placed on the flight state simulation device, a weight test device for testing the fuel weight, a vibration table for simulating the vibration state of the fuel tank to be measured under the actual service state, and the fuel quantity monitoring system; Using the simulation system to collect multiple groups of data of the fuel tank to be measured under typical flight states of an aircraft; wherein, during the collection process, the vibration state is simulated by the vibration table, and during the process of gradually draining the fuel, the liquid level image, vibration signal, two-dimensional inclination information, and the corresponding fuel weight are collected in real time. Then, the collected liquid level image, vibration signal, and two-dimensional inclination information are used as sample data, and the corresponding fuel weight is used as a label to form multiple groups of data.

7. An oil quantity monitoring system, characterized in that, The fuel quantity monitoring system includes a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor, and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the fuel tank to be measured, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be measured; wherein, The visual sensor is used to collect the liquid level image of the fuel tank to be measured, the vibration sensor is used to collect the vibration signal of the fuel tank to be measured, and the two-dimensional inclination sensor is used to collect the two-dimensional inclination information of the fuel tank to be measured; wherein, the vibration signal is a time series signal, which can record the process of the vibration amplitude changing with time; the two-dimensional inclination information refers to the inclination angles of the fuel tank to be measured around the pitch axis and the roll axis; The processor is used to input the liquid level image, the vibration signal, and the two-dimensional inclination information into a pre-trained fuel quantity prediction model, so as to use the fuel quantity prediction model to extract features from the liquid level image to obtain first high-dimensional features, and use the fuel quantity prediction model to extract features from the vibration signal to obtain second high-dimensional features; wherein, the first high-dimensional features focus on the texture features of the target area where the oil liquid contacts the wall surface. The processor is further configured to fuse the first high-dimensional feature, the second high-dimensional feature, and the two-dimensional inclination information by using the fuel quantity prediction model to obtain a global feature; The processor is further configured to predict the current fuel quantity of the fuel tank to be measured based on the global feature by using the fuel quantity prediction model; Wherein, the training process of the fuel quantity prediction model includes: Collecting multiple groups of data of the fuel tank to be measured in typical flight states of an aircraft; Constructing an initial model; wherein, the initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network; Training the initial model by using the multiple groups of data to obtain a trained fuel quantity prediction model; After training the initial model, using a Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area, and obtaining an evaluation result; Dynamically adjusting the trained fuel quantity prediction model according to the evaluation result to retain the convolutional layers effective for extracting the texture features of the target area and delete the convolutional layers unnecessary for extracting the texture features of the target area; Retraining the adjusted model, and again performing the step of using the Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area until the trained model can effectively extract the texture features of the target area.

8. The fuel quantity monitoring system according to claim 7, characterized in that Wherein, The multiple groups of data are multiple groups of data of the fuel tank to be measured under different fuel weights; the sample data of each group of data includes a real-time liquid level image, a real-time vibration signal, and real-time two-dimensional inclination information, and the label of this group of data is the fuel weight corresponding to the sample data; The first feature extraction network is configured to extract features from the input liquid level image; the second feature extraction network is configured to extract features from the input vibration signal; the feature fusion network is configured to fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the fuel quantity of the fuel tank to be measured based on the fused features.

9. The fuel quantity monitoring system according to claim 7, wherein The first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers; And / or, The second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolutional kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of the vibration signal.

Images