Method and system for monitoring fuel quantity of fuel tank of aviation aircraft

Abstract

The invention provides an aviation aircraft fuel tank fuel quantity monitoring method and a fuel quantity monitoring system. The aviation aircraft fuel tank fuel quantity monitoring method is applied to a fuel quantity monitoring system, a liquid level image is collected through a visual sensor, and meanwhile, a real-time vibration signal and two-dimensional inclination angle information of a to-be-detected fuel tank are fused; and adaptive feature extraction and fusion are carried out on the acquired liquid level picture, the real-time vibration signal and the two-dimensional inclination angle information based on a pre-trained oil mass prediction model, so that the state change of the to-be-measured oil tank is more comprehensively captured, and finally, the real-time measurement of the residual oil mass of the to-be-measured oil tank under the vibration interference and non-stable flight working conditions is realized. And high-precision measurement of the residual oil quantity of the to-be-measured oil tank in a flight jitter state can be effectively realized. In addition, the multi-information fusion can also provide the anti-interference capability of the model, so that the model can adapt to different flight attitudes and working conditions, the adaptability is improved, the robustness is high, and the accuracy of real-time prediction is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of fuel tank fuel quantity monitoring, and in particular to a fuel tank fuel quantity monitoring method and fuel quantity monitoring system for an aircraft. Background Art

[0002] Aircraft fuel measurement is one of the core functions of the aircraft fuel system. Its main function is to obtain the aircraft's fuel load and its distribution in the fuel tank in real time and accurately, which is directly related to the impact on the aircraft's long-range attack capability, agility, maneuverability and intelligent fuel management capabilities. Fuel volume is the largest variable weight of the aircraft. Real-time and accurate measurement of the remaining fuel in the fuel tank is of great significance for reasonably adjusting the distribution of fuel in each tank, effectively realizing the optimal fuel use sequence control, accurately calculating the aircraft's flight time, controlling the aircraft's center of gravity, and improving the aircraft's maneuverability and stability. It also helps to maximize the use of the aircraft's effective load and meet the requirements of real-time scientific planning of flight missions.

[0003] At present, domestic aircraft fuel level measurement still uses capacitive sensors, but the anti-electromagnetic interference ability of capacitive sensors is poor. In recent years, in order to meet the strategic requirements of air-space integration and both offense and defense, aiming at the development needs of airborne electromechanical systems in future air combat environments, and accelerating the improvement of long-range and high-intensity air combat capabilities in complex electromagnetic environments, it is imperative to develop a fuel tank fuel level monitoring system based on new principles. Summary of the invention

[0004] In view of this, the present application provides an aircraft fuel tank fuel quantity monitoring method and fuel quantity monitoring system, so as to achieve accurate prediction of the remaining fuel quantity in the aircraft onboard fuel tank under non-steady flight state.

[0005] Specifically, the present application is implemented through the following technical solutions: In a first aspect, the present application provides a method for monitoring the fuel quantity of an aircraft tank. The method is applied to a fuel quantity monitoring system, wherein the fuel quantity monitoring system comprises a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor, and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the fuel tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be tested; the method comprises: The visual sensor collects the liquid level image of the oil tank to be tested, the vibration sensor collects the vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor collects the two-dimensional inclination information of the oil tank to be tested; The processor inputs the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to extract features of the liquid surface image using the oil quantity prediction model to obtain a first high-dimensional feature, and extract features of the vibration signal using the oil quantity prediction model to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall; The processor uses the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature; The processor predicts the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

[0006] A second aspect of the present application provides an oil quantity monitoring system, the oil quantity monitoring system comprising a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor, and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the oil tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the oil tank to be tested; wherein the visual sensor is used to collect a liquid surface image of the oil tank to be tested, the vibration sensor is used to collect a vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor is used to collect two-dimensional inclination information of the oil tank to be tested; The processor is used to input the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to extract features of the liquid surface image using the oil quantity prediction model to obtain a first high-dimensional feature, and to extract features of the vibration signal using the oil quantity prediction model to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall; The processor is further used to use the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature; The processor is further configured to predict the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

[0007] The fuel quantity monitoring method and fuel quantity monitoring system for aircraft fuel tanks provided in the present application collect liquid surface images through visual sensors, and fuse the real-time vibration signal and two-dimensional inclination information of the fuel tank to be tested at the same time, and then perform adaptive feature extraction and fusion of the collected liquid surface photos, real-time vibration signals, and two-dimensional inclination information based on the pre-trained fuel quantity prediction model, so as to more comprehensively capture the state changes of the fuel tank to be tested, and finally realize the real-time measurement of the remaining fuel quantity of the fuel tank to be tested under vibration interference and non-steady flight conditions, and can effectively realize the high-precision measurement of the remaining fuel quantity of the fuel tank to be tested under the flight jitter state. In addition, multi-information fusion can also provide the model with anti-interference ability, so that it can adapt to different flight attitudes and working conditions, improve adaptability (strong robustness), and ensure the accuracy of real-time prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A flowchart of a first embodiment of a method for monitoring the fuel level in an aircraft tank provided by the present application; Figure 2 A schematic diagram of an oil level monitoring system shown as an exemplary embodiment of the present application; Figure 3 A schematic diagram of an oil quantity prediction model exemplarily shown in the present application; Figure 4 A flowchart of Embodiment 2 of the method for monitoring the fuel level in an aircraft tank provided by the present application; Figure 5 This is a flow chart of Example 3 of the aviation aircraft tank fuel quantity monitoring method provided in this application. DETAILED DESCRIPTION

[0009] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0010] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0011] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0012] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0013] Figure 1 Please refer to the flowchart of the first embodiment of the method for monitoring the fuel level of an aircraft tank provided by this application. Figure 1 The fuel quantity monitoring method for an aircraft fuel tank provided in this embodiment is applied to a fuel quantity monitoring system, wherein the fuel quantity monitoring system comprises a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor, and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the fuel tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the fuel tank to be tested; the method comprises: S101, the visual sensor collects a liquid surface image of the oil tank to be tested, the vibration sensor collects a vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor collects two-dimensional inclination information of the oil tank to be tested.

[0014] The present application provides an aircraft fuel tank fuel quantity monitoring method, which is applied to a fuel quantity monitoring system. Figure 2 This is a schematic diagram of an oil level monitoring system shown in an exemplary embodiment of the present application. Figure 2 , the oil quantity monitoring system is installed on the oil tank to be tested, and the oil quantity monitoring system includes a visual sensor (in one embodiment, the visual sensor can be a camera), a vibration sensor, a two-dimensional tilt sensor, and a processor electrically connected to the visual sensor, the vibration sensor and the two-dimensional tilt sensor; the visual sensor is arranged obliquely above the oil tank to be tested, and the vibration sensor and the two-dimensional tilt sensor are arranged at the top or bottom of the oil tank to be tested. Among them, the front end of the visual sensor is facing the inside of the oil tank to be tested, and is used to collect the liquid level image of the oil tank to be tested, which can fully collect the liquid level image from full oil load to low oil state; the vibration sensor is used to detect the vibration signal of the oil tank to be tested, and realize the measurement of the vibration interference intensity; the two-dimensional tilt sensor can collect the two-dimensional tilt information of the oil tank to be tested. It should be noted that the vibration signal is a time series signal, which can record the process of vibration amplitude changing with time; the two-dimensional tilt information refers to the tilt angle of the oil tank to be tested around the pitch axis and roll axis.

[0015] Specifically, the specific installation positions of the two-dimensional inclination sensor and the vibration sensor are set according to actual needs, and are not limited in this embodiment. For example, in this embodiment, the two-dimensional inclination sensor and the vibration sensor are both set at the bottom of the oil tank to be tested; for another example, in another embodiment, the two-dimensional inclination sensor and the vibration sensor can also be set at the top of the oil tank to be tested.

[0016] It should be noted that the vibration sensor works by detecting the acceleration or displacement of the vibration of the oil tank to be tested. It is usually an acceleration sensor made of piezoelectric materials (such as piezoelectric ceramics). When the oil tank to be tested vibrates, the piezoelectric material generates an electric charge proportional to the vibration amplitude, which is converted into an electrical signal output through the circuit. In addition, the vibration sensor is usually installed on the top or bottom of the oil tank to be tested, as close as possible to the area that may be most affected by the vibration. In addition, the part of the oil tank to be tested with strong rigidity is selected to reduce the influence of environmental noise and ensure that the vibration sensor can accurately collect the vibration signal.

[0017] Furthermore, the two-dimensional inclination sensor usually has two mutually perpendicular measuring axes (pitch axis and roll axis). When the oil tank to be tested tilts, gravity will act on the internal measuring element, causing its output electrical signal to change. By calculating the gravity components on the two axes, the inclination of the oil tank to be tested around the pitch axis and roll axis can be obtained. In addition, the two-dimensional inclination sensor needs to be installed on the top or bottom plane of the oil tank to be tested to ensure that the pitch and roll angle changes of the oil tank to be tested relative to the ground plane can be accurately sensed.

[0018] Please continue to refer to Figure 2 The visual sensor, the vibration sensor and the two-dimensional inclination sensor are all electrically connected to the processor. The processor is used to receive the liquid level image collected by the visual sensor, the vibration signal collected by the vibration sensor, and the two-dimensional inclination information collected by the two-dimensional inclination sensor, and predict the oil volume at the current moment based on the liquid level image, the vibration signal and the two-dimensional inclination information.

[0019] In specific implementation, the visual sensor collects the liquid surface image in real time, the vibration sensor collects the vibration signal in real time, and the two-dimensional inclination sensor collects the two-dimensional inclination information in real time, so as to realize the real-time online measurement of the dynamic oil volume of the oil tank to be tested.

[0020] S102, the processor inputs the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to use the oil quantity prediction model to perform feature extraction on the liquid surface image to obtain a first high-dimensional feature, and use the oil quantity prediction model to perform feature extraction on the vibration signal to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall.

[0021] S103: The processor uses the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature.

[0022] S104: The processor predicts the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

[0023] It should be noted that a pre-trained oil quantity prediction model is deployed on the processor, and the oil quantity prediction model is used to predict the oil quantity at a certain moment based on the liquid surface image, vibration signal and two-dimensional inclination information at that moment. The input of the oil quantity prediction model is the liquid surface image, vibration signal and two-dimensional inclination information, and the output is the predicted oil quantity. After the liquid surface image, vibration signal and two-dimensional inclination information are input into the oil quantity prediction model, it can perform feature extraction on the liquid surface image to obtain the first high-dimensional feature, perform feature extraction on the vibration signal to obtain the second high-dimensional feature, and then fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain the global feature, and predict the current oil quantity of the oil tank to be tested based on the global feature.

[0024] Specifically, Figure 3 This is a schematic diagram of an exemplary oil volume prediction model of this application. Figure 3 In a possible implementation, the oil quantity prediction model may include a first feature extraction network, a second feature extraction network and a feature fusion network, which may utilize the first feature extraction network to extract features from the liquid surface image, utilize the second feature extraction network to extract features from the vibration signal, and utilize the feature fusion network to fuse the first high-dimensional features extracted by the first feature extraction network, the second high-dimensional features extracted by the second feature network, and the input two-dimensional inclination information, and then utilize the feature fusion network to predict the oil quantity based on the global features.

[0025] A specific embodiment will be given below to introduce the training process of the oil quantity prediction model in detail, which will not be repeated here.

[0026] It should be noted that, in this embodiment, the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall surface, so as to improve the accuracy.

[0027] The fuel quantity monitoring method for aircraft fuel tanks provided in this embodiment collects liquid surface images through visual sensors, and simultaneously fuses the real-time vibration signal and two-dimensional inclination information of the fuel tank to be tested, and then adaptively extracts and fuses the collected liquid surface photos, real-time vibration signals, and two-dimensional inclination information based on the pre-trained fuel quantity prediction model, so as to more comprehensively capture the state changes of the fuel tank to be tested, and finally realize the real-time measurement of the remaining fuel quantity of the fuel tank to be tested under vibration interference and non-steady flight conditions, and can effectively realize the high-precision measurement of the remaining fuel quantity of the fuel tank to be tested under the flight jitter state. In addition, multi-information fusion can also provide the model with anti-interference ability, so that it can adapt to different flight attitudes and working conditions, improve adaptability (strong robustness), and ensure the accuracy of real-time prediction.

[0028] Figure 4 Please refer to the flowchart of the second embodiment of the method for monitoring the fuel level of an aircraft tank provided by this application. Figure 4 Based on the above embodiment, the method provided in this embodiment, the training process of the oil quantity prediction model may include: S401, collecting multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft.

[0029] It should be noted that the multiple sets of data are multiple sets of data of the tested fuel tank under different fuel weights, wherein the sample data of each set of data includes a real-time liquid level image, a real-time vibration signal and real-time two-dimensional inclination information, and the label of the set of data is the fuel weight corresponding to the sample data.

[0030] It is understandable that in a typical flight state of an aircraft (it should be noted that the typical flight state of an aircraft includes take-off, cruising, and landing), as the aircraft flies, the weight of the fuel in the fuel tank to be tested is constantly decreasing, and with different flight states, the distribution and flow of the fuel in the fuel tank to be tested are also different. This will cause certain difficulties in measuring the amount of oil. By collecting multiple sets of data under different fuel weights, based on the real-time liquid level image, real-time vibration signal and real-time two-dimensional inclination information in each set of data, it can help the model to better learn the relationship between fuel weight and various characteristic information, thereby improving the accuracy of the model's prediction of the amount of oil.

[0031] Optionally, in a possible implementation, the collecting of multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft includes: (1) Constructing a simulation system; wherein the simulation system includes a flight state simulation device, a fuel tank to be tested placed on the flight state simulation device, a weight test device for testing the weight of fuel, a vibration table for simulating the vibration state of the fuel tank to be tested under an actual service state, and the fuel quantity monitoring system.

[0032] (2) Using the simulation system to collect multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft; wherein, during the collection process, the vibration table is used to simulate the vibration state, and in the process of gradually draining the oil, the liquid surface image, vibration signal, two-dimensional inclination information, and the corresponding fuel weight are collected in real time, and then the collected liquid surface image, vibration signal, and two-dimensional inclination information are used as sample data, and the corresponding fuel weight is used as a label to form multiple sets of data.

[0033] Specifically, in order to collect multiple sets of data of the fuel tank to be tested under typical flight conditions of an aircraft, a simulation system is constructed to simulate multiple typical flight conditions of an aircraft and collect multiple sets of data of the fuel tank to be tested.

[0034] It should be noted that the simulation system includes a flight state simulation device, a fuel tank to be tested placed on the flight state simulation device, a weight test device for testing the weight of fuel, a vibration table for simulating the vibration state of the fuel tank to be tested under the actual service state, and a fuel quantity monitoring system. In specific implementation, the fuel tank to be tested, the fuel quantity monitoring system, the weight measurement device, and the vibration table arranged on the fuel tank to be tested can be placed on the flight state simulation device to simulate the vibration state of the fuel tank to be tested under the typical flight state of an aircraft.

[0035] Furthermore, after the simulation system is constructed, the simulation system can be used to collect multiple sets of data of the fuel tank to be tested under the typical flight state of the aircraft.

[0036] Please note that, please refer to Figure 2 The measured oil quantity also has a fuel inlet valve and a fuel outlet valve. In the process of data collection, the vibration table, the oil quantity monitoring system, the fuel outlet valve, and the weight measuring device can be turned on to drain the oil under vibration. At the same time, the oil quantity monitoring system collects the liquid surface image, vibration signal, and two-dimensional inclination signal in real time, and the weight measuring device collects the corresponding fuel weight synchronously. Furthermore, the real-time liquid surface image, real-time vibration signal, and real-time two-dimensional inclination information are used as sample data, and the corresponding fuel weight is used as a label to form a "real-time liquid surface image, real-time vibration signal, real-time two-dimensional inclination information and real-time fuel weight" data set.

[0037] The method provided in this embodiment constructs a simulation system and uses the simulation system to collect multiple sets of data on the fuel tank to be tested under typical flight conditions of an aircraft. In this way, various flight conditions of an aircraft can be simulated in a ground environment, and data similar to the actual flight conditions can be collected, thereby providing rich and accurate training data for the fuel quantity prediction model, thereby improving the accuracy of the trained fuel quantity prediction model in predicting the fuel quantity of the fuel tank to be tested.

[0038] S402: construct an initial model, wherein the initial model includes a first feature extraction network, a second feature extraction network, and a feature fusion network.

[0039] For details, please continue to refer to Figure 3 In a possible implementation, the initial model includes a first feature extraction network, a second feature extraction network and a feature fusion network.

[0040] The first feature extraction network is used to extract features from the input liquid surface image. It can be understood that the liquid surface image is a two-dimensional image, and the first feature extraction network is a two-dimensional convolutional neural network to effectively extract spatial features from the liquid surface image.

[0041] Optionally, in a possible implementation, the two-dimensional convolutional neural network includes 3 to 5 convolutional layers to focus on the texture features of the target area where the oil contacts the wall of the oil tank to be tested.

[0042] It should be noted that each convolution layer can extract features at different levels. By setting 3 to 5 convolution layers, the texture features can be focused on. In the specific implementation, the number of convolution layers can be set according to actual needs, and it is not limited here. For example, in one embodiment, the two-dimensional convolutional neural network may include 3 convolutional layers; for another example, in another embodiment, the two-dimensional convolutional neural network may include 4 convolutional layers.

[0043] It can be understood that each convolution layer may include a convolution layer, an activation function layer and a pooling layer.

[0044] Furthermore, the second feature extraction network is used to extract features from the input vibration signal. It can be understood that the vibration signal is time series data, and the second feature extraction network can be a one-dimensional convolutional neural network, so as to process the vibration signal using the one-dimensional convolutional neural network and effectively capture the energy information in the vibration signal.

[0045] Optionally, in a possible implementation, the dimension of the convolution kernel of the first convolution layer of the one-dimensional convolutional neural network should be greater than 7*7 to increase the perception area of ​​the vibration signal and make the neural network pay more attention to the energy information of the vibration signal. It should be noted that the specific dimension of the convolution kernel of the first convolution layer of the one-dimensional convolutional neural network is set according to actual needs and is not limited here. For example, in one embodiment, the dimension of the convolution kernel of the first convolution layer of the one-dimensional convolutional neural network is 32*32.

[0046] It should be noted that the number of convolutional layers included in the one-dimensional convolutional neural network is also set according to actual needs and is not limited in this embodiment.

[0047] Furthermore, the feature fusion network is used to fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the oil volume of the oil tank to be tested based on the fused features. Optionally, in a possible implementation, the feature fusion network is a fully connected neural network, which can be used to effectively fuse the first high-dimensional feature extracted by the first feature extraction network, the second high-dimensional feature extracted by the second feature extraction network, and the input two-dimensional inclination information, thereby improving the accuracy of oil volume prediction.

[0048] The method provided in this embodiment is based on a deep neural network to achieve adaptive extraction of dynamic liquid level images and vibration signal features, and uses a fully connected neural network to fuse multivariate data, avoiding manual screening and extraction of oil-sensitive features, introducing operating condition monitoring data, enhancing the robustness of the algorithm, and improving the algorithm accuracy.

[0049] S403: Using the multiple groups of data to train the initial model to obtain a trained oil quantity prediction model.

[0050] Specifically, after obtaining multiple sets of data of the fuel tank to be tested under typical flight conditions of an aircraft and constructing an initial model, the multiple sets of data can be used to train the initial model to obtain a trained fuel quantity prediction model.

[0051] In specific implementation, multiple groups of data can be divided into training sets and test sets according to a certain ratio. Furthermore, the parameters of the initial model can be trained based on the training set to obtain a trained oil quantity prediction model. Furthermore, the trained oil quantity prediction model is saved; and the measurement accuracy of the trained oil quantity prediction model is tested based on the test set to evaluate the performance of the trained oil quantity prediction model.

[0052] The fuel quantity monitoring method for a fuel tank to be tested of an aircraft provided in the present embodiment collects multiple sets of data on the fuel quantity to be tested under a typical flight state of the aircraft, thereby constructing an initial model, and using the multiple sets of data to train the initial model to obtain a trained fuel quantity prediction model. In this way, the trained fuel quantity prediction model can be combined with liquid surface images, vibration signals and two-dimensional inclination information to predict the fuel quantity, so as to improve the accuracy of the model in predicting the fuel quantity. In addition, the trained fuel quantity prediction model can be adaptively adjusted according to the characteristics of the fuel tank to be tested under different flight states, so as to accurately predict the fuel quantity in real time.

[0053] Figure 5 This is a flow chart of Embodiment 3 of the method for monitoring the fuel level of an aircraft tank provided by this application. Figure 5 Based on the above embodiment, the method provided in this embodiment further includes: S501. After the initial model is trained, a gradient weighted class activation mapping (Grad-CAM) visualization tool is used to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area to obtain an evaluation result.

[0054] It should be noted that the effectiveness of each convolutional layer in the first feature extraction network in extracting the texture features of the target area is not the same. In this step, after the initial model is trained, the correlation between the features extracted by each convolutional layer and the texture features of the target area can be evaluated, and the evaluation results can be used to determine which layers extract features that best represent the texture features of the contact area between the oil and the tank wall to be tested.

[0055] In specific implementation, the Gradient-weighted Class Activation Mapping (Grad-CAM) visualization tool can be used to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area.

[0056] It should be noted that the working principle of the Grad-CAM visualization tool is as follows: Step 1: First, the Grad-CAM visualization tool calculates the gradient of the model's output for a specific category relative to the output feature map of a specific convolutional layer. These gradients reflect the importance of the features extracted by the model in this convolutional layer to the final output.

[0057] Step 2: Use these gradients to weight the feature maps of the convolutional layer. Specifically, the gradient of each feature map is pooled through a global average operation to obtain a weight value, which reflects the contribution of the feature map to the category.

[0058] Step 3: Perform ReLU activation on the weighted feature map to generate a heat map, which indicates which areas in the image contribute more to the prediction of a specific category, where the highlighted area represents the part that has the greatest impact on the model decision.

[0059] In step 4, the heat map is superimposed on the original image to visually see which areas are focused on by the model, especially the textured areas where the oil is in contact with the wall of the tank to be tested.

[0060] It should be noted that in this step, the evaluation result is the result of superimposing the thermal map onto the original image (the original liquid surface image).

[0061] S502. Dynamically adjust the trained oil quantity prediction model according to the evaluation result to retain the convolution layers that are effective for extracting the texture features of the target area and delete the convolution layers that are unnecessary for extracting the texture features of the target area.

[0062] Specifically, based on the evaluation results, we can decide which effective convolutional layers to retain and delete those layers that are not very helpful for texture feature extraction, so as to simplify the model, improve efficiency, and ensure that the model focuses more on important features.

[0063] In specific implementation, for example, if a convolution layer has a low correlation with the texture features of the target area, it can be deleted, while the convolution layer with a high correlation with the texture features of the target area can be retained, so that the retained convolution layer can accurately extract the texture features of the target area and improve the accuracy of oil volume prediction.

[0064] S503, retraining the adjusted model, and again executing the step of using the gradient weighted class activation mapping Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area, until the trained model can effectively extract the texture features of the target area.

[0065] Specifically, after the oil quantity prediction model is adjusted, it is retrained so that it can adapt to the structural changes brought about by the deletion of convolutional layers with low correlation. Subsequently, the step of using the gradient weighted class activation mapping Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area is performed again, and then the process is continuously repeated to achieve the gradual optimization of the oil quantity prediction model until the model can effectively extract the texture features of the target area.

[0066] The method for monitoring the fuel quantity of an aircraft tank provided in this embodiment, after training the initial model, evaluates the correlation between the features extracted by each convolution layer in the first feature extraction network and the texture features of the target area by using the gradient weighted class activation mapping Grad-CAM visualization tool to obtain an evaluation result, and then dynamically adjusts the trained fuel quantity prediction model according to the evaluation result to retain the convolution layers that are effective for extracting the texture features of the target area and delete the convolution layers that are unnecessary for extracting the texture features of the target area, and then retrains the adjusted model, and again executes the step of evaluating the correlation between the features extracted by each convolution layer in the first feature extraction network and the texture features of the target area by using the gradient weighted class activation mapping Grad-CAM visualization tool until the trained model can effectively extract the texture features of the target area. In this way, not only can the final model be more focused on extracting relevant features, thereby improving its ability to capture the texture features of the target area, so as to ultimately improve the accuracy of fuel quantity prediction and the overall performance of the model, but also unnecessary convolution layer calculations can be avoided to reduce the complexity of the model.

[0067] Please continue to refer to Figure 2 The present application also provides an oil quantity monitoring system. The oil quantity monitoring system provided by the present application is introduced as follows: The oil quantity monitoring system provided by the present application comprises a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the oil tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the oil tank to be tested; The visual sensor is used to collect the liquid level image of the oil tank to be tested, the vibration sensor is used to collect the vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor is used to collect the two-dimensional inclination information of the oil tank to be tested; The processor is used to input the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to extract features of the liquid surface image using the oil quantity prediction model to obtain a first high-dimensional feature, and to extract features of the vibration signal using the oil quantity prediction model to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall; The processor is further used to use the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature; The processor is further configured to predict the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

[0068] Optionally, the training process of the oil quantity prediction model includes: Collect multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft; wherein the multiple sets of data are multiple sets of data of the fuel tank to be tested under different fuel weights; the sample data of each set of data includes a real-time liquid level image, a real-time vibration signal and real-time two-dimensional inclination information, and the label of the set of data is the fuel weight corresponding to the sample data; Constructing an initial model; wherein the initial model includes a first feature extraction network, a second feature extraction network and a feature fusion network; the first feature extraction network is used to extract features from an input liquid surface image; the second feature extraction network is used to extract features from an input vibration signal; the feature fusion network is used to fuse the first high-dimensional features extracted by the first feature extraction network, the second high-dimensional features extracted by the second feature network, and the input two-dimensional inclination information, and predict the oil volume of the oil tank to be tested based on the fused features; The initial model is trained using the multiple sets of data to obtain a trained oil quantity prediction model.

[0069] Optionally, the first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers; And / or, the second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of ​​the vibration signal.

[0070] Referring to the previous description, the oil level monitoring system provided in the present application has been introduced in the previous description and will not be described in detail here.

[0071] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for monitoring the fuel level in an aircraft tank, characterized in that: The method for monitoring the oil quantity of an aircraft tank is applied to an oil quantity monitoring system, which includes a visual sensor, a vibration sensor, a two-dimensional inclination sensor, and a processor electrically connected to the visual sensor, the vibration sensor, and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the oil tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged at the top or bottom of the oil tank to be tested; the method includes: The visual sensor collects the liquid level image of the oil tank to be tested, the vibration sensor collects the vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor collects the two-dimensional inclination information of the oil tank to be tested; The processor inputs the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to extract features of the liquid surface image using the oil quantity prediction model to obtain a first high-dimensional feature, and extract features of the vibration signal using the oil quantity prediction model to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall; The processor uses the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature; The processor predicts the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

2. The method according to claim 1, characterized in that The training process of the oil quantity prediction model includes: Collect multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft; wherein the multiple sets of data are multiple sets of data of the fuel tank to be tested under different fuel weights; the sample data of each set of data includes a real-time liquid level image, a real-time vibration signal and real-time two-dimensional inclination information, and the label of the set of data is the fuel weight corresponding to the sample data; Constructing an initial model; wherein the initial model includes a first feature extraction network, a second feature extraction network and a feature fusion network; the first feature extraction network is used to extract features from an input liquid surface image; the second feature extraction network is used to extract features from an input vibration signal; the feature fusion network is used to fuse the first high-dimensional features extracted by the first feature extraction network, the second high-dimensional features extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the oil volume of the oil tank to be tested based on the fused features; The initial model is trained using the multiple sets of data to obtain a trained oil quantity prediction model.

3. The method according to claim 2, characterized in that The method further comprises: After the initial model is trained, a Gradient Weighted Class Activation Mapping (Grad-CAM) visualization tool is used to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area to obtain an evaluation result; Dynamically adjusting the trained oil volume prediction model according to the evaluation result to retain the convolution layers that are effective for extracting the texture features of the target area and delete the convolution layers that are unnecessary for extracting the texture features of the target area; Retrain the adjusted model and re-perform the step of using the gradient weighted class activation mapping Grad-CAM visualization tool to evaluate the correlation between the features extracted by each convolutional layer in the first feature extraction network and the texture features of the target area until the trained model can effectively extract the texture features of the target area.

4. The method according to claim 2, characterized in that: The first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers.

5. The method according to claim 2 or 4, characterized in that: The second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of ​​the vibration signal.

6. The method according to claim 5, characterized in that The feature fusion network is a fully connected neural network.

7. The method according to claim 2, characterized in that The collecting of multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft includes: Constructing a simulation system; wherein the simulation system includes a flight state simulation device, a fuel tank to be tested placed on the flight state simulation device, a weight test device for testing the weight of fuel, a vibration table for simulating the vibration state of the fuel tank to be tested in an actual service state, and the fuel quantity monitoring system; The simulation system is used to collect multiple sets of data of the fuel tank to be tested under the typical flight state of the aircraft; wherein, during the collection process, the vibration table is used to simulate the vibration state, and in the process of gradually draining the oil, the liquid surface image, vibration signal, two-dimensional inclination information, and the corresponding fuel weight are collected in real time, and then the collected liquid surface image, vibration signal, two-dimensional inclination information are used as sample data, and the corresponding fuel weight is used as a label to form multiple sets of data.

8. An oil quantity monitoring system, characterized in that: The oil quantity monitoring system comprises a visual sensor, a vibration sensor, a two-dimensional inclination sensor and a processor electrically connected to the visual sensor, the vibration sensor and the two-dimensional inclination sensor; the visual sensor is arranged obliquely above the oil tank to be tested, and the vibration sensor and the two-dimensional inclination sensor are arranged on the top or bottom of the oil tank to be tested; wherein the visual sensor is used to collect the liquid surface image of the oil tank to be tested, the vibration sensor is used to collect the vibration signal of the oil tank to be tested, and the two-dimensional inclination sensor is used to collect the two-dimensional inclination information of the oil tank to be tested; The processor is used to input the liquid surface image, the vibration signal and the two-dimensional inclination information into a pre-trained oil quantity prediction model, so as to extract features of the liquid surface image using the oil quantity prediction model to obtain a first high-dimensional feature, and to extract features of the vibration signal using the oil quantity prediction model to obtain a second high-dimensional feature; wherein the first high-dimensional feature focuses on the texture features of the target area where the oil contacts the wall; The processor is further used to use the oil quantity prediction model to fuse the first high-dimensional feature, the second high-dimensional feature and the two-dimensional inclination information to obtain a global feature; The processor is further configured to predict the current oil level of the oil tank to be tested by using the oil level prediction model based on the global feature.

9. The oil level monitoring system according to claim 8, characterized in that: The training process of the oil quantity prediction model includes: Collect multiple sets of data of the fuel tank to be tested under a typical flight state of an aircraft; wherein the multiple sets of data are multiple sets of data of the fuel tank to be tested under different fuel weights; the sample data of each set of data includes a real-time liquid level image, a real-time vibration signal and real-time two-dimensional inclination information, and the label of the set of data is the fuel weight corresponding to the sample data; Constructing an initial model; wherein the initial model includes a first feature extraction network, a second feature extraction network and a feature fusion network; the first feature extraction network is used to extract features from an input liquid surface image; the second feature extraction network is used to extract features from an input vibration signal; the feature fusion network is used to fuse the first high-dimensional features extracted by the first feature extraction network, the second high-dimensional features extracted by the second feature extraction network, and the input two-dimensional inclination information, and predict the oil volume of the oil tank to be tested based on the fused features; The initial model is trained using the multiple sets of data to obtain a trained oil quantity prediction model.

10. The oil level monitoring system according to claim 9, characterized in that: The first feature extraction network is a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes 3 to 5 convolutional layers; And / or, the second feature extraction network is a one-dimensional convolutional neural network, and the dimension of the convolution kernel of the first convolutional layer of the one-dimensional convolutional neural network is greater than 7*7 to increase the perception area of ​​the vibration signal.

Images