Aircraft fuel tank fuel quantity monitoring device based on machine vision and calibration and monitoring method

Through the aircraft fuel tank oil volume monitoring device based on machine vision, the liquid line detection is performed using image sensors and calibration devices, and the existing capacitive sensor calibration problems are solved, and the oil volume is achieved with high precision and low cost Monitoring, suitable for a variety of fuel tank models, and has expanded the abnormal situation monitoring function.

CN119085800AInactive Publication Date: 2024-12-06NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411014643.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing aircraft fuel tank oil volume monitoring method relies on capacitive liquid level sensors. The calibration process is complex and costly, and is susceptible to electromagnetic interference. The existing visual methods have complex system structures and are not suitable for practical applications.

Method used

The aircraft fuel tank oil volume monitoring device based on machine vision, including image sensors and calibration devices, fix the image sensor by installing a clamp, and calibrating the imaging parameters are calibrated using illumination light sources and calibration target plates. The pixel coordinates of the liquid surface line are extracted and calculated by combining the image processing algorithm. Three-dimensional coordinates and fit the oil surface to achieve oil volume monitoring.

Benefits of technology

It realizes high accuracy, low cost, simple structure and convenient calibration of oil tank oil volume measurement. It is suitable for complex structure oil tanks, and can flexibly expand monitoring areas and functions, with good versatility and engineering application value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085800A_ABST
    Figure CN119085800A_ABST
Patent Text Reader

Abstract

The invention discloses an aircraft fuel tank fuel quantity monitoring device based on machine vision and a calibration and monitoring method. The monitoring device comprises one or more image sensors, an image sensor installation clamp and a calibration device. The image sensor is fixedly connected to the oil tank or the calibration device through the installation clamp, and the calibration device is used for completing calibration of imaging parameters and related transformation matrixes of the image sensor. Performing image processing on the internal image of the fuel tank acquired by each image sensor to obtain characteristics of a liquid level line where the fuel oil surface intersects with the tank body and accessories thereof; obtaining a three-dimensional point column of a liquid level line observed by each image sensor under a fuel tank coordinate system according to a calibration result and information such as fuel tank mathematical model, and fitting a fuel oil surface by using the three-dimensional point columns of the liquid level lines obtained by all the image sensors; and according to the fuel oil surface fitting result and the fuel tank three-dimensional digital-analog calculation, the remaining oil quantity under the fuel oil surface is obtained. The device has the advantages of being convenient in calibration process, flexible in structure layout and high in oil mass monitoring precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of machine vision detection, and in particular to an aircraft fuel tank oil quantity monitoring device based on machine vision and a calibration and monitoring method. Background Art

[0002] The aircraft fuel tank is an important component of the aircraft. The measurement accuracy of the fuel volume in the tank directly affects the pilot's judgment of the remaining flight distance. The current actual method of monitoring the fuel volume of aircraft fuel tanks is to obtain the fuel level by arranging multiple capacitive liquid level sensors inside the tank, and then find the fuel volume characteristic table based on the aircraft's attitude angle and other information, and interpolate and calculate the fuel value. However, the calibration process of capacitive sensors is complicated. Each sensor needs to be calibrated to obtain the liquid level height corresponding to the capacitance value when the tank is empty or full, resulting in high calibration, installation and maintenance costs. In addition, capacitive sensors are easily affected by electromagnetic interference, and corresponding shielding measures need to be taken during installation. Therefore, the aviation field has an urgent desire to adopt new fuel volume monitoring technology to replace the fuel volume monitoring method based on capacitive sensors.

[0003] In addition, some scholars have conducted research on aircraft fuel quantity monitoring through liquid level measuring instruments such as fiber optic sensors and magnetostrictive sensors, but they are not yet practical. Some researchers have used multiple cameras or one camera to measure the liquid level, but these existing visual methods require the use of laser and structured light equipment to monitor the specific liquid level position of the oil. Therefore, the system structure and control are complex and not convenient for installation and application in actual aircraft fuel tanks. It is of great significance and commercial value to develop an aircraft fuel tank fuel quantity monitoring system with strong applicability and good monitoring accuracy to replace the capacitive liquid level sensor currently used. Summary of the invention

[0004] In order to solve the problems of the prior art, the present invention provides an aircraft fuel tank oil quantity monitoring device and a calibration and monitoring method based on machine vision, which has the advantages of convenient calibration process, flexible structure layout and high oil quantity monitoring accuracy.

[0005] The present invention provides an aircraft fuel tank fuel quantity monitoring device based on machine vision, comprising one or more image sensors and a calibration device, each image sensor is provided with a mounting fixture, the image sensor is fixedly connected to the fuel tank or the calibration device through the mounting fixture, and is used to observe the liquid level line where the oil surface of the fuel tank intersects with the fuel tank wall; the calibration device is used to complete the calibration of the image sensor imaging parameters and the transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system, and the image sensor coordinate system and the fuel tank coordinate system.

[0006] As a further improvement, the image sensor includes an imaging camera and an imaging lens, and is provided with an illumination light source used in conjunction with the imaging camera and the imaging lens. The illumination light source is fixed to the mounting fixture via a light source bracket.

[0007] Further improvement, the calibration device includes a fixed bracket, a calibration target plate and a positioning block for positioning the calibration target plate, the image sensor is fixed above the calibration target plate by the fixed bracket, the fixed bracket and the positioning block ensure a clear positional relationship through processing accuracy, and there are more than four marking points on the calibration target plate for visual identification and positioning.

[0008] The present invention also provides a calibration method for an aircraft fuel tank fuel quantity monitoring device based on machine vision, comprising the following steps:

[0009] Step 1) Fix the image sensor on the calibration device through the mounting fixture, and calibrate the imaging parameters of each image sensor and the transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system through the calibration device.

[0010]

[0011] 1.1) The image sensor connected to the calibration device takes n images of the calibration target plate in different positions, n ≥ 3, and ensures that the nth calibration target plate position is determined by the positioning block, and uses the known position relationship between the fixed bracket and the positioning block to obtain the transformation matrix from the camera mounting fixture coordinate system to the calibration plate coordinate system in the nth position

[0012]

[0013] 1.2) Process the n calibration plate images captured by each image sensor in step 1.1) and calculate the image sensor internal parameter matrix K by plane calibration method C , distortion correction coefficient vector d C , the transformation matrix from the calibration plate coordinate system to the image sensor coordinate system

[0014] 1.3) According to the results obtained in step 1.1) and step 1.2) and Calculate the transformation matrix from each image sensor mounting fixture coordinate system to the image sensor coordinate system

[0015] Step 2) According to the coordinates of the mounting fixture in the fuel tank coordinate system, obtain the transformation matrix between the image sensor mounting fixture coordinate system and the aircraft fuel tank coordinate system according to and the transformation matrix between the image sensor coordinate system and the mounting fixture coordinate system calibrated in step 1) Get the transformation matrix between the aircraft tank coordinate system and each image sensor coordinate system

[0016] The present invention also provides an online monitoring method for the fuel level of an aircraft fuel tank based on machine vision, comprising the following steps:

[0017] Step 1) The image sensor is installed on the aircraft fuel tank through the installation fixture. Each image sensor collects the image inside the fuel tank and transmits it back to the aircraft fuel tank fuel quantity monitoring system software. The monitoring system software extracts the pixel coordinates of the liquid level line where the fuel surface intersects with the fuel tank body and the accessories inside the fuel tank in the image returned by each image sensor through an image processing algorithm.

[0018] Step 2) Calculate the three-dimensional coordinates of a series of pixel points on the liquid surface observed by each image sensor extracted in step 1) in the tank coordinate system according to the calibration results obtained by calibration and the known three-dimensional digital model of the tank;

[0019] The specific process is as follows:

[0020] According to the camera model, the two-dimensional pixel points (u i ,v i ) is the three-dimensional coordinate point (x) in the tank coordinate system after distortion correction. i ,y i ,x i )satisfy:

[0021]

[0022] In formula (1), the subscript i represents any point on the liquid surface observed by each image sensor, K C and Represents the intrinsic parameter matrix of each image sensor and the transformation matrix from the tank coordinate system to the corresponding image sensor coordinate system;

[0023] After reorganizing the above formula, we can get:

[0024]

[0025] Formula (2) is the three-dimensional point (x i ,y i ,z i ) is located, and the straight line equation is used to intersect with the three-dimensional digital model of the fuel tank body and its internal accessories. The intersection point closest to the camera center is the pixel point on the liquid surface observed by the image sensor (u i ,v i ) corresponds to the three-dimensional point (x i ,y i ,zi ), the oil surface is fitted using the three-dimensional coordinate points in the tank coordinate system corresponding to the liquid level lines observed by all image sensors.

[0026] Step 3) Using the coordinates of all three-dimensional coordinate points on the liquid surface observed by each image sensor obtained in step 2), the fuel surface in the tank coordinate system is fitted, and the remaining fuel volume below the fuel surface is calculated based on the fitted fuel surface parameters and the three-dimensional digital model of the fuel tank. The specific process is as follows:

[0027] 3.1) Fitting the spatial plane equation of the fuel surface using all l three-dimensional coordinate points corresponding to the liquid surface lines observed by all image sensors. The RANSAC algorithm is used in the plane fitting algorithm to eliminate the three-dimensional points that are far away from the fitting plane, and obtain the spatial plane equation parameter p of the fuel surface;

[0028] 3.2) Based on the three-dimensional digital model of the internal space of the fuel tank, before implementing online monitoring, the remaining fuel volume in the fuel tank under a given series of oil level parameters is calculated offline to obtain a fuel volume characteristic table;

[0029] 3.3) According to the spatial plane equation parameter p of the current fuel surface obtained in the online monitoring, the fuel volume characteristic table of the fuel tank established offline is searched, and the remaining fuel volume value under the current oil level is obtained by interpolation calculation.

[0030] The beneficial effects of the present invention are:

[0031] 1. A set of image sensors can be used to measure the amount of fuel in the tank, ensuring high-precision measurement while having the advantages of low cost, simple structure, low maintenance cost, and convenient calibration. For tanks with complex structures, multiple sets of image sensors can also be flexibly used to expand the liquid level monitoring area. There is no need for multiple sets of image sensors to have a common field of view, which is very flexible and convenient to use.

[0032] 2. As long as the aircraft fuel tank has an installation structure that can adapt to the image sensor mounting fixture, each set of image sensors only needs to complete an offline calibration through the calibration device, and can be installed on aircraft fuel tanks of different models to carry out fuel quantity monitoring. It has good versatility and is ready to use, which is very convenient and has extremely high engineering application value.

[0033] 3. Compared with the current mainstream capacitive fuel level sensor, the visual sensor-based method proposed in the present invention can directly transmit back the image inside the fuel tank, and can be expanded to monitor abnormal conditions in the fuel tank, such as oil leakage, foreign objects left behind, etc. At the same time, it can provide pilots and ground maintenance personnel with an intuitive picture of the inside of the fuel tank, which helps relevant personnel understand the various conditions in the fuel tank. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a schematic diagram of the structure of the oil tank and its internal accessories in an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the structure of an image sensor and a mounting fixture in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the structure of a calibration device in an embodiment of the present invention;

[0038] Figure 4 The present invention provides a flow chart of an algorithm for generating a fuel volume characteristic table in the method of the present invention.

[0039] In the figure, 1-imaging camera, 2-imaging lens, 3-mounting fixture, 4-illumination light source, 5-light source bracket, 6-image sensor, 7-calibration plate, 8-fixed bracket, 9-positioning block. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The fuel tank body and its internal accessories of this embodiment are shown in the attached Figure 1 As shown, it includes an irregularly shaped box body and reinforcing ribs, pipe clamps, axle seats, fuel supply pumps, ejection pumps, fuel delivery valves, fuel delivery valve control rods, fuel delivery pipelines and other internal accessories. This embodiment uses a group of image sensors to monitor the remaining fuel amount in the fuel tank.

[0042] The monitoring device includes one or more image sensors 6, the image sensor includes an imaging camera 1 and an imaging lens 2, and an illumination light source 4 used in conjunction with the imaging camera and the imaging lens. The illumination light source 4 is fixed to the mounting fixture 3 via a light source bracket 5.

[0043] Each image sensor is equipped with a mounting fixture 3 and a calibration device. The image sensor is fixedly connected to the oil tank or the calibration device through the mounting fixture, and is used to observe the liquid level line where the oil surface of the oil tank intersects with the oil tank wall; the calibration device is used to complete the calibration of the image sensor imaging parameters and the transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system, and the transformation matrix between the image sensor coordinate system and the oil tank coordinate system.

[0044] The image sensor related components include a 2 / 3 inch target size, 5 million pixel CMOS camera, an industrial lens with a focal length of 8 mm, a ring light source, and a light source bracket; the image sensor mounting fixture is connected to the camera and is used to install the image sensor on the fixed bracket of the calibration device or on the fuel tank body. The image sensor and image sensor mounting fixture are as attached. Figure 2 As shown. In this embodiment, the calibration device includes a calibration plate 7, a fixed bracket 8, and a positioning block 9. The mechanical structure on the fixed bracket for connecting the image sensor mounting fixture and the mechanical structure on the aircraft fuel tank for connecting the image sensor mounting fixture both have three positioning holes with a diameter of 8 mm distributed in a triangle. The positioning block of the calibration target plate and the three positioning holes with a diameter of 8 mm on the bracket for connecting the image sensor mounting fixture are processed to ensure that they have a certain known position relationship, as shown in the attached Figure 3 In this embodiment, there are 99 marking points on the calibration target plate that can be visually identified and positioned.

[0045] Before the image sensor and the image sensor mounting fixture are used for fuel tank monitoring, they are first calibrated by a calibration device, as follows:

[0046] (1) Connect the image sensor to the calibration device, as shown in the attached figure. Figure 3 As shown in the figure, 10 images of the calibration target plate in different positions are taken. The calibration target plate in the 10th position is placed close to the calibration target plate positioning block. Therefore, the transformation matrix from the image sensor mounting fixture coordinate system to the calibration plate coordinate system in the 10th position can be obtained according to the known position relationship between the three positioning mounting holes on the bracket for connecting the image sensor mounting fixture and the calibration target plate positioning block.

[0047] (2) The 10 calibration plate images captured by the image sensor are processed and the image sensor internal parameter matrix K is calculated using Zhang Zhengyou's plane calibration method. C , distortion correction coefficient vector d C , the transformation matrix from the calibration plate coordinate system to the image sensor coordinate system

[0048] (3) Based on and Calculate the transformation matrix from the image sensor mounting fixture coordinate system to the image sensor coordinate system

[0049] (4) According to the coordinates of the three positioning holes on the aircraft fuel tank for connecting the image sensor mounting fixture in the fuel tank coordinate system, the transformation matrix between the image sensor mounting fixture coordinate system and the aircraft fuel tank coordinate system is obtained: according to The transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system obtained by calibration Get the transformation matrix between the aircraft tank coordinate system and each image sensor coordinate system

[0050] In this embodiment, the method for online monitoring of the fuel level in an aircraft fuel tank comprises the following steps:

[0051] (1) The calibrated image sensor is installed on the three positioning holes on the aircraft fuel tank through the installation fixture. The image sensor collects the image inside the fuel tank and transmits it back to the aircraft fuel tank fuel quantity monitoring system software. The monitoring system software extracts the pixel coordinates (u) of the liquid level line where the fuel surface intersects with the fuel tank body and the accessories inside the fuel tank in the image sent back by the image sensor through the image processing algorithm. i ,v i ), i = 1, 2, ..., m, where m represents the number of extracted liquid level pixel coordinates;

[0052] In the embodiment of the present invention, the image processing algorithm for extracting the pixel coordinates of the liquid level line where the fuel surface intersects with the fuel tank body and the accessories inside the fuel tank from the image transmitted back by the image sensor is as follows: first, the median filtering technology is used to reduce the noise of the internal image of the fuel tank collected by the visual system, and then the image is enhanced by histogram equalization, and then the intersection line of the fuel surface with the tank body and its accessories is extracted by background difference method. The specific implementation method of the background difference method is: before the system works, a background image of the fuel tank in the empty oil state is collected, and the noise in the image is removed by median filtering. Then, the Canny algorithm is used to extract the contours of the internal accessories and structures in the background image, and the internal accessories in the background image are extracted by morphological closing operation, which is recorded as image B(x, y); after the fuel monitoring system starts working, each collected internal image of the fuel tank is subjected to median filtering and histogram equalization processing, and the double threshold obtained by the OSTU algorithm is used to perform Canny edge extraction to obtain image C(x, y), and the pixel points belonging to image B(x, y) in image C(x, y) are filtered out to obtain image M(x, y), and then the image is subjected to length filtering, α-Shape algorithm and other operations to obtain a series of two-dimensional pixel points (u) on the liquid surface line. i ,v i ),i=1,2,...,m.

[0053] According to the position information of the liquid level line extracted from the current image, the rectangular area surrounding the liquid level line is determined, and the area is used as the search area for extracting the liquid level line in the next image to achieve the effect of fast dynamic tracking and extraction of the liquid level line in adjacent frame images.

[0054] (2) According to the camera model, the two-dimensional pixel points (u i ,v i ) is the three-dimensional coordinate point (x) in the tank coordinate system after distortion correction. i ,y i ,x i ) satisfies formula (1):

[0055]

[0056] Here K C and represents the intrinsic parameter matrix of the image sensor and the transformation matrix from the tank coordinate system to the image sensor coordinate system;

[0057] After rearranging formula (1), we can get formula (2):

[0058]

[0059] Formula (2) is the three-dimensional point (x i ,y i ,z i ) is located, and the straight line equation is used to intersect with the three-dimensional digital model of the fuel tank body and its internal accessories. The intersection point closest to the camera center is the pixel point on the liquid surface observed by the image sensor (u i ,v i ) corresponds to the three-dimensional point (x i ,y i ,z i ), i=1,2,…,m;

[0060] (3) Using all m three-dimensional coordinate points (x i ,y i ,z i ), i = 1, 2, ..., m, fitting the spatial plane equation of the oil surface, and then calculating the remaining oil volume below the fuel surface according to the fitted parameters of the spatial plane equation of the fuel surface, the three-dimensional digital model of the fuel tank and the pre-established fuel volume characteristic table;

[0061] The pitch angle and roll angle of the fuel surface in the tank coordinate system are in the range of [α min ,α max ]、[β min ,βmax ], the change step lengths are recorded as Δα and Δβ respectively, and the initial liquid level height of the fuel surface in the tank coordinate system is d min , the change step is Δd. The plane equation where the oil surface is located is represented by the ternary number p(α, β, d). Figure 4 The fuel volume characteristic table generation algorithm shown in the figure obtains the oil volume V corresponding to each discrete sampling position p on the fuel surface, and obtains the fuel volume characteristic table. Figure 4 The V max Indicates the maximum volume of fuel that can be stored in the tank.

[0062] In this embodiment, in the spatial plane equation fitting algorithm for solving the oil surface based on the liquid level line observed in the monitoring image, the RANSAC algorithm is used to eliminate three-dimensional points with a large distance from the fitting plane, and then the least squares fitting is used to obtain the plane equation parameter p of the fuel surface; based on the three-dimensional digital model of the internal space of the fuel tank, before implementing online monitoring, the remaining oil in the fuel tank under a given series of oil spatial plane parameters is calculated offline to obtain a fuel volume characteristic table; based on the current fuel surface plane equation parameter p obtained in the online monitoring, the offline established fuel tank fuel volume characteristic table is searched, and then the remaining fuel volume value under the current oil plane is calculated using the trilinear interpolation method.

[0063] The above method was applied, and 50L, 60L and 70L of aviation fuel substitute medium were added to the fuel tank of this embodiment. The image sensor and the whole set of devices calibrated by the method of the present invention were used to carry out the oil quantity measurement experiment of the static fuel tank. In addition, a rotating mechanism was used to make the fuel tank sway slightly in the pitch and roll directions, and the dynamic fuel quantity monitoring experiment of the fuel tank under different shaking postures was carried out. The statistical results of 10 static measurements and 10 dynamic monitoring are shown in Table 1.

[0064] Table 1

[0065]

[0066] The experimental results in Table 1 show that the aircraft fuel tank fuel quantity monitoring method based on machine vision proposed in the present invention has high fuel quantity monitoring accuracy and has good engineering application prospects.

[0067] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field is within the technical scope disclosed by the present invention. For ordinary technicians in the technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An aircraft fuel tank fuel quantity monitoring device based on machine vision, characterized in that: The invention comprises one or more image sensors and a calibration device, each image sensor is provided with a mounting fixture, the image sensor is fixedly connected to the oil tank or the calibration device through the mounting fixture, and is used to observe the liquid level line where the oil surface of the oil tank intersects with the oil tank wall; the calibration device is used to complete the calibration of the image sensor imaging parameters and the transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system, and the transformation matrix between the image sensor coordinate system and the oil tank coordinate system.

2. The aircraft fuel tank fuel quantity monitoring device based on machine vision according to claim 1 is characterized in that: The image sensor comprises an imaging camera and an imaging lens, and is provided with an illumination light source used in conjunction with the imaging camera and the imaging lens.

3. The aircraft fuel tank fuel quantity monitoring device based on machine vision according to claim 2 is characterized in that: The illumination light source is fixed on the installation fixture through a light source bracket.

4. The aircraft fuel tank fuel quantity monitoring device based on machine vision according to claim 1 is characterized in that: The calibration device includes a fixed bracket, a calibration target plate and a positioning block for positioning the calibration target plate. The image sensor is fixed above the calibration target plate through the fixed bracket. The fixed bracket and the positioning block ensure a clear positional relationship through processing accuracy. There are more than four marking points on the calibration target plate for visual identification and positioning.

5. A calibration method for an aircraft fuel tank fuel quantity monitoring device based on machine vision, characterized in that The following steps are involved: Step 1) Fix the image sensor on the calibration device through the mounting fixture, and calibrate the imaging parameters of each image sensor and the transformation matrix between the image sensor coordinate system and the image sensor mounting fixture coordinate system through the calibration device. Step 2) According to the coordinates of the mounting fixture in the fuel tank coordinate system, obtain the transformation matrix between the image sensor mounting fixture coordinate system and the aircraft fuel tank coordinate system according to and the transformation matrix between the image sensor coordinate system and the mounting fixture coordinate system calibrated in step 1) Get the transformation matrix between the aircraft tank coordinate system and each image sensor coordinate system 6. The calibration method of the aircraft fuel tank fuel quantity monitoring device based on machine vision according to claim 5 is characterized by: The calibration process of step 1) is as follows: 1.1) The image sensor connected to the calibration device takes n images of the calibration target plate in different positions, n ≥ 3, and ensures that the nth calibration target plate position is determined by the positioning block, and uses the known position relationship between the fixed bracket and the positioning block to obtain the transformation matrix from the image sensor mounting fixture coordinate system to the calibration plate coordinate system in the nth position 1.2) Process the n calibration plate images captured by each image sensor in step 1.1) and calculate the image sensor internal parameter matrix K by plane calibration method C , distortion correction coefficient vector d C , the transformation matrix from the calibration plate coordinate system to the image sensor coordinate system 1.3) According to the results obtained in step 1.1) and step 1.2) and Calculate the transformation matrix from each image sensor mounting fixture coordinate system to the image sensor coordinate system 7. A method for online monitoring of the fuel level in an aircraft fuel tank based on machine vision, characterized in that The following steps are involved: Step 1) The image sensors are mounted on the aircraft fuel tank through a mounting fixture, and each image sensor collects an image of the interior of the fuel tank and transmits it back to the aircraft fuel tank fuel quantity monitoring system software. The monitoring system software extracts the pixel coordinates of the liquid level line where the fuel surface intersects with the fuel tank body and the accessories inside the fuel tank in the image transmitted back by each image sensor through an image processing algorithm; Step 2) calibrating the aircraft fuel tank fuel quantity monitoring device based on machine vision and combining it with the known three-dimensional digital model of the fuel tank to calculate the three-dimensional coordinates of a series of pixel points on the liquid surface observed by each image sensor extracted in step 1) in the fuel tank coordinate system; Step 3) Using the coordinates of all three-dimensional coordinate points on the liquid surface observed by each image sensor obtained in step 2), the fuel surface in the tank coordinate system is fitted, and the remaining oil volume below the fuel surface is calculated based on the fitted fuel surface parameters and the three-dimensional digital model of the fuel tank.

8. The method for online monitoring of the fuel level in an aircraft tank based on machine vision according to claim 7 is characterized in that: Step 2) The specific process is as follows: According to the camera model, the two-dimensional pixel points (u i ,v i ) is the three-dimensional coordinate point (x) in the tank coordinate system after distortion correction. i ,y i ,x i )satisfy: Where, the subscript i represents any point on the liquid surface observed by each image sensor, K C and Represents the intrinsic parameter matrix of each image sensor and the transformation matrix from the tank coordinate system to the corresponding image sensor coordinate system; After reorganizing the above formula, we can get: This formula is a three-dimensional point (x i ,y i ,z i ) is located, and the straight line equation is used to intersect with the three-dimensional digital model of the fuel tank body and its internal accessories. The intersection point closest to the camera center is the pixel point on the liquid surface observed by the image sensor (u i ,v i ) corresponds to the three-dimensional point (x i ,y i ,z i ), the oil surface is fitted using the three-dimensional coordinate points in the tank coordinate system corresponding to the liquid level lines observed by all image sensors.

9. The method for online monitoring of the fuel level in an aircraft fuel tank based on machine vision according to claim 7 is characterized in that: Step 3) The specific process is as follows: 3.1) Fitting the spatial plane equation of the fuel surface using all l three-dimensional coordinate points corresponding to the liquid surface lines observed by all image sensors. The RANSAC algorithm is used in the plane fitting algorithm to eliminate the three-dimensional points that are far away from the fitting plane, and obtain the spatial plane equation parameter p of the fuel surface; 3.2) Based on the three-dimensional digital model of the internal space of the fuel tank, before implementing online monitoring, the remaining fuel volume in the fuel tank under a given series of oil level parameters is calculated offline to obtain a fuel volume characteristic table; 3.3) According to the spatial plane equation parameter p of the current fuel surface obtained in the online monitoring, the fuel volume characteristic table of the fuel tank established offline is searched, and the remaining fuel volume value under the current oil level is obtained by interpolation calculation.

Citation Information

Patent Citations

  • Visual imaging sensor point-light ranging sensor measuring coordinate system unifying method

    CN109323650A

  • Robot 3D laser vision disordered grabbing control method, medium and system

    CN116587280A

  • Aircraft fuel oil measuring system based on visual sensor

    CN116754048A

  • Oil quantity measuring method and system

    CN117268498A

  • 2d area-array camera and line laser 3D sensor joint calibration method and apparatus

    WO2024138916A1