A method for early warning of whiplash in a flexible hose based on multi-sensor fusion

Abstract

This application belongs to the field of hose whiplash warning technology for refueling and receiving aircraft docking, specifically involving a hose whiplash warning method based on multi-sensor fusion. A force sensor is installed at the connection point between the refueling aircraft and the hose to monitor the force on the hose in real time; an infrared vision sensor is installed at the receiving aircraft head, and multiple infrared markers are evenly distributed on the hose surface to capture images of the overall hose motion; an inertial measurement unit is installed at the end of the hose to collect the linear and angular accelerations at the hose end; a data processing and fusion module acquires real-time data from the force sensor, inertial measurement unit, and infrared vision sensor, performs data processing and fusion, and outputs a whiplash risk assessment result.

Description

Technical Field

[0001] This application belongs to the field of early warning technology for hose whipping during refueling and receiving aircraft docking, specifically involving a hose whipping early warning method based on multi-sensor fusion. Background Technology

[0002] In modern aerial refueling and receiver aircraft docking technology, the connection system between the hose and the aircraft plays a crucial role. However, in actual close-range docking operations, due to various factors such as changes in flight status, air disturbances, and the inherent mechanical properties of the hose, the hose often exhibits whiplash, resulting in severe vibration or swaying, leading to docking failure or even collisions between the refueling and receiver aircraft, thereby endangering flight safety.

[0003] Currently, research and control methods for hose whiplash are relatively limited, mainly focusing on improvements to docking equipment. Furthermore, existing research on hose whiplash largely focuses on modeling and numerical simulation, aiming to analyze the mechanism of whiplash occurrence and provide some preventative measures and suggestions at a macroscopic level. The lack of real-time monitoring and early warning mechanisms makes it impossible to cope with complex and ever-changing flight environments.

[0004] To improve the safety of aerial docking between tanker and receiver aircraft, some studies have attempted to introduce visual sensors to monitor the hose's status in real time. By installing cameras, image processing techniques are used to identify dynamic changes in the hose and thus assess its condition. While this method provides spatial data on the hose, it is susceptible to obstruction or being outside the sensor's field of view. Furthermore, mechanical sensors such as accelerometers, gyroscopes, and force sensors can provide real-time force and motion data on the hose, monitoring specific behaviors. However, single-type sensors often only reflect one dimension of the hose's motion, and relying on a single sensor for real-time monitoring and early warning of the hose's complex movement patterns makes it difficult to effectively prevent whiplash.

[0005] This application is made in view of the aforementioned technical deficiencies. Summary of the Invention

[0006] The purpose of this application is to provide a hose whiplash warning method based on multi-sensor fusion. By fusing data from multiple sensors, the method can monitor and predict hose whiplash in a more efficient and accurate manner, thereby improving the safety of the aerial docking process between the tanker and the receiver aircraft.

[0007] The technical solution of this application is:

[0008] A method for early warning of whiplash in a flexible hose based on multi-sensor fusion, comprising:

[0009] A force sensor is installed at the connection between the fuel dispenser and the hose to monitor the force exerted on the hose in real time.

[0010] An infrared vision sensor is installed at the head of the receiver engine, and multiple infrared markers are evenly distributed on the surface of the hose to capture images of the overall movement of the hose.

[0011] An inertial measurement unit is installed at the end of the hose to collect the linear and angular accelerations at the end of the hose.

[0012] The data processing and fusion module acquires real-time data from force sensors, inertial measurement units, and infrared vision sensors, processes and fuses the data, and outputs the whip-flick risk assessment result.

[0013] Optionally, in the above-mentioned hose whip warning method based on multi-sensor fusion, the force sensor monitors the force on the hose in real time, including tension and torque.

[0014] The infrared vision sensor uses a binocular infrared camera;

[0015] The inertial measurement unit includes an accelerometer and a gyroscope.

[0016] Optionally, in the above-mentioned hose whip-whip warning method based on multi-sensor fusion, the data processing and fusion module adopts a hierarchical data fusion method, including:

[0017] At the bottom layer, extended Kalman filtering is used to fuse multi-dimensional dynamic data from accelerometers, gyroscopes and force sensors to estimate the motion state of the hose end in real time.

[0018] The middle layer uses an infrared vision sensor to capture the temporal trajectory of infrared markers on the hose. Combined with trajectory fitting and feature extraction algorithms, the dynamic characteristics of the hose are obtained, including swing amplitude, spatial displacement and curvature.

[0019] At higher levels, based on the motion state and dynamic characteristics of the hose tip, a random forest model is used to intelligently determine the current and future states of the hose, identify whether there is a whiplash tendency, and output early warning signals in a timely manner.

[0020] Optionally, in the above-mentioned method for soft hose whip-whip warning based on multi-sensor fusion, the data processing and fusion module processes the image of infrared marker points captured by the infrared visual sensor on the soft hose:

[0021] Use target detection algorithms to determine the infrared marker image region;

[0022] Local regions are cropped from the infrared marker image area, and infrared markers are extracted by combining Gaussian smoothing, threshold segmentation, and edge detection.

[0023] Using the temporal location information of infrared markers across multiple frames, hose trajectory prediction is performed through a Transformer network model.

[0024] Cubic spline curve fitting is performed on the positions of infrared markers in the current frame and the predicted frame to construct the current and future hose trajectories;

[0025] Calculate the dynamic characteristics of the hose.

[0026] Optionally, in the above-mentioned hose whip warning method based on multi-sensor fusion, the data processing and fusion module is integrated into the aircraft avionics bay, and receives data from force sensors, inertial measurement units and infrared vision sensors via wired or wireless means.

[0027] Optionally, in the above-mentioned method for early warning of whiplash in a flexible hose based on multi-sensor fusion, the data processing and fusion module calls... Perform cubic spline curve fitting on the positions of infrared marker points in the current frame and the predicted frame.

[0028] Optionally, in the above-mentioned hose whiplash warning method based on multi-sensor fusion, the input of the random forest model is the hose dynamic feature vector. :

[0029]

[0030] in:

[0031] This represents the average displacement of the hose curves in the preceding and following frames.

[0032] The acceleration modulus represents the magnitude of the linear acceleration at the end of the hose.

[0033] The velocity modulus represents the speed at the end of the hose.

[0034] ω is the angular acceleration modulus, representing the rate of change of rotation at the end of the hose;

[0035] ω is the angular velocity modulus, representing the rotational speed of the hose end;

[0036] The maximum swing amplitude represents the distance between the highest and lowest points of the hose;

[0037] The force modulus indicates the magnitude of the force on the hose;

[0038] The bending modulus of the hose represents the average degree of bending of the hose.

[0039] Optionally, in the above-mentioned hose whiplash warning method based on multi-sensor fusion, the output of the random forest model is the hose risk level, including 0, 1, and 2, where 0 indicates safety and no whiplash risk; 1 indicates moderate risk and moderate whiplash risk; and 2 indicates high risk and high whiplash risk.

[0040] Optionally, in the above-mentioned hose whip warning method based on multi-sensor fusion, a random forest model is constructed using the Scikit-learn machine learning framework, real or simulated flight data is collected, the hose risk level of each frame is manually labeled, and the random forest model is trained.

[0041] Optionally, in the above-mentioned hose whiplash warning method based on multi-sensor fusion, the number of trees in the random forest model is set. The value is 100. Each tree is trained on different sample subsets and feature subsets. Each tree outputs a classification result. The random forest model uses majority voting to determine the final classification. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the sensor and infrared marker arrangement in the hose whip-whip warning method based on multi-sensor fusion provided in the embodiments of this application;

[0043] Figure 2 This is a flowchart of the infrared marker positioning provided in an embodiment of this application;

[0044] Figure 3 This is a flowchart of hose pose feature extraction based on infrared markers provided in an embodiment of this application;

[0045] Figure 4 This is an architecture diagram of the hose whip warning method based on multi-sensor fusion provided in the embodiments of this application.

[0046] To better illustrate this embodiment, some content in the accompanying drawings may be omitted, enlarged, or reduced. They are for illustrative purposes only and should not be construed as limiting the scope of this application. Detailed Implementation

[0047] To make the technical solution and advantages of this application clearer, the technical solution of this application will be described in a clearer and more complete manner below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some embodiments of this application, and are only used to explain this application, not to limit this application. It should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, and other related parts can be referred to the general design.

[0048] Furthermore, unless otherwise defined, the technical or scientific terms used in this application description shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The word "comprising" as used in this application description indicates that the concept preceding the word encompasses the concepts listed following the word and their equivalents, without excluding other related concepts.

[0049] Furthermore, the terms indicating location used in the description of this application are only used to indicate relative directions or positional relationships. When the absolute position of the described object changes, its relative positional relationship may also change accordingly. It should also be noted that, unless otherwise explicitly specified and limited, terms such as "installation" and "connection" used in the description of this application should be interpreted broadly. For example, a connection can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand its specific meaning in this application according to the specific circumstances.

[0050] A method for hose whiplash warning based on multi-sensor fusion aims to achieve real-time perception and anomaly prediction of the dynamic state of the hose during aerial docking by fusing data from multiple sensors, providing timely and effective whiplash warnings, and improving the safety and intelligence level of refueling missions.

[0051] Configure the following core modules:

[0052] Force sensor: Installed at the connection between the fuel dispenser and the hose, it is used to monitor the tension and torque on the hose in real time, reflect the trend of stress changes on the hose, and provide a basis for identifying dangerous conditions such as excessive stretching and sudden vibration.

[0053] Infrared vision sensor and infrared markers: The infrared vision sensor employs a binocular infrared camera, mounted on the receiver engine head, to capture images of the overall motion of the hose. Multiple infrared markers are evenly distributed on the hose surface. These markers possess excellent anti-interference capabilities and can be stably identified by the binocular infrared camera under various lighting conditions. By extracting the temporal position sequence of the infrared markers and combining it with the depth information from the binocular infrared camera's vision, the spatial curvature deformation and motion trajectory of the hose can be inferred.

[0054] Inertial measurement unit: including accelerometer and gyroscope, installed at the end of the hose, collects linear acceleration and angular acceleration data at the end, reflecting the rapid swinging trend of the hose and its local oscillation behavior, and is an important source of information for judging the initial stage of whip swing.

[0055] Data processing and fusion module: responsible for receiving data from multiple sources of sensors, completing data preprocessing, state estimation and multi-dimensional feature extraction, running early warning algorithms, and outputting whiplash risk judgment results.

[0056] The specific steps for triggering a whiplash warning via a flexible hose are as follows:

[0057] Sensor data acquisition:

[0058] Real-time data is acquired from a force sensor, an inertial measurement unit (IMU), and an infrared vision sensor. The force sensor outputs the tension and torque acting on the hose. The accelerometer and gyroscope in the IMU output the acceleration and angular acceleration at the end of the hose. The infrared vision sensor acquires images of the hose and extracts the position information of infrared marker points.

[0059] Processing of visual data:

[0060] Use target detection algorithms to determine the infrared marker image region;

[0061] Local regions of the infrared marker image are cropped, and infrared markers are extracted by combining Gaussian smoothing, threshold segmentation, and edge detection techniques.

[0062] Trajectory prediction is performed using the temporal location information of infrared markers across multiple frames through a Transformer network model.

[0063] Cubic spline curve fitting is performed on the positions of infrared markers in the current frame and the predicted frame to construct the current and future hose trajectories;

[0064] Calculate the dynamic characteristics of the hose trajectory, such as the maximum swing amplitude (difference between the highest and lowest points), average displacement amplitude, and curvature.

[0065] Multi-level fusion and dynamic state estimation:

[0066] A hierarchical data fusion approach is employed. The bottom layer utilizes extended Kalman filtering to fuse multidimensional dynamic data from accelerometers, gyroscopes, and force sensors, estimating the motion state of the hose tip in real time. The middle layer captures the temporal trajectory of infrared markers on the hose using an infrared vision sensor, and combines trajectory fitting and feature extraction algorithms to obtain dynamic features such as the hose's swing amplitude, spatial displacement, and curvature. The top layer constructs a random forest-based model based on the above information to intelligently determine the current and future states of the hose, identify the presence of whiplashing tendencies, and promptly output early warning signals, providing a basis for dynamic adjustments during the docking process.

[0067] By analyzing fused data, potential danger signals during hose movement can be identified based on historical data, allowing for early assessment of the risk of whiplash. If the prediction indicates that the hose is about to enter a whiplash state, a warning signal will be issued. Based on the warning, an automatic adjustment mechanism can be activated to adjust the aircraft's flight attitude and speed to reduce the stress on the hose, thereby preventing the whiplash from worsening and ensuring the smooth completion of the operation.

[0068] Figure 1 This is a schematic diagram of the sensor and infrared marker arrangement in a hose whip warning method based on multi-sensor fusion, including force sensors, infrared vision sensors, infrared markers, and inertial measurement units.

[0069] Force sensors are installed at the connection interface between the refueling pod and the hose to monitor changes in tension and torque on the hose in real time. When the hose is subjected to abnormal loads, the force sensors will output corresponding signals to warn of potential anomalies during the docking process, ensuring that the docking operation is not damaged or causes other safety accidents due to overload or uneven stress.

[0070] An infrared vision sensor is installed at the receiver engine head, using a binocular infrared camera to capture the movement of the oil delivery hose in real time. Multiple infrared markers are placed on the hose, which can be accurately identified by the binocular infrared camera in all weather conditions. By monitoring the hose's dynamic changes in real time, the infrared vision sensor calculates the hose's bending, swaying, and other attitude information.

[0071] An inertial measurement unit (IMU) is installed at the end of the hose and includes an accelerometer and a gyroscope for real-time monitoring of changes in linear and angular acceleration at the end of the hose.

[0072] The data processing and fusion module is integrated into the aircraft's avionics bay. It receives data from various sensors via wired or wireless means and processes and fuses it in real time.

[0073] Figure 2 The precise positioning scheme for infrared marker points consists of the following steps:

[0074] 1. Infrared marker area detection and interception

[0075] Input infrared image Output bounding boxes using object detection algorithms This corresponds to the area where the infrared markers are located. Then, the corresponding portion of the image is cropped using the detected bounding box coordinates as the region of interest, resulting in an image of the infrared marker region. For the bounding box... From the original image Extract the sub-image containing the infrared marker points. :

[0076] ;

[0077] By cropping the image, only the area containing the infrared markers is retained, reducing the computational load of subsequent processing.

[0078] 2. Gaussian smoothing

[0079] Pair images The image is convolved using a Gaussian filter and a two-dimensional Gaussian kernel:

[0080] ;

[0081] in, It is the standard deviation of the Gaussian kernel, controlling the intensity of smoothing. The Gaussian kernel is then compared with the sub-image. Convolution yields a smoothed image. :

[0082] ;

[0083] in, The radius of the Gaussian convolution kernel determines the kernel size as (2... +1)×(2 +1), acceptable =2, corresponding to a 5×5 convolution kernel. For the Gaussian kernel in position The weight value at a given position indicates the degree of contribution of that pixel to the center pixel.

[0084] Gaussian filtering reduces high-frequency noise in an image, making it smoother and helping to obtain more accurate edges during thresholding.

[0085] 3. Threshold segmentation

[0086] Infrared marker regions are extracted using adaptive thresholding. First, the image is converted to a grayscale image. The grayscale value of each pixel is Next, by setting a threshold... To perform segmentation, obtain a binarized image. :

[0087] ;

[0088] By distinguishing the infrared marker areas in the image from the background area, a clear target area is provided for subsequent localization.

[0089] 4. Edge detection and subpixel-level positioning

[0090] For the smoothed image Using Canny edge detection, calculate gradients to find edges in the image:

[0091] ;in, Let be the grayscale intensity function of the image, representing the image's coordinates in space. The pixel grayscale values ​​at point I, ∂I / ∂x and ∂I / ∂y represent the image's grayscale values ​​at point I, ∂I / ∂x and ∂I / ∂y, respectively. direction and The partial derivative in the direction represents the spatial rate of change of gray level at that point. The gradient magnitude represents the intensity of the grayscale change at that point.

[0092] Assuming the center pixel value of the infrared marker is Select the left and right neighboring pixel values ​​around the infrared marker point. and The optimal infrared marker location is found by fitting a quadratic polynomial using the least squares method and then solving for the extrema of the polynomial. It is assumed that the infrared marker is located within the pixel grid. Between these points, a quadratic interpolation formula is used to estimate the precise location of the infrared markers. :

[0093] ;

[0094] in, It is the grayscale value of adjacent pixels.

[0095] Figure 3 This is a flowchart of the hose pose feature extraction process based on infrared markers. After reading image data from the infrared video stream, it uses... Figure 2 The method described in the paper extracts the positions of infrared markers in each frame of the image. Then, the encoder of the Transformer network model extracts the sequential position features of the infrared markers, and the decoder predicts their future positions.

[0096] 1. Input Data Preparation

[0097] At a given time node The infrared markers extracted from the image are , No. The coordinates of the infrared markers are: These coordinates constitute the time step. Data:

[0098] ;

[0099] Based on historical frame data from multiple time points, the location of infrared markers can be predicted for future moments.

[0100] 2. Location coding

[0101] Since the Transformer network model itself lacks the ability to process sequence information, positional encoding needs to be added to the input data so that the Transformer network model can know the order of the infrared markers in the time series. The positional encoding is calculated using the following formula:

[0102] ;

[0103] ;

[0104] in, It's about location. It is a dimensional index. It is the size of the embedded dimension.

[0105] 3. Encoder

[0106] The encoder consists of a multi-layered self-attention mechanism and a feedforward neural network. The self-attention mechanism helps the Transformer network model capture the relationships between time steps in the sequence, calculating the influence of each infrared marker on other infrared markers. Specifically, the input to the encoder is a... A matrix of dimension, where It is the number of infrared markers. This is the embedding dimension. It uses a self-attention mechanism to calculate the relationship between queries, keys, and values. The calculation formula for the self-attention mechanism is as follows:

[0107] ;

[0108] in, It is a query vector, representing a query of features at the current location; and It is a key-value pair vector, representing feature keys and feature values ​​at other locations.

[0109] 4. Decoder

[0110] The decoder generates the predicted future locations based on the features extracted by the encoder. Specifically, it transforms the high-dimensional embedding vector into the predicted locations of specific infrared marker points through a linear layer.

[0111] ;

[0112] in These are the predicted locations of the infrared markers. Indicates a future time step. To perform a linear transformation on the decoder output, the high-dimensional feature vector is mapped to the two-dimensional coordinates of specific infrared marker points.

[0113] 5. Curve Fitting and Feature Extraction

[0114] Then, using the known current position and the future position predicted by the Transformer network model, cubic spline curves are fitted respectively. (Call...) To achieve a fit. Assume... and It is the predicted location at the current and future times; the fitted hose trajectory consists of a set of multiple pixels. , and Given the pixel coordinates representing the start and end points of the trajectory, the displacement between the two frames is:

[0115] ;

[0116] in, It is the number of points on the fitted curve. This represents the longitudinal position of the hose curve points at adjacent time points. The highest and lowest points of the infrared marker trajectory within the current time period are as follows:

[0117] ;

[0118] ;

[0119] This indicates the vertical fluctuation range of the hose within the current time period. The formula for calculating the average curvature of the hose trajectory is:

[0120] ;

[0121] in, The slope of the reference straight line connecting the start and end points of the hose trajectory. As the reference straight line Axis intercept, calculated by formula, is the average distance from each point on the hose trajectory curve to the reference straight line, used to quantify the degree of hose bending.

[0122] Figure 4 This is an architecture diagram of a hose whiplash warning method based on multi-sensor fusion. Its core is a three-layer fusion architecture, including a bottom-layer hose state estimation layer, a middle-layer visual feature extraction layer, and a high-layer risk discrimination and warning layer. The specific implementation of each layer is as follows:

[0123] 1. Bottom Layer: Hose Condition Estimation

[0124] In monitoring the whiplash phenomenon in a flexible hose, it is necessary to estimate the hose's state in real time by fusing data from multiple sensors. The hose state vector is... ,in, This refers to the position of the hose tip in three-dimensional space. The speed at the end of the hose; This represents the rotational speed of the hose end.

[0125] The change in the hose's state over time can be described by kinematic equations and control inputs. The control inputs include acceleration. and angular acceleration and the force measured by the force sensor The state transition equation is used to describe the state changes of the hose, starting from time... At the time The position, angular velocity, and velocity changes of the hose end are as follows:

[0126] ;

[0127] The corresponding observation equation is:

[0128] ;

[0129] in, For the sensor at any time The actual measured value vector includes the acceleration, force, and angular acceleration at the end of the hose; The observation matrix describes the linear mapping relationship between the state vector and the measurement vector; The observation noise represents the random error during the sensor measurement process. In the Kalman filter prediction stage, the current state can be predicted based on the state estimate from the previous time step and the control input.

[0130] ;

[0131] in, The predicted current state includes the position of the hose end. angular velocity and speed , For control inputs, including acceleration and angular acceleration and the force measured by the force sensor , The state transition function represents the prediction of the current state based on the state estimate of the previous time step and the current control input, using physical laws (kinematic equations).

[0132] Then, the state covariance matrix at the current moment is predicted to measure the prediction error:

[0133] ;

[0134] in, The process noise covariance matrix is... Let be the state transition matrix, representing the uncertainty of the previous time step. Through the state transition matrix Propagation to the current moment, plus process noise The impact, Let be the covariance matrix of the posterior state estimate of the previous time step, representing the uncertainty of the state estimate of the previous time step after measurement update.

[0135] During the update phase, the Kalman gain is calculated to balance the weights of the predicted and observed values:

[0136] ;

[0137] in, The observation noise covariance matrix is ​​used. The state estimate is updated based on the Kalman gain and the observed data.

[0138] ;

[0139] Update the covariance matrix of the state estimate:

[0140] ;

[0141] in, It is an identity matrix.

[0142] Each time new sensor data arrives, EKF performs prediction and update steps, continuously refining the state estimate and maintaining a real-time estimate of the hose's motion state.

[0143] 2. Middle layer: Visual feature extraction.

[0144] This layer uses an infrared vision sensor to extract the temporal trajectory features of infrared markers on the surface of the hose as input, which is then passed to the higher-level judgment module.

[0145] 3. Senior Management: Risk Assessment and Early Warning

[0146] By introducing the random forest algorithm at higher levels, a multi-feature intelligent classification model for the whiplash phenomenon is constructed. By integrating mechanical and visual features, it can accurately determine whether there is a risk of whiplash and issue timely warnings.

[0147] 1. Feature Input Design

[0148] The input to the random forest model is at each time step. The dynamic feature vector of the hose :

[0149]

[0150] in:

[0151] This represents the average displacement of the hose curves in the preceding and following frames.

[0152] acceleration modulus The magnitude of linear acceleration reflects the acceleration or deceleration of the hose;

[0153] velocity modulus The speed at the end of the hose describes its motion.

[0154] angular acceleration modulus The angular acceleration of the hose describes its rate of rotational change;

[0155] angular velocity modulus The rotation rate of the hose tip, describing its rotation state;

[0156] Maximum swing amplitude The distance between the highest and lowest points of the hose;

[0157] Force modulus : Measure the magnitude of the force applied to the hose;

[0158] Hose curvature modulus The average degree of bending of the hose.

[0159] 2. Model Training Phase

[0160] A large amount of real or simulated flight data was collected, including sensor outputs and the corresponding hose motion status at each moment. The hose risk level for each frame was manually labeled, with 0 representing safe (no whiplash risk), 1 representing moderate risk (moderate whiplash risk), and 2 representing high risk (high whiplash risk). A random forest model was built using the Scikit-learn machine learning framework. The number of trees was set. There are 100 trees; each tree is trained on different sample subsets and feature subsets to increase generalization ability; each tree outputs a classification result, and the overall model takes the majority vote to determine the final classification.

[0161] Constructing the classification function:

[0162] ;

[0163] in, This represents the current risk level of the hose.

[0164] 3. Model Deployment Phase

[0165] In actual operation, the system works according to the following steps:

[0166] 1) Extract the current feature vector for each frame or time step. ;

[0167] 2) Input the features into the trained random forest classifier to obtain the risk level output. ;

[0168] 3) Output and linkage control mechanism: If Normal operation; if The system outputs a yellow warning, and the flight control system makes minor adjustments; if A red alert is issued, and the refueling operation is immediately suspended in the flight control system.

[0169] The multi-sensor fusion-based hose whip-whip warning method disclosed in the above embodiments has the following advantages:

[0170] 1. Multi-source fusion sensing enables real-time prediction of hose status.

[0171] A multi-source sensing system based on force sensors, an infrared vision system, accelerometers, and gyroscopes is constructed to monitor the hose status in real time from multiple dimensions, including mechanics, kinematics, and spatial attitude. The infrared vision system identifies the spatial shape of the hose based on infrared markers and predicts its future trajectory in conjunction with a sequence modeling network; the force sensor monitors the force fluctuations on the hose, reflecting the dynamic load during the docking process; and the accelerometer and gyroscope provide acceleration and angular velocity information at the hose tip, reflecting its vibration characteristics.

[0172] 2. Multi-layered data fusion architecture improves the accuracy and responsiveness of whiplash warnings.

[0173] A three-level fusion architecture is adopted to achieve accurate modeling and risk identification of hose dynamics. The bottom layer obtains high-precision state estimation of the hose tip by fusing force sensor and inertial measurement data through extended Kalman filtering; the middle layer extracts features such as hose swing amplitude and displacement based on infrared marker trajectory fitting; and the top layer uses a random forest model to achieve real-time judgment and graded early warning of whiplash risk. This fusion strategy effectively improves the accuracy of early warning while balancing real-time performance and robustness.

[0174] 3. Highly robust, supporting stable operation around the clock.

[0175] Featuring a highly robust design, the sensors possess excellent redundancy and complementarity, ensuring the system can maintain basic operation even if some sensors malfunction. The infrared vision system supports day and night operation and stable recognition in low-visibility environments, significantly improving all-weather availability and ensuring continuous stable operation under drastic changes in flight conditions, effectively mitigating whiplash risks.

[0176] 4. Designed for future aerial docking scenarios, it possesses excellent integration and engineering value.

[0177] Designed for future unmanned aerial docking scenarios, this system can autonomously identify abnormal dynamics of the flexible hose based on fused perception results, providing decision-making support for the flight control system. It possesses excellent scalability and portability, can be integrated into the docking control chain, significantly improving the safety and intelligence of the aerial docking process, and has broad engineering application prospects.

[0178] The technical solution of this application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. Those skilled in the art should understand that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for early warning of whiplash in a flexible hose based on multi-sensor fusion, characterized in that, include: A force sensor is installed at the connection between the fuel dispenser and the hose to monitor the force exerted on the hose in real time. An infrared vision sensor is installed at the head of the receiver engine, and multiple infrared markers are evenly distributed on the surface of the hose to capture images of the overall movement of the hose. An inertial measurement unit is installed at the end of the hose to collect the linear and angular accelerations at the end of the hose. The data processing and fusion module acquires real-time data from force sensors, inertial measurement units, and infrared vision sensors, processes and fuses the data, and outputs the whip-flick risk assessment result.

2. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 1, characterized in that, Force sensors monitor the forces acting on the hose in real time, including tension and torque; The infrared vision sensor uses a binocular infrared camera; The inertial measurement unit includes an accelerometer and a gyroscope.

3. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 2, characterized in that, The data processing and fusion module employs a hierarchical data fusion method, including: At the bottom layer, extended Kalman filtering is used to fuse multi-dimensional dynamic data from accelerometers, gyroscopes and force sensors to estimate the motion state of the hose end in real time. The middle layer uses an infrared vision sensor to capture the temporal trajectory of infrared markers on the hose. Combined with trajectory fitting and feature extraction algorithms, the dynamic characteristics of the hose are obtained, including swing amplitude, spatial displacement and curvature. At higher levels, based on the motion state and dynamic characteristics of the hose tip, a random forest model is used to intelligently determine the current and future states of the hose, identify whether there is a whiplash tendency, and output early warning signals in a timely manner.

4. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 3, characterized in that, In the data processing and fusion module, the image of infrared marker points captured by the infrared vision sensor on the flexible tube is processed: Use target detection algorithms to determine the infrared marker image region; Local regions are cropped from the infrared marker image area, and infrared markers are extracted by combining Gaussian smoothing, threshold segmentation, and edge detection. Using the temporal location information of infrared markers across multiple frames, hose trajectory prediction is performed through a Transformer network model. Cubic spline curve fitting is performed on the positions of infrared markers in the current frame and the predicted frame to construct the current and future hose trajectories; Calculate the dynamic characteristics of the hose.

5. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 4, characterized in that, The data processing and fusion module is integrated into the aircraft's avionics bay and receives data from force sensors, inertial measurement units, and infrared vision sensors via wired or wireless means.

6. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 5, characterized in that, In the data processing and fusion module, call Perform cubic spline curve fitting on the positions of infrared marker points in the current frame and the predicted frame.

7. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 6, characterized in that, The input to the random forest model is the hose dynamic feature vector. : in: This represents the average displacement of the hose curves in the preceding and following frames. The acceleration modulus represents the magnitude of the linear acceleration at the end of the hose. The velocity modulus represents the speed at the end of the hose. ω is the angular acceleration modulus, representing the rate of change of rotation at the end of the hose; ω is the angular velocity modulus, representing the rotational speed of the hose end; The maximum swing amplitude represents the distance between the highest and lowest points of the hose; The force modulus indicates the magnitude of the force on the hose; The bending modulus of the hose represents the average degree of bending of the hose.

8. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 7, characterized in that, The output of the random forest model is the hose risk level, including 0, 1, and 2, where 0 represents safety and there is no whiplash risk; 1 represents moderate risk and there is moderate whiplash risk; and 2 represents high risk and there is high whiplash risk.

9. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 8, characterized in that, A random forest model was built using the Scikit-learn machine learning framework. Real or simulated flight data was collected, and the risk level of the hose was manually labeled for each frame to train the random forest model.

10. The method for early warning of whiplash in a flexible hose based on multi-sensor fusion according to claim 9, characterized in that, The number of trees is set in the random forest model. The value is 100. Each tree is trained on different sample subsets and feature subsets. Each tree outputs a classification result. The random forest model uses majority voting to determine the final classification.

Images