Hose whip throwing special condition identification method and system

Abstract

The invention provides a method and a system for identifying a special condition of whip throwing of a hose. The method comprises the following steps of: generating a hose curvature heat map from continuous image frames containing a refueling hose through image preprocessing; utilizing an improved deep convolutional neural network to extract a spatial geometric feature vector representing the instantaneous form of the hose; performing short-time Fourier transform based on the curvature time sequence of the continuous frames, and extracting a frequency feature vector representing the short-time dynamic behavior of the hose; the two features are fused through an attention mechanism and then jointly classified, the hose is judged to be in a normal state, a whip-throwing precursor or a whip-throwing state at present, and the confidence coefficient is output; and finally, triggering corresponding early warning or safety intervention according to the classification result and the confidence coefficient. According to the method, multi-dimensional fusion perception of space and time movement is realized, morphological anomaly and dynamic trend analysis are considered, the accuracy of whip swinging detection is remarkably improved, false alarm and missing alarm are reduced, and effective safety guarantee is provided for air-air refueling operation.

Description

Technical Field

[0001] This invention belongs to the field of aircraft in-flight refueling control technology, and relates to a method and system for identifying hose whiplash incidents. It combines spatial geometry and temporal dynamics characteristics to intelligently identify and warn of whiplash incidents in the refueling hose during in-flight refueling operations. The hose whiplash incident described in this invention refers to the abnormal, rapid oscillation of the hose during in-flight refueling. Background Technology

[0002] Aerial refueling is a crucial step in the long-range operation of modern aviation equipment, and its operational safety is of paramount importance. During aerial refueling using the "hose-and-hook" method, the refueling hose released by the tanker aircraft may experience severe, uncontrolled shaking or large-amplitude rapid swinging under the influence of complex factors such as airflow disturbances and docking disturbances of the receiving aircraft. This phenomenon is commonly known as "whiplash."

[0003] Whiplashes pose a serious threat to aerial refueling operations. Their high frequency and large amplitude not only directly hinder successful docking of the receiver aircraft with the floating anchor, but can also cause structural damage to the receiver aircraft's fuselage, probes, and even engines, severely threatening flight safety.

[0004] Currently, the identification methods for whiplash incidents mainly suffer from the following technical bottlenecks: (1) Traditional methods rely on pilots or refueling operators to make manual judgments through video monitoring, or to use analysis algorithms based on single-frame visual images. These methods can only capture the bending shape of the hose at a certain moment, and it is difficult to effectively distinguish between normal dynamic bending and abnormal whiplash evolution, which easily leads to false alarms and insufficient identification reliability. (2) Although some methods have attempted to introduce time series analysis, such as judging anomalies by statistically analyzing the amplitude or speed of motion in historical data, whiplash usually manifests as a short-term, sudden dynamic instability process. Simple statistical methods are not sensitive enough to such rapidly evolving dynamic characteristics and cannot accurately describe the complete process from precursors to occurrence, resulting in a high risk of missed alarms. (3) Most existing technical solutions treat the spatial shape and temporal dynamics of the hose separately, failing to establish a unified and systematic theoretical framework to describe the whiplash evolution mechanism, resulting in insufficient accuracy of the identification results and a lack of sufficient interpretability, making it difficult to integrate into flight control systems with high reliability requirements.

[0005] In summary, to improve the safety and automation level of aerial refueling operations, it is urgent to develop a novel whiplash recognition method that can synergistically integrate the instantaneous geometric features and short-term dynamic evolution patterns of the hose, in order to achieve high-confidence, interpretable whiplash detection and early warning capabilities. Summary of the Invention

[0006] The purpose of this invention is to address the problems of high false alarm and false negative risks and insufficient reliability in existing methods for identifying whiplash incidents during in-flight refueling. This invention provides a method and system for identifying whiplash incidents using a flexible hose. By constructing a unified analytical framework, this invention jointly models and analyzes the spatial geometric features characterizing the instantaneous bending morphology of the hose with the temporal frequency features reflecting its short-term dynamic trends. This aims to overcome the false alarm and false negative problems caused by the single-dimensional analysis of existing methods, thereby achieving accurate identification and effective early warning of whiplash incidents.

[0007] To achieve the above objectives, the technical solution provided by this invention is:

[0008] A method for identifying whiplash issues in a flexible hose, used to distinguish between normal bending patterns and abnormal whiplash patterns; comprising the following steps:

[0009] Step 1: Image preprocessing and 2D curvature heatmap generation;

[0010] Acquire multiple consecutive frames of images containing the hose, and process each frame to generate a two-dimensional curvature heatmap reflecting the instantaneous bending shape of the hose.

[0011] Step 2, spatial geometric feature extraction;

[0012] Based on an improved deep convolutional neural network, spatial geometric feature vectors representing the instantaneous spatial geometry of the hose are extracted from the two-dimensional curvature heatmap generated in step 1. ;

[0013] The improved deep convolutional neural network uses the ResNet-18 architecture as its backbone and embeds a dual-branch geometric attention enhancement unit at the output of a specific residual block. This dual-branch geometric attention enhancement unit is configured to iteratively perform the following operations to enhance the features of high-curvature anomaly regions in the 2D curvature heatmap: receiving the feature map output from the corresponding residual block in the backbone network and simultaneously extracting global topological features and local gradient features from the feature map; employing an additive fusion mechanism based on adaptive weight allocation to fuse the topological enhancement feature map output from the two branches with the local deformation feature map, generating a fused enhanced feature vector; finally, the fused enhanced feature vector is returned to the backbone network for downsampling processing to obtain the spatial geometric feature vector. ;

[0014] Step 3, extraction of time dynamic features;

[0015] Based on the continuous multi-frame images obtained in step 1, the curvature sequence of the hose over time is calculated to form a curvature time series; and time-frequency analysis is performed on the curvature time series to extract frequency feature vectors characterizing the short-time dynamic behavior of the hose. ;

[0016] Step 4: Feature-level fusion and classification determination;

[0017] The spatial geometric feature vector obtained in step 2 Compared with the frequency eigenvector obtained in step 3 The features are fused to form a joint feature vector. ;

[0018] Joint feature vectors Input the classifier to obtain the probability distribution sequence of state categories corresponding to normal, whiplash premonition and whiplash already occurred;

[0019] The final judgment result is determined by selecting the state category with the highest probability from the state category probability distribution sequence, and the confidence level of the judgment result is determined by the highest probability value. The judgment result and the corresponding confidence level are then output.

[0020] Step 5: Early warning and intervention control;

[0021] Based on the obtained judgment results and corresponding confidence levels, appropriate early warning and intervention control operations are triggered, including:

[0022] If the determination result is normal, the current status information of the hose will be displayed in real time, and hose status identification will continue.

[0023] If the determination result is a pre-whiplash state and its confidence level exceeds the preset first warning threshold, or if the determination result is a whiplash state that has occurred and its confidence level does not exceed the preset second warning threshold, then a warning reminder will be issued and a stabilizing hose operation will be performed.

[0024] If the determination result indicates that a whiplash has occurred and its confidence level exceeds the preset second warning threshold, an emergency alarm will be triggered and corresponding safety intervention operations will be performed.

[0025] Furthermore, in step 1, the specific steps for generating a two-dimensional curvature heat map reflecting the instantaneous bending shape of the hose are as follows:

[0026] Step 1.1: Acquire the real-time video stream of the hose and decompose the real-time video stream into multiple consecutive frames of images according to a preset fixed frame rate;

[0027] Step 1.2: Perform hose instance segmentation on each frame of the image to generate a pixel-level mask;

[0028] Step 1.3: Perform morphological skeletonization on the pixel-level mask and extract the center line of its single-pixel width; parameterize the center line into arc length, and perform discrete sampling along the center line of the arc length to calculate the curvature value of each sampling point and form a one-dimensional curvature feature sequence.

[0029] Step 1.4: Map the one-dimensional curvature feature sequence to a standard-sized two-dimensional grayscale image, and use this two-dimensional grayscale image as a two-dimensional curvature heatmap of the input data of the improved deep convolutional neural network; in the two-dimensional curvature heatmap, the pixel brightness is proportional to the curvature size.

[0030] Furthermore, in step 2, a dual-branch geometric attention enhancement unit is embedded at the output of the second, third, and fourth residual blocks of the ResNet-18 backbone network; a single dual-branch geometric attention enhancement unit includes a geometric topology attention branch and a curvature gradient attention branch set in parallel, as well as a dual-branch feature fusion gate connected to the output of the two branches;

[0031] The geometric topological attention branch is configured to receive the shallow feature map output from its previous layer. And the node connection matrix obtained based on the parameterization of the hose centerline; the shallow feature map The regional features corresponding to each discrete node along the hose centerline are aggregated into an initial node feature set. Based on the connection relationships of each discrete node, a graph attention coding strategy is used to propagate and update the initial node feature set, resulting in updated node features that include information about the overall topology of the hose. The updated node features are then mapped back to two-dimensional space to generate a topology-enhanced feature map. ;

[0032] The curvature gradient attention branch is configured to receive the shallow feature map of the output of the previous layer. Extracting shallow feature maps Local gradient features; spatial attention mask generated based on local gradient features; shallow feature map Weighted by spatial attention mask, a local deformation feature map is generated. ;

[0033] The dual-branch feature fusion gate is configured to: fuse the received topological enhancement feature map through an additive fusion mechanism based on adaptive weight allocation. and local deformation feature map The fusion process generates an enhanced feature vector, which is then fed back to the backbone network for subsequent downsampling.

[0034] Furthermore, the addition and fusion mechanism based on adaptive weight allocation is as follows:

[0035] Topology-enhanced feature maps and local deformation feature map Perform splicing along the channel dimension;

[0036] Global average pooling and fully connected operations are performed sequentially on the concatenated feature maps to generate two scalar weight coefficients. and ,in ;

[0037] According to the formula Perform a pixel-wise weighted summation operation on the two feature maps to generate the fused enhanced feature vector. .

[0038] Furthermore, in step 3, the calculated curvature time series is subjected to time-frequency analysis using short-time Fourier transform, which converts it into a time-frequency spectrum. Key indicators in the time-frequency spectrum, including the proportion of high-frequency energy, the spectral centroid, and the spectral flux, are calculated, and these key indicators form a frequency feature vector. .

[0039] Furthermore, in step 4, a joint feature vector is formed. Includes the following steps:

[0040] spatial geometric feature vectors With frequency eigenvectors The corresponding channel dimensions are concatenated to form a concatenated feature vector;

[0041] The concatenated feature vectors are input into a pre-trained small self-attention network; the small self-attention network consists of multi-head self-attention layers, which learn and assign the relevance weights of the two feature vectors in classification decisions through the multi-head self-attention mechanism during training.

[0042] The concatenated feature vectors are weighted and fused according to their relevance weights to generate a joint feature vector with the same dimension as the concatenated feature vectors. Joint eigenvectors Used for subsequent classification and determination.

[0043] Furthermore, the classifier used in step 4 is a multilayer perceptron classifier, which includes an input layer, three hidden layers and an output layer connected in sequence.

[0044] Each hidden layer is followed by a ReLU activation function layer and a Dropout layer in sequence;

[0045] The output layer consists of three neurons, corresponding to the normal state, the pre-whiplash state, and the whiplash state, respectively. It uses the Softmax activation function to output a three-dimensional probability vector, representing the probability distribution of each state category.

[0046] Furthermore, it also includes a post-processing step on the continuously output state category probability distribution sequence from step 4 to obtain a stable judgment result and the corresponding confidence level. The post-processing step includes:

[0047] Perform hysteresis threshold discrimination; or perform sliding window smoothing and hysteresis threshold discrimination sequentially; wherein:

[0048] The hysteresis threshold discrimination method is as follows: based on preset state entry and state exit thresholds, hysteresis discrimination is performed on the received state category probability distribution sequence, and the current final state category is output; where:

[0049] When switching from the normal state to the pre-whiplash state, the probability of the pre-whiplash state must be no less than the first entry threshold; when returning from the pre-whiplash state to the normal state, the probability of the pre-whiplash state must be less than the first exit threshold, and the first exit threshold must be less than the first entry threshold; when switching from the pre-whiplash state to the already occurred whiplash state, the probability of the already occurred whiplash state must be no less than the second entry threshold; when switching from the already occurred whiplash state to the pre-whiplash state, the probability of the already occurred whiplash state must be no less than the second exit threshold.

[0050] Based on the hysteresis threshold, the final state category of the output is determined. The confidence level corresponding to the category is extracted from the continuous output state category probability distribution sequence or the smoothed state category probability distribution sequence in step 4, and used as the confidence level of the final state category. The final state category and its corresponding confidence level are then output.

[0051] The present invention also provides a hose whip-whip emergency identification system for implementing the above-mentioned hose whip-whip emergency identification method; the hose whip-whip emergency identification system includes:

[0052] The image preprocessing and curvature heatmap generation module is used to: receive a series of multiple frames of images containing the hose, process the series of multiple frames of images, and generate a two-dimensional curvature heatmap;

[0053] The spatial geometric feature extraction module, connected to the image preprocessing and curvature heatmap generation module, is used to: extract spatial geometric feature vectors representing the instantaneous spatial geometry of the hose from the two-dimensional curvature heatmap using an improved deep convolutional neural network. ;

[0054] The time-dynamic feature extraction module, connected to the image preprocessing and curvature heatmap generation module, is used to: acquire the curvature time series corresponding to multiple consecutive frames of images, and perform time-frequency analysis on the curvature time series to extract frequency feature vectors characterizing the short-term dynamic behavior of the hose. ;

[0055] The feature-level fusion and classification module is connected to the spatial geometric feature extraction module and the temporal dynamics feature extraction module, respectively, and is used to: fuse spatial geometric feature vectors With frequency eigenvectors By fusing the features, a joint feature vector is obtained. Based on joint feature vectors Classify the states to obtain probability distribution sequences corresponding to normal, pre-whiplash, and whiplash occurrence; use the state category with the highest probability in the probability distribution sequence as the final judgment result, and use the highest probability value as the confidence level of the judgment result; output the judgment result and the corresponding confidence level.

[0056] The early warning and intervention control module, connected to the feature-level fusion and classification judgment module, is used to: trigger corresponding early warning and intervention control operations based on the judgment result and the corresponding confidence level, including: when the judgment result is a premonition of whiplash and the confidence level exceeds the set first early warning threshold, or when the judgment result is that whiplash has occurred and the confidence level has not exceeded the set second early warning threshold, issue an early warning reminder and output the first control command to control the execution of stable hose operation;

[0057] When the determination result is that a whip has been lashed and the confidence level exceeds the set second warning threshold, an emergency alarm is triggered and a second control command is output to control the execution of the corresponding safety intervention operation.

[0058] When the determination result is normal, the current status information of the hose is displayed in real time, and the hose status continues to be identified.

[0059] Furthermore, it also includes a post-processing module, which is configured to: perform hysteresis threshold discrimination processing on the state category probability distribution sequence continuously output by the feature-level fusion and classification decision module, or sequentially perform sliding window smoothing processing and hysteresis threshold discrimination processing to output the current final state category;

[0060] Based on the hysteresis threshold, the final state category is determined and processed. The confidence level corresponding to the category is extracted from the continuous output probability distribution sequence of the state category or the smoothed probability distribution sequence of the state category from the feature-level fusion and classification determination module. This confidence level is used as the confidence level of the final state category, and the final state category and its corresponding confidence level are output.

[0061] The advantages of this invention are:

[0062] 1. The hose whiplash incident identification method provided by this invention can effectively identify normal dynamic bending and abnormal whiplash phenomena of aerial refueling hoses. Specifically, this invention deeply integrates spatial geometric field analysis and temporal dynamic field analysis, constructing a complete framework for parallel extraction, fusion, and judgment of spatial geometric feature vectors and temporal dynamic feature vectors. On the one hand, the spatial geometric feature extraction module accurately captures the instantaneous abnormal bending shape of the hose, effectively preventing missed detections caused by normal dynamic disturbances (such as the approach of the receiver aircraft) that result in a hose with a normal shape but violent movement. On the other hand, the temporal dynamic feature extraction module keenly captures short-term high-frequency dynamic burst characteristics, effectively identifying false alarms that may be caused by normal static large curvature of the hose (such as normal hose drooping). The information from the two dimensions complements and verifies each other. As shown in the experimental data in Table 1, the dual-field fusion method of this invention outperforms single visual analysis methods or traditional threshold statistical methods in terms of accuracy, missed detection rate, and false alarm rate, significantly improving the overall detection performance.

[0063] 2. This invention explicitly defines the whiplash precursor state category in the feature-level fusion and classification judgment module and specifically designs a time-dynamic feature extraction module. The time-dynamic feature extraction module uses short-time Fourier transform to extract the time-frequency features of the curvature sequence and focuses on calculating key spectral indicators that are highly sensitive to sudden high-frequency oscillations, such as high-frequency energy ratio and spectral flux. This enables the hose whiplash incident identification system of this invention to identify abnormal precursor states at the dynamic level before the macroscopic geometry of the hose undergoes drastic and uncontrollable whiplash deformation, thus providing valuable early warning and intervention control time for pilots or automatic control systems, fundamentally improving the proactive safety protection level of aerial refueling operations.

[0064] 3. This invention quantifies the instantaneous geometry of the hose through spatial field analysis and captures its high-frequency dynamic characteristics through temporal field analysis. This decouples the complex phenomenon of hose whiplash into two observable dimensions: spatial geometry and temporal dynamics. These two dimensions have clear physical meanings. In the final classification, especially the pre-whiplash state, it directly corresponds to the clear physical situation of "significant high-frequency dynamics detected, but the overall geometry of the hose is still within a controllable range." This gives the judgment results clear physical meaning (interpretability), facilitating technical personnel's understanding and verification of the system's judgment logic. It also provides a clear basis for subsequent system debugging, performance optimization, and fault tracing.

[0065] 4. The spatial geometry + temporal dynamics dual-field fusion analysis framework proposed in this invention has high versatility. It is not only applicable to aerial refueling hoses, but can also be easily extended to the abnormal motion monitoring of other flexible structures, such as stability monitoring of UAV tethered cables, wind vibration analysis of large bridge cables, and vibration monitoring of flexible components in wind tunnel experiments.

[0066] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0067] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0068] Figure 1 This is a flowchart of the method for identifying special situations involving a flexible hose and whip in this invention.

[0069] Figure 2 This is a diagram illustrating a hose whipping emergency during aerial refueling.

[0070] Figure 3 This is a schematic diagram of the normal dynamic bending of the hose during aerial refueling. Detailed Implementation

[0071] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0072] To effectively identify the normal bending state and abnormal whiplash situation of the aerial refueling hose, the present invention provides a preferred embodiment to illustrate the specific identification process of the hose characteristic identification method of the present invention.

[0073] Reference Figure 1 A method for identifying whiplash incidents involving flexible hoses includes the following steps:

[0074] Step 1: Perform image data preprocessing to generate a two-dimensional curvature heatmap.

[0075] Step 1.1, Image sequence acquisition.

[0076] This embodiment acquires a real-time video stream of the refueling hose from an airborne monitoring camera, with the video frame rate set to 30fps. The video stream is then broken down into consecutive image frames according to this fixed frame rate.

[0077] Step 1.2: Perform hose instance segmentation on a single frame image to generate a pixel-level mask.

[0078] In a preferred embodiment of the present invention, for each frame of image, a pre-trained deep learning semantic segmentation network is used for pixel-level segmentation to accurately extract the pixel-level mask of the refueling hose. This embodiment preferably employs the well-known U-Net network, trained specifically for hose images in aerial refueling scenarios. The U-Net network takes a single frame image as input and outputs a mask image of the same size as the input image. Pixels belonging to the target hose are labeled as foreground, and background pixels are labeled as background. This step achieves robust separation of the hose from the complex background, providing an accurate pixel-region basis for subsequent spatial geometric analysis.

[0079] Step 1.3: Perform morphological skeletonization on the generated pixel-level mask to extract the center line of the hose with a single pixel width.

[0080] To eliminate interference from variations in hose thickness caused by lighting or shadows in the original mask, ensuring that subsequent curvature calculations only consider the geometric centroid trajectory of the hose, this invention employs a well-known morphological skeletonization algorithm (typically the Zhang-Suen thinning algorithm) to skeletonize the pixel-level mask of the hose. By iteratively stripping the boundary pixels of the mask while preserving its core topology, a single-pixel-width connected path (skeleton line) is obtained. This skeleton line is represented by a discrete set of single-pixel centerline coordinates, accurately describing the geometric center trajectory of the hose in the image. This effectively eliminates interference caused by variations in apparent width, thus ensuring the stability of the subsequent deformation datum.

[0081] Step 1.4: Parameterize the center line of a single pixel and divide it into equal-interval segments along its arc length.

[0082] After obtaining the discrete set of single-pixel coordinates, a parameterized representation is needed for accurate continuous spatial geometric analysis. This transforms the scattered pixel coordinates into a continuous geometric description with order and physical dimensions, enabling the calculation of geometric derivatives through numerical differentiation. This step first arranges the single-pixel coordinate set in connection order, representing the hose centerline as an arc length... Planar curve functions with parameters:

[0083]

[0084] In the formula, It is the arc length extending along the centerline of the hose. ; This is the total arc length of the hose centerline; These are the coordinates of a single pixel.

[0085] To balance computational efficiency (real-time performance) and feature fidelity (the degree of restoration of curve details), especially to effectively capture the minute local deformations that may occur in the early stages of the whiplash phenomenon, this embodiment of the invention parameterizes the curve... Perform discrete sampling with equal arc length. In a preferred embodiment, N sampling points are set at equal intervals along the arc length direction of the parameterized centerline. The number of sampling points N can be set according to the real-time and accuracy requirements of the actual application, and its value range is usually from 50 to 300. In this embodiment, N=200 is preferred.

[0086] Step 1.5: Based on discrete sampling points, calculate the curvature at each position of the hose centerline to generate a one-dimensional curvature feature sequence.

[0087] For each sampling point obtained in step 1.4, the curvature of that sampling point is calculated using the three-point local fitting method or the numerical differentiation method. The specific calculation formula is as follows:

[0088]

[0089] In the formula, , Centerline coordinates for parameters The first derivative, , Centerline coordinates for parameters The second derivative; in discrete space, the corresponding first and second derivative values ​​can be obtained by performing finite difference operations on adjacent sampling points.

[0090] After traversing all sampling points, a one-dimensional curvature feature sequence reflecting the degree of bending at various parts of the hose is obtained. , This represents the number of sampling points.

[0091] Step 1.6: Map the one-dimensional curvature feature sequence into a two-dimensional curvature heatmap for recognition by a deep convolutional neural network.

[0092] Since deep convolutional neural networks typically possess powerful feature extraction and pattern recognition capabilities on two-dimensional image data, this step, to further realize intelligent classification and early warning of hose dynamic behavior (such as normal swinging and abnormal whipping), will use the one-dimensional curvature feature sequence generated in step 1.5. Convert to a standard-sized two-dimensional image format, namely a two-dimensional curvature heatmap. The specific conversion process includes:

[0093] Step 1.6.1, data normalization processing.

[0094] To convert the physical quantity of curvature into grayscale values ​​of image pixels, it needs to be normalized. First, the theoretical minimum curvature value of the hose within its safe operating range is defined. (can be approximated as 0) and the possible limiting maximum curvature value Then, through the following linear mapping function... Each curvature value is mapped to a pixel grayscale range [0, 255] to convert the curvature value into grayscale brightness:

[0095]

[0096] In the formula, and These are the preset maximum and minimum curvature values ​​for the hose bending.

[0097] Thus, a one-dimensional grayscale value sequence is obtained, where the level of grayscale value directly corresponds to the magnitude of curvature value.

[0098] Step 1.6.2, scale resampling.

[0099] To meet the size input requirements of the deep neural network, the aforementioned one-dimensional grayscale sequence needs to be scaled and resampled. In this embodiment, an interpolation algorithm (e.g., bilinear interpolation) is used to resample the original sequence of length N to the target size width W (e.g., W=224 pixels), ensuring that hoses of different lengths have a consistent feature dimension in the heatmap space, and their curvature distribution features are unified to the same feature dimension before being input into the deep neural network.

[0100] Step 1.6.3, Two-dimensional matrix construction and curvature heatmap generation.

[0101] The purpose of this step is to convert the scale-resampled one-dimensional grayscale value sequence into a two-dimensional image matrix to obtain the desired curvature heatmap. This can be achieved using two well-known methods: The first is the vertical expansion method, which involves copying and expanding the resampled one-dimensional grayscale value sequence vertically until it fills to a preset height of H (H=224 pixels in this embodiment), thereby generating a two-dimensional matrix with a width of W and a height of H. This two-dimensional matrix is ​​the two-dimensional curvature heatmap. The second method is the spatial layout mapping method. First, the original coordinates of each sampling point in step 1.4 are normalized across all pixels in the image to obtain the corresponding normalized coordinates. Then, a blank image matrix with a width of W and a height of H is created (all pixels are initially 0). The normalized coordinates corresponding to each sampling point are mapped to the corresponding pixel position in this blank image matrix, and the grayscale value of the sampling point is set to the grayscale value of that pixel position. Finally, a Gaussian kernel is used to perform spatial smoothing on the mapped matrix, making the discrete sampling points continuous in the image to simulate the physical morphological continuity of the hose.

[0102] Using the aforementioned arbitrary two-dimensional matrix construction method, a two-dimensional curvature heatmap with a width of W and a height of H is finally generated. This two-dimensional curvature heatmap visually reflects the distribution of the degree of curvature at each point from the root to the end of the hose. The brightness of a pixel is proportional to the curvature, with brighter areas corresponding to the more curved sections of the hose. This two-dimensional curvature heatmap serves as input to the spatial geometric feature extraction module, used to extract high-frequency spatial texture features.

[0103] Step 2, spatial geometric feature extraction.

[0104] This step is implemented by the spatial geometric feature extraction module, aiming to extract spatial geometric feature vectors that can characterize the instantaneous abnormal bending morphology of the hose from the two-dimensional curvature heatmap obtained in step 1 through spatial field analysis. This allows for precise quantification and identification of whether the instantaneous bending shape of the hose at any given moment exceeds the normal range, providing crucial spatial morphological basis for subsequent whip-whip judgment.

[0105] The spatial geometric feature extraction module designed in this invention uses a pre-trained improved deep convolutional neural network to extract spatial geometric features from the input two-dimensional curvature heatmap, so as to effectively amplify the local high bending morphology in the whiplash situation and intuitively reflect the bending degree distribution of the hose from its root to its end at the current moment.

[0106] Specifically, this embodiment selects the well-known deep convolutional neural network ResNet-18 as the backbone network for feature extraction, and uses a two-dimensional curvature heatmap as the input to this backbone network. To achieve accurate capture of the complex deformation of the hose, this invention, based on the sparse and bright characteristics of the two-dimensional curvature heatmap, makes targeted improvements to the ResNet-18 network structure, embedding a designed G²ANet unit (dual-branch geometric attention enhancement unit) between specific residual blocks to replace the traditional attention module. This dual-branch geometric attention enhancement unit is designed with two parallel geometric topology attention branches and curvature gradient attention branches, used to simultaneously extract global topological features and local gradient features from the input feature map. This allows the backbone network to adaptively learn and enhance feature channels related to high curvature anomaly regions, sensitively capturing whiplash precursors or instantaneous geometric anomalies, rather than relying on the generalization ability of known networks, while also suppressing irrelevant background noise.

[0107] Furthermore, considering that the curvature anomaly of the hose includes both subtle kinks (shallow and intermediate features) and overall deformation (deep features), this embodiment does not simply stack G²ANet units between each residual block of the ResNet-18 network. Instead, it embeds G²ANet units at the outputs of the second residual block (Layer 2), the third residual block (Layer 3), and the fourth residual block (Layer 4) of the ResNet-18 network, respectively. This hierarchical embedding method enables the ResNet-18 backbone network to capture abrupt changes in the geometric shape of the hose image at different scales.

[0108] In this embodiment of the invention, the input to the ResNet-18 network is a 224×224 two-dimensional curvature heatmap generated in step 1, which is received by the first layer of the ResNet-18 network. The feature maps output by the second residual block Layer2, the third residual block Layer2, and the fourth residual block Layer4 of the ResNet-18 network are used as the input to the G²ANet units at the corresponding positions. The G²ANet units reweight the received feature maps through their geometric topology attention branch and curvature gradient attention branch. The feature maps enhanced by the G²ANet units flow back to the backbone network for subsequent downsampling processing. Through this repeated cycle of extraction-enhancement-re-extraction, adaptive enhancement of feature channels in high curvature anomaly regions is achieved.

[0109] Specifically, the G²ANet unit includes a parallel geometric topology attention branch and a curvature gradient attention branch, as well as a two-branch feature fusion gate following both branches. The geometric topology attention branch is configured to receive the shallow feature map output from its previous layer. And the node connection matrix (a data structure representing whether nodes are connected) obtained based on the parameterization of the hose centerline; and the shallow feature map The regional features corresponding to each discrete node along the centerline of the hose are aggregated into an initial node feature set. Based on the connection relationships (node ​​connection matrix) of each discrete node, the initial node feature set is propagated and updated using a graph attention coding strategy to obtain an updated node feature set containing information about the overall topology of the hose (e.g., node connections, loop structure). The updated node feature set is then mapped back to two-dimensional space to generate a topology-enhanced feature map. The geometric topology attention branch enhances the perception of overall topological features through the propagation of attention weights between graph nodes, enabling the detection of global topological anomalies such as loop self-intersections or local overlaps in flexible tubes. The specific execution strategy of the geometric topology attention branch is as follows:

[0110] Step s1: Construct the graph structure and initialize node features: Treat each discrete point on the hose centerline in the received shallow feature map as a graph node. Based on the physical continuity of the hose centerline, construct a node connection matrix and define the connections between nodes. The shallow feature map output from the previous layer... In the process, the spatial location vector corresponding to each graph node is extracted to form an initial node feature set.

[0111] Step s2, Graph Attention Encoding and Feature Update: Calculate the attention coefficient using the similarity between node features. :

[0112]

[0113] In the formula, , They are nodes and nodes The original input feature vector; It is a learnable linear transformation weight matrix, whose function is to map the low-dimensional geometric features of the input to a higher-dimensional feature space to enhance the linear expressive power of the features. It is the weight vector of a learnable single-layer feedforward neural network, which maps the concatenated high-dimensional vector to a scalar fraction; It is a non-linear activation function.

[0114] Step s3, attention weight normalization: using the Softmax function to normalize the attention weights of the nodes. all adjacent nodes The attention coefficients are normalized to obtain the node. For adjacent nodes Final attention weights ,

[0115] Step s4, based on the final attention weight The feature vectors of the nodes in the neighborhood are weighted and summed to generate an enhanced feature vector for each sampling point.

[0116] Step s5: The geometric topology attention branch identifies global topology anomalies in the hose by propagating and aggregating attention weights among nodes. The criterion is the consistency between the attention weight distribution and the physical topology structure, as detailed below:

[0117] Under normal conditions, the hoses are streamlined, and attention weights are highly concentrated between physically adjacent nodes, resulting in a clear and regular diagonal band structure in the corresponding weight matrix. However, when a global topology anomaly occurs, two sampling points that were originally physically far apart (e.g., nodes near the root) become disjointed. With the point near the end Attention weights between ) Abnormal peaks may occur due to reduced spatial distance or enhanced geometric correlation, when the attention weight... If the threshold is exceeded (e.g., 0.4) for several consecutive frames, it can be determined that the hose topology is abnormal.

[0118] It should be noted that the ring-shaped self-crossing described in this section specifically refers to the phenomenon where, under severe airflow disturbance or sudden stress change, the refueling hose's trajectory folds or wraps due to excessive local curvature, causing one segment of the hose to cross over and approach or even overlap another segment. Visually, this resembles the hose forming a temporary shape or a closed ring outline. Because this morphology disrupts the normal force transmission path of the hose, in graph attention mechanisms, it manifests as a very strong characteristic coupling response between non-adjacent nodes.

[0119] Step s6: Map the updated node feature set back to two-dimensional space and output the topology-enhanced feature map. This topology-enhanced feature map strengthens the representation of long-distance spatial connectivity relationships in hoses.

[0120] Specifically, the curvature gradient attention branch is configured to receive the shallow feature map output from the previous layer. Real-time calculation and extraction of curvature gradient changes in local regions, i.e., local gradient features; generation of spatial attention masks based on local gradient features; and processing of shallow feature maps. The attention weight map is generated by weighting the spatial attention mask and generating an attention weight map that is positively correlated with the rate of curvature change, which is the local deformation feature map. This feature map focuses on capturing local geometric deformations such as minute bends and kinks. The specific execution strategy of the curvature gradient attention branch is as follows:

[0121] First, gradient operations are performed on the received feature map using a first-order Laplacian operator (such as the Sobel operator or a differential convolution kernel) to extract the rate of change of curvature, i.e., the curvature gradient feature. This curvature gradient feature characterizes the abrupt change in curvature along the centerline of the hose. (Includes the corresponding curvature mutation value and mutation location).

[0122] Subsequently, based on the extracted curvature gradient features, a corresponding gradient attention mask is constructed using the following formula. :

[0123]

[0124] in, This represents the Sigmoid function. This gradient attention mask... The value at each position represents the importance weight of the local curvature change at the corresponding spatial location. This step, by constructing a gradient attention mask, can automatically amplify areas in the curvature heatmap where grayscale values ​​(reflecting changes in curvature values) fluctuate drastically, thus focusing on capturing minute kinks or sudden large-angle bends caused by the whiplash of the hose.

[0125] Finally, the gradient attention mask is used to reweight the received shallow feature map, and the local deformation feature map is output. It exhibits extremely high response values ​​at geometric anomaly points, providing crucial local anomaly geometric clues for subsequent fusion and classification.

[0126] The dual-branch feature fusion gate employs an additive fusion mechanism based on adaptive weight allocation, combining the topology-enhanced feature maps output from the two branches. and local deformation feature map Adaptive fusion is performed to obtain the enhanced feature map. The specific method is:

[0127] Topology-enhanced feature maps and local deformation feature map The feature maps are concatenated along the channel dimension. The concatenated feature maps are then processed through a global average pooling layer and a fully connected layer within the dual-branch feature fusion gate, performing global average pooling and fully connected operations sequentially to generate two scalar weight coefficients. and ,satisfy ; using the obtained scalar weighting coefficients and According to the formula Perform a pixel-wise weighted summation operation on the two feature maps to generate a fused enhanced feature vector that includes topological complexity, local curvature distribution, and cross-regional geometric correlation. The data is then fed back to the backbone network for further downsampling. This dual-branch feature fusion process achieves the goal of dynamically adjusting the focus on global deformation or local kinks based on the current shape of the hose.

[0128] After processing by the ResNet-18 network with embedded G²ANet units, the last global average pooling layer of the ResNet-18 network structure performs global average pooling on the final output feature vector, resulting in a high-dimensional feature vector (a 512-dimensional graph feature vector in this embodiment), which is the final output spatial geometric feature vector. Each dimension of the feature vector represents an abstract response value of the hose at a specific spatial scale or morphological pattern. The 512-dimensional length not only retains sufficient information richness to distinguish complex whip-like patterns, but also avoids computational redundancy caused by excessive dimensionality, ensuring the real-time performance of feature-level fusion and classification.

[0129] Step 3, Temporal Dynamics Feature Extraction

[0130] This step is implemented by the time-dynamics feature extraction module. Its purpose is to capture the dynamic characteristics of the hose motion from multiple consecutive frames of images of the hose motion through time field (or frequency field) analysis, particularly to identify the presence of short-term high-frequency oscillations that foreshadow the impending whip-flicking, i.e., frequency feature vectors reflecting its short-term dynamic behavior. This enables early warning of whip cracking. The specific process includes the following:

[0131] Step 3.1, Curvature Sequence Construction: From the M consecutive frames of images generated by the preprocessing in Step 1 (M=128 in this embodiment), the average curvature value of the hose centerline in each frame is extracted, thus forming a curvature time series of length M. Compared with the traditional method of selecting curvature at a single sampling point, the average curvature can characterize the overall energy fluctuation state of the hose, effectively filtering out the instantaneous noise caused by single-point image detection errors, making the time series more stable in reflecting the dynamic evolution trend of the hose from a calm state to a disordered state.

[0132] Step 3.2, Time-Frequency Analysis: Apply a Short-Time Fourier Transform (STFT) to the formed curvature time series to analyze its frequency characteristics over time. This curvature time series is converted into a time-frequency spectrum, where the horizontal axis represents time frames and the vertical axis represents frequency. The intensity of each point represents the amplitude of the frequency component at that moment. In this embodiment, a Hanning window is used to reduce spectral leakage, and the overlap rate is set to 50%. This setting ensures sufficient smoothness on the time axis, enabling the sharp capture of non-stationary signal characteristics at the moment of whiplash. Unlike traditional time-domain statistics, the time-frequency analysis described in this invention calculates key spectral indicators based on the obtained time-frequency spectrum, which together constitute a frequency feature vector. The key spectral indicators mentioned above include at least:

[0133] High-frequency energy percentage: Calculated above a preset frequency threshold (In this embodiment, the frequency is set to 5Hz based on the typical frequency characteristics of a refueling hose whip.) The ratio of the sum of the energies to the total energy is used. This indicator directly reflects the intensity of the high-frequency oscillation. In this embodiment, the frequency... The frequency was set to 5Hz, a value determined based on the natural frequency of the refueling hose physical model and experimental observations of actual whiplash events. The total energy refers to the sum of energy across the entire frequency band (from 0Hz to the Nyquist frequency) on the spectrum.

[0134] Spectral centroid: the frequency centroid of the energy spectrum This reflects the main distribution location of energy in the frequency domain. The calculation formula is: ,in, For frequency The energy power at a given point. The centroid of the spectrum reflects the main distribution center of energy in the frequency domain. In the pre-whiplash stage, due to the introduction of minute high-frequency vibrations, the centroid will shift significantly towards the high-frequency direction; this indicator can effectively distinguish between normal low-frequency oscillations and abnormal high-frequency oscillations, and is an important a priori indicator for identifying the whiplash outbreak point.

[0135] Spectral flux: Calculates the Euclidean distance difference between spectral values ​​of adjacent time frames. This is used to measure the abrupt changes (i.e., dynamic bursts) in the spectrum. The calculation formula is:

[0136]

[0137] In the formula, and They are respectively and The spectral values ​​at time t, and the difference in Euclidean distance between them. It directly quantifies the rate of evolution of the spectrum over time (i.e., spectral abruptness). When a whiplash occurs, the spectrum will instantly generate a cross-band energy burst, resulting in a surge in spectral flux, thereby enabling a rapid response to sudden special events.

[0138] Ultimately, a frequency eigenvector is obtained, which includes the high-frequency energy proportion, the spectral centroid, and the spectral flux. .

[0139] Step 4: Feature-level fusion and classification determination.

[0140] Feature-level fusion step: This embodiment achieves deep coupling of heterogeneous features through a feature-level attention mechanism, adaptively balancing the weight of instantaneous geometric distortion and time-frequency energy bursts in the classification and determination step. Specifically, the spatial geometric feature vector obtained in step 2 is... (In this embodiment, the feature vector is 512-dimensional) and the frequency feature vector obtained in step 3. (Based on spectral index extraction, the feature vector in this embodiment is 64-dimensional.) The feature vectors are concatenated along the channel dimension to obtain a concatenated feature vector of length 576. This concatenated feature vector is then input into a pre-trained small self-attention network, which consists of multi-head self-attention layers; in this embodiment, a small self-attention network with four attention heads is used. During training, the small self-attention network calculates the autocorrelation between the two feature vectors, learns the correlation weights between the two feature vectors generated from different dimensions, and automatically adjusts the contribution of the two different feature vectors in the final classification decision. After weighted fusion processing of the input concatenated feature vector according to the assigned correlation weights, a joint feature vector of 576 dimensions is formed. .

[0141] Classification determination steps: The weighted and fused joint feature vector... The input is a multilayer perceptron (MLP) classifier, which outputs a probability distribution sequence corresponding to the state categories of normal, whiplash precursor, and whiplash occurred. Preferably, this embodiment uses a multilayer perceptron (MLP) classifier containing three hidden layers with 256, 128, and 64 neurons respectively. Each hidden layer is followed by a ReLU activation function layer and a Dropout layer. The ReLU activation function and Dropout technique are used to prevent overfitting when the fused feature dimension is high. The output layer of the classifier contains three neurons, corresponding to the three categories of normal, whiplash precursor, and whiplash occurred. The output layer uses a Softmax activation function and outputs a three-dimensional probability vector, corresponding to the probability distribution sequence of the state categories of normal, whiplash precursor, and whiplash occurred (e.g., ...). ).

[0142] The decision unit in the feature-level fusion and classification module determines whether the confidence level of each state category exceeds the corresponding confidence threshold based on the state category probability distribution sequence output by the classifier. It takes the state category with the highest probability as the decision result and defines the highest probability value as the confidence level of that category; it then outputs the decision result and its corresponding confidence level. Simultaneously, it triggers the execution of subsequent warning and intervention control steps. Because the state category probability distribution sequence output by the classifier is an instantaneous classification probability sequence, there may be state category jumps at state critical points, such as frequent switching between the normal state and the whiplash precursor state, causing the onboard control system to receive unstable state signals.

[0143] To further improve the stability of the output results of the hose whiplash emergency detection system, this invention introduces a post-processing module. This module performs sliding window smoothing and hysteresis threshold discrimination processing on the probability distribution sequence continuously output by the feature-level fusion and classification decision module. Alternatively, it can directly perform hysteresis threshold discrimination processing to obtain stable judgment results. The most preferred processing method includes sequentially performing sliding window smoothing and hysteresis threshold discrimination processing.

[0144] (1) Sliding window smoothing: To filter out isolated false alarms caused by instantaneous noise or fluctuations and ensure the smoothness of hose state switching, this embodiment performs sliding window smoothing on the three-dimensional probability vector output by the classifier. Specific operation steps include:

[0145] Window parameter settings: In this specific application, the length L of the sliding window is set to 15 frames (corresponding to a real-time sampling time of approximately 0.5 seconds). This length selection effectively balances the sensitivity of recognition and the stability of output, suppressing false alarms caused by single-frame image jitter without introducing excessive warning delay.

[0146] Smoothing Mechanism Selection: The smoothing process can employ either the well-known sliding window averaging mechanism or the voting mechanism. The sliding window averaging mechanism calculates the mean by weighted summation of the confidence scores for each category within the window, which serves as the basis for determining the current frame. The voting mechanism, on the other hand, uses the category label with the highest frequency within the window as the final determination result. The specific processing steps of these two methods are well-known to those skilled in the art and will not be elaborated upon here.

[0147] Smoothing Output: After sliding window smoothing, the instantaneous classification probability sequence output by the classifier is transformed into a smooth state class probability distribution sequence and corresponding confidence scores. This process effectively suppresses jumps between state classes, thus ensuring that the output hose state curve remains continuous in time and is more physically plausible.

[0148] (2) Hysteresis Threshold Decision Processing: To further improve the robustness of the whiplash emergency identification system, this invention introduces a confidence-based hysteresis threshold decision unit after the classifier output layer. This unit, based on the sliding window smoothing process, further performs hysteresis threshold decision processing on the smoothed state category probability distribution sequence and its corresponding confidence level, or on the three-dimensional probability vector output by the classifier. By setting an asymmetric threshold (confidence threshold) for state switching, it effectively suppresses frequent jumps in output state caused by small fluctuations in confidence level near the classification critical point, thereby increasing the stability and reliability of state determination. Based on the statistical characteristics of the aerial refueling test data, different state entry and exit thresholds are set for each state. This embodiment sets the following logical thresholds:

[0149] Transitioning from normal state to whiplash precursor state: Set the confidence threshold for entering the whiplash precursor state to 0.8. The hose will only switch to the whiplash precursor state when the system detects that the confidence level of the whiplash precursor state category has increased and exceeded 0.8.

[0150] Returning from whiplash precursor state to normal state: Set the confidence threshold for exiting the whiplash precursor state to 0.6. The hose will only switch to the normal state when the system detects that the confidence of the whiplash precursor state category has dropped below 0.6.

[0151] Transition from pre-whiplash warning state to actual whiplash state: Set the confidence threshold for entering the actual whiplash state to 0.85. Only when the system detects that the confidence level of the actual whiplash state category has increased and exceeded 0.85 will the current state of the hose be switched to the actual whiplash state, and the corresponding warning operation be triggered.

[0152] Reverting from an already occurred whiplash state to a pre-whiplash state: Set the confidence threshold for reverting to an already occurred whiplash state to 0.65. Only when the system detects that the confidence level of the already occurred whiplash state category has dropped below 0.65 will the current state of the hose be switched to the pre-whiplash state, and the corresponding warning operation be triggered.

[0153] In a preferred embodiment of the present invention, a sliding window smoothing process and a hysteresis threshold determination process are used to further transform the instantaneous determination result of feature-level fusion and classification into a stable state determination result, thus avoiding state jumps between state transitions. The following uses the normal state and the whiplash precursor state as examples to illustrate its anti-jump mechanism:

[0154] If a single threshold is used (e.g., a threshold set to 0.70), when the hose is in a critical jitter state, the corresponding confidence level oscillates slightly between 0.69 and 0.71. The hose whiplash incident identification system of this invention will frequently alternate between outputting normal state signals and whiplash precursor state signals, causing the pilot to receive unstable alarm prompts. However, using the hysteresis threshold discrimination processing of this invention, when the hose state enters the whiplash precursor state (confidence level reaches 0.80), even if the confidence level temporarily drops to 0.75 due to noise interference, because it falls below the exit threshold of 0.60 for the whiplash precursor state, the hose whiplash incident identification system still maintains the output result as the whiplash precursor state, thereby ensuring the logical stability and physical consistency of subsequent warning and intervention control.

[0155] Based on this, the final state category (stable determination result) is processed according to the hysteresis threshold. The confidence level corresponding to this category is extracted from the continuously output state category probability distribution sequence or the smoothed state category probability distribution sequence in step 4, and used as the confidence level of the final state category. The final state category and its corresponding confidence level are then output. Simultaneously, the early warning and intervention control module executes corresponding early warning and intervention control steps based on the stable final state category and its corresponding confidence level obtained through post-processing.

[0156] Step 5: Early warning and intervention control steps.

[0157] Based on the judgment result and corresponding confidence level in step 4, the early warning and intervention control module performs the following tiered early warning and intervention control operations:

[0158] When the classification result is normal, there is no warning. The system continues to output a normal signal for the pilot's reference and continuously monitors the hose status.

[0159] When the classification result indicates a premonition of whiplash and the confidence level exceeds the warning threshold (0.8 in this embodiment), or when the classification result indicates that whiplash has occurred but the confidence level does not exceed the alarm threshold (0.85 in this embodiment), the identification system issues a warning to the pilot, reminding them to take evasive action or perform a stabilizing hose maneuver. Specifically, the warning may include a flashing yellow indicator on the head-up display (HUD) accompanied by a voice prompt: "Caution: Abnormal hose oscillation, it is recommended to stabilize airspeed."

[0160] When the classification result indicates that whiplash has occurred and the confidence level exceeds the alarm threshold, the system triggers an emergency alarm. The emergency alarm may display a red text message on the cockpit display screen: "Emergency: Hose Whiplash, Perform Automatic Disengagement!" accompanied by continuous, rapid warning prompts. Simultaneously, the identification system triggers an emergency intervention mode via the data bus in conjunction with the flight control system. The receiver aircraft automatically performs a slight deceleration or slight descent to tighten the hose, or directly releases the coupling lock for emergency disengagement. When the hose switches back to normal from the pre-whiplash state or the state where whiplash has already occurred, the flight control system immediately clears the corresponding visual and / or auditory warning prompts or emergency alarms.

[0161] To verify the effectiveness of the method of this invention, this embodiment collected 100 hours of aerial refueling simulation test data, covering scenarios of normal hose operation, pre-whiplash warning signs, and whiplash occurrence. Figure 2 and Figure 3 As shown, where, Figure 2 This is a diagram illustrating the state of the hose when a whiplash has occurred. Figure 3 This diagram illustrates the normal swinging state of the hose. To highlight the advantages of the dual-field fusion recognition method of this invention, this embodiment sets up three methods for comparative experiments. The first method uses only the spatial geometric feature extraction module to classify the single-frame curvature heatmap, with the input of the MLP classifier being the spatial geometric feature vector. The second method is the threshold method (focus counting method) commonly used in traditional engineering, which determines the whip-whip effect by detecting whether the number of connected components in the highlighted area of ​​the curvature heatmap exceeds a set threshold. The third method is the dual-field fusion method provided by this invention. The experiment used accuracy, false negative rate, and false positive rate as evaluation indicators for whip-whip performance, and the results are shown in Table 1.

[0162] Table 1: Comparison of Whip-Swing Recognition Performance of Different Methods

[0163]

[0164] Traditional threshold methods, due to their simple logic, rely on a single preset geometric deformation threshold. When the number of detected abnormal focal points exceeds the threshold, it is determined that whiplash has occurred. This method depends solely on simple point counting, cannot identify the topological structure of hose deformation, and is highly susceptible to interference from normal large-amplitude swings (generating a large number of false focal points). This results in its coarse logic, insensitivity to the dynamic process of the hose, and high false alarm and false negative rates. While spatial field-only methods can effectively identify morphological anomalies and reduce the false alarm rate, the false alarm rate is still as high as 10.1% because they cannot distinguish between normal deformation of the hose with large static curvature and dynamic high-frequency whiplash. The dual-field fusion method of this invention, by introducing temporal dynamics features, significantly reduces the false alarm rate (to 6.2% in this experiment). At the same time, the complementarity of spatial and temporal features further reduces the false alarm rate, which is reduced to 6.5% in this embodiment, thus achieving the highest accuracy of 87.3%. This fully demonstrates the necessity and superiority of fusing spatial geometric information with temporal dynamics information for accurately identifying hose whiplash events and whiplash precursors.

[0165] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A method for identifying whiplash-like special conditions in a flexible hose, used to distinguish between normal bending patterns and abnormal whiplash patterns in a flexible hose; characterized in that, Includes the following steps: Step 1: Image preprocessing and 2D curvature heatmap generation; Acquire multiple consecutive frames of images containing the hose, and process each frame to generate a two-dimensional curvature heatmap reflecting the instantaneous bending shape of the hose. Step 2, spatial geometric feature extraction; Based on an improved deep convolutional neural network, spatial geometric feature vectors representing the instantaneous spatial geometry of the hose are extracted from the two-dimensional curvature heatmap generated in step 1. ; The improved deep convolutional neural network uses the structure of deep convolutional neural network ResNet-18 as the backbone network and embeds a dual-branch geometric attention enhancement unit at the output of a specific residual block. The dual-branch geometric attention enhancement unit is configured to perform the following operations in a loop to enhance the high curvature anomaly region features in the two-dimensional curvature heatmap: receive the feature map output by the corresponding residual block in the backbone network, and simultaneously extract global topological features and local gradient features from the feature map. An additive fusion mechanism based on adaptive weight allocation is employed to fuse the topological enhancement feature maps and local deformation feature maps output from the two branches, generating a fused enhanced feature vector. Finally, the fused enhanced feature vector is returned to the backbone network for downsampling processing to obtain the spatial geometric feature vector. ; Step 3, extraction of time dynamic features; Based on the continuous multi-frame images obtained in step 1, the curvature sequence of the hose over time is calculated to form a curvature time series; and time-frequency analysis is performed on the curvature time series to extract frequency feature vectors characterizing the short-term dynamic behavior of the hose. ; Step 4: Feature-level fusion and classification determination; The spatial geometric feature vector obtained in step 2 Compared with the frequency eigenvector obtained in step 3 The features are fused to form a joint feature vector. ; The joint feature vector Input the classifier to obtain the probability distribution sequence of state categories corresponding to normal, whiplash premonition and whiplash already occurred; The state category with the highest probability in the state category probability distribution sequence is taken as the final judgment result, and the highest probability value is used as the confidence level of the judgment result. Output the determination result and the corresponding confidence level; Step 5: Early warning and intervention control; Based on the obtained judgment results and corresponding confidence levels, appropriate early warning and intervention control operations are triggered, including: If the determination result is normal, the current status information of the hose will be displayed in real time, and hose status identification will continue. If the determination result is a pre-whiplash state and its confidence level exceeds the preset first warning threshold, or if the determination result is a whiplash state that has occurred and its confidence level does not exceed the preset second warning threshold, then a warning reminder will be issued and a stabilizing hose operation will be performed. If the determination result indicates that a whiplash has occurred and its confidence level exceeds the preset second warning threshold, an emergency alarm will be triggered and corresponding safety intervention operations will be performed.

2. The method for identifying whiplash incidents using a flexible hose according to claim 1, characterized in that, The specific steps for generating a two-dimensional curvature heat map reflecting the instantaneous bending shape of the hose in step 1 are as follows: Step 1.1: Obtain the real-time video stream of the hose and decompose the real-time video stream into multiple consecutive frames of images according to a preset fixed frame rate; Step 1.2: Perform hose instance segmentation on each frame of the image to generate a pixel-level mask; Step 1.3: Perform morphological skeletonization on the pixel-level mask and extract the center line of its single-pixel width; parameterize the center line as arc length, and perform discrete sampling along the center line of the arc length to calculate the curvature value of each sampling point, forming a one-dimensional curvature feature sequence. Step 1.4: Map the one-dimensional curvature feature sequence to a standard-sized two-dimensional grayscale image, and use the two-dimensional grayscale image as the two-dimensional curvature heatmap of the input data of the improved deep convolutional neural network; in the two-dimensional curvature heatmap, the pixel brightness is proportional to the curvature size.

3. The method for identifying whiplash-related emergencies using a flexible hose according to claim 2, characterized in that, In step 2, a dual-branch geometric attention enhancement unit is embedded at the output of the second residual block, the third residual block, and the fourth residual block of the ResNet-18 backbone network. Each of the aforementioned bi-branch geometric attention enhancement units includes a geometric topology attention branch and a curvature gradient attention branch arranged in parallel, as well as a bi-branch feature fusion gate connected to the output of the two branches; The geometric topology attention branch is configured to receive the shallow feature map output from its previous layer. And the node connection matrix obtained based on the hose centerline parameterization; the shallow feature map The regional features corresponding to each discrete node of the hose centerline are aggregated into an initial node feature set; based on the connection relationship of each discrete node, the initial node feature set is propagated and updated through a graph attention coding strategy to obtain updated node features containing the overall topology information of the hose. The updated node features are mapped back to two-dimensional space to generate a topology-enhanced feature map. ; The curvature gradient attention branch is configured to receive the shallow feature map output from its previous layer. Extract the shallow feature map Local gradient features; spatial attention mask generated based on the local gradient features; shallow feature map The local deformation feature map is generated by weighting the spatial attention mask. ; The dual-branch feature fusion gate is configured to perform additive fusion on the received topology-enhanced feature map using an adaptive weight allocation-based weight allocation mechanism. and the local deformation feature map The features are fused to generate an enhanced feature vector, which is then fed back to the backbone network for subsequent downsampling processing.

4. The method for identifying whiplash-related emergencies using a flexible hose according to claim 3, characterized in that, The addition and fusion mechanism based on adaptive weight allocation is as follows: The topology-enhanced feature map and the local deformation feature map Perform splicing along the channel dimension; Global average pooling and fully connected operations are performed sequentially on the concatenated feature maps to generate two scalar weight coefficients. and ,in ; According to the formula Perform a pixel-wise weighted summation operation on the two feature maps to generate the fused enhanced feature vector. .

5. The method for identifying whiplash-related emergencies using a flexible hose according to claim 1, characterized in that, In step 3, the calculated curvature time series is subjected to time-frequency analysis using short-time Fourier transform, which converts it into a time-frequency spectrum. Key indicators in the time-frequency spectrum, including the proportion of high-frequency energy, the spectral centroid, and the spectral flux, are calculated, and these key indicators are used to construct the frequency feature vector. .

6. The method for identifying whiplash-related emergencies using a flexible hose according to claim 1, characterized in that, In step 4, a joint feature vector is formed. Includes the following steps: spatial geometric feature vectors With frequency eigenvectors The corresponding channel dimensions are concatenated to form a concatenated feature vector; The concatenated feature vector is input into a pre-trained small self-attention network; the small self-attention network consists of multi-head self-attention layers, which learn and assign the relevance weights of the two feature vectors in classification judgment through the multi-head self-attention mechanism during training. The concatenated feature vectors are weighted and fused according to the relevance weights to generate a joint feature vector with the same dimension as the concatenated feature vectors. The joint feature vector Used for subsequent classification and determination.

7. The method for identifying whiplash incidents using a flexible hose according to claim 6, characterized in that, The classifier used in step 4 is a multilayer perceptron classifier, which includes an input layer, three hidden layers and an output layer connected in sequence. Each of the hidden layers is followed by a ReLU activation function layer and a Dropout layer in sequence; The output layer includes three neurons, corresponding to the normal state, the pre-whiplash state, and the whiplash state, respectively, and uses the Softmax activation function to output a three-dimensional probability vector, representing the probability distribution of each state category.

8. The method for identifying whiplash incidents using a flexible hose according to claim 7, characterized in that, It also includes a post-processing step on the continuously output state category probability distribution sequence from step 4 to obtain a stable determination result and the corresponding confidence level. The post-processing step includes: Perform hysteresis threshold discrimination; or perform sliding window smoothing and hysteresis threshold discrimination sequentially; wherein: The hysteresis threshold discrimination method is as follows: based on preset state entry thresholds and state exit thresholds, hysteresis discrimination is performed on the received state category probability distribution sequence, and the current final state category is output; wherein: When switching from the normal state to the pre-whiplash state, the probability of the pre-whiplash state must be no less than the first entry threshold; when returning from the pre-whiplash state to the normal state, the probability of the pre-whiplash state must be less than the first exit threshold, and the first exit threshold must be less than the first entry threshold; when switching from the pre-whiplash state to the already occurred whiplash state, the probability of the already occurred whiplash state must be no less than the second entry threshold; when switching from the already occurred whiplash state to the pre-whiplash state, the probability of the already occurred whiplash state must be no less than the second exit threshold. Based on the hysteresis threshold, the final state category is determined and processed. The confidence level corresponding to the category is extracted from the continuous output probability distribution sequence of the state category or the smoothed probability distribution sequence of the state category in step 4. This confidence level is used as the confidence level of the final state category, and the final state category and its corresponding confidence level are output.

9. A special situation identification system for a flexible hose whip cracking device, characterized in that, A method for identifying whiplash incidents using a hose as described in any one of claims 1 to 8; the whiplash incident identification system includes: The image preprocessing and curvature heatmap generation module is used to: receive a series of multiple frames of images containing a hose, process the series of multiple frames of images, and generate a two-dimensional curvature heatmap; The spatial geometric feature extraction module, connected to the image preprocessing and curvature heatmap generation module, is used to: extract spatial geometric feature vectors representing the instantaneous spatial geometric shape of the hose from the two-dimensional curvature heatmap using an improved deep convolutional neural network. ; The time-dynamic feature extraction module, connected to the image preprocessing and curvature heatmap generation module, is used to: acquire the curvature time series corresponding to the consecutive multi-frame images, and perform time-frequency analysis on the curvature time series to extract frequency feature vectors characterizing the short-term dynamic behavior of the hose. ; The feature-level fusion and classification module, connected to the spatial geometric feature extraction module and the temporal dynamics feature extraction module respectively, is used to: fuse the spatial geometric feature vector... With the frequency feature vector By fusing the features, a joint feature vector is obtained. Based on the joint feature vector Classify the states to obtain probability distribution sequences corresponding to normal, pre-whiplash, and whiplash occurrence; use the state category with the highest probability in the probability distribution sequence as the final judgment result, and use the highest probability value as the confidence level of the judgment result; output the judgment result and the corresponding confidence level. The early warning and intervention control module, connected to the feature-level fusion and classification judgment module, is used to: trigger corresponding early warning and intervention control operations based on the judgment result and the corresponding confidence level, including: when the judgment result is a premonition of whiplash and the confidence level exceeds the set first early warning threshold, or when the judgment result is that whiplash has occurred and the confidence level has not exceeded the set second early warning threshold, issuing an early warning reminder and outputting a first control command to control the execution of stable hose operation; When the determination result is that a whip has been lashed and the confidence level exceeds the set second warning threshold, an emergency alarm is triggered and a second control command is output to control the execution of the corresponding safety intervention operation. When the determination result is normal, the current status information of the hose is displayed in real time, and the hose status continues to be identified.

10. The hose whip-whip emergency identification system according to claim 9, characterized in that, It also includes a post-processing module, configured to: perform hysteresis threshold discrimination processing on the state category probability distribution sequence continuously output by the feature-level fusion and classification decision module, or sequentially perform sliding window smoothing processing and hysteresis threshold discrimination processing to output the current final state category; Based on the final state category output by the hysteresis threshold discrimination processing, the confidence level corresponding to the category is extracted from the state category probability distribution sequence or the smoothed state category probability distribution sequence continuously output by the feature-level fusion and classification determination module, and used as the confidence level of the final state category. The final state category and the corresponding confidence level are then output.

Images