A hydraulic system drip visual detection method and system

Abstract

This invention belongs to the field of computer vision technology and relates to a visual detection method and system for hydraulic system dripping. The detection method includes the following steps: inputting the acquired hydraulic oil leakage video data into a trained GSSR-YOLO target detection model for target detection, extracting candidate detection results of hydraulic oil droplets in complex backgrounds; using the candidate detection results as initial input, passing them to the TSDR-Track target tracking algorithm to obtain the motion trajectory of the oil droplets in the spatiotemporal sequence. This invention designs a GSSR-YOLO target detection model and a TSDR-Track target tracking algorithm, achieving high-precision identification of small target oil droplets in complex backgrounds, and further realizing accurate identification of dynamic dripping.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a visual detection method and system for dripping in hydraulic systems. Background Technology

[0002] With the widespread application of hydraulic technology, fault detection in hydraulic systems faces numerous challenges. Among these, external leakage of hydraulic oil is one of the main factors affecting the stable operation of hydraulic systems. In the early stages of leakage, the leakage rate is usually slow, mainly manifesting as dripping at components or pipelines, making it difficult to detect in time. Once the leakage intensifies, it indicates that the component may have been damaged. External leakage of hydraulic oil can cause oil waste, equipment and environmental pollution, and even lead to safety accidents such as fires and personal injuries. Therefore, timely and accurate detection of oil droplets and identification of leakage phenomena has become an important guarantee for the stable operation of hydraulic systems.

[0003] Currently, traditional methods for detecting external hydraulic oil leaks mainly rely on manual inspection. However, this method is time-consuming, labor-intensive, and susceptible to subjective factors, failing to meet the timeliness requirements of detection, especially for early dripping leaks, which are prone to being missed. Secondly, while signal processing technology can detect oil leaks, the information revealed by the initial dripping phenomenon is very limited. At this stage, the changes in the system's operating status are not obvious, making it difficult to determine whether a leak has occurred based on changes in pressure, sound, and other signals. Furthermore, this type of method is highly susceptible to environmental factors. In addition, some studies have used thermal imaging technology or added fluorescent substances to the oil to enhance its visual effect. However, these methods are costly, have a high rate of missed detection for minor leaks, and in some situations (such as aircraft hydraulic systems and large equipment hydraulic systems), the addition of fluorescent substances to the oil is not permitted.

[0004] With the rapid development of computer vision technology, detection techniques based on convolutional neural networks and deep learning have received widespread attention due to their powerful feature extraction capabilities. Meanwhile, deep learning-based object detection methods are also widely used in engineering problems such as defect detection, demonstrating significant advantages over traditional methods in both accuracy and efficiency. However, for hydraulic oil external leakage detection tasks, the small size of the leaking target and the complex background increase the difficulty of identification. Furthermore, current methods are limited to the static detection of oil droplets and lack research on the identification of the entire leakage process, resulting in insufficient accuracy and applicability to fully meet industrial needs.

[0005] Therefore, how to achieve high-precision detection of oil droplets under complex backgrounds and accurately determine whether oil droplets are leaking is a problem that must be solved to achieve accurate detection of external leaks, which is of great significance for the stable operation of hydraulic systems. Summary of the Invention

[0006] The purpose of this invention is to provide a visual detection method and system for dripping in hydraulic systems, which solves the problem of high-precision detection of dripping phenomena under complex backgrounds.

[0007] This invention is achieved through the following technical solution: This invention discloses a visual detection method for dripping in a hydraulic system, comprising the following steps: The collected hydraulic oil leakage video data is fed into the trained GSSR-YOLO target detection model to perform target detection and extract candidate detection results of hydraulic oil droplets in complex backgrounds; The candidate detection results are used as initial inputs and passed to the TSDR-Track target tracking algorithm to obtain the motion trajectory of the oil droplets in the spatiotemporal sequence.

[0008] Furthermore, the GSSR-YOLO target detection model includes a backbone network, a feature fusion neck network, and a detection head network; The backbone network consists of a series of convolutional layers, a multi-scale feature fusion module, a dual attention mechanism module, a convolutional layer, an enhanced feature extraction module, a convolutional layer, an enhanced feature extraction module, a convolutional layer, a multi-scale feature fusion module, and a fast spatial pyramid pooling layer, connected in sequence. The feature fusion neck network includes a top-down path and a bottom-up path. The top-down path contains three sets of cascaded feature enhancement units, each of which includes an upsampling layer, a feature concatenation layer, and a multi-scale feature fusion module connected in sequence. The bottom-up path contains three sets of cascaded feature downsampling units, each of which includes a convolutional layer, a spatial depth transformation convolutional layer, a feature concatenation layer, and a multi-scale feature fusion module connected in sequence. The detection head network consists of three detection layers and a small target detection layer. The three detection layers are respectively connected to the multi-scale feature fusion module in the bottom-up path, and the small target detection layer is connected to the last multi-scale feature fusion module in the top-down path.

[0009] Furthermore, the enhanced feature extraction module includes an input module, a local feature extraction module, a global feature extraction module, a feature fusion module, and an output module connected in sequence; the local feature extraction module and the feature fusion module are connected; the local feature extraction module includes a depthwise separable convolutional layer and a pointwise convolutional layer connected in sequence; the global feature extraction module includes a lightweight encoder module and a pointwise convolutional layer connected in sequence; the feature fusion module includes a concatenation layer, a convolutional layer, and a residual connection layer connected in sequence; the input module and the residual connection layer are connected.

[0010] Furthermore, the feature extraction process of the enhanced feature extraction module is as follows: The input characteristics of the input module are X , ;in C , H and W These represent the number of channels, height, and width of the input feature, respectively. The input features are passed through the input module and then fed into a depthwise separable convolutional layer to capture global information of the input image. Subsequently, a pointwise convolutional layer projects the output back to the original channel dimension, outputting local features. Y ; Next, dimensionality reduction is performed in the global feature extraction module, and the output is recorded as the global features. Z ; In the feature fusion module, the stitching layer combines local features Y With global features Z The data is concatenated along the channel dimension and then passed through a convolutional layer to output features. F The residual connection layer will feature F Input features X Perform residual connection output features O .

[0011] Furthermore, the calculation formula for the local feature extraction module is as follows: ; The calculation formula for the global feature extraction module is as follows: ; The calculation formula for the splicing layer is as follows: ; The formula for calculating convolutional layers is as follows: ; The calculation formula for the residual connection layer is as follows: ; Here, DWConv-5×5 represents a depthwise separable convolution with a kernel size of 5×5, and Transformer represents a lightweight encoder module.

[0012] Furthermore, the dual attention mechanism module includes a channel attention mechanism module and a spatial attention mechanism module.

[0013] Furthermore, the multi-scale feature fusion module includes convolutional layers, feature splitting layers, Bottleneck_MCA layers, splicing layers, and convolutional layers; After the input features are processed by the convolutional layer, they are divided into two branches in the feature splitting layer. One branch directly enters n Bottleneck_MCA layers, while the other branch skips the concatenation process. Finally, the outputs of all branches are concatenated and restored to the number of channels through the convolutional layer to output refined features.

[0014] Furthermore, the Bottleneck_MCA layer includes an MCA module, which includes a global pooling module, a local pooling module, and sequentially connected scale fusion module, feature concatenation module, dilated convolution module, normalization and nonlinear activation module, feature splitting module, depthwise separable convolution module, normalization module, and reweighting module; the global pooling module and the local pooling module are respectively connected to the scale fusion module; The input features decompose the target's position information into two one-dimensional feature codes in the global pooling module and the local pooling module, respectively; the two one-dimensional feature codes include horizontal dimension feature codes and vertical dimension feature codes; The two one-dimensional feature codes output by the global pooling module and the local pooling module are fused in the scale fusion module to output a fused feature map; Perform feature concatenation on the fused feature map to output an aggregated feature map; The aggregated feature map is effectively compensated by dilated convolutional layers, and the compensated feature map is output. After being processed by the standardization and nonlinear activation modules, the compensated feature map is mapped to the compressed latent feature space to obtain the activated feature vector. Perform feature splitting to restore the activated feature vectors into two independent branch vectors, one horizontal and one vertical; Attention weights are calculated using depth-separable convolutional layers along the horizontal and vertical directions, respectively, and then processed by normalization layers to generate horizontal and vertical weight maps. The horizontal and vertical weight maps are reweighted with the residuals of the original input feature map through element-wise multiplication to obtain the output feature map.

[0015] Furthermore, the TSDR-Track target tracking algorithm includes a detection branch, a trajectory prediction branch, and a matching update module; The detection branch is used to perform target detection and feature extraction on the input video frame to obtain a candidate detection set of oil droplets; the trajectory prediction branch is used to perform state prediction based on the detection results of the previous frame using a Kalman filter based on the motion parameters of the leaking oil droplets, and obtain the prediction result. The matching update module is used to fuse the detection set and the prediction results through features, and to associate the target through the multi-dimensional matching cost calculation module. The multi-dimensional matching cost calculation module integrates two cost indicators: IoU cost and anchor proximity cost. The anchor proximity cost is constructed by the anchor matching remediation mechanism. After the target association is completed, the Hungarian matching algorithm is used for global optimal allocation to ensure a one-to-one correspondence between the predicted trajectory and the detected target; Subsequently, a Kalman state update is performed on the successfully matched targets, while a certain number of frames of prediction are retained for the unmatched trajectories.

[0016] This invention also discloses a visual inspection system for dripping in hydraulic systems, comprising the following steps: The target detection module is used to input the collected hydraulic oil leakage video data into the trained GSSR-YOLO target detection model to perform target detection and extract candidate detection results of hydraulic oil droplets in complex backgrounds. The target tracking module is used to take the candidate detection results as initial input and pass them to the TSDR-Track target tracking algorithm to obtain the motion trajectory of the oil droplet in the spatiotemporal sequence.

[0017] Compared with the prior art, the present invention has the following beneficial technical effects: This invention proposes a visual detection method for hydraulic system leaks. It designs a GSSR-YOLO target detection model and a TSDR-Track target tracking algorithm. The GSSR-YOLO target detection model performs target detection on acquired hydraulic oil leak video data, extracting candidate detection results of hydraulic oil droplets in complex backgrounds. These candidate detection results are used as initial input to the TSDR-Track target tracking algorithm to obtain the trajectory of the oil droplets in the spatiotemporal sequence. This invention effectively integrates the GSSR-YOLO detection model and the TSDR-Track tracking algorithm, achieving leak status identification based on multiple frames of images, beyond single-frame oil droplet recognition, effectively avoiding false detections of leaks from single-frame images.

[0018] Furthermore, the designed GSSR-YOLO target detection model incorporates an enhanced feature extraction module in the backbone network, combined with a dual attention mechanism, to improve the model's global perception of leakage features and mitigate background interference. A small target preservation mechanism based on SPD-Conv is built in the feature fusion neck network to prevent the loss of oil information from small targets. A multi-scale feature fusion module is designed to enhance the model's feature extraction capability in complex backgrounds. A small target detection branch is added to the model's head network to improve small target detection performance. Compared with traditional detection models, this method significantly improves the accuracy of small oil droplet detection in complex backgrounds and exhibits strong robustness.

[0019] Furthermore, the designed TSDR-Track target tracking algorithm includes a detection branch, a trajectory prediction branch, and a matching update module. The trajectory prediction branch utilizes a Kalman filter based on the motion parameters of the leaking oil droplets, enabling the model prediction stage to predict the trajectory based on the actual motion state of the oil droplets, significantly improving the stability and scene adaptability of the leaking oil droplet trajectory. The matching update module uses a prediction matching remediation strategy based on anchor point proximity to reduce the loss and breakage of oil droplet trajectory and enhance the robustness of the leakage trajectory data association. A trajectory post-processing and detector pre-filtering mechanism is designed to adaptively identify and remove stationary clutter, further ensuring the closed-loop reliability of the leaking target tracking link. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the visual inspection method for dripping in hydraulic systems. Figure 2 This is a structural diagram of the GSSR-YOLO detection model, a visual inspection method for hydraulic system leaks. Figure 3 This is a structural diagram of the Enhanced Feature Extraction Module (EFE) for visual detection of leaks in hydraulic systems. Figure 4 This is a structural diagram of the multi-scale feature fusion module (MFF) for visual detection of leaks in hydraulic systems. Figure 5 This is a flowchart of the TSDR-Track oil droplet target tracking algorithm, a visual detection method for hydraulic system leaks. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the present invention, and not all of them.

[0022] The detailed description of the embodiments of the present invention provided in the following figures is not intended to limit the scope of the claimed invention, but merely to illustrate one selected embodiment of the invention. All other embodiments obtained by those skilled in the art based on the figures and embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0023] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0024] From a design perspective, this invention relates to a visual detection method for hydraulic system leaks based on oil droplet and target tracking and recognition, the process of which is as follows: Figure 1 As shown, it includes the following steps: S1. Establish a dataset of hydraulic oil leakage images of small targets against a complex background, and perform data augmentation on the dataset. This consists of the following two sub-steps: S1.1 Establish an image dataset of small-target hydraulic oil leakage under complex backgrounds, specifically including a detection dataset and a target tracking dataset. This includes the following steps: (1) Field data collection: An experimental environment for external leakage of hydraulic oil was built based on a hydraulic test bench. During the experiment, parameters such as the pressure, flow rate and oil outlet sealing status of the hydraulic system were adjusted to simulate the external leakage phenomenon of the hydraulic system under different working conditions. A high-resolution industrial camera was used to collect video data of the external leakage process of hydraulic oil, and the entire process of oil droplet formation, dripping and disappearance was obtained.

[0025] The complex background consists of background elements included in the experimental scene, such as hydraulic pipelines, hydraulic pumps, connecting joints, and background structural components, in order to recreate the real and ever-changing working environment of industrial equipment.

[0026] (2) Constructing the detection dataset: Based on the collected video data, continuous frame image samples were obtained by extracting images frame by frame. Among the extracted image samples, the image set containing externally leaking oil droplets was selected. To ensure the high quality of the dataset, the LabelImg tool was used to annotate the images, marking the position and size of the oil droplets in each frame and generating corresponding annotation files to construct a standardized image dataset for the target detection task. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. This dataset is mainly used to train and validate the GSSR (Global Sensing and Small-target Retention)-YOLO hydraulic oil external leakage detection model, i.e., the target detection model.

[0027] (3) Constructing a target tracking dataset: Based on the collected video data, video sequences that can record the complete dynamic changes of oil droplets were selected. DarkLabel software was used to annotate the video sequences frame by frame. By assigning the same identification number to the same oil droplet in different frames, the trajectory annotation of oil droplets across frames was achieved. The generated dataset contains information on the size and position changes of oil droplets in the time series, the trajectory continuity relationship and motion characteristics, which are used to verify the Time-Space Dynamic Recognition-Track (TSDR) algorithm for tracking leaking targets in spatiotemporal sequences.

[0028] S1.2, Perform data augmentation on the target tracking dataset, specifically as follows: Based on the acquired video data, 100-150 consecutive frames are selected as an enhancement segment. Different enhancement methods are applied to different enhancement segments to construct a diverse video sample set. The enhancement methods include: adding filtered noise, simulating lens shake, changing lighting conditions, adding static targets, adding occlusion during the leakage process, and constructing multi-target tracking samples.

[0029] The construction of multi-target detection samples involves copying and pasting the target in the image, which is a type of enhancement method.

[0030] The construction of multi-target tracking samples involves copying video clips containing leakage phenomena, pasting them into an enhanced clip, and ensuring that all leakage phenomena contained in the processed enhanced clip do not overlap.

[0031] Data augmentation of the target tracking dataset can significantly improve its diversity, thereby enhancing the robustness and generalization performance of the target tracking algorithm under different environmental conditions.

[0032] S2. Construct the GSSR-YOLO target detection model.

[0033] The GSSR-YOLO target detection model is an improvement on the YOLOv8 network model, and its structure is as follows: Figure 2 As shown in the figure, this improved strategy mainly enhances the model's ability to extract and preserve features of small global targets, and strengthens its resistance to interference from complex background information, thereby achieving high-precision detection of small target oil droplets in hydraulic oil leakage images.

[0034] The GSSR-YOLO object detection model comprises a backbone network, a feature fusion neck network, and a detection head network. The backbone network consists of a series of convolutional layers, a multi-scale feature fusion module, a dual attention mechanism module, a convolutional layer, an enhanced feature extraction module, a convolutional layer, a multi-scale feature fusion module, and a fast spatial pyramid pooling (SPPF) layer.

[0035] The feature fusion neck network includes a top-down path and a bottom-up path, wherein: The top-down path contains three sets of cascaded feature enhancement units. Each set of feature enhancement units includes an upsampling layer, a feature concatenation module, and a multi-scale feature fusion module connected in sequence. The bottom-up path contains three sets of cascaded feature downsampling units. Each set of feature downsampling units includes a convolutional layer, a spatial depth transformation convolutional layer (SPDConv), a feature concatenation layer (Concat), and a multi-scale feature fusion module (MFF) connected in sequence.

[0036] The detection head network consists of three detection layers and a small object detection layer. The three detection layers are connected to the corresponding multi-scale feature fusion modules in the bottom-up path, and the small object detection layer is connected to the last multi-scale feature fusion module in the top-down path.

[0037] Introduce an Enhanced Feature Extraction (EFE) module into the backbone network: The structure of the enhanced feature extraction module is as follows: Figure 3 As shown, the system includes an input module, a local representation block, a global representation block, a fusion block, and an output module connected in sequence. The output of the local representation block is also connected to the fusion block. The local representation block includes a depthwise separable convolutional layer and a pointwise convolutional layer connected in sequence. The global representation block includes a lightweight encoder module and a pointwise convolutional layer connected in sequence. The fusion block includes a concatenation layer, a convolutional layer, and a residual connection layer connected in sequence. The input module is connected to the residual connection layer.

[0038] The feature extraction process of the enhanced feature extraction module is as follows: The input characteristics of the input module are X , ,in C , H and W These represent the number of channels, height, and width of the input feature, respectively. The local feature extraction module focuses on capturing fine-grained information in the image but lacks an understanding of the global context. To fully leverage the role of local feature information and improve the performance of subsequent feature extraction, a DWConv-5×5 convolution is used in the local feature extraction module. This increases the receptive field of the module, helping the model to focus on capturing global information of the input image and reducing the negative impact of dominant background features on detection performance. Subsequently, a pointwise Conv-1×1 convolution is performed to project the output back to the original channel dimension, outputting the local features. Y ; Subsequently, dimensionality reduction is performed in the global feature extraction module, and the output is recorded as the global features. Z ; In the feature fusion module, local features Y With global features Z The features are concatenated along the channel dimension and then output after Conv-3×3 convolution. F Ultimately, the characteristics F Input features X Perform residual connection output features O .

[0039] Specifically, the calculation formula for the local feature extraction module is as follows: ; The calculation formula for the global feature extraction module is as follows: ; The calculation formula for the splicing layer is as follows: ; The formula for calculating convolutional layers is as follows: ; The calculation formula for the residual connection layer is as follows: .

[0040] Figure 2 In this context, DWConv-5×5 represents a depthwise separable convolution with a kernel size of 5×5, Concat represents channel concatenation, and Transformer represents a lightweight encoder module used to enhance global feature extraction and modeling capabilities while ensuring model efficiency. The final output features... O The EFE module is able to retain sufficient local and global feature information with minimal parameter overhead, making it suitable for small target detection tasks in complex backgrounds.

[0041] Introduce a dual attention mechanism module (CBAM) into the backbone network: Hydraulic oil leak images contain numerous components, pipelines, and equipment. Detecting the same scene from different angles can lead to significant background differences, making the features expressed by tiny leaking oil droplets easily interfered with by background information. To address this issue, the CBAM attention mechanism is introduced, which can weaken the influence of complex background information on the detection results.

[0042] The dual attention mechanism module includes a channel attention mechanism module and a spatial attention mechanism module. These modules effectively filter out meaningful channel and spatial features of the oil droplets to be detected. Using this dual attention mechanism allows the network to selectively emphasize informational features and suppress background noise, which is particularly important for detecting tiny oil targets in complex industrial scenarios.

[0043] The structure of the multi-scale feature fusion module (MFF) is as follows: Figure 4 As shown, it is an improvement based on the C2f module used in YOLOv8, and the specific improvement method is as follows: In the YOLOv8 model, the C2f module, by combining depthwise separable convolution and cross-layer connections, can provide higher detection accuracy while maintaining computational efficiency. However, when the model is applied to complex scenarios such as detecting hydraulic oil leaks, the C2f module struggles to provide sufficiently detailed feature information. This is because the cross-connection mechanism used by the C2f module primarily transmits information along the channel dimension, limiting its ability to effectively focus on key regions, resulting in insufficient feature selection capabilities in complex contexts. To address this, this invention proposes a multi-scale pooling attention mechanism (MCA) based on Coordinate Attention (CA).

[0044] Figure 4 The upper part is the multi-scale feature fusion module (MFF module); the middle part is the structure of the Bottleneck_MCA layer in the MFF; the lower part is the MCA module in the Bottleneck_MCA layer.

[0045] like Figure 4 As shown, the multi-scale feature fusion module includes convolutional layers, feature splitting layers, Bottleneck_MCA layers, concatenation layers, and convolutional layers. In the MFF module, after the input features undergo convolutional block operations, they are split into two branches in channel splitting. One branch directly enters n Bottleneck_MCA layers, while the other branch skips and participates in the concatenation process. Finally, the outputs of all branches are concatenated, and the number of channels is restored through convolutional blocks to output refined features.

[0046] In the MFF module, the Bottleneck structure is reconstructed using the MCA attention mechanism, resulting in the Bottleneck_MCA structure. The MFF module effectively enhances the model's ability to capture key features in complex environments without significantly increasing the number of model parameters. In Bottleneck_MCA, the input features pass through two convolutional blocks, followed by MCA attention processing. The resulting features are then element-wise added to the input features to obtain the output features.

[0047] like Figure 4As shown, the MCA module in the Bottleneck_MCA layer includes a global pooling module, a local pooling module, and a scale fusion module, a feature concatenation module, a dilated convolution module, a normalization and nonlinear activation module, a feature splitting module, a depthwise separable convolution module, a normalization module, and a reweighting module connected in sequence; the global pooling module and the local pooling module are respectively connected to the scale fusion module.

[0048] In the MCA module, the size of the input feature is C × H × W In the global pooling module, the target's location information is decomposed into two one-dimensional feature codes, which helps the attention module capture the target's precise location information. However, when facing extreme sizes or complex backgrounds, single-dimensional global projection can easily lead to feature dilution. To address this, the local pooling module of this invention divides the features into multiple sub-regions in a single direction for adaptive average pooling, achieving multi-scale pooling operations. In this branch, the input features are still encoded according to the horizontal and vertical dimensions, but during pooling, adaptive average pooling operations Adp Avg Pool(X, (H, 2)) and Adp Avg Pool(X, (2, W)) are used to operate on the two dimensions respectively. At this time, the output feature size is... C × H ×2 and C ×2× W Next, local compression is performed to obtain features incorporating locally sampled information, with a size of [size missing]. C × H ×1 and C ×1× W .

[0049] Subsequently, the output features from the global pooling module and the local pooling module are fused in the scale fusion module to output a fused feature map. The feature size of the fused feature map is... C × H ×1 and C ×1× W Using multi-scale pooling can reduce the interference of abnormal noise in the image, making it easier to accurately locate the attention weights on small targets during training.

[0050] Then, feature concatenation is performed to output an aggregated feature map, the feature size of which is [value missing]. C ×1× (W+H) .

[0051] Next, by using dilated convolution, the mechanism can obtain a larger receptive field with the same number of parameters and computational cost, effectively compensating for the information loss problem caused by pooling operations. After the dilated convolution operation, the compensated feature map is output, and the feature size of the compensated feature map is [missing information]. C / r ×1× (W+H) ; After being processed by the standardization and non-linear activation module (BatchNorm+Non-linear), the compensated feature map is mapped to the compressed latent feature space to obtain the activated feature vector. Then, feature splitting is performed to restore the activated feature vectors into two independent branch vectors, one horizontal and one vertical. Next, attention weights are calculated using depthwise separable convolutions along the horizontal and vertical directions, respectively. This design can significantly reduce the computational complexity and number of parameters of the model, while preserving the module's ability to discriminate fine-grained features to the maximum extent. After normalization (Sigmoid) processing, an attention weight map is generated with a range between 0 and 1. The attention weight map includes a horizontal weight map and a vertical weight map. Finally, the obtained horizontal and vertical weight maps are re-weighted with the residual of the original input feature map through element-wise multiplication to obtain the output feature map.

[0052] Constructing a small target preservation mechanism in the feature fusion neck network: Striking convolutions and pooling operations are commonly used in deep learning models. These methods can effectively reduce the computational cost and complexity of the model while acquiring more feature information. However, excessive use can reduce the spatial resolution of the feature map, ultimately causing the model to focus more on suboptimal feature information during detection, especially leading to missed detections of small objects. To address the issue of preserving the positive improvements brought by methods such as striding convolutions and pooling while avoiding the loss of key information about small objects, a Spatial Depth Transformation Convolutional Layer (SPD-Conv) is introduced before feature fusion in deeper networks. This layer serves as a small object preservation mechanism in the feature fusion step, retaining more feature information about small objects. SPD-Conv consists of a Spatial to Depth (SPD) layer and a non-striking convolutional layer. The SPD layer remaps the spatial dimension of the input feature map to the channel dimension, allowing the model to retain as much spatial information as possible when processing the features of small oil objects. Furthermore, this design helps avoid information loss caused by downsampling operations in deep learning models. Non-striking convolutions can extract features without reducing the size of the feature space, thus preserving the fine features present in the image.

[0053] Add a small target detection branch to the head detection network: In the YOLO model, three detectors are used by default to achieve multi-scale target detection. To enhance the detection capability of small targets in hydraulic oil leak images, this invention adds a small target detection branch, Detect-x. This branch has a feature map scale of 160×160, targeting tiny targets of 4×4 pixels and above. By fusing shallow features, this branch can retain more hydraulic oil droplet contour and position information, thereby significantly improving the model's detection accuracy for small targets.

[0054] S3. After constructing the GSSR-YOLO model, train the GSSR-YOLO model based on the detection dataset to obtain the trained network model. Specifically, based on the previously constructed dataset of small-target hydraulic oil leakage images against a complex background, train the GSSR-YOLO target detection model to obtain the trained detection network weight model.

[0055] The training environment uses an NVIDIA GeForce RTX 4060Ti GPU with 8GB of video memory, an Intel(R) Core(TM) i5-12400F CPU, and Windows 11 as the operating system.

[0056] The software development and experimental environment is the PyCharm integrated development platform. The model is based on the PyTorch 2.1.0 deep learning framework and runs with the support of Python 3.8 and CUDA 12.1.

[0057] The model input image resolution was set to 640×640, the training epochs were 300, and the batch size was 4. No pre-trained weights were used during training, and all other hyperparameters were set to system defaults.

[0058] The model weight file generated after training is used as the final output of the detection model and serves as the input basis for the subsequent oil droplet recognition and target tracking algorithm TSDR-Track, so as to realize the dynamic detection and status judgment of hydraulic oil external leakage.

[0059] S4. Construct the TSDR-Track target tracking algorithm The characteristics of hydraulic oil leakage dictate that it is a dynamic and continuous process, with the trajectory of oil droplets changing over time. Therefore, relying solely on target detection within a single frame image is insufficient to fully reflect the actual dynamic situation of hydraulic oil leakage. This invention proposes the TSDR-Track target tracking algorithm, based on the DeepSORT target tracking algorithm and specifically optimized for the dynamic characteristics of hydraulic oil leakage. This algorithm can continuously track the dynamic movement trajectory of oil within the leakage area, achieving accurate tracking of hydraulic oil leakage and improving the real-time monitoring and detection capabilities for continuous leakage.

[0060] like Figure 5 As shown, the TSDR-Track target tracking algorithm includes a detection branch, a trajectory prediction branch, and a matching update module.

[0061] The detection branch is used to perform target detection and feature extraction on the input video frames to obtain a candidate detection set of oil droplets; the trajectory prediction branch is used to predict the state based on the trajectory result of the previous frame using a Kalman filter based on the motion parameters of the leaking oil droplets, and obtain the prediction result.

[0062] The TSDR-Track target tracking algorithm process is as follows: Figure 5 As shown, after the input video frame is input, the detection branch performs target detection and feature extraction on the input video frame to obtain a candidate detection set of oil droplets; the trajectory prediction branch uses a Kalman filter based on the motion parameters of the leaking oil droplets to predict the state based on the trajectory result of the previous frame, and obtains the prediction result. After the prediction result is output, it is paired with the detection set of the detection branch and participates in the subsequent matching and update process.

[0063] The Kalman filter based on the motion parameters of the leaking oil droplets can more accurately describe the motion of oil droplets in the spatiotemporal sequence. The improvement includes the following two aspects: State Vector: The state vector of a Kalman filter based on the motion parameters of leaking oil droplets includes [ x,y,s,r,v x ,v y , v s ,v r ],in x,y Indicates the coordinates of the center point of the detection box. s Indicates the area of ​​the detection frame. r Indicates the aspect ratio of the detection frame. v x , , v y This represents the rate of change of the center point coordinates of the detection box across consecutive frames; vs This represents the rate of change of the detection box area across consecutive frames; v r This represents the rate of change of the aspect ratio of the detection frame across consecutive frames. This method enables the filter to simultaneously characterize the spatial position, size changes, and morphological changes of the oil droplet, thereby more accurately fitting the true motion state of the oil droplet during its fall.

[0064] Non-dominant direction motion penalty mechanism: A non-dominant direction motion penalty mechanism was constructed in the prediction stage of the Kalman filter based on the leakage oil droplet motion parameters. The calculation process is as follows: ; ; In the formula: This is a motion direction monitoring function that can extract the historical cumulative motion trend of the target motion phase. , and the posterior velocity obtained in the previous update phase In the previous update phase, when a trajectory collision occurred in the detected value, causing the posterior velocity to... With historical movement trends If the directions are inconsistent, a conflict motion penalty factor is constructed in the prediction phase at the current moment. Correct the prior state at the current moment. The corrected prior state at the current moment is obtained. This mechanism effectively ensures the accuracy of the Kalman filter in predicting oil droplet motion.

[0065] In the update phase of the Kalman filter based on the motion parameters of leaking oil droplets, the conflict motion penalty factor affects the state covariance matrix. The velocity noise is used to perform feedback expansion, making the observations more reliable in the prediction phase of the next time step. The constructed feedback expansion process is as follows: .in, This represents the element in the nth row and nth column of the state covariance matrix. After the non-dominant direction motion penalty mechanism is introduced more than twice during the prediction phase, the tracking of that target will be directly removed during the update phase. This effectively filters out falsely detected random moving targets and prevents continuous ID interference.

[0066] Matching and updating mechanism: After feature fusion of the detection set and prediction results, target association is performed through a multi-dimensional matching cost calculation module. This module integrates two cost metrics: IoU cost and anchor point proximity cost, with the anchor point proximity cost constructed by an anchor point matching compensation mechanism. Because oil droplets exhibit low IoU matching during free fall or brief occlusion, frequent ID switching can occur. In such cases, compensating for the matching cost by calculating the distance difference in the Euclidean distance between anchor points enables trajectory recovery and improves the stability of trajectory association.

[0067] After target association is completed, the Hungarian matching algorithm is used for global optimal allocation to ensure a one-to-one correspondence between the predicted trajectory and the detected target. Subsequently, Kalman state updates are performed on successfully matched targets, and a certain number of frames of prediction are retained for unmatched trajectories to prevent trajectory interruption due to temporary frame loss.

[0068] To further improve the accuracy of the TSDR-Track target tracking algorithm in identifying real leak targets, a closed-loop optimization mechanism for pre-detector filtering and post-trajectory processing was constructed.

[0069] The trajectory post-processing mechanism enables the TSDR-Track target tracking algorithm to dynamically maintain each historical tracking region after trajectory updates. If the position, size, and velocity changes of a target within a region are below a set threshold across multiple consecutive frames, it is determined to be a stationary target. In this case, the trajectory is removed, and the region is marked as a "filtered area." In subsequent frames, the detection branch, after detection output, uses the detector pre-filtering mechanism to determine whether the target appears within the constructed "filtered area." If so, the target is removed, and the region's last detection time is updated. If it does not appear within the "filtered area," the target detection history is updated, and a target trajectory is created. Furthermore, during the trajectory post-processing stage, filtering region timeout detection is performed based on the region's last detection time. If a timeout occurs, the filtering state of the region is automatically lifted, enabling dynamic region reuse. This closed-loop optimization strategy establishes an information feedback relationship between the detector and the tracker: detection results guide trajectory updates, and trajectory information inversely optimizes the detection input, thereby significantly improving the robustness and recognition accuracy of trajectory detection.

[0070] Ultimately, this invention uses the trained GSSR-YOLO target detection model as a detector and the TSDR-Track target tracking algorithm to dynamically determine leakage behavior.

[0071] This invention constructs a visual detection and dynamic identification system for hydraulic oil external leakage based on the GSSR-YOLO target detection model and the TSDR-Track target tracking algorithm. Through the coordinated operation of the detection and tracking models, this system achieves real-time monitoring and dynamic judgment of the shape and location of oil droplet leaks. The overall process is as follows: First, the GSSR-YOLO target detection model performs target detection on the input video frames, extracting candidate detection results of hydraulic oil droplets against a complex background. The detection results are then passed as initial input to the TSDR-Track target tracking algorithm to establish the motion trajectory of the oil droplets in the spatiotemporal sequence.

[0072] The TSDR-Track target tracking algorithm continuously updates the position, velocity, and shape features of oil droplets in time by matching the detection results of consecutive frames. Through a closed-loop optimization mechanism of pre-detector filtering and post-trajectory processing, it eliminates stationary targets, reflective interference, or non-leaking moving targets, thereby preserving the motion trajectory that conforms to the real leakage characteristics and avoiding false detection of leaking targets.

[0073] This detection-tracking joint strategy transforms oil droplet identification from static recognition to dynamic behavior judgment. Compared with traditional methods that rely solely on single-frame detection, this method significantly improves the robustness of leak detection and the accuracy of discrimination, enabling automatic detection and dynamic monitoring of external hydraulic oil leaks in complex industrial environments.

[0074] Experimental comparison: To evaluate the beneficial effects of this invention, a performance comparison experiment was conducted using commonly used detection network models and target tracking network models. The results of the detection network comparison experiment are shown in Table 1, and the results of the target tracking network comparison experiment are shown in Table 2.

[0075] Table 1. Experimental Results of Comparison of Detection Network Models

[0076] Table 1 shows that on the hydraulic oil leak detection image dataset, the GSSR-YOLO model outperforms other networks in three metrics: P, mAP@0.5, and mAP@0.50-0.95. The YOLOv8n model uses a dynamic label allocation strategy to optimize the balance of positive and negative samples, but its default anchor box size has poor adaptability to small targets, resulting in a high false negative rate. Furthermore, its mAP@0.5 score is only 70%, indicating insufficient ability to locate extremely small oil droplets. Compared to the original YOLOv8n network, the improved GSSR-YOLO network achieves a 9.4% higher P, a 6.6% higher R, and 16.5% and 11.2% higher mAP@0.5 and mAP@0.50-0.95, respectively. The YOLOv5s and YOLOv7tiny models exhibit strong localization and recognition capabilities, but their large parameters and slow training speed make them less effective. In contrast, the YOLOv10n and YOLOv11n models employ an anchor-free detection mechanism, which reduces anchor box dependence but performs poorly in detecting small, single targets, resulting in poor adaptability to small-target oil leaks. The GSSR-YOLO model still outperforms YOLOv11n in four metrics by 1.8%, 4.6%, 10.4%, and 3.1%, respectively. The GSSR-YOLO model demonstrates higher accuracy in detecting small-target oil leaks in complex backgrounds, further validating the superiority of our proposed method in hydraulic oil leak detection.

[0077] Table 2 Comparison of Target Tracking Network Models Experimental Results

[0078] Table 2 shows that the TSDR-Track model achieves an IDF1 score of 70.4%, which is 22.2% higher than the higher-performing DeepSORT and 22.8% higher than ByteTrack. This indicates that the TSDR-Track model has stronger ability to maintain trajectory continuity and identity consistency, effectively reducing missed matches and identity switching issues. The IDP and IDR scores reached 71.6% and 69.3% respectively, significantly higher than other algorithms. This demonstrates that the TSDR-Track model can stably identify oil droplet trajectories under complex backgrounds and high dynamic conditions. Although it slightly lacks an advantage over the StrongSORT model in MOTP, the TSDR-Track model achieves 43.5% in the MOTA score, which reflects the overall tracking accuracy and matching performance, far exceeding other models, indicating stronger overall matching accuracy and temporal stability.

[0079] In summary, the visual detection and dynamic recognition method for hydraulic oil external leakage proposed in this invention significantly outperforms existing mainstream algorithms in both detection accuracy and target tracking performance. After the detection and tracking models work together, the method can transform the identification of hydraulic oil leakage phenomena from single-frame static recognition to time-series dynamic judgment, exhibiting higher robustness and stability in complex industrial environments. Experimental results fully verify the high accuracy, high robustness, and practicality advantages of this invention in hydraulic oil external leakage detection tasks under complex backgrounds. It provides reliable technical support for online quantitative assessment and early warning of hydraulic system external leakage, reduces the investment of inspection manpower, and effectively reduces the missed detection rate of minor external leakage in hydraulic systems and the probability of accidents caused by external leakage.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A visual inspection method for dripping in a hydraulic system, characterized in that, Includes the following steps: The collected hydraulic oil leakage video data is fed into the trained GSSR-YOLO target detection model to perform target detection and extract candidate detection results of hydraulic oil droplets in complex backgrounds; The candidate detection results are used as initial inputs and passed to the TSDR-Track target tracking algorithm to obtain the motion trajectory of the oil droplets in the spatiotemporal sequence.

2. The method for visually detecting dripping in a hydraulic system according to claim 1, characterized in that, The GSSR-YOLO target detection model comprises a backbone network, a feature fusion neck network, and a detection head network. The backbone network consists of a series of convolutional layers, a multi-scale feature fusion module, a dual attention mechanism module, a convolutional layer, an enhanced feature extraction module, a convolutional layer, an enhanced feature extraction module, a convolutional layer, a multi-scale feature fusion module, and a fast spatial pyramid pooling layer, connected in sequence. The feature fusion neck network includes a top-down path and a bottom-up path. The top-down path contains three sets of cascaded feature enhancement units, each of which includes an upsampling layer, a feature concatenation layer, and a multi-scale feature fusion module connected in sequence. The bottom-up path contains three sets of cascaded feature downsampling units, each of which includes a convolutional layer, a spatial depth transformation convolutional layer, a feature concatenation layer, and a multi-scale feature fusion module connected in sequence. The detection head network consists of three detection layers and a small target detection layer. The three detection layers are respectively connected to the multi-scale feature fusion module in the bottom-up path, and the small target detection layer is connected to the last multi-scale feature fusion module in the top-down path.

3. The method for visually detecting dripping in a hydraulic system according to claim 2, characterized in that, The enhanced feature extraction module includes an input module, a local feature extraction module, a global feature extraction module, a feature fusion module, and an output module connected in sequence; the local feature extraction module and the feature fusion module are connected; the local feature extraction module includes a depthwise separable convolutional layer and a pointwise convolutional layer connected in sequence; the global feature extraction module includes a lightweight encoder module and a pointwise convolutional layer connected in sequence; the feature fusion module includes a concatenation layer, a convolutional layer, and a residual connection layer connected in sequence; the input module and the residual connection layer are connected.

4. The method for visually detecting dripping in a hydraulic system according to claim 3, characterized in that, The feature extraction process of the enhanced feature extraction module is as follows: The input characteristics of the input module are X , ;in C , H and W These represent the number of channels, height, and width of the input feature, respectively. The input features are transmitted to the depthwise separable convolutional layer after passing through the input module, capturing the global information of the input image; Subsequently, through pointwise convolutional layers, the output is projected back to the original channel dimension, outputting local features. Y ; Next, dimensionality reduction is performed in the global feature extraction module, and the output is recorded as the global features. Z ; In the feature fusion module, the stitching layer combines local features Y With global features Z The data is concatenated along the channel dimension and then passed through a convolutional layer to output features. F The residual connection layer will feature F Input features X Perform residual connection output features O .

5. The method for visually detecting dripping in a hydraulic system according to claim 4, characterized in that, The calculation formula for the local feature extraction module is as follows: ； The calculation formula for the global feature extraction module is as follows: ； The calculation formula for the splicing layer is as follows: ； The formula for calculating convolutional layers is as follows: ； The formula for calculating the residual connection layer is as follows: ； Here, DWConv-5×5 represents a depthwise separable convolution with a kernel size of 5×5, and Transformer represents a lightweight encoder module.

6. The method for visually detecting dripping in a hydraulic system according to claim 2, characterized in that, The dual attention mechanism module includes a channel attention mechanism module and a spatial attention mechanism module.

7. The method for visually detecting dripping in a hydraulic system according to claim 2, characterized in that, The multi-scale feature fusion module includes convolutional layers, feature splitting layers, Bottleneck_MCA layers, splicing layers, and convolutional layers. After the input features are processed by the convolutional layer, they are divided into two branches in the feature splitting layer. One branch directly enters n Bottleneck_MCA layers, while the other branch skips the concatenation process. Finally, the outputs of all branches are concatenated and restored to the number of channels through the convolutional layer to output refined features.

8. The method for visually detecting dripping in a hydraulic system according to claim 7, characterized in that, The Bottleneck_MCA layer contains an MCA module, which includes a global pooling module, a local pooling module, and a scale fusion module, a feature concatenation module, a dilated convolution module, a normalization and non-linear activation module, a feature splitting module, a depthwise separable convolution module, a normalization module, and a reweighting module connected in sequence. The global pooling module and the local pooling module are respectively connected to the scale fusion module. The input features decompose the target's position information into two one-dimensional feature codes in the global pooling module and the local pooling module, respectively; the two one-dimensional feature codes include horizontal dimension feature codes and vertical dimension feature codes; The two one-dimensional feature codes output by the global pooling module and the local pooling module are fused in the scale fusion module to output a fused feature map; Perform feature concatenation on the fused feature map to output an aggregated feature map; The aggregated feature map is effectively compensated by dilated convolutional layers, and the compensated feature map is output. After being processed by the standardization and nonlinear activation modules, the compensated feature map is mapped to the compressed latent feature space to obtain the activated feature vector. Perform feature splitting to restore the activated feature vectors into two independent branch vectors, one horizontal and one vertical; Attention weights are calculated using depth-separable convolutional layers along the horizontal and vertical directions, respectively, and then processed by normalization layers to generate horizontal and vertical weight maps. The horizontal and vertical weight maps are reweighted with the residuals of the original input feature map through element-wise multiplication to obtain the output feature map.

9. A visual inspection method for dripping in a hydraulic system according to claim 1, characterized in that, The TSDR-Track target tracking algorithm includes a detection branch, a trajectory prediction branch, and a matching update module; The detection branch is used to perform target detection and feature extraction on the input video frame to obtain a candidate detection set of oil droplets; the trajectory prediction branch is used to perform state prediction based on the trajectory result of the previous frame using a Kalman filter based on the motion parameters of the leaking oil droplets, and obtain the prediction result. The matching update module is used to fuse the detection set with the prediction results and associate the target through the multi-dimensional matching cost calculation module. The multi-dimensional matching cost calculation module integrates two cost indicators: IoU cost and anchor proximity cost. The anchor proximity cost is constructed by the anchor matching remediation mechanism. After the target association is completed, the Hungarian matching algorithm is used for global optimal allocation to ensure a one-to-one correspondence between the predicted trajectory and the detected target; Subsequently, a Kalman state update is performed on the successfully matched targets, while a certain number of frames of prediction are retained for the unmatched trajectories.

10. A visual inspection system for dripping in a hydraulic system, characterized in that, Includes the following steps: The target detection module is used to input the collected hydraulic oil leakage video data into the trained GSSR-YOLO target detection model to perform target detection and extract candidate detection results of hydraulic oil droplets in complex backgrounds. The target tracking module is used to take the candidate detection results as initial input and pass them to the TSDR-Track target tracking algorithm to obtain the motion trajectory of the oil droplet in the spatiotemporal sequence.

Images