Rail fastener intelligent oiling detection method and device based on lightweight visual recognition

Abstract

The application provides a kind of track fastener intelligent oiling detection method and device based on lightweight visual recognition, it is related to rail transit operation and maintenance technical field, wherein, the method comprises: collecting the original image of track fastener, and the original image is preprocessed, to obtain enhanced image;The foreground fastener identification and region of interest, fast screening are carried out to the enhanced image, to determine the suspected fastener position;The corrosion state grading classification identification is carried out to the track fastener in the region of interest using image recognition model;According to the corrosion state grade, determine the target oiling amount of the track fastener, and the track fastener is sprayed to apply the target oiling amount;The oiling quality of the track fastener is evaluated, and when the evaluation result is unqualified, the track fastener is re-sprayed.By the application, a closed-loop control system of "recognition-execution-evaluation-correction" is constructed, efficient, safe and accurate maintenance of track fastener is realized.

Description

Technical Field

[0001] This invention relates to the field of rail transit operation and maintenance technology, and in particular to a method and device for intelligent oiling detection of rail fasteners based on lightweight visual recognition. Background Technology

[0002] Rail fasteners are key components connecting rails and sleepers. Long-term exposure to the natural environment makes them prone to corrosion and loosening, affecting train safety. Currently, the oiling and maintenance of rail fasteners mainly relies on manual labor with tools or railcar-assisted operations.

[0003] However, existing technologies have the following drawbacks: First, railway maintenance "windows" are limited, resulting in low efficiency for manual on-site operations and posing significant safety risks to workers. Second, the complex track environment, small fastener targets, and similar color and texture to the background ballast and sleepers lead to high false positive and false negative rates and poor robustness for general visual recognition algorithms. Third, existing spraying technologies often employ a single spraying strategy, failing to adjust the amount of oil applied based on the actual degree of corrosion of the fasteners, easily resulting in oil waste or insufficient penetration. Finally, the lack of an effective closed-loop quality control and re-inspection mechanism after operation makes it impossible to ensure that every fastener meets maintenance standards. Therefore, existing technologies face pressing issues in terms of operational efficiency, identification accuracy, spraying precision, and quality control. Summary of the Invention

[0004] This invention provides a method and device for intelligent oiling detection of track fasteners based on lightweight visual recognition, which solves the defects of existing technologies such as low efficiency of track fastener oiling and maintenance operations, high safety risks, inaccurate fastener identification and condition assessment, rough control of oiling amount, and lack of effective quality evaluation closed loop.

[0005] In a first aspect, the present invention provides an intelligent oiling detection method for track fasteners based on lightweight visual recognition, the method being executed by an unmanned aerial vehicle (UAV), the method comprising: Acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image; The enhanced image is subjected to foreground fastener identification and region of interest, and rapid filtering to determine the location of suspected fasteners; The corrosion status of the track fasteners within the region of interest is classified and identified using an image recognition model. Based on the corrosion status level, determine the target amount of oil to be applied to the rail fastener, and spray the rail fastener to apply the target amount of oil. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-sprayed.

[0006] According to the present invention, a method for intelligent oiling detection of rail fasteners based on lightweight visual recognition is provided, which involves acquiring original images of rail fasteners and preprocessing the original images to obtain enhanced images, including: The original image is subjected to bad pixel detection and correction, and bad pixels with brightness differences exceeding a preset standard are replaced by the average value of neighboring pixels. An adaptive nonlocal means algorithm is used to filter out Gaussian noise in the original image; A contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast in dark areas, and neighborhood mean smoothing is applied to overexposed pixels.

[0007] According to the present invention, a method for intelligent oiling detection of track fasteners based on lightweight visual recognition is provided, which performs foreground fastener identification and region of interest identification, rapid filtering, and determination of suspected fastener locations on the enhanced image, including: Background subtraction is performed on the enhanced image using an adaptive Gaussian mixture model to extract the moving foreground mask; Based on the prior physical dimensions and geometric shape of the track fastener, the motion foreground mask is pre-screened to remove interference areas that do not conform to the scale and shape characteristics. A lightweight object detection network is used to quickly scan the pre-screened area to generate candidate regions of interest for fasteners and determine the location of suspected fasteners.

[0008] According to the present invention, a smart oiling detection method for track fasteners based on lightweight visual recognition, after determining the location of a suspected fastener, further includes: Calculate the actual physical distance between two consecutive track fasteners based on the suspected fastener positions; The actual physical distance is compared with the preset standard fastener distance to obtain the distance deviation; The flight speed or position of the UAV is adjusted based on the spacing deviation.

[0009] According to the present invention, a smart oiling detection method for rail fasteners based on lightweight visual recognition is provided, which uses an image recognition model to classify and identify the corrosion status of the rail fasteners within the region of interest, including: Extract the image color features of the region of interest and calculate the percentage of pixels with a hue in the orange-brown range; Extract the geometric features of the region of interest from the image, and analyze the edge continuity and thread visibility; By integrating the image color features and the image geometric features, the corrosion status level of the track fastener is classified.

[0010] According to the present invention, a smart oiling detection method for rail fasteners based on lightweight visual recognition is provided, which determines the target oiling amount for the rail fasteners according to the corrosion state level, and applies the target oiling amount by spraying the rail fasteners. The method includes: The model of the track fastener is identified, and the corresponding geometric parameters are retrieved to calculate the effective coated surface area; The porosity correction factor is determined based on the corrosion status level of the track fastener; Based on the effective coating surface area, the preset oil film thickness, and the porosity correction coefficient, the standard oil volume of a single fastener is calculated. The track fasteners are sprayed with paint, and the actual oil output is monitored in real time using a flow sensor. The spraying time or nozzle valve opening is adjusted based on the monitoring results.

[0011] According to the present invention, a smart oiling detection method for rail fasteners based on lightweight visual recognition is provided. When spraying the rail fasteners, a dual-pressure fan-shaped nozzle is used, and the spraying process selects the corresponding spraying mode according to the corrosion status level of the rail fasteners. The spraying modes include penetration mode and coating mode.

[0012] According to the present invention, a smart oiling detection method for track fasteners based on lightweight visual recognition is provided, which evaluates the oiling quality of the track fasteners and performs re-spraying on the track fasteners when the evaluation result is unqualified, including: A secondary image of the track fastener after oiling is performed to extract the image features of the oil film area; Calculate the proportion of the oil film coverage area to the total area of ​​the track fasteners, and generate a coverage score; The equivalent capacitance value of the track fastener surface is measured using an electrostatic induction ring to evaluate the adhesion thickness of the oil film. Infrared thermal imagers were used to monitor temperature changes before and after spraying to evaluate the penetration performance of the oil. If the evaluation result is unqualified, the track fasteners shall be re-sprayed.

[0013] According to the present invention, a smart oiling detection method for track fasteners based on lightweight visual recognition is provided, the method further comprising: The available track maintenance window time is automatically analyzed based on the railway timetable. Based on the drone's endurance and operating speed, the optimal operating route is dynamically planned to ensure that the drone leaves the track area before the train passes.

[0014] Secondly, the present invention also provides an intelligent oiling detection device for track fasteners based on lightweight visual recognition, comprising: The acquisition module is used to acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image; The positioning module is used to identify foreground fasteners and regions of interest in the enhanced image, quickly filter them, and determine the location of suspected fasteners. The identification module is used to classify and identify the corrosion status of the track fasteners within the region of interest using an image recognition model; The spraying module is used to determine the target amount of oil to be applied to the rail fastener based on the corrosion status level, and to spray the rail fastener to apply the target amount of oil. The correction module is used to evaluate the oiling quality of the track fasteners and to re-spray the track fasteners if the evaluation result is unqualified.

[0015] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the intelligent oiling detection method for track fasteners based on lightweight visual recognition as described in the first aspect above.

[0016] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent oiling detection method for track fasteners based on lightweight visual recognition as described in the first aspect above.

[0017] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent oiling detection method for track fasteners based on lightweight visual recognition as described in the first aspect above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. By using drones to perform automated operations, the traditional manual on-line operations are replaced, which significantly improves the efficiency of operations and avoids the high safety risks brought about by manual on-line operations. This makes it possible to carry out efficient maintenance during the limited railway "maintenance window".

[0019] 2. By performing multi-stage preprocessing on the original image and combining background subtraction and lightweight network for region of interest filtering, image noise and background interference in complex track environments are effectively suppressed, laying the foundation for subsequent accurate recognition and improving the accuracy and robustness of fastener recognition.

[0020] 3. By dynamically calculating the target amount of oil based on parameters such as the fastener's model, corrosion level, and porosity, and using a flow sensor for closed-loop feedback control, "on-demand oil supply" is achieved for each fastener, avoiding oil waste or insufficient coating and improving the accuracy of spraying.

[0021] 4. By conducting multi-dimensional evaluation of the coating quality after spraying and automatically re-spraying unqualified areas, a complete closed-loop control system of "identification-execution-evaluation-correction" was constructed to ensure the final maintenance quality and realize quantitative management and guarantee of the operation effect. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the intelligent oiling detection method for track fasteners based on lightweight visual recognition provided by the present invention. Figure 2 This is a detailed schematic diagram illustrating the process of applying oil to the track fasteners in an embodiment of the present invention; Figure 3 This is a flowchart of the closed-loop control of oil application amount based on surface area and permeability in an embodiment of the present invention; Figure 4 This is a flowchart of the oiling quality evaluation and touch-up triggering logic in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] This invention provides an intelligent oiling inspection method for track fasteners based on lightweight visual recognition. The method is executed by a drone. The drone flight platform adopts a multi-rotor configuration and is equipped with a high-precision real-time dynamic differential (RTK) positioning module. The airborne visual sensing unit includes a high-definition visible light camera, an infrared thermal imager, and a millimeter-wave radar. The airborne computing unit includes a built-in edge computing AI chip and deploys an image recognition model. The adaptive spraying module includes a dual-pressure fan-shaped nozzle and an oil supply system. The background control system deploys an intelligent coating inspection human-machine interface for data archiving and risk warning.

[0026] Example 1 Figure 1 This is a flowchart of the intelligent oiling detection method for track fasteners based on lightweight visual recognition provided by the present invention, as follows: Figure 1 As shown, the method includes the following steps: Step S101: Acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image; Step S102: Foreground fastener identification and region of interest identification are performed on the enhanced image, and the location of suspected fasteners is determined quickly. Step S103: Use an image recognition model to classify and identify the corrosion status of the track fasteners within the region of interest. Step S104: Determine the target amount of oil to be applied to the rail fasteners based on the corrosion status level, and spray the rail fasteners to apply the target amount of oil. Step S105: Evaluate the oiling quality of the track fasteners, and if the evaluation result is unqualified, re-spray the track fasteners.

[0027] In this method, the original images of the track fasteners are first acquired and preprocessed to obtain enhanced images, improving image quality and effectively ensuring the subsequent recognition and localization results. Then, the enhanced images are used for foreground fastener identification and region of interest (ROI) screening to determine the locations of suspected fasteners. Next, an image recognition model (such as MobileNetV4) is used to classify and identify the corrosion status of the track fasteners within the ROI, helping to execute corresponding oiling operations based on the degree of corrosion, improving oiling efficiency and accuracy. Then, based on the corrosion status level, the target oiling amount for the track fasteners is determined, and the fasteners are sprayed with the target oiling amount. Finally, the oiling quality of the track fasteners is evaluated, and if the evaluation result is unsatisfactory, the fasteners are re-sprayed. Through the above operations, a complete closed-loop control system of "identification-execution-evaluation-correction" is constructed, ensuring the final maintenance quality and achieving quantitative management and guarantee of the operation effect.

[0028] Example 2 Figure 2 This is a detailed schematic diagram illustrating the process of applying oil to the track fasteners in an embodiment of the present invention. Next, as follows... Figure 2 The above method is described in detail as shown below: In some embodiments, step S101 involves acquiring the original image of the track fastener and preprocessing the original image to obtain an enhanced image, including: performing bad pixel detection and correction on the original image, replacing bad pixels with brightness differences exceeding a preset standard using the average value of neighboring pixels; using an adaptive nonlocal mean algorithm to filter out Gaussian noise in the original image; using a contrast-limited adaptive histogram equalization algorithm to enhance the contrast of dark areas, and performing neighborhood mean smoothing on overexposed pixels.

[0029] In this embodiment, the original image is subjected to bad pixel detection and correction. Bad pixels with a brightness difference exceeding 8% of the full scale are replaced by the average value of the surrounding 8 neighboring pixels. An adaptive nonlocal mean algorithm is used to filter out Gaussian noise in the image, and the filtering intensity parameter is adaptively adjusted according to the flight speed of the UAV. A contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast of dark areas, and neighborhood mean smoothing is performed on overexposed pixels with a brightness exceeding 240 and a saturation below 30.

[0030] For example, due to the influence of propeller vibration, high-speed airflow, and changes in ambient light when the UAV cruises above the orbit, the original images acquired by the airborne visual sensing unit often contain motion blur, Gaussian noise, salt-and-pepper noise, and strong reflective spots, which can easily mislead the subsequent fastener detection model. Therefore, this embodiment adds a multi-stage image denoising and background interference suppression module to the front end of the MobileNetV4 model. Specifically, after the airborne visible light camera on the UAV outputs the original Bayer format image, it first performs bad pixel detection and interpolation replacement, and sets a brightness threshold. Its value is taken as 8% of the pixel's full scale (for an 8-bit image, an empirical value is 20). Iterate through each pixel in the image and calculate the average brightness value of its surrounding 8 neighboring pixels. If the current pixel's brightness value P satisfy If the pixel is not found to be defective, bilinear interpolation is used to replace it with the pixel values ​​from its eight neighboring pixels. Black level correction is employed to deduct the sensor's dark current offset, preventing low-frequency noise from reaching the backend.

[0031] Spatial domain denoising is performed on the RGB-converted image using adaptive nonlocal means denoising. Let the current pixel be... p Take a similar window with size (2N+1)×(2N+1) centered on it. In a search window with a size of (2M+1)×(2M+1) Searching inside and Similar pixel blocks Weights are calculated based on the Euclidean distance between pixel blocks. Pixel values ​​after noise reduction This is a weighted average of all pixels within the search window, with the weights determined by a similarity exponential decay function, where the decay parameter... h The drone's ground-to-ground flight speed is adaptively increased to suppress fuzzy residuals caused by high-speed motion. The Adaptive Non-Local Means (ANLM) algorithm effectively filters out ballast pseudo-textures and Gaussian noise while preserving fastener edge information. The specific formula is as follows:

[0032]

[0033] in, Represents pixels p grayscale value; , respectively with p and q Similar search blocks centered on the center; h This represents the filter strength parameter, which is adaptively adjusted based on the UAV's current flight speed (the higher the speed, the stronger the filter strength). h (Appropriately increase to suppress motion blur residuals) The normalization factor ensures that the weight sum is 1, avoiding brightness drift. Adaptive nonlocal mean denoising effectively filters out high-frequency random noise and pseudo-textures caused by ballast particles while preserving the edges of the fasteners.

[0034] When millimeter-wave radar provides feedback on the instantaneous speed of the drone Or, if the inertial measurement unit (IMU) detects angular vibration exceeding a threshold, based on the exposure time... With flight speed v This yields the relative displacement on the sensor during shutter opening; then, the actual physical size of a single pixel is used to obtain this displacement. Normalization is performed to convert the displacement distance into an equivalent pixel span. L Estimating the length of the fuzzy kernel The direction is along the flight path. If potential motion blur is detected, and the blur level is light (kernel length less than 3 pixels), the restoration is skipped to reduce the ringing effect.

[0035] In track scenes, backlighting at tunnel entrances or direct sunlight often causes high light saturation on fasteners, resulting in reduced contrast between the fasteners and the background. To address this, a Contrast-Limited Adaptive Histogram Equalization (CLAHE) algorithm is used to locally enhance image contrast, adaptively stretching the grayscale dynamic range of dark areas to highlight the outline of the fasteners obscured by shadows. A bilateral filter is introduced to separate the low-frequency illumination component and the high-frequency reflection component of the image, enhancing contrast while preserving the sharpness of the fastener edges. Saturation is also addressed... With brightness Overexposed pixels exceeding a set threshold undergo highlight cropping and neighborhood mean smoothing to prevent areas of strong light reflection from being misidentified as "metallic reflective fasteners".

[0036] In some embodiments, step S102, which involves identifying foreground fasteners and regions of interest (ROIs) in the enhanced image and quickly filtering them to determine the locations of suspected fasteners, includes: performing background subtraction on the enhanced image using an adaptive Gaussian mixture model to extract a moving foreground mask; pre-filtering the moving foreground mask based on the prior physical size and geometric shape of the track fasteners to remove interference areas that do not conform to scale and shape characteristics; and quickly scanning the pre-filtered areas using a lightweight object detection network to generate candidate regions of interest (ROIs) for fasteners to determine the locations of suspected fasteners.

[0037] For example, the enhanced image undergoes foreground segmentation and coarse extraction of the region of interest. Background interference is removed through geometric priors, and potential candidate bounding boxes for fasteners are initially delineated. Specifically, this includes: firstly, quickly locking the candidate region for fasteners using coarse-grained detection. Considering the characteristic of a stationary background (rails, sleepers, ballast) while the track fasteners are in relative motion, an adaptive Gaussian mixture model is used for background subtraction. During initialization, N frames (N=50) of stable images are continuously acquired, and a Gaussian distribution model is established for each pixel. The Mahalanobis distance between the current frame pixel and the background model is calculated in real time. If the distance is greater than a threshold... (If the empirical value is 20), then it is determined to be a foreground pixel. Perform opening (to remove noise) and closing (to fill holes) operations on the foreground mask to obtain the complete moving foreground connected component.

[0038] Since the track fasteners have fixed physical dimensions and specific geometric shapes, this embodiment utilizes perspective projection relationships for pre-screening. Based on the altitude H fed back by the UAV's altitude-holding radar, the theoretical pixel size range of the fastener in the image is calculated. The theoretical pixel dimensions of the track fastener are as follows:

[0039] in, Indicates the theoretical pixel size. F For camera focal length, This refers to the actual dimensions of the fastener (e.g., a spring-loaded type I fastener is approximately 129mm long and 74mm wide). If the side length of the bounding rectangle of the connected domain is not within this range, it is discarded directly.

[0040] To further reduce computational load, a lightweight YOLO-Tiny network is run for rapid scanning before the fine-grained MobileNetV4 inspection. This network contains only three inspection heads, has fewer than 1MB of parameters, and achieves an inference speed of up to 100 FPS. It does not classify corrosion levels; it only outputs bounding boxes for "suspected fasteners." Based on the bounding boxes output by YOLO-Tiny, corresponding sub-images (patches) are cropped from the original image.

[0041] MobileNetV4 only processes the cropped smaller image, instead of the entire image, reducing computation by approximately 90% and significantly lowering the power consumption and heat generation of the onboard computing unit. This method eliminates interference from large areas of ballast background, allowing the MobileNetV4 model to focus more on the local rust texture features of the fasteners, further improving recognition accuracy.

[0042] Furthermore, after determining the suspected fastener location, the process also includes: calculating the actual physical distance between two consecutive track fasteners based on the suspected fastener location. Compare actual physical distances Spacing with the preset standard fastener To obtain the spacing deviation According to the spacing deviation Adjust the drone's flight speed or position.

[0043] For example, first establish a fastener spatial coordinate system, with the X-axis along the direction of the rail extension (the direction of the drone's flight), the Y-axis perpendicular to the rail (the direction of left and right adjustment), and the origin O as the center of the fastener currently in operation.

[0044] In standard railway lines, the spacing of fasteners The value is typically a fixed 600mm. The MobileNetV4 model identifies all visible fasteners in the current frame and calculates the Euclidean distance between two adjacent fasteners in the image pixel coordinate system. Based on the altitude feedback from the drone's altitude-keeping radar. H and camera intrinsic parameter matrix K , pixel distance Converted to actual physical distance The specific formula is as follows:

[0045] in, Let be the camera focal length. Calculate the deviation between the measured distance and the standard distance:

[0046] like If the threshold is 20mm, it is determined that there is a positioning deviation, and the correction process is triggered.

[0047] when At this time, it means the actual distance is too large and the drone is flying too fast, which may lead to missed spraying; when This means the distance between the drone and the aircraft is too small, and the drone is flying too slowly, which could lead to a double-jet. The onboard computing unit sends a speed compensation command to the flight control system:

[0048] By adjusting the flight speed, ensure that the next spray point lands in the expected physical location.

[0049] The curvature of the rail at bends or the yaw angle of the drone can cause lateral alignment errors between the nozzle and the fastener. This is addressed by using the fastener pixel center output from MobileNetV4. Calculate its deviation from the image center (the projection point directly below the nozzle). Just before painting, compensation is achieved by controlling the drone's gimbal or fine-tuning its body attitude. Directional deviations should be checked to ensure the nozzle is aligned with the center of the fastener.

[0050] When detected When the distance is much greater than the standard spacing (such as when there is a fork in the track, a broken rail, or a large gap), the system automatically pauses the spraying operation, switches to panoramic scanning mode, and re-finds a continuous sequence of fasteners to prevent wasting oil in invalid areas.

[0051] In some embodiments, step S103 involves using an image recognition model to classify and identify the corrosion status of rail fasteners within the region of interest. This includes: extracting image color features from the region of interest and calculating the percentage of pixels with hues in the orange-brown range; extracting image geometric features from the region of interest and analyzing edge continuity and thread visibility; and fusing image color features and image geometric features to classify the corrosion status of the rail fasteners. In this embodiment, the corrosion status of the rail fasteners is divided into four levels.

[0052] For example, to establish a standard dataset for model training, the corrosion status of fasteners is defined into four levels. Type 0 (normal) is an intact surface coating with no obvious corrosion and visible metallic luster; Type 1 (slight corrosion) is sporadic surface rust or spot-like corrosion, covering an area of ​​less than 30%; Type 2 (moderate corrosion) is large-area corrosion, covering an area of ​​30%-70%, accompanied by slight peeling; Type 3 (severe corrosion) is severe corrosion, covering an area of ​​more than 70%, accompanied by obvious rust accumulation, thread blockage, or metal deformation.

[0053] The MobileNetV4 model receives a pre-processed image of the region of interest. The first few convolutional layers focus on extracting the physical contours of the fastener. Since some channels directly retain the original edge information while others undergo convolutional transformations, the sharpness of the fastener's edges is ensured not to be lost as the network deepens. In the intermediate layers, the model blends color and texture channel information through channel rearrangement. At the end of the network, a global average pooling layer compresses the feature map into a vector, and a fully connected layer maps this vector to the four rust levels mentioned above according to the weight matrix.

[0054] The model not only relies on deep learning features but also incorporates features from traditional image processing as auxiliary decision-making. The region of interest is transformed from the RGB space to the HSV (hue, saturation, brightness) space. Statistical hue... H In the orange-brown range And saturation S Pixel percentage above the threshold .like If it is determined to be level 0-1; It is classified as Level 3.

[0055] Use the Canny operator to extract the fastener edges and check if the bolt head edges are continuous. If the edges are broken or disappear (covered by thick rust), increase the rust level. Check the clarity of the thread texture around the nut; the more blurred the texture, the more severe the rust.

[0056] The airborne computing unit integrates the above characteristics and outputs the final corrosion level through a Softmax classifier, using the following formula:

[0057] in, Deep features extracted for the MobileNetV4 model. For color characteristics, Geometric features These represent four levels: no rust, light rust, moderate rust, and heavy rust.

[0058] In some embodiments, step S104, determining the target oil application amount for the track fastener based on the rust state level and spraying the track fastener to apply the target oil application amount, includes: identifying the model of the track fastener and retrieving the corresponding geometric parameters to calculate the effective coating surface area; determining the porosity correction coefficient based on the rust state level of the track fastener; calculating the standard oil amount for a single fastener based on the effective coating surface area, the preset oil film thickness, and the porosity correction coefficient; spraying the track fastener and using a flow sensor to monitor the actual oil output in real time, and adjusting the spraying time or nozzle valve opening based on the monitoring results.

[0059] For example, Figure 3 This is a flowchart illustrating the closed-loop control of oil application amount based on surface area and permeability in an embodiment of the present invention, as shown below. Figure 3 As shown, firstly, the MobileNetV4 model dynamically calculates the standard amount of oil required for a single fastener based on the identified fastener type and corrosion level. The system pre-stores the geometric parameters (length) of different fastener models. L ,Width W ,high H The effective surface area to be coated is calculated using the following formula:

[0060] in, This is the drone pitch angle compensation factor (when the drone tilts, the actual spraying area will be smaller, so compensation is needed).

[0061] The degree of corrosion on metal surfaces directly affects the amount of oil adsorbed. A porosity correction factor is introduced. When corrosion type 0 (normal), (Dense surface, low adsorption); When corrosion type 1 (slight), When corrosion is type 2 (moderate), =1.5; When corrosion type 3 (severe), (The surface is loose and porous, resulting in a large adsorption capacity).

[0062] The standard oil quantity calculation formula for a single track fastener is:

[0063] in, Indicates standard oil volume. This indicates the preset thickness of the oil film applied in a single coat. This indicates the specific gravity of the rust-preventive oil.

[0064] Because the drone experiences slight forward and backward swaying due to wind during operation, resulting in changes in relative speed, the system employs speed-flow coupled control to ensure a constant oil application rate per unit length. The rated flow rate of the nozzle is assumed to be... The onboard computing unit reads the drone's ground speed in real time. .when When the wind suddenly increases (headwind), the system controls the flow valve to open wider to ensure... .when Reducing the (tailwind) will decrease the flow rate and prevent fuel buildup.

[0065] Furthermore, a dual-pressure fan-shaped nozzle is used when spraying the track fasteners, and the corresponding spraying mode is selected according to the corrosion status level of the track fasteners during the spraying process; the spraying modes include penetration mode (mode A) and coating mode (mode B).

[0066] In this embodiment, when the rust level is high, mode A is activated, the working pressure is set to 0.8MPa-1.2MPa, and large-diameter atomized particles (80-120μm) are output; when the rust level is low, mode B is activated, the peak working pressure reaches 2.0MPa, and small-diameter atomized particles (20-50μm) are output; a -15kV high-voltage electrostatic field is applied during the spraying process to make the oil mist particles negatively charged, thereby improving their adhesion rate on the surface of the metal fastener.

[0067] For example, when type 0-1 (coating), oil volume Allocate all fuel to Mode B (high pressure, low flow) for more uniform atomization. When using Type 2-3 (penetration), the fuel volume... The injection is divided into two stages. The first stage injects 70% of the total oil volume as base oil (mode A, low pressure and high flow rate). After a 0.5-second interval, the second stage injects the remaining 30% (mode B, high pressure and compaction) to ensure that the oil penetrates deep into the threads without overflowing.

[0068] Furthermore, a flow sensor is installed at the nozzle tip to monitor whether the actual oil output matches the command due to nozzle blockage or pipeline leakage. When the system issues a "spraying" command... The flow sensor monitors the actual oil flow in real time using the "ml" command. ,like ,in For instruction traffic, The system immediately marks the fastener as "painting failed" and prioritizes repainting during the re-inspection phase. Simultaneously, the system dynamically adjusts the instruction paint volume for the next fastener based on the deviation, achieving adaptive calibration.

[0069] Based on the above embodiments, step S105 evaluates the oiling quality of the track fasteners, and if the evaluation result is unqualified, the track fasteners are re-sprayed, including: performing secondary imaging on the track fasteners after oiling to extract image features of the oil film area; calculating the proportion of the oil film coverage area to the total area of ​​the track fasteners to generate a coverage score; measuring the equivalent capacitance value of the track fastener surface using an electrostatic induction ring to evaluate the adhesion thickness of the oil film; monitoring the temperature change before and after spraying using an infrared thermal imager to evaluate the penetration performance of the oil; and re-spraying the track fasteners if the evaluation result is unqualified.

[0070] For example, Figure 4 This is a flowchart illustrating the oil coating quality evaluation and touch-up triggering logic in an embodiment of the present invention, as follows: Figure 4 As shown, after the painting is completed, the drone triggers a high-definition camera at the rear to take a secondary image of the completed fasteners. The image after painting is then displayed. Image before spraying Perform differential operations to extract the oil film region. Calculate the percentage of the oil film area relative to the total fastener area. The specific formula is as follows:

[0071] when It was determined to be "unqualified".

[0072] Analyze the variance of gray values ​​in the oil film region Let the oil film area be defined. The total number of pixels contained therein is N , No. i The grayscale value of each pixel is (The range is typically 0-255), the average gray value of this area is The specific calculation formula is as follows:

[0073] If variance A value >300 indicates the presence of "sagging" (oil buildup) or "dry spots" (not sprayed). Check the edges of the oil film for jagged or feather-like spread to determine if the oil is evaporating too quickly or if the spraying pressure is appropriate.

[0074] The equivalent capacitance value of the fastener surface is measured after spraying using an electrostatic induction ring at the nozzle tip. .when If the oil film is close to the baseline value for dry metal (5pF~10pF), it indicates that the oil film is too thin or not adhered at all; if If the value is within the preset range (11pF to 16pF), it indicates that the electrostatic adsorption effect is good.

[0075] An airborne infrared thermal imager was used to monitor temperature changes before and after coating. Rust-preventive oil seeps into threaded gaps, accompanied by minute heat exchange. As the rate of temperature decrease on the fastener surface slows after coating, a monitoring window was set. Calculate the difference between the rate of temperature drop after spraying and the reference natural cooling rate. .when At that time, among them The oil film indicates that it has achieved a good heat insulation / sealing effect, and is rated as excellent. A rapid temperature increase within a short period (due to solvent evaporation absorbing heat) indicates that the oil is evaporating too quickly, and adjustments to the formulation or spray volume may be necessary.

[0076] This embodiment also constructs a quality scoring model (QSM) for coating coverage. Uniformity Adsorption strength The score is fused with the penetration depth (infrared) index, and the specific formula is as follows:

[0077] in, Indicates the rating value. Indicates the weight of a single indicator, coverage rate Weight =0.4, uniformity Weight =0.3, Adsorption strength Weight =0.2, Penetration Depth (Infrared) Weight =0.1, This represents the score for a single indicator. When... A score of ≥90 is considered excellent; no further processing is needed, just record and archive. When 80 ≤ A score less than 90 is considered acceptable; the record is archived and marked as an observation object. If the value is less than 80, it is considered unqualified. The re-spraying process should be triggered immediately, and the cause of the failure should be recorded (such as nozzle blockage or insufficient oil).

[0078] When the system determines the fastener to be "unqualified," the drone hovers directly above it and, based on the location of the uncovered area, fine-tunes the nozzle angle to perform localized respraying. After respraying is completed, the first three evaluation steps are repeated. If two consecutive resprays still fail to meet the standard, the system marks the fastener as a "difficult-to-diagnose fault," uploads its coordinates to the backend system, and notifies a human to handle it online.

[0079] Furthermore, the method also includes: automatically analyzing the available track maintenance window based on the railway timetable; and dynamically planning the optimal operation route based on the drone's endurance and operating speed to ensure that the drone leaves the track section before the train passes.

[0080] In summary, this method first utilizes a multi-rotor UAV to cruise along a track, acquiring real-time track environment data using millimeter-wave radar and visual sensors. Second, it performs adaptive non-local mean denoising and illumination equalization on the original images, and employs a lightweight network for foreground fastener identification and ROI selection. Third, it constructs a MobileNetV4 deep learning model, introducing a channel rearrangement mechanism and hardware perception architecture to perform rust classification identification of fasteners within the ROI area, achieving an inference speed of 30 FPS. Simultaneously, it performs trajectory correction based on prior knowledge of equal fastener spacing and calculates the standard oil volume based on surface area and permeability, driving a dual-pressure fan-shaped nozzle to perform closed-loop spraying. Finally, it evaluates the oiling quality through multimodal perception (vision + capacitance + infrared), triggering respraying if the quality is unsatisfactory.

[0081] This method deeply integrates lightweight AI, fluid control, and unmanned aerial vehicle platforms, ensuring a 15% improvement in train throughput efficiency. While achieving a 35% improvement, it also realizes low-power, high-real-time automated track maintenance, which has high practical application value.

[0082] Example 3 This invention also provides a lightweight visual recognition-based intelligent oiling detection device for rail fasteners. The following describes the lightweight visual recognition-based intelligent oiling detection device for rail fasteners provided by this invention. The lightweight visual recognition-based intelligent oiling detection device described below can be referred to in correspondence with the lightweight visual recognition-based intelligent oiling detection method for rail fasteners described above. The device includes: The acquisition module is used to acquire the original image of the track fastener and preprocess the original image to obtain the enhanced image; The positioning module is used to identify foreground fasteners and regions of interest in the enhanced image, quickly filter them, and determine the location of suspected fasteners. The recognition module is used to classify and identify the corrosion status of track fasteners within the region of interest using an image recognition model. The spraying module is used to determine the target amount of oil to be applied to the rail fasteners based on the level of rust, and to spray the rail fasteners to apply the target amount of oil. The correction module is used to evaluate the oiling quality of the track fasteners and to re-spray the track fasteners when the evaluation result is unqualified.

[0083] In operation, this device first acquires original images of the track fasteners and preprocesses them to obtain enhanced images, improving image quality and effectively ensuring subsequent image recognition and localization results. Then, the localization module identifies foreground fasteners and regions of interest (ROIs) within the enhanced images, quickly filtering them to determine suspected fastener locations. The recognition module then uses an image recognition model (such as MobileNetV4) to classify and categorize the track fasteners within the ROI based on their corrosion status, helping to execute corresponding oiling operations according to the degree of corrosion, improving oiling efficiency and accuracy. Next, the spraying module determines the target oiling amount for the track fasteners based on the corrosion status level and applies the target amount of oil. Finally, the correction module evaluates the oiling quality of the track fasteners and performs re-spraying if the evaluation result is unsatisfactory. Through these operations, a complete closed-loop control system of "recognition-execution-evaluation-correction" is constructed, ensuring the final maintenance quality and achieving quantitative management and guarantee of the work effect.

[0084] Example 4 Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504. The processor 501 can call logical instructions from the memory 503 to execute a lightweight vision recognition-based intelligent oiling detection method for track fasteners. This method includes: Acquire the original image of the track fastener and preprocess the original image to obtain the enhanced image; Foreground fasteners and regions of interest are identified and quickly filtered in enhanced images to determine the location of suspected fasteners; Image recognition models are used to classify and identify the corrosion status of track fasteners within the region of interest. Based on the corrosion status level, determine the target amount of oil to be applied to the rail fasteners, and then spray the rail fasteners to apply the target amount of oil. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-coated.

[0085] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] Example 5 On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the intelligent oiling detection method for track fasteners based on lightweight vision recognition provided by the above methods. The method includes: Acquire the original image of the track fastener and preprocess the original image to obtain the enhanced image; Foreground fasteners and regions of interest are identified and quickly filtered in enhanced images to determine the location of suspected fasteners; Image recognition models are used to classify and identify the corrosion status of track fasteners within the region of interest. Based on the corrosion status level, determine the target amount of oil to be applied to the rail fasteners, and then spray the rail fasteners to apply the target amount of oil. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-coated.

[0087] Example 6 In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described intelligent oiling detection method for track fasteners based on lightweight vision recognition, the method comprising: Acquire the original image of the track fastener and preprocess the original image to obtain the enhanced image; Foreground fasteners and regions of interest are identified and quickly filtered in enhanced images to determine the location of suspected fasteners; Image recognition models are used to classify and identify the corrosion status of track fasteners within the region of interest. Based on the corrosion status level, determine the target amount of oil to be applied to the rail fasteners, and then spray the rail fasteners to apply the target amount of oil. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-coated.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent oiling detection of track fasteners based on lightweight visual recognition, wherein the method is executed by a drone, characterized in that, The method includes: Acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image; The enhanced image is subjected to foreground fastener identification and region of interest, and rapid filtering to determine the location of suspected fasteners; The corrosion status of the track fasteners within the region of interest is classified and identified using an image recognition model. Based on the corrosion status level, determine the target amount of oil to be applied to the rail fastener, and spray the rail fastener to apply the target amount of oil. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-sprayed. Based on the corrosion condition level, determine the target oil application amount for the rail fasteners, and apply the target oil application amount by spraying the rail fasteners, including: The model of the track fastener is identified, and the corresponding geometric parameters are retrieved to calculate the effective coated surface area; The porosity correction factor is determined based on the corrosion status level of the track fastener; Based on the effective coating surface area, the preset oil film thickness, and the porosity correction coefficient, the standard oil volume of a single fastener is calculated. The track fasteners are sprayed with oil, and the actual oil output is monitored in real time using a flow sensor. The spraying time or nozzle valve opening is adjusted based on the monitoring results. The quality of the oil coating on the track fasteners is evaluated, and if the evaluation result is unqualified, the track fasteners are re-sprayed, including: A secondary image of the track fastener after oiling is performed to extract the image features of the oil film area; Calculate the proportion of the oil film coverage area to the total area of ​​the track fasteners, and generate a coverage score; The equivalent capacitance value of the track fastener surface is measured using an electrostatic induction ring to evaluate the adhesion thickness of the oil film. Infrared thermal imagers were used to monitor temperature changes before and after spraying to evaluate the penetration performance of the oil. If the evaluation result is unqualified, the track fasteners shall be re-sprayed.

2. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, Acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image, including: The original image is subjected to bad pixel detection and correction, and bad pixels with brightness differences exceeding a preset standard are replaced by the average value of neighboring pixels. An adaptive nonlocal means algorithm is used to filter out Gaussian noise in the original image; A contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast in dark areas, and neighborhood mean smoothing is applied to overexposed pixels.

3. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, The enhanced image is subjected to foreground fastener identification and region of interest (ROI) and rapid filtering to determine the location of suspected fasteners, including: Background subtraction is performed on the enhanced image using an adaptive Gaussian mixture model to extract the moving foreground mask; Based on the prior physical dimensions and geometric shape of the track fastener, the motion foreground mask is pre-screened to remove interference areas that do not conform to the scale and shape characteristics. A lightweight object detection network is used to quickly scan the pre-screened area to generate candidate regions of interest for fasteners and determine the location of suspected fasteners.

4. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, After identifying the suspected fastener location, the following is also included: Calculate the actual physical distance between two consecutive track fasteners based on the suspected fastener positions; The actual physical distance is compared with the preset standard fastener distance to obtain the distance deviation; The flight speed or position of the UAV is adjusted based on the spacing deviation.

5. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, Using an image recognition model, the corrosion status of the track fasteners within the region of interest is classified and identified, including: Extract the image color features of the region of interest and calculate the percentage of pixels with a hue in the orange-brown range; Extract the geometric features of the region of interest from the image, and analyze the edge continuity and thread visibility; By integrating the image color features and the image geometric features, the corrosion status level of the track fastener is classified.

6. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, When spraying the track fasteners, a dual-pressure fan-shaped nozzle is used, and the spraying process selects the corresponding spraying mode according to the corrosion status level of the track fasteners. The spraying modes include penetration mode and coating mode.

7. The intelligent oiling detection method for track fasteners based on lightweight visual recognition according to claim 1, characterized in that, The method further includes: The available track maintenance window time is automatically analyzed based on the railway timetable. Based on the drone's endurance and operating speed, the optimal operating route is dynamically planned to ensure that the drone leaves the track area before the train passes.

8. A lightweight visual recognition-based intelligent oiling detection device for rail fasteners, used to implement the lightweight visual recognition-based intelligent oiling detection method for rail fasteners as described in any one of claims 1-7, characterized in that, include: The acquisition module is used to acquire the original image of the track fastener and preprocess the original image to obtain an enhanced image; The positioning module is used to identify foreground fasteners and regions of interest in the enhanced image, quickly filter them, and determine the location of suspected fasteners. The identification module is used to classify and identify the corrosion status of the track fasteners within the region of interest using an image recognition model; The spraying module is used to determine the target amount of oil to be applied to the rail fastener based on the corrosion status level, and to spray the rail fastener to apply the target amount of oil. The correction module is used to evaluate the oiling quality of the track fasteners and to re-spray the track fasteners if the evaluation result is unqualified.

Images