A dynamic spraying control method and system based on an intelligent spraying vehicle

Abstract

The embodiment of the present application relates to the maintenance technical field, disclose a kind of dynamic spraying control method and system based on intelligent oil spraying vehicle, the method includes: corresponding target bolt surface two-dimensional color texture image is collected by image acquisition module and three-dimensional laser point cloud data of target bolt is collected by laser radar;Two-dimensional color texture image is analyzed by HSV color space analysis, and the rust grade of target bolt surface is determined;According to three-dimensional laser point cloud data, the rust state information of target bolt surface is determined;According to rust grade and rust state information, current environmental temperature and humidity data are combined, and the oil injection strategy of corresponding target bolt is generated by rust-oil film coupling decision model;According to oil injection strategy, control signal is generated to drive oil injection nozzle to execute oil injection operation to target bolt.The method of the present application can accurately identify the rust degree and surface state of bolt by visual and laser radar fusion analysis, and improve the accuracy of oil injection protection.

Description

Technical Field

[0001] This invention relates to the field of maintenance technology, specifically to a dynamic spraying control method and system based on an intelligent spraying vehicle. Background Technology

[0002] Currently, railway rails are fixed to sleepers via a fastening system (elastic clips, bolts, washers, etc.), among which the fastening bolts are key components ensuring the stability of the track's geometric dimensions. Bolts are exposed to the outdoor environment for extended periods, suffering from wind, rain, and train vibrations, making them highly susceptible to corrosion. This leads to reduced fastening force, difficulty in disassembly, and even compromises the safety of the track structure. Therefore, regular lubrication of bolts is an important part of railway maintenance. Currently, bolt lubrication is primarily done manually: workers bend over, heads down, and use brushes to apply lubricant to each bolt individually. This method has the following significant problems: Problem 1: Poor working environment and high labor intensity. Workers spend long hours bending over and kneeling or squatting on the ballast, causing serious damage to their lumbar and cervical spine. This is a typical job that is dirty, tiring, and dangerous, making recruitment difficult.

[0003] Problem 2: Low work efficiency. The manual oiling speed is about 2-4 bolts per minute. There are about 1600-2000 bolts per kilometer of track (calculated based on 4 bolts per sleeper, about 1667 sleepers per kilometer). It takes a single work team (4-6 people) 3-5 days to complete one kilometer, which is extremely inefficient.

[0004] Problem 3: The quality of oiling cannot be guaranteed. The uniformity and coverage of manual oiling depend entirely on the experience and sense of responsibility of the operators, resulting in a large number of missed areas, insufficient oiling, or excessive waste. Moreover, the quality is difficult to inspect and trace.

[0005] Problem 4: Lack of intelligent oiling control methods. Existing mechanized oiling equipment (if available) mostly operates with fixed parameters, unable to dynamically adjust based on the actual position of the bolts, their corrosion state, and environmental conditions. This results in poor spraying accuracy and significant oil waste. Therefore, designing a method for dynamic spraying control has become a pressing technical problem for those skilled in the art. Summary of the Invention

[0006] To address the aforementioned shortcomings, this invention discloses a dynamic spraying control method based on an intelligent spraying vehicle, which can improve the accuracy of bolt identification and enhance the bolt spraying effect.

[0007] The first aspect of this invention discloses a dynamic spraying control method based on an intelligent fuel injection vehicle, comprising: The image acquisition module acquires two-dimensional color texture images of the target bolt surface, and the lidar acquires three-dimensional laser point cloud data of the target bolt. The two-dimensional color texture image was analyzed using HSV color space analysis to determine the corrosion level of the target bolt surface; The corrosion status information of the target bolt surface is determined based on the three-dimensional laser point cloud data; Based on the corrosion level and corrosion status information, combined with the current ambient temperature and humidity data, the corresponding oil spraying strategy for the target bolt is generated through the corrosion-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. A control signal is generated according to the oil injection strategy to drive the oil injector to perform an oil injection operation on the target bolt.

[0008] As an optional implementation, in a first aspect of the present invention, the corrosion state information includes the depth of the corrosion pits; The step of determining the corrosion state information of the target bolt surface based on the three-dimensional laser point cloud data further includes: Statistical filtering and radius filtering are performed on the three-dimensional laser point cloud data; Perform plane fitting on the filtered point cloud to determine the reference plane of the target bolt surface; Calculate the normal distance of each point in the filtered point cloud relative to the reference plane to generate a depth map of the target bolt surface; Based on the depth map, the maximum depression depth, average depression depth, and total area of ​​the depression region on the surface of the target bolt are extracted, and the depth of the rust pit is determined based on the maximum depression depth and the average depression depth.

[0009] As an optional implementation, in the first aspect of the present invention, the corrosion state information further includes surface porosity characteristics; the surface porosity characteristics are calculated in the following manner: Perform connected component analysis on the depth map to identify all depression regions whose depth values ​​exceed a preset depth threshold; Calculate the area and volume of each of the said recessed regions, whereby the volume is obtained by integrating the depth value of the corresponding region in the depth map; The ratio of the total volume of all recessed areas to the theoretical smooth surface area of ​​the region of interest in the bolt head is used as the first porosity index. The ratio of the sum of the areas of all recessed regions to the total area of ​​the region of interest in the bolt head is used as the second porosity index. The weighted combination of the first porosity index and the second porosity index is output as the surface porosity feature.

[0010] As an optional implementation, in the first aspect of the present invention, the spraying control method further includes: Read the historical depth map and historical corrosion level of the same target bolt at the historical operation time from the bolt cascade database; The current depth map is compared with the historical depth map point by point to generate a corrosion depth variation map; The corrosion rate index of the target bolt surface during the interval between two operations is calculated based on the corrosion depth variation diagram. The corrosion rate index is the average increment of the corrosion pit depth per unit time. When the corrosion rate index exceeds the preset rate threshold, the target bolt is marked as an accelerated corrosion target in the list of targets to be sprayed, and the maintenance priority of the target bolt is increased in the subsequent oil spraying strategy generation.

[0011] As an optional implementation, in the first aspect of the present invention, after acquiring a two-dimensional color texture image of the target bolt surface via an image acquisition module and acquiring three-dimensional laser point cloud data of the target bolt via a lidar, the method further includes: Obtain calibration parameters between the image acquisition module and the lidar, the calibration parameters including rotation matrix and translation vector; According to the calibration parameters, each spatial point in the three-dimensional laser point cloud data is projected onto the pixel coordinate system of the two-dimensional color texture image to establish a point-by-point mapping relationship between the three-dimensional laser point cloud data and the two-dimensional color texture image. Based on the point-to-point mapping relationship, the rust level information and rust pit depth information of the target bolt surface are fused at the pixel level to generate a rust state fusion map. Each pixel position in the rust state fusion map contains both a rust level label and a depth value label. Based on the rust state fusion map, the surface of the target bolt is divided into regions, and multiple sub-regions with different combinations of rust levels and rust pit depths are identified. The expected oil spraying demand for each sub-region is calculated separately, and the expected oil spraying demand for each sub-region is positively correlated with the rust grade and average rust pit depth of the sub-region. The estimated oil injection requirements of all sub-regions are summed to generate the comprehensive estimated oil injection requirement of the target bolt. When driving the injector to perform the oil spraying operation, the oil spraying pulse width and oil spraying pressure are controlled according to the comprehensive expected oil spraying volume, and the scanning trajectory of the injector is controlled according to the spatial distribution of each sub-region, so as to achieve differentiated spraying on the surface of the target bolt.

[0012] As an optional implementation, in a first aspect of the present invention, the step of analyzing the two-dimensional color texture image through HSV color space analysis and determining the corrosion level of the target bolt surface includes: The two-dimensional color texture image is converted from the RGB color space to the HSV color space, and the hue component, saturation component and lightness component are extracted. Histogram statistics are performed on the hue components to identify the distribution ratio of each pixel on the surface of the target bolt in the orange-yellow, reddish-brown, and dark brown ranges on the hue ring; The corrosion color feature vector is calculated based on the distribution ratio, and the corrosion color feature vector includes the proportion of orange-yellow, reddish-brown and blackish-brown. The corrosion color feature vector is input into a pre-trained corrosion grading classifier, which outputs the initial corrosion level of the target bolt surface. The training process of the pre-trained rust grading classifier includes: Multiple color images of bolt surfaces labeled with corrosion grade tags were collected. The corrosion grade tags were either manually labeled or calibrated by a laboratory corrosion degree measuring instrument. For each sample, extract the histogram of hue components, the mean value of saturation components, and the mean value of lightness components in the HSV color space to form a training feature set; Using the training feature set as input and the corrosion level label as supervision signal, a support vector machine classifier or a lightweight convolutional neural network classifier is trained, and the classifier hyperparameters are optimized through cross-validation during the training process. The trained classifier is deployed in the edge computing unit for real-time corrosion level determination.

[0013] As an optional implementation, in the first aspect of the present invention, the spraying control method further includes: Based on the first detection component mounted on the first-level vehicle, the bolts along the route are quickly located and their initial condition is judged, and a list of targets to be sprayed is generated. The list of targets to be sprayed includes the geographical location information of the bolts suspected of needing maintenance and a rough rust status mark; wherein, the first-level vehicle travels within a first speed threshold range; The second-level vehicle receives the list of targets to be sprayed sent by the first-level vehicle and navigates sequentially to the locations of suspected bolts requiring maintenance based on the geographical location information in the list. This triggers the edge computing unit to perform a dynamic spraying operation. The second-level vehicle travels within a second speed threshold range and is equipped with a second perception kit and an execution kit. The second perception kit includes an image acquisition module and a lidar, and the execution kit includes fuel injectors. The upper limit of the second speed threshold range is lower than the lower limit of the first speed threshold range.

[0014] As an optional implementation, in the first aspect of the present invention, before acquiring the two-dimensional color texture image of the corresponding target bolt surface through the image acquisition module, the method further includes: The initial detection image captured during the movement of the intelligent fuel injection vehicle is acquired. The initial detection image is then identified using a first-level detection model, and the fastener boundary box and fastener category identifier in the initial detection image are determined. The spatial relationship between fasteners and bolts is determined based on the fastener category identifier and the pre-built fastener-bolt mapping table, and the bolt detection area is dynamically generated based on the fastener boundary box and the spatial relationship between fasteners and bolts. The bolt detection area is input into the second-level detection model for bolt identification to obtain the current status information of the bolt at the corresponding position and the target position of the bolt; Output the target position of the bolt and the current status information of the bolt.

[0015] A second aspect of this invention discloses a dynamic spraying control system based on an intelligent spraying vehicle, comprising: Acquisition module: used to acquire two-dimensional color texture images of the target bolt surface through the image acquisition module and to acquire three-dimensional laser point cloud data of the target bolt through the lidar; First analysis module: used to analyze the two-dimensional color texture image through HSV color space analysis and determine the corrosion level of the target bolt surface; The second analysis module is used to determine the corrosion status information of the target bolt surface based on the three-dimensional laser point cloud data. The generation module is used to generate an oil spraying strategy for the target bolt based on the corrosion level and corrosion status information, combined with the current ambient temperature and humidity data, through a corrosion-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. Injection module: Used to generate control signals according to the injection strategy to drive the injector to perform the injection operation on the target bolt.

[0016] A third aspect of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the dynamic spraying control method based on an intelligent spraying vehicle disclosed in the first aspect of the present invention.

[0017] A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the dynamic spraying control method based on an intelligent spraying vehicle disclosed in the first aspect of the present invention.

[0018] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The dynamic spraying control method based on an intelligent spraying vehicle in this embodiment of the invention uses a fusion analysis of vision (two-dimensional images) and lidar (three-dimensional point clouds) to accurately identify the individual corrosion degree and surface condition of each bolt, rather than treating the entire area indiscriminately. This improves the accuracy of spraying protection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of the dynamic spraying control method based on an intelligent spraying vehicle disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of the process for rust identification disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a dynamic spraying control system based on an intelligent spraying vehicle provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," "third," "fourth," etc., in the specification and claims of this invention are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "having," and any variations thereof, in the embodiments of this invention are intended to cover non-exclusive inclusion. Exemplarily, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0023] Example 1 Please see Figure 1 , Figure 1This is a flowchart illustrating the dynamic spraying control method based on an intelligent spraying vehicle disclosed in an embodiment of the present invention. The execution entity of the method described in this embodiment is an execution entity composed of software and / or hardware. This execution entity can receive relevant information via wired or / or wireless means and can send certain instructions. It may also have certain processing and storage functions. This execution entity can control multiple devices, such as remote physical servers or cloud servers and related software, or local hosts or servers and related software that perform related operations on devices located in a certain location. In some scenarios, multiple storage devices can also be controlled; these storage devices may be placed in the same location as the devices or in different locations. Figure 1 As shown, the dynamic spraying control method based on an intelligent fuel injection vehicle includes the following steps: S101: Acquire two-dimensional color texture images of the target bolt surface through the image acquisition module and acquire three-dimensional laser point cloud data of the target bolt through the lidar; S102: Analyze the two-dimensional color texture image using HSV color space analysis and determine the corrosion level of the target bolt surface; S103: Determine the corrosion status information of the target bolt surface based on the three-dimensional laser point cloud data; S104: Based on the rust level and rust state information, and combined with the current ambient temperature and humidity data, generate the corresponding oil spraying strategy for the target bolt through the rust-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. S105: Generate a control signal according to the oil injection strategy to drive the oil injector to perform oil injection operation on the target bolt.

[0024] The solution of this invention overcomes the limitations of a single vision by fusing dual-modal data of two-dimensional color images (identifying color and texture) and three-dimensional laser point clouds (identifying geometric shape and depth). It can not only detect surface color changes, but also detect three-dimensional corrosion features such as volume expansion and pits in bolt edges, thread grooves, and other parts through point cloud data, providing a more complete corrosion profile for subsequent decision-making.

[0025] Specifically, it uses the HSV (Hue, Saturation, Lightness) color space instead of the traditional RGB, making rust color analysis more consistent with human visual characteristics and less sensitive to changes in lighting. It can more accurately separate the typical tonal range of rust from the image, effectively distinguishing between rust, oil, dust, and the natural color of the metal, thus reliably quantifying the degree of rust.

[0026] In practical implementation, a rust-oil film coupled decision model is adopted to achieve precise quantification of the spraying strategy. This model does not simply spray oil based on area, but couples the severity of rust (level and state) with ambient temperature and humidity to dynamically calculate the required oil film thickness and coverage requirements. It can generate differentiated target spraying quantities (more spray for severe rust, increased oil film density for humid environments) and spraying pressure parameters, avoiding underspray leading to protection failure or overspray causing waste and pollution.

[0027] It replaces manual visual inspection and manual oiling, avoiding human error; at the same time, it can spray each bolt array individually, at specific points, and in specific quantities, making it especially suitable for high-altitude, confined, or hazardous work scenarios, ensuring that each bolt receives the exact protective treatment it needs.

[0028] The value of the method in this invention lies in on-demand oil supply. By accurately sensing the actual corrosion risk of each bolt and combining it with environmental factors, the optimal oil spraying parameters are dynamically output. This ensures the anti-corrosion effect while maximizing oil savings, reducing environmental pollution, and extending the service life of bolts and connecting pairs.

[0029] More preferably, the corrosion status information includes the depth of the corrosion pits; The step of determining the corrosion state information of the target bolt surface based on the three-dimensional laser point cloud data further includes: S1031: Perform statistical filtering and radius filtering on the three-dimensional laser point cloud data; S1032: Perform plane fitting on the filtered point cloud to determine the reference plane of the target bolt surface; S1033: Calculate the normal distance of each point in the filtered point cloud relative to the reference plane, and generate a depth map of the target bolt surface; S1034: Based on the depth map, extract the maximum depression depth, average depression depth, and total area of ​​the depression region on the surface of the target bolt, and determine the depth of the rust pit based on the maximum depression depth and the average depression depth.

[0030] This invention employs a combined strategy of statistical filtering (removing outliers) and radius filtering (removing sparse isolated points). This effectively filters out flying spots caused by differences in lidar reflectivity, interference points from environmental suspended matter, and stray points on bolt edges. This allows subsequent calculations to be based on a clean, continuous surface point cloud, significantly improving the signal-to-noise ratio of depth measurements and avoiding misjudgments of corrosion depth due to isolated anomalies.

[0031] By performing planar fitting on the filtered point cloud, an ideal reference surface (equivalent to the original intact surface) is constructed for the bolt surface, unaffected by corrosion. Since the bolt surface is not perfectly flat (it has curvature or edges), this step adaptively fits the actual geometry of the bolt, establishing a dynamic measurement reference zero point for each local area. Compared to a fixed horizontal plane, this method accurately isolates the bolt's own shape, extracting only the concave deformation caused by corrosion, and avoiding miscalculating structural curvature as rust pits.

[0032] The normal distance calculation and depth map generation embodiments of this invention achieve quantitative mapping from three-dimensional to two-dimensional. It calculates the normal (vertical) distance from each point cloud point to the reference plane and projects it to generate a two-dimensional depth map. This transforms complex three-dimensional spatial data into an intuitive pixel-level grayscale depth map, where each pixel value directly represents the corrosion depression depth at that point. This facilitates subsequent image processing algorithms (such as thresholding and region growing), making the statistical analysis of corrosion distribution across the entire scanned area fast and repeatable. Simultaneously, it extracts three key indicators: maximum depression depth, average depression depth, and total area of ​​the depression region, and determines the depth of the corrosion pits accordingly.

[0033] Specifically, the following parameters are considered: Maximum Depth: Reflects the most severe risk of localized perforation (determining whether emergency treatment is needed). Average Depth: Reflects the overall degree of rust damage (determining the base amount of paint spraying). Total Area of ​​Indentation: Reflects the extent of rust spread (determining the area to be covered by paint spraying). Combining these three parameters allows for the differentiation between two different hazard modes: localized deep pits and large-area shallow rust, avoiding overly aggressive or insufficient strategies based solely on a single maximum value.

[0034] The solution in this invention transforms qualitative observation of rust into quantitative analysis of rust pit volume and morphology. The final determined pit depth is not an abstract value, but rather a realistic wear amount based on physical benchmarks, after noise reduction and geometric calculation. This allows the model to accurately calculate the required oil film thickness based on the pit volume (e.g., deeper pits require higher viscosity or greater pressure to force the oil to the bottom), thus truly achieving on-demand repair spraying based on three-dimensional damage conditions.

[0035] More preferably, the corrosion state information further includes surface porosity characteristics; the surface porosity characteristics are calculated as follows: Perform connected component analysis on the depth map to identify all depression regions whose depth values ​​exceed a preset depth threshold; Calculate the area and volume of each of the said recessed regions, whereby the volume is obtained by integrating the depth value of the corresponding region in the depth map; The ratio of the total volume of all recessed areas to the theoretical smooth surface area of ​​the region of interest in the bolt head is used as the first porosity index. The ratio of the sum of the areas of all recessed regions to the total area of ​​the region of interest in the bolt head is used as the second porosity index. The weighted combination of the first porosity index and the second porosity index is output as the surface porosity feature.

[0036] This invention performs connected component analysis on the depth map by setting a preset depth threshold, rather than simply identifying all depressions. This effectively filters out minute machining textures, scratches, or measurement noise on the bolt surface, extracting only genuine rust pores that reach a certain depth and meet protective requirements.

[0037] The actual volume of each rust-affected hole is calculated by integrating the depth values ​​of the recessed areas in the depth map. This represents a qualitative change from area to volume. It directly quantifies the amount of material loss caused by rust peeling or expansion. This volume value directly corresponds to the volume of oil required to fill the hole, providing the most direct physical basis for calculating the amount of oil sprayed (for example, the larger the hole volume, the more rust-preventive oil is required).

[0038] In this embodiment of the invention, a first porosity based on volume (total volume / theoretical surface area) and a second porosity based on area (total area / total area) are calculated simultaneously, and the final feature is output by weighted combination.

[0039] Specific manifestations: First porosity (volume ratio): reflects the average depth of rust; a higher value indicates that the surface absorbs more oil, requiring a larger amount of oil per unit area to fill the pits. Second porosity (area ratio): reflects the spread and density of rust; a higher value indicates a wider area that needs to be covered by oil spraying. Weighted combination: allows the system to adjust the weights of the two indicators according to actual working conditions (e.g., increasing the volume weight when deep pits pose a greater hazard), flexibly adapting to different types of rust morphology.

[0040] The volumetric porosity is calculated using the theoretically smoothed surface area of ​​the region of interest on the bolt head as the denominator. This avoids the increased surface area caused by manufacturing marks, chamfers, or edges on the bolt head itself, ensuring that the porosity calculation only reflects the additional volume caused by corrosion.

[0041] The solution in this invention elevates the visual inspection results from image features to physical requirements. The final output is the surface porosity feature, which represents the total volume of voids per unit area that need to be filled with anti-rust oil. When this feature is input into the rust-oil film coupled decision model, the model can directly calculate the theoretically required oil film thickness based on this value, and then make corrections based on ambient temperature and humidity, thereby achieving true volumetric oil filling and greatly improving the physical rationality of the spraying strategy and material utilization.

[0042] More preferably, the spraying control method further includes: Read the historical depth map and historical corrosion level of the same target bolt at the historical operation time from the bolt cascade database; The current depth map is compared with the historical depth map point by point to generate a corrosion depth variation map; The corrosion rate index of the target bolt surface during the interval between two operations is calculated based on the corrosion depth variation diagram. The corrosion rate index is the average increment of the corrosion pit depth per unit time. When the corrosion rate index exceeds the preset rate threshold, the target bolt is marked as an accelerated corrosion target in the list of targets to be sprayed, and the maintenance priority of the target bolt is increased in the subsequent oil spraying strategy generation.

[0043] This invention upgrades single static detection to dynamic monitoring across time, introduces corrosion rate as the core basis for predictive maintenance, and achieves a leap from on-demand oiling to proactive early warning and graded maintenance.

[0044] By storing historical depth maps and corrosion levels of the same target bolt at different times in a cascaded bolt database, a one-to-one health record is established. This breaks away from the traditional isolated detection mode of one-time testing and no correlation between different bolts, enabling the system to track the corrosion trajectory of each bolt over time. This provides the data foundation for personalized maintenance and trend analysis.

[0045] This invention employs pixel-by-pixel subtraction between the current depth map and historical depth maps to generate a corrosion depth variation map. This subtraction operation can precisely offset the inherent structural irregularities of the bolt itself, extracting only the newly generated or aggravated corrosion increments within the interval between two operations. This temporal subtraction technique greatly enhances the system's sensitivity to subtle changes, accurately capturing corrosion progress even if the initial state has defects.

[0046] The average increment of rust pit depth per unit time is calculated from the change graph, which is the rust rate index. This reflects the dynamic characteristics of rust development; a bolt that is currently only slightly rusted but has an extremely rapid rust rate poses a far greater risk than a bolt that is severely rusted but has stabilized. This rate value provides the acceleration dimension input to the decision model.

[0047] When the corrosion rate exceeds a preset threshold, the system marks the bolt as an accelerated corrosion target in the list of targets to be sprayed. This achieves a shift from passive response to proactive early warning. Marked bolts will receive higher maintenance priority in subsequent spraying strategies (e.g., priority processing, earlier maintenance cycles).

[0048] In batch bolt maintenance scenarios, the solution of this invention can dynamically allocate limited oiling resources (such as rust-preventive oil and operation time), prioritizing resources for bolts with high growth risk, while maintaining routine maintenance for stable bolts. This ensures the safety of the overall structure.

[0049] The solution in this invention endows the system with time-aware capabilities. It no longer makes decisions solely based on the current corrosion snapshot, but instead predicts future corrosion risks through the key indicator of corrosion rate. Combined with environmental temperature and humidity corrections, the system can identify abnormal bolts that suddenly accelerate corrosion under specific environmental conditions, providing reliable technical support for predictive maintenance and safety early warning of infrastructure.

[0050] More preferably, after acquiring a two-dimensional color texture image of the target bolt surface via the image acquisition module and acquiring three-dimensional laser point cloud data of the target bolt via the lidar, the method further includes: Obtain calibration parameters between the image acquisition module and the lidar, the calibration parameters including rotation matrix and translation vector; According to the calibration parameters, each spatial point in the three-dimensional laser point cloud data is projected onto the pixel coordinate system of the two-dimensional color texture image to establish a point-by-point mapping relationship between the three-dimensional laser point cloud data and the two-dimensional color texture image. Based on the point-to-point mapping relationship, the rust level information and rust pit depth information of the target bolt surface are fused at the pixel level to generate a rust state fusion map. Each pixel position in the rust state fusion map contains both a rust level label and a depth value label. Based on the rust state fusion map, the surface of the target bolt is divided into regions, and multiple sub-regions with different combinations of rust levels and rust pit depths are identified. The expected oil spraying demand for each sub-region is calculated separately, and the expected oil spraying demand for each sub-region is positively correlated with the rust grade and average rust pit depth of the sub-region. The estimated oil injection requirements of all sub-regions are summed to generate the comprehensive estimated oil injection requirement of the target bolt. When driving the injector to perform the oil spraying operation, the oil spraying pulse width and oil spraying pressure are controlled according to the comprehensive expected oil spraying volume, and the scanning trajectory of the injector is controlled according to the spatial distribution of each sub-region, so as to achieve differentiated spraying on the surface of the target bolt.

[0051] The solution of this invention realizes a complete closed loop from overall rough estimation to pixel-level precise registration and then to sub-regional differentiated execution, so that the spraying operation can truly achieve the level of precision in construction according to the drawing and precise delivery.

[0052] Specifically, by utilizing the calibration parameters (rotation matrix and translation vector) between the image acquisition module and the LiDAR, the 3D laser point cloud is accurately projected onto the pixel coordinate system of the 2D color image. This solves the engineering challenges of inconsistent spatial coordinate systems, different resolutions, and field-of-view deviations between the two sensors. The established point-to-point mapping relationship ensures that each pixel obtains its corresponding 3D spatial coordinate information.

[0053] Based on the mapping relationship, the corrosion level (from HSV image analysis) and the corrosion pit depth (from point cloud depth map) are fused at the pixel level to generate a fused map. Each pixel in the fused map carries both a corrosion level label (qualitative) and a depth value label (quantitative). This data structure makes the analysis no longer fragmented; the system can not only determine whether a certain location is not only deeply corroded but also severely corroded, providing the most complete data foundation for fine-grained partitioning.

[0054] The solution in this invention divides the fused image into regions, identifying multiple sub-regions with different combinations of rust grades and pit depths. It can segment the surface into severely pitted areas, lightly rusted areas, moderately pitted areas, and intact areas. This multi-dimensional clustering partitioning is more scientific than partitioning by area or a single indicator because, at the same depth, the repair needs for high-grade rust are far greater than those for low-grade rust. The finer the partitioning granularity, the more precise the subsequent oil spraying strategy.

[0055] The solution in this invention calculates the estimated paint spraying requirement for each sub-region, which is positively correlated with the rust level and average pit depth. This breaks away from the traditional, crude method of applying a single total paint spraying amount to each bolt. The paint requirement per unit area can vary significantly between different regions. Deep pits with heavy rust require high pressure, high flow rate, and long pulses for filling and covering, while shallow rust areas only require light sweeping protection. This spatially differentiated requirement calculation makes the composition of the total paint spraying amount more reliable.

[0056] During the execution phase, the injection pulse width and injection pressure are controlled based on the comprehensive estimated injection quantity, while the injector scanning trajectory is controlled according to the spatial distribution of each sub-region.

[0057] The spray nozzles no longer apply sprays at fixed points or sweep evenly; instead, they automatically plan their trajectories, prioritizing the scanning of heavily rusted areas with longer dwell times, lightly sweeping shallow rust areas, and skipping intact areas. This path planning and parameter-linked execution method achieves truly spatially differentiated spraying, ensuring each sub-area receives an oil film thickness and coverage density precisely matched to its own damage. Using the minimum total oil volume and optimal spatial distribution, every drop of rust-preventive oil lands where it's most needed, covering the most suitable thickness, maximizing rust prevention while minimizing oil consumption and environmental costs.

[0058] More preferably, the step of analyzing the two-dimensional color texture image through HSV color space analysis and determining the corrosion level of the target bolt surface includes: The two-dimensional color texture image is converted from the RGB color space to the HSV color space, and the hue component, saturation component and lightness component are extracted. Histogram statistics are performed on the hue components to identify the distribution ratio of each pixel on the surface of the target bolt in the orange-yellow, reddish-brown, and dark brown ranges on the hue ring; The corrosion color feature vector is calculated based on the distribution ratio, and the corrosion color feature vector includes the proportion of orange-yellow, reddish-brown and blackish-brown. The corrosion color feature vector is input into a pre-trained corrosion grading classifier, which outputs the initial corrosion level of the target bolt surface. The training process of the pre-trained rust grading classifier includes: Multiple color images of bolt surfaces labeled with corrosion grade tags were collected. The corrosion grade tags were either manually labeled or calibrated by a laboratory corrosion degree measuring instrument. For each sample, extract the histogram of hue components, the mean value of saturation components, and the mean value of lightness components in the HSV color space to form a training feature set; Using the training feature set as input and the corrosion level label as supervision signal, a support vector machine classifier or a lightweight convolutional neural network classifier is trained, and the classifier hyperparameters are optimized through cross-validation during the training process. The trained classifier is deployed in the edge computing unit for real-time corrosion level determination.

[0059] This invention converts images from RGB to HSV color space and extracts three independent components: hue (H), saturation (S), and lightness (V). Hue (H) directly corresponds to the color category (rust falls within the orange-yellow, reddish-brown, and dark brown range on the color wheel), is insensitive to changes in light intensity, and is more robust than RGB. Saturation (S) reflects color purity; the more severe the corrosion, the lower the saturation (the color tends to be grayish and dark). Lightness (V) reflects surface brightness and helps distinguish between fresh and old rust. Separating these three components allows subsequent analysis to process color category, color purity, and surface brightness separately, greatly improving the physical interpretability of feature extraction.

[0060] The hue histogram statistics and interval distribution ratio calculation of this invention accurately capture the hue characteristics of rust, perform histogram statistics on the hue components, and identify the distribution ratio of each pixel in the orange-yellow interval, reddish-brown interval, and dark brown interval.

[0061] The evolution path of rust is usually metal color, orange-yellow (initial rust), reddish-brown (moderate rust), and blackish-brown (severe / old rust). The distribution ratio of these three intervals directly constitutes the color fingerprint of the rusting process. By statistically analyzing the proportions rather than individual pixel values, noise interference such as uneven surface texture and local reflections is effectively suppressed. The extracted three-dimensional proportion feature vector is concise and complete, providing high-quality input for the subsequent classifier.

[0062] In this embodiment of the invention, a pre-trained classifier replaces a fixed threshold to achieve adaptive intelligent grading. The rust color feature vector is input into the pre-trained rust grading classifier, which outputs the initial rust level.

[0063] This invention abandons the traditional fixed judgment method of manually setting color thresholds and instead adopts a data-driven classification model. The classifier can learn the nonlinear decision boundary of different corrosion levels in the HSV feature space, and deal with complex situations such as overlapping and gradation of corrosion colors in actual working conditions. The output is a discrete level, which is convenient for subsequent decision models to use directly.

[0064] More preferably, the spraying control method further includes: Based on the first detection component mounted on the first-level vehicle, the bolts along the route are quickly located and their initial condition is judged, and a list of targets to be sprayed is generated. The list of targets to be sprayed includes the geographical location information of the bolts suspected of needing maintenance and a rough rust status mark; wherein, the first-level vehicle travels within a first speed threshold range; The second-level vehicle receives the list of targets to be sprayed sent by the first-level vehicle and navigates sequentially to the locations of suspected bolts requiring maintenance based on the geographical location information in the list. This triggers the edge computing unit to perform a dynamic spraying operation. The second-level vehicle travels within a second speed threshold range and is equipped with a second perception kit and an execution kit. The second perception kit includes an image acquisition module and a lidar, and the execution kit includes fuel injectors. The upper limit of the second speed threshold range is lower than the lower limit of the first speed threshold range.

[0065] The solution of this invention constructs a two-stage joint operation system of coarse screening and fine screening, achieving optimal decoupling and synergy between detection efficiency and maintenance accuracy, and significantly improving the engineering feasibility and economy of large-area bolt group maintenance. The traditional serial mode in which a single vehicle completes all the work is split into two independent operation units: the first stage (high-speed inspection / preliminary judgment) and the second stage (low-speed fine spraying / execution).

[0066] By trading speed difference for detection coverage and operational accuracy, the upper limit of the second speed threshold range is lower than the lower limit of the first speed threshold range, forming a clear speed range separation.

[0067] Specifically, the system features two levels of high-speed travel: First-level high-speed travel: This allows for the initial screening and coverage of bolts over a large area (such as an entire railway line or a large bridge) in a short time, sacrificing the depth of single-point detection for the efficiency of full-area scanning; Second-level low-speed travel: This slows down or even stops near suspected problem points, providing a stable working platform for fine scanning by the lidar, high-resolution image acquisition, and precision oil injection, avoiding point cloud distortion or oil injection deviation caused by high-speed vibration; The speed difference design ensures that both levels of vehicles can achieve both fast and precise scanning within the same working window.

[0068] Information relay: The list of targets to be sprayed serves as a data contract for two-level collaboration. The first level generates the list of targets to be sprayed (including geographical location and rough rust markings) and sends it to the second level as the basis for operation.

[0069] Specifically, the list reduces the complexity of the task, eliminating the need for blind, full-coverage inspection of all bolts in the second stage. Instead, it focuses on the suspected targets marked in the list. The rough rust markings in the list can serve as a priori reference for the more detailed inspection in the second stage (e.g., prioritizing the use of higher-precision scanning parameters for bolts marked with heavy rust). The list also supports dynamic updates. If the second stage detects a false alarm on-site, the result can be sent back to optimize the initial judgment model of the first stage.

[0070] The solution of this invention optimizes resources and reduces the ineffective idle running of the second-level high-precision equipment. The second level only navigates and operates on the positions in the list of targets to be sprayed, rather than checking each target along the way.

[0071] Specifically, the expensive and power-consuming equipment on the second stage, such as lidar, high-resolution camera, and precision spraying system, only activates operations at effective target points, avoiding equipment wear and energy waste caused by idle operation on a large number of healthy bolts; the consumption of spraying materials (rust-preventive oil) is also strictly limited to targets suspected of needing maintenance, significantly reducing consumable costs.

[0072] The above method shortens the overall operation cycle, allowing the first and second levels to overlap in time (the first level inspects ahead, and the second level follows up with processing). It eliminates the need for sequential waiting times before dispatching maintenance vehicles after all inspections are completed. For extremely long lines (such as railways hundreds of kilometers long), rolling operations can be implemented: the first level continuously advances and generates new lists, while the second level follows closely, processing each point systematically. The overall completion time is approximately equal to the inspection time and the time for each individual point processing, rather than the sum of the two. The independent setting of the vehicle speed thresholds for the two levels allows them to adapt to different road conditions.

[0073] Specifically, the first-level high-speed vehicle can quickly pass through main roads with good road conditions; the second-level low-speed vehicle can safely operate at low speeds in complex / narrow sections such as bridge connections and tunnel entrances, and even has the ability to reverse or rotate in place to align with bolt positions, improving the system's passability and operational accessibility in complex infrastructure environments. This invention's solution, through a two-level division of labor, resolves the fundamental contradiction between efficiency and accuracy in the maintenance of large-area bolt groups, enabling the system to be applied in real-world industrial scenarios.

[0074] More preferably, before acquiring the two-dimensional color texture image of the corresponding target bolt surface through the image acquisition module, the method further includes: The initial detection image captured during the movement of the intelligent fuel injection vehicle is acquired. The initial detection image is then identified using a first-level detection model, and the fastener boundary box and fastener category identifier in the initial detection image are determined. The spatial relationship between fasteners and bolts is determined based on the fastener category identifier and the pre-built fastener-bolt mapping table, and the bolt detection area is dynamically generated based on the fastener boundary box and the spatial relationship between fasteners and bolts. The bolt detection area is input into the second-level detection model for bolt identification to obtain the current status information of the bolt at the corresponding position and the target position of the bolt; Output the target position of the bolt and the current status information of the bolt.

[0075] The solution presented in this invention significantly improves detection efficiency and real-time performance. The first-level detection (global fastener location) has low computational complexity and high speed, while the second-level detection (local bolt location) only needs to process a very small cropped area, avoiding sliding windowing or high-precision convolution on the entire large image. This significantly reduces overall inference latency and is more suitable for the high-speed, real-time processing requirements of intelligent fuel injection vehicles in motion.

[0076] The two-level detection model effectively addresses the issues of small targets and background interference. Bolts are typical small targets in railway scenarios, making them prone to missed detection by direct inspection. This scheme narrows the search range from the entire image to a local high-confidence area by utilizing the spatial relationship between fasteners and bolts, effectively removing complex background noise such as rails, track bed, and oil stains. The high input signal-to-noise ratio of the second-level detection model significantly improves the classification accuracy of bolt states (loose, missing, corroded) and the regression accuracy of bolt target locations.

[0077] Because the fuel injection vehicle vibrates during movement, the relative position of the camera and the track is not fixed. This solution does not rely on fixed pixel coordinates, but instead calculates the fastener bounding box of the current frame in real time and then dynamically generates the bolt detection area. This gives the system adaptive anti-shake capability, ensuring that the bolt detection area can always accurately follow the fastener movement under different lighting conditions and different operating speeds, avoiding detection box offset failure due to vehicle bumps.

[0078] The final output bolt target position can be directly used to drive the robotic arm or fuel injector for fixed-point operation, while the current status information is sent back to the management backend. This direct conversion of visual perception results into execution commands and maintenance records ensures the accuracy of the fuel spraying operation (avoiding mis-spraying and missed spraying) and provides quantitative data support for subsequent bolt lifecycle health management.

[0079] Existing solutions involve acquiring bolt images, training a detection model, and directly outputting the bolt's position and status. While feasible in static scenarios, this approach has drawbacks in dynamic railway operation scenarios. Bolts are small, easily obscured by oil and rust, and their imaging is unstable due to vehicle vibrations, resulting in extremely high false negative and false positive rates. However, rusted areas require more maintenance, which increases the risk to rail safety. The solution in this invention employs a different technical concept: first, identifying structurally stable and characteristic fasteners; then, utilizing the fastener-bolt spatial constraint relationship; dynamically calculating the bolt's location; and finally, precisely locating the bolt within a small area.

[0080] The purpose of this invention is to transform the identification of a small, easily disturbed target into the identification of a stable, large target and geometric deduction, which is a spatial relationship-anchored indirect identification method. Fasteners, as fixing components of railway tracks, are larger in size, have obvious structural features, are less affected by oil contamination, and their detection stability is far higher than that of bolts.

[0081] The problem with existing technologies is that bolts, being small targets, occupy a very small percentage of pixels in an image, making it difficult for existing target detection algorithms to reliably detect them under conditions such as oil stains, changes in lighting, and vehicle vibration. This application addresses the inherent limitations of direct detection of small targets by employing indirect spatial reasoning to achieve stable and accurate bolt positioning in dynamic environments.

[0082] Specifically, the first-level detection model performs spatial anchoring, identifying the fastener bounding box and category identifier; the second-level detection model performs fine positioning within the local area where the fastener is anchored, generating and accurately locating the bolt detection area based on spatial relationships. There is a clear division of functions and information transmission between the two, which is not simply a matter of connecting two existing models, but rather the construction of a new spatial perception method.

[0083] This application reduces the search range of the second-level detection model from the entire image to a local area near the fastener by using fastener bounding boxes and spatial mapping. Assuming the image size is 1920×1080, and the fastener area occupies approximately 5%~10% of the image area, the search range is reduced to 1 / 10~1 / 20 of the original, significantly decreasing the false negative rate and significantly improving the positioning accuracy.

[0084] It is important to emphasize that when identifying fasteners, one can identify the fastener as a whole or directly identify the elastic clip within the fastener. The elastic clip's shape is relatively easier to identify than that of a bolt. In railway fastening systems, the elastic clip has a specific elongated, curved rod-like structure with highly geometrical specificity (specific aspect ratio, radius of curvature, and spatial extension direction), maintaining a recognizable outline even in harsh environments such as oil stains and corrosion. In contrast, bolts have a near-circular, compact structure with a single feature dimension. Once the surface texture is covered by oil or the edges are blurred by corrosion, their visual recognizability drops sharply. This application utilizes the elastic clip, with its higher morphological recognizability and stronger anti-interference ability, as the initial anchoring target, rather than directly detecting bolts whose shape is easily degraded and confused. This constitutes a fundamental difference in technical approach from existing technologies that directly detect bolts.

[0085] More preferably, the first-level detection model is a lightweight object detection model enhanced by an attention mechanism, and the first-level detection model includes a convolutional neural network and a weight calculation network; The step of identifying the initial detection image using a first-level detection model and determining the fastener bounding box and fastener category identifier in the initial detection image includes: The initial feature map is obtained by extracting features from the initial detection image using the convolutional neural network. The initial feature map is forward-propagated in the convolutional neural network to obtain a multi-scale feature map generated by a preset convolutional layer based on the initial feature map. The multi-scale feature map is input into a weight calculation network, which calculates the weight value of each spatial location in the initial detection image. The weight value represents the probability confidence that each spatial location belongs to the fastener region. An attention weight map of the initial detection image is constructed based on the obtained weight values. The attention weight map is a weight matrix with the same spatial resolution as the initial detection image. The weight values ​​of the fastener candidate region in the attention weight map are higher than those of the non-fastener region. The initial detection image is fused with the attention weight map pixel by pixel to obtain a weighted initial detection image; The weighted initial detection image is input into the detection head of the convolutional neural network for target detection, and the output fastener bounding box and fastener category identifier are generated.

[0086] In railway scenarios, fasteners are often obscured by oil, rust, or shadows. Traditional CNNs treat all pixels equally, easily misclassifying obscured objects as background. This solution generates an attention weight map through a weighted calculation network, actively strengthening candidate fastener regions (with high weight values) while suppressing irrelevant backgrounds (rails, ballast). This ensures that even if the fastener's appearance is blurred, its spatial location features can still be perceived by the network, significantly reducing the false negative rate and preventing the inability to generate subsequent bolt detection regions due to missed fasteners.

[0087] Non-fastener areas (such as metal reflections and gravel on the track) are assigned low weights in the attention weight map. Even if the detection head responds in these areas, its contribution is attenuated after pixel-by-pixel weighted fusion. Through a soft masking mechanism, false targets are effectively filtered out, improving the confidence purity of fastener category identification and reducing resource waste caused by generating invalid bolt detection areas due to false detections.

[0088] Conventional detection models simultaneously perform the dual tasks of location finding and classification, which often interfere with each other. In this proposed solution, the weighted calculation network focuses on spatial location confidence (probability map), while the detection head focuses on bounding box regression and classification. This decoupled design allows different parts of the network to perform their respective functions, reducing optimization difficulty and enabling the model to maintain high accuracy with lightweight parameters, making it suitable for deployment on edge computing devices in fuel injection vehicles.

[0089] The solution in this invention explicitly extracts multi-scale feature maps (from preset convolutional layers) and calculates weights based on these feature maps. This means that the attention weight map integrates global semantic (deep) and local detail (shallow) information. This makes the model more robust to different shooting distances or changes in fastener size (such as sudden changes in target scale caused by vehicles going uphill or downhill), ensuring that fasteners at both near and far distances receive reasonable attention allocation.

[0090] The solution of this invention allows the attention weight calculation network and the detection head to perform joint backpropagation optimization under the same loss function, so that the generation of the attention weight map can be adaptively adjusted based on the final detection accuracy, rather than relying on manually preset rules.

[0091] More preferably, the convolutional neural network of the first-level detection model is an improved YOLOv8n, whose backbone network adopts the CSPDarknet53 structure, and whose neck network adopts a weighted bidirectional feature pyramid network for cross-scale feature fusion; the multi-scale feature map includes a first-scale feature map, a second-scale feature map and a third-scale feature map, which correspond to fastener targets of different scales in the initial detection image, respectively.

[0092] In practical implementation, YOLOv8n is the version with the fewest parameters and lowest computational cost in the YOLOv8 series (approximately 3.2M parameters). Combined with the cross-stage partial connection structure of CSPDarknet53, it ensures the effective flow of gradient information while reducing redundant gradient calculations. This allows the model to be deployed on embedded or edge computing devices in intelligent fuel injection vehicles, maintaining a high frame rate even with limited computing power, meeting the real-time processing requirements while in motion. Simultaneously, the feature reuse capability of the CSP structure prevents a sharp drop in accuracy due to its lightweight design.

[0093] Because the scale of fasteners in an image varies drastically with the distance between the fuel injection truck and the track (larger when closer, smaller when farther away), traditional FPN only performs top-down unidirectional fusion, limiting information flow. BiFPN introduces bidirectional (top-down and bottom-up) weighted fusion and learns trainable weight parameters for each fusion path to distinguish the importance of features at different levels. This allows small-scale fasteners (at a distance) to obtain high-resolution shallow detail information (edges, textures), while large-scale fasteners (at a distance) can obtain deep semantic information (class discrimination), achieving stable detection across all distance scales and avoiding missed detections due to distance variations.

[0094] The solution of this invention achieves accurate positioning of coarse, medium, and fine targets (three-scale feature map division of labor). The feature map at each scale is enhanced with BiFPN cross-scale information. This forms a mechanism of collaborative division of labor among the three-level detection heads. Compared with single-scale or dual-scale detection, it significantly improves the bounding box regression accuracy of fasteners of various sizes, ensuring that the output fastener bounding box closely fits the actual target, and providing a more accurate spatial reference for the subsequent dynamic generation of bolt detection areas.

[0095] In practice, BiFPN's learnable weights not only act between different feature layers but also implicitly suppress the propagation of background noise features. For example, when a feature map in a certain layer is rich in rail texture rather than fastener semantics, its fusion weights will be trained to be lower. This enhances the model's feature purity in complex railway backgrounds (ballast, glare, oil stains), enabling the subsequent attention weight calculation network to generate accurate spatial probability maps based on cleaner multi-scale features.

[0096] More preferably, the second-level detection model is a keypoint detection model based on heatmap regression, which includes a convolutional neural network with an encoder-decoder structure and a heatmap prediction head; The bolt detection area is input into the second-level detection model for bolt identification to obtain the target position of the bolt at the corresponding location, including: The local image after cropping the bolt detection area is input into the convolutional neural network of the encoder-decoder structure for feature extraction and upsampling reconstruction, and the output is a feature reconstruction map with the same spatial resolution as the local image. The feature reconstruction map is input into the heatmap prediction head, and the output is a heatmap of the bolt center point. The value of each pixel in the heatmap of the bolt center point represents the probability that the position is the bolt center point. The position with the largest response value in the heat map of the bolt center point is obtained as the pixel-level coordinate of the bolt center point; Using the pixel-level coordinates as the center, Gaussian surface fitting is performed on the thermal response values ​​within its neighborhood, and the peak position of the Gaussian surface is taken as the target bolt position of the bolt center point.

[0097] Traditional object detection (such as YOLO) requires pre-setting a large number of anchor boxes on the feature map and regressing the width and height offsets of the bounding boxes, which is computationally intensive and requires manual parameter tuning. Heatmap regression, on the other hand, is essentially pixel-by-pixel binary classification and coordinate regression. The encoder-decoder network only needs to output a single-channel heatmap, without needing to pre-set any prior boxes. This allows the second-level model to be designed to be extremely lightweight, with very fast inference speed on cropped local images, ensuring ultra-low latency response during the movement of the fuel-injecting vehicle.

[0098] Conventional detection heads directly output integer pixel coordinates, resulting in quantization errors. The solution in this invention first extracts the integer positions of the peak values ​​from the heatmap, then performs Gaussian surface fitting on the neighborhood response values, using a continuous function to find the extreme points, thus improving coordinate accuracy to the sub-pixel level. For small targets like bolts, sub-pixel accuracy can reduce positioning errors, directly determining the subsequent fuel injector alignment accuracy and preventing lubricant spray deviation due to pixel-level deviations.

[0099] Bolts in images may have blurred edges or even be partially obscured due to rust, oil, or other contaminants. Traditional bounding box regression relies on sharp edges, making it prone to failure. Heatmaps, however, predict the probability density distribution of the bolt's central region; even with contaminated edges, the Gaussian response of the central region remains stable. The model learns the semantic understanding of where the bolt's center should be, rather than the geometric fit of where the bolt's edges are, significantly improving detection robustness and continuity under harsh conditions.

[0100] The encoder is responsible for downsampling to extract high-level semantic features (compressing the receptive field), while the decoder upsamples to restore the resolution to the input size, outputting a reconstructed feature map. This process decouples semantic understanding from spatial localization, and the reconstructed map remains spatially aligned with the local image. This allows the heatmap prediction head to perform probability estimation simply by performing pixel-wise convolutions on the spatially aligned feature map, greatly simplifying training, resulting in fast convergence, and insensitivity to scale changes in the input image.

[0101] During training, only the coordinates of the bolt center point (single pixel) need to be labeled, rather than the complete bounding box (which requires 4.8 coordinates). Furthermore, the Gaussian kernel radius of the heatmap can be adaptively adjusted. Compared to traditional detection box labeling, the labeling cost is reduced, and the labeling consistency is higher, facilitating rapid expansion of the training dataset to adapt to the recognition needs of different lines and bolt models.

[0102] In specific implementation, the fuel injection strategy generation operation of this embodiment of the invention further includes temperature-viscosity compensation logic, which is configured as follows: Obtain the temperature value from the current ambient temperature and humidity data; Based on the pre-stored database of oil viscosity-temperature-optimal oil film thickness relationship, query the viscosity compensation coefficient corresponding to the temperature value; The target injection quantity is dynamically adjusted based on the viscosity compensation coefficient. When the temperature value is lower than a preset temperature threshold, the target injection quantity is increased to compensate for the decrease in oil fluidity caused by low temperature. This method achieves more precise injection operation.

[0103] In practical implementation, the first sampling data of the inertial measurement unit and the second sampling data of the odometer can be obtained. The global image, local image, first sampling data and second sampling data are assigned timestamps under a unified time reference and spatiotemporally aligned. Based on the spatiotemporally aligned data, the bolt target position is continuously tracked through a Kalman filter. Forward prediction is performed with the system full-link delay as the prediction step size. The forward prediction result is corrected in real time by combining the vehicle body vibration information detected by the inertial measurement unit to obtain the bolt prediction position at the moment of injection execution. Based on the predicted bolt position, the adjustable nozzle is driven to adjust to the target spray posture, triggering the high-pressure supply device to spray grease into the contact area between the bolt and the fastener spring according to the target spray parameters.

[0104] Based on the real-time detected oil temperature and a pre-established mathematical model of grease temperature-viscosity-injection pressure, the output pressure of the high-pressure injection pump is controlled by feedforward, and the grease temperature between the oil storage container and the nozzle is maintained within a predetermined range through a full-link heat tracing pipeline.

[0105] The dynamic spraying control method based on an intelligent spraying vehicle in this embodiment of the invention uses a fusion analysis of vision (two-dimensional images) and lidar (three-dimensional point clouds) to accurately identify the individual corrosion degree and surface condition of each bolt, rather than treating the entire area indiscriminately. This improves the accuracy of spraying protection.

[0106] Example 2 Please see Figure 3 , Figure 3 This is a schematic diagram of the dynamic spraying control system based on an intelligent spraying vehicle disclosed in an embodiment of the present invention. Figure 3 As shown, the dynamic spraying control system based on the intelligent spraying vehicle may include: Acquisition module: used to acquire two-dimensional color texture images of the target bolt surface through the image acquisition module and to acquire three-dimensional laser point cloud data of the target bolt through the lidar; First analysis module: used to analyze the two-dimensional color texture image through HSV color space analysis and determine the corrosion level of the target bolt surface; The second analysis module is used to determine the corrosion status information of the target bolt surface based on the three-dimensional laser point cloud data. The generation module is used to generate an oil spraying strategy for the target bolt based on the rust level and rust state information, combined with the current ambient temperature and humidity data, through a rust-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. Injection module: Used to generate control signals according to the injection strategy to drive the injector to perform the injection operation on the target bolt.

[0107] The dynamic spraying control method based on an intelligent spraying vehicle in this embodiment of the invention uses a fusion analysis of vision (two-dimensional images) and lidar (three-dimensional point clouds) to accurately identify the individual corrosion degree and surface condition of each bolt, rather than treating the entire area indiscriminately. This improves the accuracy of spraying protection.

[0108] Example 3 Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device can be a computer, a server, etc. Of course, in certain cases, it can also be a mobile phone, tablet computer, monitoring terminal, or other smart device, as well as an image acquisition device with processing capabilities. Figure 4 As shown, the electronic device may include: Memory 510 storing executable program code; Processor 520 coupled to memory 510; The processor 520 calls the executable program code stored in the memory 510 to execute some or all of the steps in the dynamic spraying control method based on the intelligent spraying vehicle in Embodiment 1.

[0109] This invention discloses a computer-readable storage medium storing a computer program that enables a computer to perform some or all of the steps in the dynamic spraying control method based on an intelligent spraying vehicle in Embodiment 1.

[0110] This invention also discloses a computer program product, wherein when the computer program product is run on a computer, the computer executes some or all of the steps in the dynamic spraying control method based on an intelligent spraying vehicle in Embodiment 1.

[0111] This invention also discloses an application publishing platform, which is used to publish computer program products. When the computer program products are run on a computer, the computer executes some or all of the steps in the dynamic spraying control method based on an intelligent spraying vehicle in Embodiment 1.

[0112] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of the present invention.

[0116] In the embodiments provided by this invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0117] Those skilled in the art will understand that some or all of the steps in the various methods of the embodiments described can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0118] The above provides a detailed description of the dynamic spraying control method, system, electronic device, and storage medium based on an intelligent spraying vehicle disclosed in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dynamic spraying control method based on an intelligent spraying vehicle, characterized in that, include: The image acquisition module acquires two-dimensional color texture images of the target bolt surface, and the lidar acquires three-dimensional laser point cloud data of the target bolt. The two-dimensional color texture image was analyzed using HSV color space analysis to determine the corrosion level of the target bolt surface; The corrosion status information of the target bolt surface is determined based on the three-dimensional laser point cloud data; Based on the corrosion level and corrosion status information, combined with the current ambient temperature and humidity data, the corresponding oil spraying strategy for the target bolt is generated through the corrosion-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. A control signal is generated according to the oil injection strategy to drive the oil injector to perform oil injection operation on the target bolt; after acquiring the two-dimensional color texture image of the target bolt surface through the image acquisition module and the three-dimensional laser point cloud data of the target bolt through the lidar, the method further includes: Obtain calibration parameters between the image acquisition module and the lidar, the calibration parameters including rotation matrix and translation vector; According to the calibration parameters, each spatial point in the three-dimensional laser point cloud data is projected onto the pixel coordinate system of the two-dimensional color texture image to establish a point-by-point mapping relationship between the three-dimensional laser point cloud data and the two-dimensional color texture image. Based on the point-to-point mapping relationship, the rust level information and rust pit depth information of the target bolt surface are fused at the pixel level to generate a rust state fusion map. Each pixel position in the rust state fusion map contains both a rust level label and a depth value label. Based on the rust state fusion map, the surface of the target bolt is divided into regions, and multiple sub-regions with different combinations of rust levels and rust pit depths are identified. The expected oil spraying demand for each sub-region is calculated separately, and the expected oil spraying demand for each sub-region is positively correlated with the rust grade and average rust pit depth of the sub-region. The estimated oil injection requirements of all sub-regions are summed to generate the comprehensive estimated oil injection requirement of the target bolt. When driving the injector to perform the injection operation, the injection pulse width and injection pressure are controlled according to the comprehensive expected injection demand, and the scanning trajectory of the injector is controlled according to the spatial distribution of each sub-region, so as to achieve differentiated spraying on the surface of the target bolt. The method further includes: The system acquires the first sampled data from the inertial measurement unit and the second sampled data from the odometer. It assigns timestamps under a unified time reference to the global image, local image, first sampled data, and second sampled data and performs spatiotemporal alignment. Based on the spatiotemporally aligned data, it continuously tracks the bolt target position through a Kalman filter. It performs forward prediction with the system's full-link delay as the prediction step size and combines the vehicle body vibration information detected by the inertial measurement unit to correct the forward prediction result in real time, thereby obtaining the bolt prediction position at the moment of injection execution.

2. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 1, characterized in that, The corrosion status information includes the depth of the corrosion pits; The step of determining the corrosion state information of the target bolt surface based on the three-dimensional laser point cloud data further includes: Statistical filtering and radius filtering are performed on the three-dimensional laser point cloud data; Perform plane fitting on the filtered point cloud to determine the reference plane of the target bolt surface; Calculate the normal distance of each point in the filtered point cloud relative to the reference plane to generate a depth map of the target bolt surface; Based on the depth map, the maximum depression depth, average depression depth, and total area of ​​the depression region on the surface of the target bolt are extracted, and the depth of the rust pit is determined based on the maximum depression depth and the average depression depth.

3. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 2, characterized in that, The corrosion status information also includes surface porosity characteristics; these surface porosity characteristics are calculated as follows: Perform connected component analysis on the depth map to identify all depression regions whose depth values ​​exceed a preset depth threshold; Calculate the area and volume of each of the said recessed regions, whereby the volume is obtained by integrating the depth value of the corresponding region in the depth map; The ratio of the total volume of all recessed areas to the theoretical smooth surface area of ​​the region of interest in the bolt head is used as the first porosity index. The ratio of the sum of the areas of all recessed regions to the total area of ​​the region of interest in the bolt head is used as the second porosity index. The weighted combination of the first porosity index and the second porosity index is output as the surface porosity feature.

4. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 2, characterized in that, The spraying control method further includes: Read the historical depth map and historical corrosion level of the same target bolt at the historical operation time from the bolt cascade database; The current depth map is compared with the historical depth map point by point to generate a corrosion depth variation map; The corrosion rate index of the target bolt surface during the interval between two operations is calculated based on the corrosion depth variation diagram. The corrosion rate index is the average increment of the corrosion pit depth per unit time. When the corrosion rate index exceeds the preset rate threshold, the target bolt is marked as an accelerated corrosion target in the target list to be sprayed, and the maintenance priority of the target bolt is increased in the subsequent oil spraying strategy generation. The target list to be sprayed is used to display the target bolts in a list.

5. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 1, characterized in that, The step of analyzing the two-dimensional color texture image using HSV color space analysis and determining the corrosion level of the target bolt surface includes: The two-dimensional color texture image is converted from the RGB color space to the HSV color space, and the hue component, saturation component and lightness component are extracted. Histogram statistics are performed on the hue components to identify the distribution ratio of each pixel on the surface of the target bolt in the orange-yellow, reddish-brown, and dark brown ranges on the hue ring; The corrosion color feature vector is calculated based on the distribution ratio, and the corrosion color feature vector includes the proportion of orange-yellow, reddish-brown and blackish-brown. The corrosion color feature vector is input into a pre-trained corrosion grading classifier, which outputs the initial corrosion level of the target bolt surface. The training process of the pre-trained rust grading classifier includes: Multiple color images of bolt surfaces labeled with corrosion grade tags were collected. The corrosion grade tags were either manually labeled or calibrated by a laboratory corrosion degree measuring instrument. For each sample, extract the histogram of hue components, the mean value of saturation components, and the mean value of lightness components in the HSV color space to form a training feature set; Using the training feature set as input and the corrosion level label as supervision signal, a support vector machine classifier or a lightweight convolutional neural network classifier is trained, and the classifier hyperparameters are optimized through cross-validation during the training process. The trained classifier is deployed on the edge computing unit for real-time corrosion level determination.

6. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 1, characterized in that, The spraying control method further includes: Based on the first detection component mounted on the first-level vehicle, the bolts along the route are quickly located and their initial condition is judged, and a list of targets to be sprayed is generated. The list of targets to be sprayed includes the geographical location information of the bolts suspected of needing maintenance and a rough rust status mark; wherein, the first-level vehicle travels within a first speed threshold range; The second-level vehicle receives the list of targets to be sprayed sent by the first-level vehicle and navigates sequentially to the locations of suspected bolts requiring maintenance based on the geographical location information in the list. This triggers the edge computing unit to perform a dynamic spraying operation. The second-level vehicle travels within a second speed threshold range and is equipped with a second perception kit and an execution kit. The second perception kit includes an image acquisition module and a lidar, and the execution kit includes fuel injectors. The upper limit of the second speed threshold range is lower than the lower limit of the first speed threshold range.

7. The dynamic spraying control method based on an intelligent spraying vehicle as described in claim 1, characterized in that, Before acquiring the two-dimensional color texture image of the corresponding target bolt surface through the image acquisition module, the method further includes: The initial detection image captured during the movement of the intelligent fuel injection vehicle is acquired. The initial detection image is then identified using a first-level detection model, and the fastener boundary box and fastener category identifier in the initial detection image are determined. The spatial relationship between fasteners and bolts is determined based on the fastener category identifier and the pre-built fastener-bolt mapping table, and the bolt detection area is dynamically generated based on the fastener boundary box and the spatial relationship between fasteners and bolts. The bolt detection area is input into the second-level detection model for bolt identification to obtain the current status information of the bolt at the corresponding position and the target position of the bolt; Output the target position of the bolt and the current status information of the bolt.

8. A dynamic spraying control system based on an intelligent spraying vehicle, characterized in that, include: Acquisition module: used to acquire two-dimensional color texture images of the target bolt surface through the image acquisition module and to acquire three-dimensional laser point cloud data of the target bolt through the lidar; First analysis module: used to analyze the two-dimensional color texture image through HSV color space analysis and determine the corrosion level of the target bolt surface; The second analysis module is used to determine the corrosion status information of the target bolt surface based on the three-dimensional laser point cloud data. The generation module is used to generate an oil spraying strategy for the target bolt based on the rust level and rust state information, combined with the current ambient temperature and humidity data, through a rust-oil film coupled decision model. The oil spraying strategy includes the target oil spraying quantity and oil spraying pressure parameters. The oil injection module is used to generate control signals according to the oil injection strategy to drive the oil injector to perform oil injection operation on the target bolt; after the image acquisition module acquires the two-dimensional color texture image of the target bolt surface and the lidar acquires the three-dimensional laser point cloud data of the target bolt, it also includes: Obtain calibration parameters between the image acquisition module and the lidar, the calibration parameters including rotation matrix and translation vector; According to the calibration parameters, each spatial point in the three-dimensional laser point cloud data is projected onto the pixel coordinate system of the two-dimensional color texture image to establish a point-by-point mapping relationship between the three-dimensional laser point cloud data and the two-dimensional color texture image. Based on the point-to-point mapping relationship, the rust level information and rust pit depth information of the target bolt surface are fused at the pixel level to generate a rust state fusion map. Each pixel position in the rust state fusion map contains both a rust level label and a depth value label. Based on the rust state fusion map, the surface of the target bolt is divided into regions, and multiple sub-regions with different combinations of rust levels and rust pit depths are identified. The expected oil spraying demand for each sub-region is calculated separately, and the expected oil spraying demand for each sub-region is positively correlated with the rust grade and average rust pit depth of the sub-region. The estimated oil injection requirements of all sub-regions are summed to generate the comprehensive estimated oil injection requirement of the target bolt. When driving the injector to perform the injection operation, the injection pulse width and injection pressure are controlled according to the comprehensive expected injection demand, and the scanning trajectory of the injector is controlled according to the spatial distribution of each sub-region, so as to achieve differentiated spraying on the surface of the target bolt. The system also includes: The system acquires the first sampled data from the inertial measurement unit and the second sampled data from the odometer. It assigns timestamps under a unified time reference to the global image, local image, first sampled data, and second sampled data and performs spatiotemporal alignment. Based on the spatiotemporally aligned data, it continuously tracks the bolt target position through a Kalman filter. It performs forward prediction with the system's full-link delay as the prediction step size and combines the vehicle body vibration information detected by the inertial measurement unit to correct the forward prediction result in real time, thereby obtaining the bolt prediction position at the moment of injection execution.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program causes a computer to execute the dynamic spraying control method based on an intelligent spraying vehicle as described in any one of claims 1 to 7.

Images