A method and device for automatically starting and stopping a lance based on path node information

Abstract

The application provides a method and device for automatically starting and stopping a spray gun based on path node information, and relates to the technical field of automatic control. The method for automatically starting and stopping a spray gun based on path node information comprises the following steps: collecting three-dimensional point cloud data of a target work object; constructing an initial three-dimensional geometric model of the target work object based on the three-dimensional point cloud data, and planning and generating an initial work path based on the initial three-dimensional geometric model; detecting and inserting one or more geometric feature nodes on the initial work path according to the geometric features of the initial three-dimensional geometric model; configuring a work parameter set containing a spraying instruction for the geometric feature nodes; and controlling a UAV to fly along the initial work path, and when the UAV reaches the geometric feature nodes, performing a spraying start-stop operation according to the spraying instruction in the work parameter set.

Description

Technical Field

[0001] This invention relates to the field of automation control technology, and more particularly to a method and apparatus for automatically starting and stopping spray guns based on path node information. More specifically, this invention relates to a method and apparatus for path generation, node information extraction, and dynamic control of operation parameters for automated spraying, cleaning, and other operations on the surface of large structures, combining three-dimensional geometric information and machine vision information. Background Technology

[0002] Large metal structures, such as ocean-going vessels, large storage tanks, bridge steel structures, and power plant cooling towers, are exposed to complex natural environments for extended periods, making their surfaces highly susceptible to corrosion and contamination. To ensure structural safety and extend their service life, regular maintenance operations such as rust removal, cleaning, and application of anti-corrosion coatings are necessary. Traditional maintenance methods rely primarily on manual labor, typically employing scaffolding or the use of slings for high-altitude work. This approach has obvious limitations: working at height carries extremely high safety risks, easily leading to injuries or fatalities due to careless operation or sudden environmental changes; simultaneously, it is inefficient, limited by manpower, weather, and the working environment, resulting in long construction cycles and high labor costs; furthermore, ensuring uniform quality is difficult, as key indicators such as coating thickness and uniformity highly depend on the experience and responsibility of the workers, easily leading to problems such as missed areas, over-spraying, or uneven coating, thus affecting the protective effect.

[0003] To overcome the many drawbacks of manual methods, automated and intelligent operation technologies have emerged. Among them, drones, as a flexible and efficient aerial mobile platform, have shown great application potential in the surface maintenance of large structures. Existing drone painting technology has solved safety and accessibility issues to some extent, but its intelligence and precision are still insufficient, especially in the intelligent aspects of path planning and operation execution, where there are urgent technical bottlenecks to be addressed. Early drone operation schemes mostly relied on remote control by pilots. While this method is intuitive, its quality and efficiency are still strongly correlated with the pilot's skill and fatigue level, making it difficult to achieve standardized, high-quality continuous operation. Furthermore, flight stability and safety are challenged near complex structures with poor GPS signals.

[0004] To improve automation, subsequent technical solutions introduced autonomous path planning. This type of technology typically first scans the target structure using sensors to build a digital twin model, and then plans a flight path covering the entire surface based on this model. However, this purely geometric-based path planning method has deep limitations. The path planning is based solely on the macroscopic geometric contour of the target, treating the work surface as a homogeneous, undifferentiated geometric surface. This planning method cannot perceive or respond to the actual working conditions of the structure's surface, such as localized severe corrosion, weld areas, oil stains, or salt deposits. Therefore, the drone can only perform a one-size-fits-all, uniform-speed, constant-flow spraying, unable to perform differentiated operations on areas requiring focused treatment, leading to paint waste and a weak link in protective effectiveness.

[0005] This type of technology lacks the ability to perceive and avoid fine obstacles in the environment. Small attachments on the surface of structures, such as railings, cables, exposed pipes, or small sensors, are often overlooked due to scanning accuracy issues, leading to distorted models. Paths planned based on these distorted models pose a significant collision risk in actual execution. Even when some obstacles are identified, existing technologies often employ a conservative strategy of large-scale avoidance, creating large blind spots around the obstacles and reducing operational coverage, requiring manual re-spraying. Furthermore, current path planning and operation execution are separate and unidirectional, lacking a mechanism for deep integration and dynamic adjustment of geometric path planning with actual operational requirements. It also lacks a closed-loop process for verifying and iteratively optimizing the accuracy of the planned path and model, resulting in the overall robustness, efficiency, and operational quality of the automated operation system falling far short of ideal levels. Summary of the Invention

[0006] This invention provides a method for automatically starting and stopping a spray gun based on path node information. The method includes the following steps: acquiring three-dimensional point cloud data of the target work object and simultaneously acquiring a high-resolution image sequence of the target work object; constructing an initial three-dimensional geometric model of the target work object based on the three-dimensional point cloud data, and planning and generating an initial work path based on the initial three-dimensional geometric model; detecting and inserting one or more geometric feature nodes on the initial work path according to the geometric features of the initial three-dimensional geometric model; configuring a set of work parameters containing spraying instructions for the geometric feature nodes; controlling the drone to fly along the initial work path, and performing a spraying start and stop operation according to the spraying instructions in the set of work parameters when reaching the geometric feature node.

[0007] Based on high-resolution image sequences, a surface semantic map is generated, and based on the surface semantic map, a hierarchical panoramic image path is planned and generated. ;

[0008] Panoramic Image Path The generation involves ranking process patches in the surface semantic map, which is achieved by minimizing the scheduling cost function within a micro-region. Cost function:

[0009]

[0010] in, It refers to the arrangement of patches within region s. It is the total number of process patches within the region. and These are the weighting coefficients. This is the normalized geodesic distance. A function to quantify the adjustment range of operation parameters. and These are the process category labels corresponding to the b-th and b+1-th process patches, respectively. and These represent the centroids of the b-th and b+1-th process patches, respectively.

[0011] Based on the initial 3D geometric model, plan and generate a 3D point cloud path. .

[0012] Before generating the final job path, it also includes: the initial 3D geometric model On the surface, according to any point Based on local characteristics, calculate a dynamically changing fusion weight function in space. The fusion weight function is determined by the following formula:

[0013]

[0014] in, For point The normalized geometric complexity index at the location, For point The normalized process complexity index at the location, and Weights are positive constants. Based on the bias weights.

[0015] Normalized geometric complexity index Based on the initial three-dimensional geometric model At point The two principal curvatures at the point and Obtained through calculation and normalization;

[0016] Normalized process complexity index Based on point The radius on the surface is Within the neighborhood of, statistical panoramic image paths The number of defined process patches is obtained by calculation and normalization.

[0017] Panoramic Image Path The generation is based on the generation of 3D point cloud paths. The process involves segmenting multiple sub-regions with different technological properties, which form the framework; panoramic image path. The planning adopts a hierarchical approach, which includes:

[0018] a) Perform macro-regional scheduling between sub-regions to determine a region access sequence that minimizes the cost of job mode switching;

[0019] b) Within each sub-region, micro-region scheduling is performed, and a process patch execution sequence is planned based on the surface semantic map to minimize the adjustment of operation parameters and movement distance.

[0020] Function for quantifying the adjustment range of operation parameters Defined as two process categories and The corresponding normalized parameter vector and Euclidean distance between them:

[0021] .

[0022] In the generated final job path The above extracts the composite key nodes, including:

[0023] Along the final job path The process involves iterating through the nodes and, when a position that meets the preset triggering conditions is detected, inserting a key node containing the node type and metadata at that position.

[0024] The types of key nodes include at least:

[0025] a) Based on the final job path Its own local curvature or its underlying three-dimensional geometric model Surface Gaussian curvature Geometric feature nodes that are triggered when the threshold is exceeded. Indicates the arc length along the final operation path;

[0026] b) When the final job path When crossing the boundaries of different process patches, the process boundary switching node triggered by the surface semantic map contains the metadata of the process category being entered and the process category being exited.

[0027] c) When the final job path The distance from the geodesic line of the predefined paint partition feature line is less than the warning threshold. When, the composite constraint node is triggered.

[0028] The present invention also provides a device for automatically starting and stopping spray guns based on path node information, the device comprising:

[0029] Acquisition module: Acquires 3D point cloud data of the target object;

[0030] Path generation module: Based on 3D point cloud data, constructs an initial 3D geometric model of the target operation object, and plans and generates an initial operation path based on the initial 3D geometric model;

[0031] Node extraction module: On the initial job path, based on the geometric features of the initial 3D geometric model, detect and insert one or more geometric feature nodes; configure a set of job parameters containing spraying instructions for the geometric feature nodes;

[0032] Execution module: controls the UAV to fly along the initial operation path, and when it reaches the geometric feature node, executes the spraying start and stop operation according to the spraying instructions in the operation parameter set.

[0033] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention provides a highly intelligent and adaptive automated operation method for unmanned aerial vehicles (UAVs). Through the deep dynamic integration of geometric safety planning and process flow planning, it significantly improves the overall efficiency, quality, and safety of surface maintenance operations on large structures. Firstly, this invention employs a parallel planning design for 3D point cloud paths and panoramic image paths. The generation of the 3D point cloud path is deeply coupled with specific geometric constraints, such as coating partition feature lines and hydrodynamic critical areas, thereby pre-planning a flight trajectory that meets all safety and quality requirements at the macroscopic geometric level. Simultaneously, the generation of the panoramic image path is entirely based on the perspective of operational process optimization, intelligently sorting and scheduling surface defects and working conditions. More importantly, this invention fundamentally ensures the consistency of the two paths in macroscopic logic by using the sub-regions defined in the 3D point cloud path planning stage as the prior framework for panoramic image path planning. This avoids potential regional conflicts and constraint violations during subsequent fusion, greatly enhancing the robustness and reliability of the entire system.

[0034] Unlike existing technologies that simply overlay paths or assign static weights, this invention introduces a spatially dynamically changing fusion weight function. This function can perceive the local characteristics of the area where the path points are located in real time. In areas with rapidly changing geometry, such as the bulbous bow, the weight automatically favors the strategy represented by the 3D point cloud path, which prioritizes safety and precise fit. This ensures stable UAV attitude and constant spraying distance, thereby guaranteeing the coating quality of critical areas. In areas with dense rust patches, such as the sides of the hull, the weight automatically favors the strategy represented by the panoramic image path, which prioritizes process efficiency. This guides the final path to strictly follow the optimal task execution sequence, minimizing ineffective adjustments to equipment parameters and long-distance unloaded flights, significantly improving overall operational efficiency. This adaptive, site-specific fusion method ensures that the final generated operational path is not a simple compromise between two single strategies, but rather achieves a dynamic optimal balance of safety, quality, and efficiency in each local area.

[0035] Furthermore, this invention extracts complex key nodes on the final generated path, providing a solid foundation for refined process control. By precisely marking event points such as process boundary switching, geometric sharp turns, and approach to critical feature lines on the path, the system can transform a continuous flight plan into a series of discrete and explicit control commands. This enables the UAV not only to fly along the optimal route but also to execute the most appropriate operations at the most precise locations. For example, it can simultaneously adjust its speed reduction and flow rate increase the instant it enters a heavily corroded area, thereby achieving adaptive processing for different operating conditions, ensuring the uniformity and effectiveness of the coating, and reducing paint waste. Finally, by compiling these information-rich nodes into a time-synchronized instruction sequence executable by the airborne system, this invention constructs a complete closed loop from high-level planning to low-level control, ensuring high-precision reproduction of complex intelligent algorithms in the real physical world and comprehensively improving the level of automated operations. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the automatic start / stop of the spray gun based on path node information in this application. Detailed Implementation

[0037] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0039] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0040] Example 1

[0041] This embodiment provides a method for automatically starting and stopping spray guns based on path node information. This method is mainly applied to the automated anti-corrosion coating spraying operation on the outer plating of large ships.

[0042] Collect 3D point cloud data;

[0043] In this step, an industrial-grade drone platform with high-precision positioning capabilities is deployed. Preferably, a quadcopter with an octagonal rotor and a rated payload capacity of 50 kg is deployed. The drone integrates a data acquisition system, which includes at least: a high-frequency LiDAR scanner, an inertial measurement unit (IMU) connected to the drone's flight control system, and a Global Navigation Satellite System (GNSS) receiver supporting real-time dynamic differential (RTK) technology to ensure that the acquired attitude information achieves centimeter-level accuracy.

[0044] Before data acquisition begins, a three-dimensional cuboid bounding box is established centered on the ship to be painted. The UAV, in autonomous flight mode, maintains a constant safe distance along the outside of this bounding box, performing a single "bow"-shaped reciprocating scan to initially acquire the overall outline data of the ship's main body. Subsequently, to ensure the data integrity of complex curved areas such as the bow and stern, the UAV performs two more orbital scans around the ship at different altitudes. Throughout the flight, the lidar continuously emits laser beams and receives reflected signals, forming dense raw point cloud data. Simultaneously, the RTK-GNSS / IMU integrated navigation system records the UAV's six-degree-of-freedom pose information (including three-dimensional coordinates (x, y, z) and attitude angles (roll, pitch, yaw)) in the global coordinate system at a frequency of 100Hz, and strictly timestamps and binds each frame of point cloud data with the pose information at the corresponding moment. After acquisition, a raw point cloud dataset containing a series of precise pose labels is output.

[0045] Constructing a 3D model and path planning;

[0046] This step is performed on a ground workstation to process and analyze the collected raw point cloud dataset in order to generate an executable job path.

[0047] Point cloud preprocessing: A statistical outlier removal algorithm is applied to all raw point cloud data to remove isolated noise points caused by airborne particles or measurement errors. Next, to improve subsequent processing efficiency, a voxel downsampling method is used to adjust the point cloud density to 5000 points per cubic meter, reducing data redundancy while preserving key geometric features.

[0048] Point cloud registration and fusion: Using the RTK-IMU pose information recorded during data acquisition as the initial transformation matrix, coarse registration is performed on the point clouds of each frame. Then, based on this, the generalized iterative nearest-point algorithm is used to perform fine registration on all point cloud frames. By iteratively calculating and minimizing the geometric errors between point clouds, all discrete point cloud data are finally fused into a unified and complete global 3D point cloud model of the ship.

[0049] Surface Model Reconstruction: Based on the fused global point cloud model, the Poisson Surface Reconstruction algorithm is applied. This algorithm estimates an indicator function by solving a Poisson equation, thereby generating a smooth and watertight triangular mesh model. This mesh model accurately represents the continuous curved surface morphology of the ship's outer plating and is stored in a standard format (such as .STL).

[0050] Job Path Planning: Based on the generated triangular mesh model, the coverage path planning program is initiated. The planning goal is to generate a flight path that guides the UAV end effector (i.e., the spray gun nozzle) to completely cover all areas to be sprayed at a constant distance and speed. The planning algorithm divides the entire ship mesh surface into several sub-regions with relatively uniform geometric features. Within each sub-region, a bow-shaped reciprocating path is generated. The path consists of a series of dense path points, each containing a six-DOF pose parameter. The position parameter ensures that the distance between the nozzle and the work surface is constant, while the pose parameter ensures that the nozzle axis is always perpendicular to the normal vector of the mesh surface at its location, ensuring optimal spraying results. Path planning also identifies depressions or protrusions in the model that are larger than a preset size and treats them as obstacles, avoiding them during path generation. The final generated initial job path is a temporal three-dimensional spatial curve containing high-density pose points.

[0051] Extract path nodes and set parameters;

[0052] On the initial job path, key geometric feature nodes are automatically detected and inserted. Geometric feature nodes include:

[0053] Curvature variation node: Calculates the principal curvature of the underlying mesh surface corresponding to the path point. When the rate of change of surface curvature between two consecutive points on the path exceeds a preset threshold, a curvature variation node is inserted at this location.

[0054] Region boundary nodes: Insert a region start node and a region end node at the beginning and end of each sub-region path, respectively.

[0055] Edge transition node: When the path crosses a significant angle or edge on the surface of the ship, an edge transition node is inserted in the transition area.

[0056] After node extraction, a specific set of operational parameters is configured for each node and the path segments between nodes. This parameter set is structured data and includes: Flight speed: in meters per second; Spraying instructions: an enumeration type, including {START, STOP, MAINTAIN}; Paint flow rate: in liters per minute; Spraying pressure: in Pa.

[0057] Perform the task;

[0058] The final mission path file, containing nodes and parameter sets, is uploaded to the UAV's onboard mission computer. The UAV takes off and autonomously flies to the starting point of the path.

[0059] After the operation begins, the UAV's flight control system controls the UAV's position and attitude in real time according to the continuous pose instructions in the path file. The onboard mission computer continuously monitors the UAV's travel status along the path. When the UAV's position reaches the coordinates of a geometric feature node in the path, the mission computer immediately triggers an interrupt event. This event reads and parses the set of operation parameters bound to that node and sends the corresponding instructions to the load control unit via serial communication or CAN bus.

[0060] Example 2

[0061] This embodiment provides a preferred automated operation method. Based on Embodiment 1, this method achieves intelligent operation on complex targets such as large ships by fusing and iteratively optimizing three-dimensional geometric information and two-dimensional image information. First, the synchronous data acquisition of this method is described in detail. In a pre-set flight mission, three-dimensional point cloud data and high-resolution surface image data of the target ship are acquired synchronously, and the data from heterogeneous sensors are strictly aligned with spatiotemporal references.

[0062] This embodiment uses the same 50kg-class industrial drone platform as Embodiment 1. On this platform, a heterogeneous sensor assembly is integrated, including: a three-dimensional sensing unit, i.e., a lidar, whose coordinate system is denoted as... The visual sensing unit, an array of five industrial-grade high-resolution CMOS cameras, is arranged in a horizontal "I" shape. The central camera serves as the main camera, and its optical center coordinate system is denoted as... All cameras underwent internal parameter calibration, resulting in their respective intrinsic parameter matrices. The pose sensing unit, i.e., the integrated navigation system comprising an IMU and RTK-GNSS, has its coordinate system (IMU body coordinate system) denoted as... .

[0063] Before performing data acquisition tasks, a one-time extrinsic parameter calibration must be performed on the heterogeneous sensor assembly to determine the relative pose relationships between the coordinate systems of each sensor. The extrinsic parameter calibration process is carried out in a controlled environment by acquiring data from the calibration board under different poses. The calibration results are two sets of rigid body transformation relationships, with... The homogeneous transformation matrix representation:

[0064] Radar-camera extrinsic matrix Indicates from the lidar coordinate system To the main camera coordinate system The transformation.

[0065]

[0066] in, It is The rotation matrix, It is The translation vectors describe respectively... Compared to The rotation and displacement.

[0067] Radar-IMU extrinsic parameter matrix Indicates from the lidar coordinate system To IMU body coordinate system The transformation.

[0068]

[0069] in, and They described respectively Compared to Rotation and translation.

[0070] The aforementioned extrinsic parameter matrix remains constant while the sensor assembly structure remains unchanged, and it forms the geometric basis for multi-sensor data fusion.

[0071] For applications involving large ships, characterized by their enormous size, complex structure, and rich surface details, a single data acquisition path is insufficient to simultaneously meet the needs of global geometry construction and local detail perception. Therefore, this invention employs a two-stage data acquisition strategy:

[0072] Phase 1: The goal of the global structural scanning route is to quickly acquire the complete three-dimensional outline of the ship. The UAV performs a circling flight at a distance of approximately 10 meters from the ship's outer plating, supplemented by a bow-shaped back-and-forth flight from bow to stern. Flight speed is relatively high in this phase, prioritizing efficiency.

[0073] Phase Two: The goal of the surface detail acquisition route is to acquire high-quality images for identifying welds, rust, stains, and fine obstructions. The UAV approaches to a distance of 3 meters from the ship's outer plating and executes a vertical reciprocating route parallel to the hull surface. The surface detail acquisition route should ensure at least 70% overlap between camera fields of view and an angle of less than 15 degrees between the camera's principal optical axis and the local normal to the work surface.

[0074] During the execution of the aforementioned flight path, the mission computer on the UAV controls all sensors to operate synchronously. The system timestamps each frame of point cloud data from the lidar, each exposure of the camera, and each reading of the IMU via a master clock triggered by a second pulse signal provided by RTK-GNSS.

[0075] After the flight mission is completed, the ground station receives the raw dataset containing timestamps and generates a fused dataset with a completely unified spatiotemporal reference. For any given data acquisition time point... The following procedures must be followed:

[0076] Pose acquisition: Obtain the IMU body coordinate system at that moment from the results calculated by the integrated navigation system. In the global world coordinate system The pose is represented by the homogeneous transformation matrix. .

[0077] Global world coordinate positioning of a point cloud: obtaining the coordinates of the point cloud. A point cloud frame captured in real time ,in In radar coordinate system The next 3D point is represented by N, where N represents the total number of points in the point cloud for that frame. Through chained matrix multiplication, all points in the point cloud of that frame are transformed to the global world coordinate system. Down:

[0078]

[0079] in, It is a point Homogeneous coordinates in the global world coordinate system. The above transformation was performed on the point cloud data at all time points, and the data was accumulated to form the preliminary global 3D point cloud of the ship.

[0080] Point cloud and image-geometric association: To establish a direct link between geometry and vision, for each point in the point cloud... The time point of data collection at that point closest image acquisition time The corresponding image Then, transfer this point from the global world coordinate system. Transform back to the main camera coordinate system :

[0081]

[0082] Obtain the coordinates in the main camera coordinate system Using the camera's intrinsic parameter matrix Projecting the 3D point onto the image plane yields its pixel coordinates. :

[0083]

[0084]

[0085] in For camera focal length, The coordinates of the main point.

[0086] After the above processing, a point cloud model with global coordinate alignment is finally generated.

[0087] With global pose labels (i.e., each image is associated with a global pose label) A high-resolution image sequence, which will be used to generate panoramic images and perform visual analysis.

[0088] Example 3

[0089] Next, this embodiment will describe in detail the process of receiving the generated 3D shaded point cloud model and high-resolution image sequence, and generating two independent preliminary operation paths in parallel based on different data modalities and optimization principles. These paths are the 3D point cloud path (path A) which focuses on geometric safety and the panoramic image path (path B) which focuses on process efficiency.

[0090] S2.1 Generation of 3D point cloud path (path A);

[0091] The input for generating the 3D point cloud path is the generated spatiotemporally aligned complete dataset. The final output is a 3D point cloud path (path A) with flight safety and geometric coverage efficiency as the primary optimization objectives.

[0092] S2.1.1 Surface model reconstruction;

[0093] To generate a continuous surface model suitable for path planning, the global point cloud data representing the entire ship needs to be converted into a triangular mesh model. .

[0094] The input data is not a point cloud from any single frame, but a global 3D point cloud accumulated from all acquired frames. It is all local point cloud frames After their respective moments After pose transformation in global world coordinate system Union in:

[0095]

[0096] in, The total number of frames captured. At any moment LiDAR coordinate system The point cloud collection below.

[0097] Global point cloud collection Each point in ( Estimate the local surface normal vector for the global point index. , forming a directed point cloud ,in It represents the total number of points in the global point cloud.

[0098] Scalar indicator function Its three-dimensional zero isosurface defines the ship surface model. The scalar indicator function is obtained by solving the following Poisson equation:

[0099]

[0100] in, It is the Laplace operator. It is based on directed point cloud A three-dimensional vector field is constructed.

[0101] S2.1.2 Path planning objective function combining ship painting characteristics;

[0102] The goal of path planning is to generate an optimal flight path for the drone. Unlike general spray painting, the anti-corrosion coating of large ships has clearly defined coating zones based on hydrodynamic and structural durability requirements. These zones have different requirements for coating thickness, uniformity, and paint type, and therefore must be considered during the pure geometry path planning stage.

[0103] In the generated three-dimensional geometric model Above, identify and mark the key paint zoning feature lines. These feature lines include, but are not limited to, the design waterline and the boundary line of the antifouling paint on the hull bottom. These feature lines cover the entire hull surface. It is divided into multiple sub-regions with different process attributes, and each sub-region includes at least the freeboard area above the waterline. and the antifouling paint area below the waterline .

[0104] The objective function of path planning is to minimize the total operation time and reduce energy consumption during the spraying process. This objective function is expressed as:

[0105]

[0106] in, These are the start and end times of the task, respectively; and These are drones in Linear velocity and angular velocity at time t; and This is the corresponding energy consumption weighting coefficient. The minimization process is subject to the following constraints:

[0107] Constraint 1: Independent coating zone operation constraint. To avoid cross-contamination between different types of coatings (anti-corrosion paint and anti-fouling paint), separate sub-zones ( and The generated paths must be spatially separated. That is, for Planned path any point on Its Z-axis coordinate must always be higher than the Z-axis coordinate of the design waterline. :

[0108]

[0109] Constraint 2: Mass reinforcement constraint for critical fluid dynamic regions. The areas near the bulbous bow, rudder, and propeller of a ship are regions experiencing significant drag and wear during navigation. These sub-regions are pre-defined as critical fluid dynamic regions. When performing path planning, stricter constraints on operational parameters must be imposed. Specifically, this means a higher tolerance for errors in constant offset distances. From the general area Rice tightened to Meters, the included angle threshold of the vertical attitude constraint Tighten from 5 degrees to 2 degrees.

[0110] Constraint 3: Reinforced coating constraint in the structural weld zone. Large ships are constructed from massive amounts of welded steel plates, and the welds are high-risk initiation zones for corrosion. (Model) Long-distance, low-curvature linear protrusion regions were identified and used as the geometric approximation region of the main structural welds. When the path When passing through these areas, the drone's flight speed must be limited to ensure that the coating has enough time to penetrate and cover.

[0111] Constraint 4: General geometric and safety constraints, including constant offset, attitude verticality and collision avoidance constraints.

[0112] S2.1.3 Generate paths with multiple constraints;

[0113] To generate paths that satisfy the aforementioned complex constraints, this embodiment employs a trajectory generation method with variable hierarchical partitioning parameters.

[0114] The first step is to partition the model, and based on the feature lines of the paint partition, divide the overall three-dimensional geometric model. Divided into A collection of multiple sub-regions.

[0115] The second step is parameter mapping. Each sub-region is assigned its own set of path planning constraint parameters.

[0116] The third step is trajectory generation within the region. Within each independent sub-region, a trajectory segment covering that region is generated. Trajectory generation is based on the viewpoint network growth method. Starting from a boundary point of the region, an initial viewpoint (a 6-DOF pose) that satisfies the current region constraints is generated. Then, using this viewpoint as the center, a greedy search is performed within its neighborhood to find the next subsequent viewpoint that satisfies the constraints, maximizes the new coverage area, and minimizes flight energy consumption. This process is iterated until the entire sub-region is completely covered, forming a smooth trajectory segment connecting all viewpoints.

[0117] The fourth step is path stitching and smoothing. All trajectory segments generated from the sub-regions are stitched together in an overall optimized order. At the junctions of different trajectory segments, trajectory smoothing is performed to generate continuous transition segments in velocity and acceleration, ensuring the stability and safety of the UAV flight. Through these steps, a 3D point cloud path is finally generated. ).

[0118] S2.2 Generate panoramic image path (path B);

[0119] The input to the panoramic image path is the generated high-resolution image sequence and its corresponding pose labels, as well as the set of hull surface sub-regions with clear process attributes divided in step S2.1 for generating path A. The final output is a hierarchical sequence of job tasks designed to optimize job quality and process efficiency, namely the panoramic image path (path B).

[0120] 2.2.1 Generate a partitioned surface semantic map;

[0121] To generate accurate surface condition maps while avoiding huge computational overhead, this embodiment is based on a semantic mapping method that combines 3D surface sampling with multi-view feature fusion. This method first performs sparse but representative sampling in 3D space, analyzes the sampled points from multiple perspectives, and then interpolates the analysis results to extend them to the entire region.

[0122] The planning of path A strictly adheres to the boundaries and specific constraints of each sub-region. The planning of path B is also carried out within this framework, avoiding the possibility of logical conflicts between process planning (path B) and geometric safety planning (path A).

[0123] The specific process is as follows: In each sub-region On its triangular mesh surface, according to the preset sampling density This generates a spatially uniform set of three-dimensional sampling points. ,in This represents the total number of sampling points in the area. For each sampling point, determine its process category label. :

[0124] Determine all points that can be measured in this 3D model. Image collection .

[0125] for Each image in ,Will Projecting the image onto the pixel coordinates yields the pixel coordinates. .

[0126] In the image Above, with Extract a dimension of centered on the target. Image blocks This image patch provides information about the points. Visual information of the local neighborhood.

[0127] This image block Input into a pre-trained convolutional neural network In this process, the network processes the entire image patch and ultimately outputs a representation of the core content of that image patch. 3D feature vector .

[0128] The quality of a viewpoint depends on its observation angle. The closer the observation direction is to being perpendicular to the surface normal vector, the smaller the image distortion and the more reliable the extracted features. Feature vectors extracted from all effective observation viewpoints are fused. Weights Used to evaluate the quality of the observation perspective. Among them, It is a point Surface normal vector, From point Pointing at the camera The unit observation vector of the optical center. It is a weighting index greater than 1, used to amplify the impact of high-quality perspectives.

[0129] Sampling points The final fused semantic feature vector The calculation is as follows:

[0130]

[0131] The fused high-dimensional feature vector The input is fed into a classifier, which outputs the final process category label for that 3D sampling point. .

[0132] To obtain a dense semantic map covering the entire sub-region ,for any point on the surface Its category label is assigned to the label of the nearest sampling point:

[0133]

[0134]

[0135] in In the triangular mesh model The geodesic distance is calculated above. This gives each sub-region... It generated its own complete semantic map. h_idx is an index variable. This represents the index number obtained by taking the minimum value parameter.

[0136] 2.2.2 Generate a process path based on partition priors;

[0137] Based on the generated semantic maps of each partition A globally optimal job execution sequence is planned. The planning is performed hierarchically:

[0138] First layer: Macro-regional scheduling, determining the execution order of tasks across large sub-regions. For These are two areas that both require painting but use different types of paint, and a collection of stained areas that need cleaning. The macro-scheduling will output the optimal region access sequence, and the preferred access sequence is... This ensures the continuity of the spraying operation and minimizes the costly number of times the spray gun and water gun need to be changed.

[0139] Second layer: Scheduling within micro-regions

[0140] Given a defined macroscopic sequence, for each sub-region Internally, a detailed work sequence planning is carried out.

[0141] In sub-region semantic map The above will have labels with the same process category. Connected pixels are aggregated into a series of process patches, resulting in a set. .

[0142] In the region Internally, the execution sequence of planned process patches is also aimed at minimizing operating costs. Since the macro-level task types within the region are already determined (e.g., in...), (The images show spraying with different parameters). The cost function mainly focuses on the smoothness of switching between operating parameters and the travel distance. Its cost function can be expressed as:

[0143]

[0144] in, It refers to the arrangement of patches within region s. It is the total number of process patches within the region. and These are the weighting coefficients. This is the normalized geodesic distance. A function to quantify the adjustment range of operation parameters. and These are the process category labels corresponding to the b-th and b+1-th process patches, respectively. and These represent the centroids of the b-th and b+1-th process patches, respectively.

[0145] The ΔParam function is used to quantify the cost of adjusting operating parameters when switching from one process patch to another. It is based on a preset process-parameter mapping table for specific calculations.

[0146] Define a process parameter mapping table as a label for each process category in the semantic map. Optimal operating parameters are pre-configured for each process. In this embodiment, two key operating parameters are considered: paint flow rate (L / min) and spraying pressure (bar). Therefore, for any process category... Each of them has a corresponding parameter vector. :

[0147]

[0148] The specific mapping relationships are shown in Table 1 below:

[0149]

[0150] Because the physical units and numerical ranges of different parameters vary (flow rate between 1-3, pressure between 100-150), directly calculating the difference is meaningless. Therefore, the core of the ΔParam function is to calculate the distance between two parameter vectors in the normalized space. For each parameter dimension, it is linearly normalized to its maximum and minimum values ​​across all process categories. Interval. For any parameter vector Its normalized vector A function for quantifying the adjustment range of operation parameters Defined as two process categories and The corresponding normalized parameter vector and Euclidean distance between them:

[0151]

[0152] in, and For vectors The specific normalized expression and For vectors The expression after specific normalization.

[0153] For example, from an intact coating ( Switch to severe corrosion ( The cost of ) is far greater than that of light rust ( Switch to weld ( The cost of the former is higher because the former allows for greater adjustments to its flow and pressure parameters. If ,but This indicates that no parameter adjustment is needed, and the switching cost is zero. In this way, the cost function... It can guide path planning to optimize in a way that reduces drastic adjustments to equipment parameters and ensures smooth and continuous operation.

[0154] The generated panoramic image path B ( It is a structured, hierarchical job plan. Its top layer is the macro-level region access sequence, and each sequence node contains the optimized execution order of process patches within that region.

[0155]

[0156] and Representing the first visited sub-region and the second visited sub-region The optimal scheduling sequence for internal process patches.

[0157] Example 4

[0158] This embodiment illustrates the dynamic fusion generation of the final job path. The present invention uses a dynamic path optimization method based on a local condition adaptive adjustment fusion strategy to generate a unique final job path. This allows it to dynamically balance and integrate the geometric safety represented by path A with the process quality and efficiency represented by path B in different regions.

[0159] S3: Final path generation based on dynamic fusion strategy;

[0160] The final path generation input is a 3D point cloud path. And the set of constraint parameters on which it was planned, and the panoramic image path Initial three-dimensional geometric model and a set of partitioned surface semantic maps ,in Index for sub-regions.

[0161] S3.1 Fusion Strategy Weights Dynamic calculation;

[0162] The final path selection in this invention varies depending on the region. In areas with complex geometries, path A should be prioritized for safety and accuracy; in areas with dense process patches, path B should be prioritized for process efficiency. To achieve this, we introduce a spatially dynamic fusion weighting function. Its range is This weight value is determined by any point on the hull surface. It is determined by the local characteristics.

[0163] Define the point for measurement Indicators of geometric complexity of the region This indicator is based on a three-dimensional geometric model. At point The two principal curvatures at the point and Joint decision:

[0164]

[0165] The more curved the surface (such as a bulbous bow), the better. The larger the value, the better. Normalization is performed on the entire hull surface to obtain the normalized geometric complexity index. .

[0166] Define a point for measurement Indicators of process complexity in the region This index is determined by the local density of the process patch. (Based on point...) Centered on, with a radius of on its surface Within the neighborhood, count the number of process patches belonging to different process categories (severe corrosion, stains, etc.). The more types of process patches and the denser their distribution, the higher the number of process patches. The larger the value, the better. Similarly, Normalization is performed to obtain the normalized process complexity index. .

[0167] Dynamic fusion weight function Defined as a linear combination of the process complexity index and the geometric complexity index, and normalized to ensure that its value is within a certain range. Within the range:

[0168]

[0169] in, and These are positive constant weights that control the degree of influence of geometric and manufacturing complexity, respectively. It is a basic bias weight. The dynamically fused weight function operates in geometrically complex regions ( big), A value approaching 1 indicates that strategy A should be prioritized. In areas with complex processes ( big), A value approaching 0 indicates that strategy B should be prioritized. In flat regions with simple processes, Take an intermediate value to represent a balanced approach.

[0170] Finally, by piecing together all the generated transition segments and work segments according to the macroscopic order of path B, the final work path is formed. .

[0171] Example 5

[0172] The following section describes the extraction of composite key nodes for the final path. This involves generating a geometrically continuous final operation path. This is transformed into a sequence of instructions with discrete event trigger points that can be directly parsed and executed by the control system. These event trigger points are the key nodes, and their extraction process deeply integrates information from the 3D geometric model and the surface semantic map.

[0173] S4: Extraction of composite key nodes in the final path;

[0174] The output is the final job path with annotations. It contains the final job path and an ordered list of critical nodes associated with the path points.

[0175] S4.1 Define the data structure of the key nodes;

[0176] To accommodate the rich control information, each extracted key node is defined as a structured data object containing the following fields: The node is on the final path The position of the arc length on the top. The six-degree-of-freedom pose of the node in space . The node type is an enumeration value that indicates the reason why the node was triggered. A metadata collection is used to store specific parameters related to the node type.

[0177] S4.2 Systematic process for node extraction;

[0178] Node extraction is the process of extracting the final job path. The process of traversing and analyzing. From the final job path. Starting from the beginning, with a tiny step size. Proceed along the path, checking each point to see if it meets any of the following node triggering conditions. Once a condition is met, generate a node object of the corresponding type at the current location and store it in the node list.

[0179] S4.3 Node triggering conditions and types; extraction of geometric feature nodes based on the geometric properties of the path itself or its relationship with the underlying 3D model. Triggered by geometric relationships, it is mainly used to control the flight attitude and speed of the drone, ensuring smooth and safe movement. Path turn nodes calculate the path. Its own local curvature , This represents the arc length along the final working path. When the curvature value exceeds a preset dynamic safety threshold... When that happens, insert a sudden turn node here.

[0180] Triggering conditions:

[0181] : {'curvature_value': }`

[0182] Surface corner node calculation path points The model below Gaussian curvature of the surface Insert this node when the value shows a sharp positive jump, indicating that the path is passing through a prominent edge or corner. A preset threshold representing the Gaussian curvature of the surface, used to trigger surface corner nodes.

[0183] Triggering conditions:

[0184] : {'gaussian_curvature': }

[0185] The extraction of process flow nodes is achieved through path and surface semantic map. Triggered by this relationship, it is the core of achieving automated and precise spraying operations, directly controlling the start-up, shutdown, and parameter switching of the spraying system. The process boundary switching node is along the path... During traversal, continuously query the current point. In the corresponding semantic map Process category label When the category label changes, it indicates that the drone is crossing the boundary between two different process patches.

[0186] Triggering conditions:

[0187] : {'entering_class': , 'exiting_class': }

[0188] Process boundary switching nodes are the most important node type. When the drone leaves the intact coating area (…), the switching node is crucial. ) flew into a heavily corroded area ( When this node is triggered, the control system will automatically adjust the spraying flow rate from 1.8 L / min to 2.5 L / min and the pressure from 100 bar to 120 bar according to the process parameter mapping table (see Table 1), achieving adaptive and refined operation for different working conditions. Similarly, when entering or leaving a stained area that needs cleaning, this node will trigger the switching between spraying and cleaning modes.

[0189] The extraction of composite constraint nodes is triggered by the relationship between the path and a predefined geometric or semantic region with special meaning, used to handle process constraints. The paint partition boundary is close to the node, when the path point... With the defined key paint zone feature line (design waterline) The geodesic distance is less than a preset warning threshold. Insert this node when needed.

[0190] Triggering conditions:

[0191] : {'feature_line_type': 'Waterline', 'distance': }

[0192] This node serves as an early warning, indicating that the flight control system is about to enter an area where operational rules have changed significantly (e.g., freeboard paint is prohibited below the waterline). This allows the controller to prepare in advance, ensuring that switching or stopping commands are executed precisely the instant the boundary is crossed, avoiding costly paint contamination errors.

[0193] Through the above process, the system reaches the final path The annotations highlight key nodes providing contextual information. The final output is the annotated path. It is a complete job instruction package containing continuous geometric trajectories and discrete control events, providing complete input for the next stage of fully automated, high-precision job execution.

[0194] Example 6

[0195] This embodiment details the final step of the workflow of the present invention: parameter configuration and job execution.

[0196] S5: Parameter configuration and job execution;

[0197] S5.1 Generation of job task instruction files;

[0198] The labeled final operation path is a task-oriented planning result that cannot be directly executed by the UAV's low-level controller. Therefore, an offline processing step is needed to compile it into a low-level operation task instruction file containing precise timestamps. First, the purely geometric final operation path... Time parameterization is performed to generate a flyable trajectory with velocity and acceleration profiles. This process uses key nodes extracted along the path as constraint boundaries. For any path segment between two consecutive key nodes, the system plans a smooth velocity curve that satisfies the UAV's dynamic constraints (maximum velocity, maximum acceleration) based on the segment's geometry and process requirements. For example, a higher cruising speed is planned for a long, straight segment in a well-coated area, while in a heavily corroded area, the upper speed limit is forcibly restricted to the low speed value required by the process. After this process, each densely packed pose point on the path is assigned a precise execution timestamp. .

[0199] The system traverses the time-parameterized path and the list of embedded key nodes. Based on the time sequence, it generates a unified instruction sequence. This sequence contains two types of instructions: high-frequency pose instructions, generated at the control frequency of the UAV flight control system. Each instruction contains (timestamp, x, y, z, roll, pitch, yaw) to guide the UAV to accurately track its geometric trajectory. Discrete event instructions, generated at the timestamp of each key node based on the node's type and metadata, provide specific load control instructions.

[0200] Assuming at timestamp At this location, there exists a node of type Process Boundary Switching with metadata {'entering_class': c_1, 'exiting_class': c_5} (transitioning from intact coating to heavy corrosion). The system will perform the following operations:

[0201] a. Query parameters: Access the process parameter mapping table (see Table 1) to query the target process category. corresponding parameter vector .

[0202] b. Generate load instructions: in the instruction sequence At any given time, insert a load control command with the format CMD_PAYLOAD(timestamp=t_{node}, command='SET_PARAMS', flow_rate=2.5, pressure=120).

[0203] c. Adjust subsequent speed: In the process of... When parameterizing the time for subsequent path segments, the upper limit of speed is constrained to the operating speed required by the process. The final generated job task instruction file is a text file containing precise instructions for every millisecond from the start to the end of the task, and it is the final carrier of the entire planning result.

[0204] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0205] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the descriptions of the embodiments described later are relatively simple, and relevant parts can be referred to the descriptions of the foregoing embodiments.

[0206] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatically starting and stopping a spray gun based on path node information, characterized in that, The method includes: acquiring three-dimensional point cloud data of the target object and simultaneously acquiring a high-resolution image sequence of the target object; constructing an initial three-dimensional geometric model of the target object based on the three-dimensional point cloud data, and planning and generating an initial operation path based on the initial three-dimensional geometric model; detecting and inserting one or more geometric feature nodes on the initial operation path according to the geometric features of the initial three-dimensional geometric model; configuring an operation parameter set containing spraying instructions for the geometric feature nodes; controlling the UAV to fly along the initial operation path, and performing a spraying start / stop operation according to the spraying instructions in the operation parameter set when reaching the geometric feature node; Based on high-resolution image sequences, a surface semantic map is generated, and based on the surface semantic map, a hierarchical panoramic image path is planned and generated. ; Panoramic Image Path The generation involves ranking process patches in the surface semantic map, which is achieved by minimizing the scheduling cost function within a micro-region. Cost function: in, It refers to the arrangement of patches within region s. It is the total number of process patches within the region. and These are the weighting coefficients. This is the normalized geodesic distance. A function to quantify the adjustment range of operation parameters. and These are the process category labels corresponding to the b-th and b+1-th process patches, respectively. and These represent the centroids of the b-th and b+1-th process patches, respectively.

2. The method for automatically starting and stopping a spray gun based on path node information according to claim 1, characterized in that, Based on the initial 3D geometric model, plan and generate a 3D point cloud path. .

3. The method for automatically starting and stopping a spray gun based on path node information according to claim 2, characterized in that, Before generating the final job path, it also includes: the initial 3D geometric model On the surface, according to any point Based on local characteristics, calculate a dynamically changing fusion weight function in space. The fusion weight function is determined by the following formula: in, For point The normalized geometric complexity index at the location, For point The normalized process complexity index at the location, and Weights are positive constants. Based on the bias weights.

4. A method for automatically starting and stopping a spray gun based on path node information according to claim 3, characterized in that: Normalized geometric complexity index Based on the initial three-dimensional geometric model At point The two principal curvatures at the point and Obtained through calculation and normalization; Normalized process complexity index Based on point The radius on the surface is Within the neighborhood of, statistical panoramic image path The number of defined process patches is obtained by calculation and normalization.

5. The method for automatically starting and stopping a spray gun based on path node information according to claim 4, characterized in that, Panoramic Image Path The generation is based on the generation of 3D point cloud paths. The process involves segmenting multiple sub-regions with different technological properties, which form the framework; panoramic image path. The planning adopts a hierarchical approach, which includes: a) Perform macro-regional scheduling between sub-regions to determine a region access sequence that minimizes the cost of job mode switching; b) Within each sub-region, micro-region scheduling is performed, and a process patch execution sequence is planned based on the surface semantic map to minimize the adjustment of operation parameters and movement distance.

6. A method for automatically starting and stopping a spray gun based on path node information according to claim 5, characterized in that, Function for quantifying the adjustment range of operation parameters Defined as two process categories and The corresponding normalized parameter vector and Euclidean distance between them: 。 7. The method for automatically starting and stopping a spray gun based on path node information according to claim 6, characterized in that, In the generated final job path The above extracts the key composite nodes, including: Along the final job path The process involves iterating through the nodes and, when a position that meets the preset triggering conditions is detected, inserting a key node containing the node type and metadata at that position. The types of key nodes include at least: a) Based on the final job path Its own local curvature or its underlying three-dimensional geometric model Surface Gaussian curvature Geometric feature nodes that are triggered when the threshold is exceeded. Indicates the arc length along the final operation path; b) When the final job path When crossing the boundaries of different process patches, the process boundary switching node triggered by the surface semantic map contains the metadata of the process category being entered and the process category being exited. c) When the final job path The distance from the geodesic line of the predefined paint partition feature line is less than the warning threshold. When, the composite constraint node is triggered.

8. A device for automatically starting and stopping a spray gun based on path node information, used to execute the method for automatically starting and stopping a spray gun based on path node information as described in claim 1, characterized in that, The device includes: Acquisition module: Acquires 3D point cloud data of the target object; Path generation module: Based on 3D point cloud data, constructs an initial 3D geometric model of the target operation object, and plans and generates an initial operation path based on the initial 3D geometric model; Node extraction module: On the initial job path, based on the geometric features of the initial 3D geometric model, detect and insert one or more geometric feature nodes; configure a set of job parameters containing spraying instructions for the geometric feature nodes; Execution module: controls the UAV to fly along the initial operation path, and when it reaches the geometric feature node, executes the spraying start and stop operation according to the spraying instructions in the operation parameter set.

Images