Automatic inspection planning method for differential inspection robot based on greedy covering and without esdf

Abstract

This invention discloses an automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection, belonging to the field of robot path planning and sensor network coverage technology. The inspection site is discretized into a grid map, and visible polygons are constructed through ray casting. The optimal camera orientation is quickly solved using residual masks, angle histograms, and FFT circular convolution. A greedy strategy is used to select camera positions and orientations, and the inspection visit order is determined by the traveling salesman problem. Collision-free guiding paths are generated for adjacent camera positions. B-splines are used to parameterize the trajectory, and trapezoidal integrals are used to strictly satisfy the nonholonomic constraints of the differentially driven robot. Adjoint recursion is used to calculate the linear complexity gradient. This invention eliminates trajectory collisions through iterative local obstacle information, combines wheel space constraints and time scaling to adapt to the chassis hardware capabilities, and performs collision pre-detection during operation, enabling stable output of robot-executable trajectories that meet the inspection operation requirements.

Description

Technical Field

[0001] This invention relates to the technical field, specifically to an automatic inspection planning method for robots based on greedy coverage and ESDF-free differential inspection. Background Technology

[0002] Differential-driven inspection robots are widely used in construction sites, warehouses, security environments, and other operational scenarios for autonomous inspection. These robots are equipped with vision cameras and require prior planning of camera observation points and orientations within the site. Then, a smooth, collision-free trajectory is generated for the robot to sequentially reach each observation point and acquire images. However, the actual working environment contains numerous polygonal obstacles and complex free-space morphology. Inspection planning must simultaneously consider camera visual coverage, point access order, robot kinematic constraints, and obstacle avoidance safety requirements. Existing inspection planning solutions typically treat camera coverage planning and robot trajectory optimization as two independent modules. This leads to practical problems such as high computational overhead for camera orientation evaluation, high memory consumption for trajectory optimization, and poor handling of nonholonomic kinematic constraints, making direct deployment on the edge computing units of ground-based inspection robots with limited computing power difficult.

[0003] At the academic research level, existing published literature such as "Beyond ShortsightedNavigation: Merging Best View Trajectory Planning with Robot Navigation" (arXiv:2408.12513) published by Tankasala et al. proposes a long-term viewpoint planning framework to achieve joint solution of observation viewpoint and navigation trajectory in ground robot inspection scenarios. This method can balance the environmental coverage area and robot motion cost during the inspection process and is suitable for security and monitoring ground robot tasks. However, this paper uses traditional ray projection to evaluate the visible coverage of each candidate camera orientation one by one. Each time the orientation is changed, ray projection and grid statistical calculations need to be repeated. When the sampling density of candidate orientations is high and the size of the field grid is large, the computational load of the camera orientation evaluation stage increases sharply. At the same time, the trajectory optimization module of this paper does not make a targeted design for the nonholonomic motion characteristics of differentially driven robots. In the optimization process, the planar position coordinates are treated as independent optimization variables, which requires the introduction of large-weight soft constraints to approximate the robot's kinematic equations. This can easily lead to ill-conditioned optimization and unsatisfied constraints. Furthermore, the trajectory gradient solution adopts the method of explicitly constructing a dense Jacobian matrix. The gradient calculation complexity increases quadratically with the number of trajectory sampling points. When the sampling density increases, the calculation time increases significantly, which is difficult to meet the real-time computing requirements of complex large-field inspection tasks.

[0004] The domestic patent publication number CN117075612A, entitled "A Fast Path Planning Method Based on Improved Visual Map and Trajectory Optimization Method Based on ESDF Map", first generates an initial path through a visual map algorithm, then constructs a global ESDF symbolic distance field map, performs trajectory optimization based on obstacle distance information provided by ESDF, and achieves trajectory obstacle avoidance by relying on distance field gradient. It can output a smooth robot motion trajectory and has certain practical value in the field of indoor robot navigation. However, this patented solution has obvious limitations: on the one hand, the patent requires the pre-construction of a global ESDF symbol distance field for the entire grid map. The larger the map size, the higher the memory resources and computation time consumed in constructing the distance field. However, in the actual trajectory optimization process, only a small amount of distance information from local areas is used, and most of the distance field computation resources are ineffective overhead. On the other hand, the patent does not carry out upper-level camera greedy coverage planning for inspection tasks, lacks an automatic solution process for camera positions and orientations, and does not establish constraints on the hardware limits of wheel speed and wheel acceleration of the differential drive chassis in the trajectory optimization stage, nor does it adopt a gradient solution method with linear complexity. When directly applied to differential inspection robots with visual observation tasks, the defects of relying on manual camera deployment, insufficient optimization computation efficiency, and difficulty in matching the actual execution capability of the chassis will still occur. Summary of the Invention

[0005] To address the aforementioned technical issues, this application discloses an automatic inspection planning method for robots based on greedy coverage and ESDF-free differential inspection, including:

[0006] The rectangular field to be inspected is discretized into a uniform grid map. Ray projection is performed on each candidate camera deployment point to construct the visible polygon of that point, thus obtaining the grid set that each candidate position can cover under different orientations.

[0007] Construct the current residual mask, discretize the residual mask into angle histograms according to angles, perform cyclic convolution of the angle histograms with the rectangular window functions corresponding to the field of view angles, and calculate the coverage gain value of all candidate orientations at once through fast Fourier transform;

[0008] Based on the submodularity of coverage gain, a greedy strategy is used to select the camera deployment location and corresponding orientation with the largest marginal coverage gain in round by round until the preset coverage threshold is met; for the selected camera deployment points, the inspection and access order is determined by solving the traveling salesman problem.

[0009] For each pair of adjacent camera deployment points in the inspection path, an A* search is performed in the SE(2) state lattice space to generate a collision-free guidance path, and the guidance path is parameterized using uniform cubic B-splines. The arc length control point and the heading control point are used as optimization variables to obtain the initial trajectory.

[0010] The vehicle pose sequence is recovered from the linear velocity and heading of the trajectory using trapezoidal integrals, ensuring that the differential drive nonholonomic kinematic constraints are strictly satisfied structurally, without treating the planar position coordinates as independent optimization variables; the adjoint recursion algorithm is used to calculate the gradient of the objective function with respect to the linear velocity and heading at the sampling time, and the gradient calculation complexity is linear to the number of trajectory sampling points.

[0011] In the trajectory optimization iteration, a multi-circle model is used to represent the vehicle body outline. For the sampling point where a collision is detected, a ray is emitted from the preset collision-free reference path to the center of the current collision circle, and only the local obstacle base point, outward normal, and safety distance information are extracted. Within the safety distance range, a C¹ consecutive cubic penalty function is applied to generate a collision elimination gradient along the outward normal direction. Historical collision constraints are accumulated through the Rebound outer loop, and the collision penalty weight is amplified round by round until the trajectory is collision-free throughout.

[0012] A secondary soft penalty term for exceeding the limit is added to the optimization objective for the speed and acceleration of the left and right wheels; after optimization, if the wheel speed or wheel acceleration still exceeds the physical limit, the speed curve is scaled proportionally by increasing the total trajectory time until the chassis physical constraints are met.

[0013] During the inspection process, the robot performs collision pre-checks on the planned trajectory for the future preset time period. If a new obstacle is detected that causes a collision, a local replanning or emergency stop state machine is triggered.

[0014] Preferably, the step of constructing the visible polygon of the point includes two processing steps: ray-obstacle line segment intersection processing and visible polygon contour generation.

[0015] The intersection process of the ray-obstacle line segment is as follows: An observation point is set. A ray is emitted outward from the observation point, with a unit vector in the direction of the ray. The ray parameter equation is denoted as , parameters in the formula Take any obstacle boundary line segment, with its two endpoints as... , Line segment direction vector The parametric equation of a line segment is denoted as , parameters in the formula Define the two-dimensional cross product operation. Let vector Solve the parametric equations of the ray and the line segment simultaneously, and solve for the intersection parameters using cross product elimination:

[0016]

[0017] when When the ray is parallel to the boundary line segment of the obstacle, it is determined that the ray is parallel to the boundary line segment of the obstacle; when At the same time satisfy , When determining whether a ray intersects with an obstacle boundary segment, the parameters are selected after a single ray traverses all obstacle boundary segments. The effective intersection point with the smallest value is taken as the nearest obstacle intersection point in the direction of that ray;

[0018] The visible polygon outline is generated by emitting a set of equally spaced probe rays at the observation point, obtaining the nearest obstacle intersection point corresponding to each probe ray, and connecting all the nearest obstacle intersection points in sequence according to the order of the ray emission angles to form a closed polygon, thus obtaining the visible polygon corresponding to the observation point.

[0019] Preferably, the step of constructing the current residual mask and discretizing the residual mask into an angle histogram by angle includes residual mask construction and angle histogram statistics;

[0020] The residual mask is constructed as follows: This is the omnidirectional view mask corresponding to the current candidate camera deployment point. Mark whether the grid is inside the omnidirectional view polygon of the candidate camera; The union overlay mask corresponding to all cameras that have been deployed. Check if the marker grid has been covered by deployed cameras; For obstacle masking, The marker grid indicates whether a region belongs to an obstacle; the residual mask expression is:

[0021]

[0022] residual mask A value of true indicates that the corresponding grid cell is an uncovered, non-obstacle grid cell that can be observed by the current candidate camera.

[0023] The angle histogram statistics are obtained by traversing the residual mask. All grid cells with a value of true are selected, and the center coordinates of the grid cells are obtained. Calculate the center of the grid relative to the observation point. azimuth ; azimuth angle Divide the data into corresponding angle bins, and calculate the cumulative sum of the faces of all valid grid cells within each angle bin to obtain an angle histogram. .

[0024] Preferably, the step of performing a circular convolution between the angle histogram and the rectangular window function corresponding to the field of view, and calculating the coverage gain value of all candidate orientations at once through a fast Fourier transform, includes configuring a circular rectangular window and performing an FFT-based circular convolution operation to solve for the coverage gain corresponding to each candidate orientation.

[0025] Configure a cyclic rectangular window and calculate the total number of angle bins occupied by the field of view (FOV) based on the camera's FOV. Set the period equal to the total number of angle discretizations. Circular rectangular window , Expression for a looping rectangular window:

[0026]

[0027] In the formula The index of the cyclic window is 1. The index range of the cyclic rectangular window with an index of 1 corresponds to the angle bin range covered by the camera's sector view.

[0028] Perform a circular convolution operation on the angle histogram. With cyclic rectangular window Perform a circular convolution operation and output the convolution. , Take the first camera The residual coverage gain corresponding to each candidate orientation; according to the circular convolution theorem, convolution calculation is performed through Fourier transform:

[0029]

[0030] In the formula Represents Fast Fourier Transform, Represents the inverse fast Fourier transform. This represents the element-wise multiplication of a complex array; iterate through the convolution output. Take all values ​​and use the index corresponding to the maximum value to determine the optimal camera orientation.

[0031] Preferably, the step of determining the inspection visit order for the selected camera deployment points by solving the traveling salesman problem includes:

[0032] The initial access sequence of the open TSP is generated using the greedy nearest neighbor rule, with the specified robot starting point as the first access node of the sequence; at each step, the point with the smallest Euclidean distance from the current node is selected from the set of unvisited camera points as the next access node, until all camera points are included in the access sequence.

[0033] The 2-opt algorithm is used to iteratively optimize the initial access sequence. During the iteration, disconnection and reconnection operations are performed on two sets of edges in the sequence. The iteration is terminated when the total path length no longer decreases, and the open inspection access sequence of the camera points is output.

[0034] Preferably, the step of parameterizing the guide path using uniform cubic B-splines and using arc length control points and heading control points as optimization variables to obtain the initial trajectory includes constructing a trajectory expression using uniform cubic B-spline basis functions;

[0035] Four compactly supported basis functions for a uniform cubic B-spline:

[0036]

[0037] In the formula These are local parameters within a single segment of the B-spline;

[0038] Control points by arc length Heading control points As optimization variables, the expressions for the arc length of the trajectory changing with time and the expression for the heading changing with time are:

[0039]

[0040] The first, second, and third derivative bases are obtained by analytically differentiating the B-spline basis functions. Combined with the time interval of a single trajectory segment, the linear velocity, angular velocity, linear acceleration, and angular acceleration corresponding to the trajectory sampling time are calculated from the derivative bases.

[0041] Preferably, the step of recovering the vehicle pose sequence from the linear velocity and heading of the trajectory using trapezoidal integral includes:

[0042] Set a fixed sampling time interval , For sampling time index, For the first Linear velocity at the sampling time, For the first Heading angle at sampling time; define heading vector ; Representing the The robot's two-dimensional planar position at the sampling moment; trapezoidal integral iterative formula:

[0043]

[0044] From the initial position Iterate every time step, accumulating the position increments to obtain the robot's planar position at all sampling times; differential driving kinematic constraints. , It is achieved by the structural constraints of this integral iterative formula.

[0045] Preferably, the step of using the adjoint recursion algorithm to calculate the gradient of the objective function with respect to the linear velocity and heading at the sampling time includes defining the position adjoint quantity, setting boundary conditions, recursively calculating the adjoint quantity in reverse, and then solving for the gradient at the sampling point using the adjoint quantity.

[0046] Define the cost function as , Cost function For the first Robot planar position at sampling time The gradient; defining the positional adjoint. The location-related quantity is from the sampling time. Start by summing the vectors of gradients at all subsequent positions: ;

[0047] Set recursive boundary conditions, and denote the index of the last sampling time in the sampling sequence as... Boundary conditions ;

[0048] Perform recursive calculations backwards from the end of the sequence to the beginning of the sequence. The associated recursive relation is: Based on the already obtained covariates , Calculate the cost function for the first Sampling time linear velocity gradient: Similarly, calculate the cost function for the first... Sampling time heading The gradient at the sampling time is calculated; after the gradient calculation is completed, the gradient at the sampling time is scattered and mapped to the corresponding arc length control point and heading control point based on the local tight support characteristics of the B spline.

[0049] Preferably, the steps of applying a continuous cubic penalty function C¹ within the safe distance range to generate a collision elimination gradient along the outward normal direction; accumulating historical collision constraints through the Rebound outer loop and amplifying the collision penalty weight round by round include the construction of the cubic penalty function and the iterative processing of the Rebound outer loop;

[0050] The set of all active collision constraints; Represents the trajectory sampling index. Represents the circle index within the multi-circle model of the vehicle body; For the first Sampling time number The directional spacing corresponding to each circle of the vehicle body. , Let the world coordinates be the center of the vehicle body circle. As the barrier base point, The unit vector normal to the obstacle. To establish a preset safe distance; Collision penalty weights; cubic penalty function:

[0051]

[0052] For the preset weight growth coefficient and , For each iteration of the outer loop, before the start of each iteration of the Rebound outer loop, the collision penalty weight is updated, and the value of the collision penalty weight is set to the sum of the current weight and the value of the outer loop. of The product of the two is raised to the power of ...

[0053] Preferably, the step of adding a secondary soft penalty term for the left and right wheel speeds and accelerations to the optimization target; after optimization, if the wheel speed or wheel acceleration still exceeds the physical limit, the step of scaling the speed curve proportionally by increasing the total trajectory time includes performing wheel space mapping, secondary soft penalty, and time scaling processing.

[0054] For differential robot wheelbase, Let the linear velocity of the center of mass be denoted as . Let ω be the angular velocity of the center of mass. For the acceleration along the center of mass, Here is the angular acceleration of the center of mass; the formula for mapping the motion parameters of the center of mass to the motion parameters of the left and right wheels is:

[0055]

[0056] For the revolver speed obtained from the mapping Right wheel speed Revolver acceleration Right wheel acceleration Construct two soft penalty terms for exceeding the limit respectively;

[0057] Time scaling processing calculates the wheel speed over-limit ratio. Wheel acceleration over-limit ratio ;based on and Calculate the scaling factor ;Complete trajectory time scale scaling to change the original total trajectory time With scaling factor Multiply by the product to obtain the scaled total time of the trajectory, thus completing the trajectory time-scale scaling.

[0058] Compared with the prior art, the technical solution of this application has the following technical effects:

[0059] This invention can complete the camera deployment planning of the site to be inspected, construct a visible polygon through ray projection, solve the optimal camera orientation based on residual mask and FFT circular convolution, and obtain the camera deployment points and orientation by combining a greedy strategy; it can efficiently solve the site visual coverage problem and output camera configuration results that adapt to the distribution of obstacles in the scene, providing a reliable observation point basis for subsequent inspection tasks.

[0060] After completing the planning of the access order of camera points, the present invention obtains the guiding path by searching the state grid A* of SE(2), and completes the trajectory parameterization by using uniform cubic B-splines; the robot pose is recovered by trapezoidal integral, which satisfies the nonholonomic kinematic constraints of the differential driven robot at the structural level, without the need to set additional constraint approximation terms, and ensures that the generated trajectory conforms to the robot's own motion characteristics.

[0061] This invention employs an adjoint recursive approach to perform gradient calculations, achieving a linear relationship between gradient calculation complexity and the number of trajectory sampling points. Simultaneously, it does not construct a global ESDF distance field, but only extracts local obstacle information at collision locations to perform collision penalty iterative optimization, which can effectively eliminate trajectory collisions, reduce the computational resource consumption in the planning process, and improve the computational efficiency of the trajectory optimization stage.

[0062] This invention introduces wheel space constraints during the optimization process and adjusts the trajectory scale through time scaling. Combined with collision pre-detection during the operation phase, it achieves online replanning. This enables the output trajectory to adapt to the hardware capabilities of the differential robot chassis, allowing the robot to cope with environmental changes during execution and ensuring the stable and reliable operation of the overall inspection work.

[0063] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0064] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0066] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0067] Figure 1 The flowchart shows the automatic inspection planning method for a greedy coverage and ESDF-free differential inspection robot.

[0068] Figure 2 This is a flowchart of the greedy coverage planning process for the upper-layer camera in this invention.

[0069] Figure 3 This is a flowchart of the differential trajectory planning process for the differential-driven inspection robot without ESDF according to the present invention.

[0070] Figure 4 This is a schematic diagram of the inspection and patrol results according to an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0072] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0073] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0074] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0075] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0076] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0077] Example 1 mainly describes an automatic inspection planning method for a greedy coverage and ESDF-free differential inspection robot, such as... Figure 1 As shown, it includes:

[0078] The rectangular field to be inspected is discretized into a uniform grid map. Ray projection is performed on each candidate camera deployment point to construct the visible polygon of that point, thus obtaining the grid set that each candidate position can cover under different orientations.

[0079] Construct the current residual mask, discretize the residual mask into angle histograms according to angles, perform cyclic convolution of the angle histograms with the rectangular window functions corresponding to the field of view angles, and calculate the coverage gain value of all candidate orientations at once through fast Fourier transform;

[0080] Based on the submodularity of coverage gain, a greedy strategy is used to select the camera deployment location and corresponding orientation with the largest marginal coverage gain in round by round until the preset coverage threshold is met; for the selected camera deployment points, the inspection and access order is determined by solving the traveling salesman problem.

[0081] For each pair of adjacent camera deployment points in the inspection path, an A* search is performed in the SE(2) state lattice space to generate a collision-free guidance path, and the guidance path is parameterized using uniform cubic B-splines. The arc length control point and the heading control point are used as optimization variables to obtain the initial trajectory.

[0082] The vehicle pose sequence is recovered from the linear velocity and heading of the trajectory using trapezoidal integrals, ensuring that the differential drive nonholonomic kinematic constraints are strictly satisfied structurally, without treating the planar position coordinates as independent optimization variables; the adjoint recursion algorithm is used to calculate the gradient of the objective function with respect to the linear velocity and heading at the sampling time, and the gradient calculation complexity is linear to the number of trajectory sampling points.

[0083] In the trajectory optimization iteration, a multi-circle model is used to represent the vehicle body outline. For the sampling point where a collision is detected, a ray is emitted from the preset collision-free reference path to the center of the current collision circle, and only the local obstacle base point, outward normal, and safety distance information are extracted. Within the safety distance range, a C¹ consecutive cubic penalty function is applied to generate a collision elimination gradient along the outward normal direction. Historical collision constraints are accumulated through the Rebound outer loop, and the collision penalty weight is amplified round by round until the trajectory is collision-free throughout.

[0084] A secondary soft penalty term for exceeding the limit is added to the optimization objective for the speed and acceleration of the left and right wheels; after optimization, if the wheel speed or wheel acceleration still exceeds the physical limit, the speed curve is scaled proportionally by increasing the total trajectory time until the chassis physical constraints are met.

[0085] During the inspection process, the robot performs collision pre-checks on the planned trajectory for the future preset time period. If a new obstacle is detected that causes a collision, a local replanning or emergency stop state machine is triggered.

[0086] This invention is divided into an upper-layer coverage planning module and a lower-layer EGO-DDR trajectory planning module, with the two modules connected by a unit conversion coefficient. Complete the unification of data units from centimeters to meters. The conversion factor represents the conversion from centimeters to meters. The overall workflow of the method is as follows: First, a greedy overlay calculation is used to output the camera position and orientation information. Then, obstacle information is sent to the grid mapping module to generate an occupied grid map. Next, the selected set of camera points is sent to the TSP path sorting module to obtain the inspection access order of the camera points. Then, the access order is input into the EGO-DDR trajectory planning module to generate a smooth robot inspection trajectory segment by segment. Finally, the trajectory data is saved and visualized. This invention defines the overall system architecture, setting the inspection site as a rectangular area. ,in Represents the entire inspection site. Represents the length of one side of the playing field; the playing field contains a set of obstacles composed of polygons. , For the collection of all obstacles, The total number of obstacles; free space is defined as the area of ​​the entire site after removing all obstacles. , This represents the free space available for robot movement and camera observation. This free space is the effective area for both robot movement and camera observation. The upper-level coverage planning module is responsible for determining the camera deployment locations and corresponding observation orientations that meet the coverage requirements. Figure 2 As shown, the lower-level trajectory planning module receives the coordinates of the points after unit conversion and completes the solution for the robot's motion trajectory between various camera points, such as... Figure 3 As shown, the two modules work together to fully realize the entire process from environmental input to inspection trajectory output.

[0087] After defining the system architecture, this invention performs ray casting and visible polygon calculation, which consists of two processing stages: parametric solution for ray-line segment intersection and visible polygon construction. During the parametric solution for ray-line segment intersection, the camera's observation point is set. , Candidate deployment locations for cameras, , These are the horizontal and vertical world coordinates of the observation point, respectively; a probe ray is emitted outward from this observation point, and the unit vector of the ray's direction is denoted as . , Let be the unit direction vector of the ray. Let be the angle of ray emission, and the corresponding parametric equation for the ray is written as:

[0088]

[0089] In the formula The distance on the ray from the observation point is Spatial points, parameters This represents the distance the ray travels from the observation point. A value greater than or equal to zero indicates that the ray extends only outward from the observation point and does not trace back. Select any obstacle boundary segment within the site, with its two endpoints being... and , , These are the two vertices of the polygonal edge of the obstacle. The x and y coordinates of the two vertices are respectively; the direction vector of the line segment is... , Representing the edge of the obstacle point to The direction vector, the parametric equation of the line segment is denoted as...

[0090]

[0091] In the formula Points representing the edges of obstacles, parameters The value is between 0 and 1, ensuring that the point falls on the obstacle segment itself and does not extend beyond the segment. Solving the parametric equations of the ray and the obstacle segment simultaneously yields the equation... After sorting and deformation, the result is Define vector , This represents the direction from the observation point to the apex of the obstacle. The vector; and define the rules for the two-dimensional cross product operation. The two-dimensional cross product is used to determine the relative orientation of two two-dimensional vectors and to solve for the intersection of a line segment and a ray. It involves cross-multiplying both sides of the equation by the direction vector of the line segment. Utilizing the properties of the cross product itself If the cross product of the same vector is zero, it can be solved to obtain...

[0092]

[0093] Then cross-multiply both sides of the equation by the ray direction vector. Using the cross product property The cross product of the same vector is zero, and By swapping the order of the two vectors and inverting their cross product, we can obtain the solution.

[0094]

[0095] When calculated When the value is zero, it means that the probe ray and the obstacle boundary segment are parallel to each other, and there is no effective intersection between them; when... Not equal to zero, and simultaneously satisfying Greater than or equal to zero If the ray is within the range of 0 to 1, then the ray and the obstacle line segment are determined to have a valid intersection point, and the coordinates of the intersection point are... For a single probe ray, it is necessary to traverse the boundary segments of all obstacles within the site, solve for all valid intersection points, and select parameters from them. The intersection point with the smallest value is the closest obstacle intersection point to the observation point along this ray direction, representing the boundary position where the camera's line of sight is obstructed by the obstacle in this direction. After completing the ray-line segment intersection operation, a visible polygon is constructed, generating a number of polygons at the observation point location. Detection rays divided into equal angles, Represents the total number of emitted rays; the formula for calculating the ray angle is: , , Use ray indexing; iterate through all probe rays to find the nearest obstacle intersection point for each ray. , For the first The intersection points obtained from the ray are connected end-to-end in order of the ray's emission angle to form a closed polygon. This closed polygon is the omnidirectional visible polygon. , This represents the 360-degree unobstructed visible area of ​​the observation point. The grid cells contained within the polygon are the grid cells that can be observed from that observation point without obstruction under a 360-degree field of view. When the camera used has a limited field of view, the field of view... Less than , Let be the camera's physical field of view, in radians; then, let be the camera's target orientation. As the center of the sector To determine the desired observation direction of the camera, half the field of view is taken on each side of the angle. Probe rays are emitted only within this fan-shaped angle interval, and the sequence of intersection points obtained from the fan-shaped interval is connected end-to-end back to the original observation point. This yields a sector-shaped visible polygon. The grid inside the sector polygon represents the set of grids that the camera can observe and cover under that specified orientation.

[0096] After constructing the visible polygon, this invention executes an optimal orientation solution process based on FFT circular convolution. This part is a core innovation of upper-layer coverage planning, assuming the current camera deployment position has been determined and the front... Assuming the combined coverage area of ​​the two cameras has been solved, This represents the current camera index to be calculated, and the optimal orientation for the current camera is determined. , To calculate the optimal viewing orientation of the camera, maximizing the residual new coverage area that the camera can provide, a residual mask is constructed, and the entire inspection site is rasterized into a grid. The grid, This represents the number of vertical grid cells in a raster map. This represents the number of horizontal grid cells in a raster map; Represents the physical area corresponding to a single grid cell; Representing the The omnidirectional view mask of the camera at the current observation point is used to mark whether each grid cell is inside the omnidirectional view polygon at that point. Represents the index position of the raster; This indicates that the previous deployment has been completed. The union overlay mask of the cameras is used to mark whether the grid has been covered by an existing camera; For obstacle masks, marking whether a grid belongs to an obstacle region, the expression for the residual mask is defined as follows:

[0097]

[0098] That is, residual mask, logical AND This represents simultaneous fulfillment of the conditions, logical NOT. Invert the representation; residual mask A grid cell with a true value is one that simultaneously meets three conditions: it is within the current camera's view area, it is not covered by any existing camera, and it is not located in an obstacle area. In other words, it is a target grid cell that the current camera can add coverage to. After constructing the residual mask, angle histogram statistics are performed, traversing the residual mask. All grid cells that return a true value are retrieved, and the center coordinates of each grid cell are extracted. subscript This represents the valid raster number, used to calculate the center point of that raster relative to the observation point. The azimuth angle represents the angle from the observation point to the center of the grid. The azimuth angle is calculated as shown in the formula.

[0099]

[0100] For the first The azimuth angle of each effective grid cell. It is the arctangent function in the fourth quadrant. This represents taking the modulus of the angle, limiting the angle to 0 to... Range. The calculated azimuth angle is then divided into... From each angle of bin, The bin represents the total number of discrete orientations, and the bin index is the discrete interval of the angle. The bin index is calculated as follows:

[0101]

[0102] Representing the The angle bin number to which each grid cell belongs. This represents the floor function (rounding down). Angle histogram. Accumulate the results for all valid grid cells within each angle bin. Represents the bin index; the expression is shown in the formula.

[0103]

[0104] Numerical representation of azimuth angle The size of the remaining area that can still be covered by the camera in the direction above. Then, based on the camera's field of view... Calculate the number of angle bins occupied by the field of view, as shown in the formula.

[0105]

[0106] This represents the total number of bins spanned by the camera's field of view. This is a rounding function. The period size is equal to the total number of angle discretizations. Circular rectangular window function , To simulate the window function of a camera's fan-shaped field of view, For window function indexes, see equation.

[0107]

[0108] Representing the cyclic integer modulo space, it implements index looping; the index interval where the window function takes a value of 1 corresponds to the angle bin range covered by the camera's fan-shaped field of view. The angle histogram and the cyclic rectangular window undergo cyclic convolution operations; when the camera is facing... , As candidate orientation indexes, the camera's fan-shaped field of view covers the angular range. The residual coverage area facing downwards is shown in the formula.

[0109]

[0110] Representing the The residual cover area corresponding to each candidate orientation. This summation operation is equivalent to an angle histogram. and cyclic rectangular window Circular convolution at index The output is shown in the equation.

[0111]

[0112] symbol This represents the circular convolution operation. According to the circular convolution theorem, time-domain circular convolution is equivalent to frequency-domain pointwise multiplication followed by inverse Fourier transform. The calculation formula is shown in equation [equation missing].

[0113]

[0114] Fast Fourier Transform (FFT) transforms the time-domain signal to the frequency domain. The inverse fast Fourier transform restores the frequency domain signal to the time domain; This represents a point-by-point multiplication operation at corresponding positions in the array. This method allows for the simultaneous calculation of the residual coverage gain for all candidate orientations by iterating through all values ​​in the convolution output and taking the index corresponding to the maximum value. , This represents the index that maximizes the function; the optimal orientation is calculated as shown in the equation.

[0115]

[0116] By employing this FFT convolution processing method, the traditional naive approach, which requires re-rasterization and evaluation for each candidate orientation, is eliminated. Computational complexity, Big O notation represents the asymptotic time complexity of the algorithm, reduced to ,in The overhead of constructing residual masks and angle histograms is reduced to a one-time construction cost. Gain evaluation of all candidate orientations only requires FFT-related operations, which significantly reduces the computational cost of orientation evaluation.

[0117] After obtaining the optimal orientation calculation method for a single camera location, this invention performs a greedy coverage selection process, setting the set of all candidate camera locations as follows: , Deploy a set of candidate locations for all available cameras; each candidate camera within the set Each corresponds to a visible area Define the overriding function ,gather This function represents a subset of cameras that have been selected. The area of ​​the intersection of the entire camera's visible region and the site's free space is defined by the coverage function shown in the equation.

[0118]

[0119] This represents the calculation of geometric area. The union operation represents the set operation. This represents the intersection operation of sets. In actual engineering implementation, complex geometric area calculations are not performed directly. Instead, a union operation is performed using a raster mask, and the total number of valid raster cells is counted and then multiplied by the area of ​​a single raster cell to approximate the area. The target coverage rate is set to... , The preset free space target coverage ratio; the total area of ​​free space is In the greedy iterative process, each round selects the camera that brings the maximum marginal coverage gain from the set of candidate cameras that have not yet been selected. The representative added the first The additional coverage area brought by the camera, that is, the marginal gain. This represents the set of cameras selected after the previous iteration. The selection rules are shown in the formula.

[0120]

[0121] Representing the The greedy iterative selection of camera numbers, set symbol. This represents removing cameras that have already been selected from the candidate set. The iterative process has multiple termination conditions; iteration stops as soon as any one of these conditions is met. The termination conditions are: overall coverage meets... Single-step marginal gain , This represents the minimum threshold of marginal gain, indicating that the maximum number of cameras allowed to be deployed in the system has been reached. Simultaneously, this invention performs visual mask caching optimization. Since the omnidirectional visual mask corresponding to a candidate camera remains unchanged when its deployment position remains constant, the omnidirectional visual mask for each candidate point is pre-calculated and cached. In each round of greedy iteration, only the global coverage mask needs to be updated. After updating, the angle histogram and FFT convolution are recalculated, eliminating the need to repeatedly perform ray casting operations and further reducing the overall computational cost of the algorithm.

[0122] After greedily iteratively selecting a number of camera locations, the TSP (Traveling Salesman Problem) path sorting process is performed. The greedy algorithm outputs a number of... The set of camera locations This represents the total number of cameras deployed. The coordinates of the points are uniformly expressed in meters. This section solves an open-form Traveling Salesman Problem, where the open path does not need to return to the initial starting point. The objective is to minimize the total Euclidean distance of the camera point inspection paths. Represents the point access sequence. Represents the first in the sequence The coordinates of the access points are given, and the objective function is shown in the equation.

[0123]

[0124] This represents finding the access sequence that minimizes the total sum. This represents the Euclidean distance norm operation. During the solution process, the starting point given by the robot is used as the first node of the visit sequence. The initial solution for the access sequence is generated using a nearest neighbor strategy. At each step, the point with the smallest Euclidean distance from the current node is selected from the set of unvisited camera points as the next access node. The selection rule is shown in the equation.

[0125]

[0126] Number the candidate points. This indicates that the point has not yet been visited. Add this point to the visit sequence until all camera points are included in the sequence. After obtaining the initial visit sequence, use the 2-opt algorithm to iteratively optimize the sequence. During the iteration process, disconnect and reconnect the two sets of edges in the path to continuously reduce the total path length. When the total path length no longer decreases significantly, the iteration ends, and the final camera point access order is output.

[0127] After obtaining the inspection and access order of camera locations, it is necessary to construct an occupation grid map, convert the obstacle polygons described at the centimeter scale to the meter unit scale, and set the grid resolution. , Represents the physical side length of a single raster; iterate through each raster in the raster map, calculate the center coordinates of the raster, and then... , For raster row index, For the raster column index, the formula for calculating the world coordinates of the corresponding raster center is shown in the equation.

[0128]

[0129] The world coordinates of the grid center point This is the world coordinate of the map origin, plus 0.5 to represent the center of the grid cell. It determines whether the grid center point falls inside an obstacle polygon. This logic supports handling polygon obstacles with holes. If the grid center is inside an obstacle, the grid is marked as occupied. Simultaneously, an occupation layer of a certain thickness is added to the four edges of the map. , The thickness of the boundary layer is used to constrain the planned trajectory by relying on the obstacles in this layer, so as to prevent the robot's planned trajectory from exceeding the map boundary.

[0130] After completing the grid mapping, for each pair of adjacent camera points after TSP sorting, a front-end search of the SE(2) state grid A* is performed. SE(2) represents the two-dimensional planar rigid body pose space, including the plane x and y coordinates and the heading angle. The input of this module is the starting pose and ending pose corresponding to a pair of camera points, and the output is a collision-free dense reference guidance path, which serves as the reference benchmark for the back-end trajectory optimization stage. The robot's SE(2) pose is then analyzed. Perform three-dimensional discretization. This represents the robot's complete pose. Plane coordinates Robot heading angle; converting continuous robot poses into 3D grid keys. Plane coordinates Row index is obtained after rasterization. and column indexes The formula for calculating the discrete heading index is shown in equation [formula missing].

[0131]

[0132] For the course discrete index, The total number of discrete segments representing the heading direction, the angle wrapping function. Used to map any angle to The range eliminates angle jumps. Circular motion primitives are used during path expansion; each primitive represents a motion pattern the robot can execute in one step. A starting pose is given. directional arc length of motion , Represents the arc length of the robot's trajectory, with positive values ​​for forward movement and negative values ​​for reverse movement; the change in heading. , Represents the change in heading angle during motion; radius of circular motion. ; After the movement ends, the course The coordinates of the endpoint plane are shown in the formula.

[0133]

[0134]

[0135] This represents the planar coordinates after the motion ends. When... , With an extremely small numerical threshold, the heading changes very little, and the circular motion degenerates into straight-line motion. , ;when This represents the robot's reversing motion. The heuristic function for A* search is shown in the formula.

[0136]

[0137] Represents A's heuristic cost estimation. For the current pose, The target endpoint pose, The planar coordinates representing the pose, where the angle difference... , This is the heading weighting coefficient, used to adjust the proportion of heading difference in the cost. Search cost A combines arc length, turning cost, reversing penalty, and direction-switching penalty. Each circular motion primitive is subdivided into... Steps, This represents the number of subdivision verification steps for a single motion primitive. Each substep performs a multi-circle vehicle collision check; if a collision occurs in any substep, the expansion branch is pruned and abandoned. The search terminates when the search reaches the endpoint tolerance range or the maximum number of node expansions is reached; a densely sampled collision-free reference path is output for backend trajectory optimization.

[0138] After obtaining the guidance path output by the front-end search, this invention uses uniform cubic B-splines to complete the trajectory parameterization. A B-spline is a piecewise smooth parametric curve. The total number of control points is set. , Represents the total number of B-spline control points; the number of samples per trajectory segment. , representing the number of sampling points within a single B-spline segment; total trajectory time This represents the total duration of the entire inspection route; from this, the number of route segments is obtained. Single segment duration Sampling time interval Local parameters of each B-spline segment , These are the normalized local parameters within a single curve segment. The four tightly supported basis functions are shown in the equation.

[0139]

[0140]

[0141] , , , The four basis functions are uniform cubic B-splines. Compact support means that each basis function is non-zero only in local intervals; the basis functions satisfy the unity decomposition condition. This represents the sum of four basis functions at the same location equal to 1, possessing non-negativity and locally compact support properties, where each sampling point depends only on its four adjacent control points. Analytical differentiation of the basis functions yields the first, second, and third derivative bases. The derivative basis corresponds to velocity, acceleration, and jerk. Time normalization is performed by combining the derivative basis with the time interval of a single trajectory segment. This invention sets two sets of optimized control points: arc length control points. Heading control points ; Representing the One arc length control point Representing the There are 1 heading control point; the trajectory endpoint control points are fixed, representing that the poses of the trajectory start and end points are constrained, while intermediate control points are treated as free optimization variables. The arc length varying with time is calculated using B-spline basis functions. ,course See formula

[0142]

[0143] It represents the arc length traversed over time. This represents the robot's heading as it changes over time. The linear velocity and angular velocity at each sampling time are then calculated using the derivative basis, as shown in the equation.

[0144]

[0145]

[0146] Let the linear velocity of the center of mass be denoted as . This represents the robot's angular velocity during heading; the superscript indicates the derivative with respect to time. Linear acceleration. angular acceleration ; jerk , It is obtained from the third derivative basis, where two superscripts represent the second time derivative and three superscripts represent the third time derivative.

[0147] After parameterizing the B-spline trajectory, differential-driven kinematics integration is performed. The inherent nonholonomic kinematic constraints of the differential-driven vehicle are shown in the equation.

[0148]

[0149] Represents velocity in the x-direction. This represents the velocity in the y-direction, and this constraint means that the differential robot has no lateral velocity perpendicular to the vehicle's orientation. Define the heading vector. The superscript T represents the transpose of a vector, so the velocity vector... , This represents the robot's planar velocity vector. This invention does not use planar coordinates. Instead of setting them as independent optimization variables, trapezoidal Heun integrals are used to iteratively solve for the robot's planar position based on the linear velocity and heading at sampling time intervals. During the sampling time interval... Inside, The time step represents the time step between two adjacent sampling points. The iterative formula for the trapezoidal integral is shown in equation [equation missing].

[0150]

[0151] Subscript Representing the Each sampling time, for Planar position at time, for Timeline velocity, for Heading at any given moment. Position increment. All positions at any given time are from the initial position. The increment is obtained by accumulating it over time, as shown in the formula.

[0152]

[0153] This represents the initial position of the trajectory. The structure of this integral naturally satisfies the difference-driven nonholonomic kinematic constraints, eliminating the need to introduce large-weight soft constraints for approximation and avoiding the problem of ill-conditioned optimization. Ill-conditioned optimization refers to the approximation of constraints by weight penalties, which easily leads to unstable solutions.

[0154] To address the high computational overhead associated with explicitly constructing the Jacobian matrix, this invention... Accompanying the gradient derivation work, This represents the total number of trajectory sampling points; if we directly and explicitly construct the position-velocity Jacobian matrix, which is a matrix composed of all partial derivatives, the complexity will be... Dense matrices; this invention uses reverse adjoint recursion, and the overall gradient computation complexity is... In other words, the computational load is directly proportional to the number of sampling points. (Position increment) Sampling time Will participate at the same time , The partial derivatives for calculating the increments at the two positions are shown in the equation.

[0155]

[0156] Partial derivative sign This represents the partial derivative operation. Define the vertical heading vector. The vector is perpendicular to the robot's orientation; similarly, the partial derivative of the heading is shown in the equation.

[0157]

[0158] ; Only affects The position, that is, the velocity at the current moment, only affects the position at this moment and thereafter. The total derivative is shown in the equation.

[0159]

[0160]

[0161] Define the cost function , This represents the total cost of optimization, taking into account comprehensive coverage, trajectory smoothing, collisions, and hardware constraints. Representational cost function For the first Robot planar position at sampling time The gradient represents the direction in which the cost changes with position. Chain expansion of the cost versus velocity gradient is shown in equation [equation missing].

[0162]

[0163] Define location companion quantity The accompanying variable is an intermediate variable in the reverse recursion, representing the result from... The sum of gradient vectors at all positions backward at time t is given by equation [equation missing].

[0164]

[0165] Set recursive boundary conditions: The index of the last sampling time in the sampling sequence is denoted as... That is, the end point of the trajectory, boundary conditions. The reverse recursive formula is shown in the equation.

[0166]

[0167] Iterate from the end of the trajectory towards the starting point. Substitute the values ​​to obtain the cost pairs. The gradient expression is shown in equation [equation missing].

[0168]

[0169] The gradient allocation by interval is implemented as shown in the equation.

[0170]

[0171] This represents cumulative assignment. Similarly, the heading adjoint gradient is shown in the formula.

[0172]

[0173] Simultaneously superimposed with direct costs from the heading. , This represents the cost gradient directly related to the heading. (Reverse scan) ,common Each step of the calculation has constant time complexity, and the overall gradient calculation has linear complexity. B-spline gradient scattering and multi-circle collision detection of the vehicle body also have corresponding computational complexities. The overall computational cost of the entire gradient solution process is controllable. The sampled-level gradient is obtained. , Subsequently, based on the local compact support characteristics of the B-spline, the gradient scattering of the sampling points is mapped onto the arc length control points and the heading control points. This represents the set of control point support indices corresponding to the sampling point, as shown in the formula.

[0174]

[0175]

[0176] superscript This represents the derivative basis of the corresponding order; the fixed endpoint control points will not receive gradients, only the intermediate free control points participate in the subsequent optimization solution.

[0177] After gradient derivation, this invention performs ESDF-free rebound collision elimination processing. Throughout the entire process, a global ESDF symbolic distance field is not pre-constructed. ESDF is a Euclidean symbolic distance field that stores the distance from each point on the entire map to obstacles; obstacle avoidance optimization is only performed by extracting obstacle-related information at local locations where collisions are detected. A multi-circle model is used to approximate the robot's body contour. The result was obtained by fitting the shapes together with the circles. This represents the total number of equivalent circles; each circle has a fixed offset vector in the vehicle body coordinate system. and the corresponding radius subscript Representing the The vehicle body is equivalent to a circle; coordinate transformation is achieved using a rotation matrix, which is shown in equation [equation missing].

[0178]

[0179] This is the vehicle attitude rotation matrix, used to transform the vehicle's local coordinates to the world coordinate system. World coordinate system center coordinates: , Representing the The center of the circle in the world coordinate system The robot's current pose. Here are the world coordinates of the robot's center. In engineering implementation, multiple circles are typically arranged along the longitudinal direction of the vehicle body, with the radius of each circle being half the width of the robot body. The robot pose is then iterated through all sampled points along the trajectory, multi-circle collision detection is performed, and the corresponding collision mask is output. , A Boolean flag is used to indicate whether a collision occurred at the current sampling time. Then, consecutively colliding sampling points are divided into independent collision segments. For each collision segment, the center position of the vehicle circle that intruded into the obstacle is extracted from the non-collision reference path corresponding to the current collision circle. , The center of the reference trajectory is the center of the circle corresponding to the current optimized trajectory; the collision center is the point from this reference center to the current optimized trajectory. Emit grid rays, The center of the current trajectory to be optimized is used as the reference point. The transition point where the ray first crosses free space and occupies the grid is found along the ray, and three local obstacle information items are extracted from this, namely the obstacle base point. (The point in space where the ray crosses the free-obstacle boundary), the outward normal unit vector pointing to free space. Safe distance , This represents an additional safety margin; the entire extraction process only processes the local area where the collision occurred and does not perform distance field calculations on the entire map. It defines the directed distance between the vehicle's circle and the obstacle's base point. , For signed distance; constructed on the activation boundary Place The penalty function for three consecutive collisions. Continuity represents the continuity of the function itself and its first derivative, as shown in the equation.

[0180]

[0181] This represents the penalty for collision. The set of all active collision constraints stores the sampling time of the collision and the circle number of the vehicle body. Collision penalty weights are applied; higher weights result in stronger collision penalties. The weights are iteratively updated according to a rule that is applied in each outer loop iteration. , This is the weighting growth factor. This represents the number of iterations in the Rebound loop. The collision gradient is obtained by differentiating the penalty function. This gradient, along the opposite direction of the obstacle's outward normal, propels the vehicle away from the obstacle into free space. The derivative with respect to the center of the vehicle's circle is calculated as follows:

[0182]

[0183] The gradient at the center plane is transferred to the robot's planar position and heading using the chain rule. , It is the identity matrix; take the derivative with respect to the heading. , is a 90-degree rotation matrix. The heading gradient is shown in the formula.

[0184]

[0185] This gradient simultaneously supports translating and rotating the vehicle to avoid obstacles. The position gradient is then fed into the previously derived adjoint recursive link, further propagating to the control points of the B-spline. The Rebound outer loop iterative processing is then initiated. Rebound is the iterative obstacle avoidance outer loop designed in this application. Before each iteration, the collision penalty weights are updated. During the outer loop iteration, the collision constraint set obtained in the previous round is not directly discarded; instead, historical constraints are accumulated and added to the constraint set of the current round. This prevents collision points from migrating back and forth between adjacent sampling points. Backend optimization and collision detection are repeatedly executed until there are no collisions along the entire trajectory or the algorithm reaches the pre-set maximum number of iterations.

[0186] After completing the collision elimination iterations, wheel space constraint processing is performed. This invention directly applies constraints at the wheel space level of the differential drive chassis, where wheel space is the physical space formed by the velocities and accelerations of the left and right wheels; rather than simply restricting linear and angular velocities at the center of mass level. This processing method is more in line with the actual execution characteristics of the differential drive chassis hardware. The robot's wheelbase is known. , Represents the physical distance between the centers of the robot's left and right wheels; determined by the linear velocity of the center of mass. angular velocity The velocities of the robot's left and right wheels are obtained by mapping, based on the acceleration along the center of mass. angular acceleration The acceleration of the robot's left and right wheels can be mapped, and the lateral acceleration can also be calculated. Lateral acceleration represents the equivalent acceleration perpendicular to the vehicle's orientation. The formulas for calculating the speeds of the left and right wheels are shown in equation [formula missing].

[0187]

[0188] For the speed of the left wheel, Let be the velocity of the right wheel. The formulas for calculating the acceleration of both wheels are shown in the equation.

[0189]

[0190] For the acceleration of the revolver, For the right wheel acceleration, a second-order soft penalty term is established for each of the physical quantities, including left wheel velocity, right wheel velocity, left wheel acceleration, right wheel acceleration, and lateral acceleration. The soft penalty means that the cost only applies when the limit is exceeded, without forcibly truncating the limit. All penalty terms are incorporated into the overall optimization objective. Generally refers to restricted physical quantities. This represents the upper limit of the hardware for this physical quantity. Represents the penalty weight; penalty terms are shown in the formula.

[0191]

[0192] This means that when the value is less than zero, the cost is zero, meaning the cost is zero if the limit is not exceeded. Only when... Penalty activation, gradient expression is shown in equation.

[0193]

[0194] Solve for the gradient corresponding to each excess penalty term, and denote the round space gradient. , Then, using the chain rule, the gradient of the wheel space dimension is mapped back to the linear velocity of the center of mass. angular velocity And the corresponding acceleration variables, see equation.

[0195]

[0196]

[0197] Similarly, acceleration gradient , superscript The gradient represents the acceleration; the superposition of lateral acceleration gradients. , , This represents the gradient corresponding to the lateral acceleration.

[0198] After wheel space constraints are applied, time scaling is performed. Once trajectory optimization is complete, wheel space hard verification is conducted. Hard verification means that physical verification is performed directly without penalties. The wheel speed over-limit ratio is calculated separately. and wheel acceleration over-limit ratio , Wheel speed hardware limit, The upper limit of wheel acceleration hardware is shown in the formula.

[0199]

[0200]

[0201] Due to the magnitude of the linear velocity and the total trajectory time Inversely proportional to the magnitude of acceleration and the total trajectory time The square of the equation is inversely proportional to the speed of the wheel; physically, this lengthens the total time, thus slowing down the overall motion. To ensure that both wheel speed and wheel acceleration meet the hardware's upper limit requirements, a safety factor is introduced, taking into account the wheel speed over-limit ratio and wheel acceleration over-limit ratio. A safety margin is reserved, and the scaling factor is calculated. The scaling factor represents the total time magnification factor, as shown in the formula.

[0202]

[0203] The total time after scaling meets the requirements. Increase total time By reducing the wheel speed and wheel acceleration corresponding to all sampling points to conform to the physical limitations of the chassis hardware, the time scaling operation can be executed at most. wheel, The maximum number of iterations is set for time scaling to prevent the trajectory duration from being infinitely amplified.

[0204] When the robot is actually performing inspection tasks, it relies on a finite state machine to complete the state transitions of the overall work process. The finite state machine includes the following states: INIT (system initialization), WAIT_GOAL (waiting for the task target), GEN_NEW_TRAJ (generating a new trajectory), EXEC_TRAJ (executing the trajectory), REPLAN_TRAJ (local replanning), and EMERGENCY_STOP (emergency stop). When the robot is in the EXEC_TRAJ trajectory execution state, it selects the future... Predicted time window Represents the time length for forward prediction; the set of sampling indices within the window. , Represents the current actual time of the robot; the trajectory sampling within the window performs a multi-circle vehicle collision pre-detection operation. If a collision is detected within the predicted time window, the system switches to the REPLAN_TRAJ state, takes the robot's current actual pose as the new starting point, and re-executes the SE(2) state grid A* front-end search and back-end trajectory optimization process; if the algorithm cannot solve for a safe and feasible collision-free trajectory, the system switches to the EMERGENCY_STOP state, controls the robot to perform an emergency stop operation, and ensures on-site operation safety.

[0205] Each group of adjacent camera points obtained by sorting the TSP output , Represents the starting point pose, Representing the final pose, the complete planner call steps are as follows: Call the SE(2) state grid A* search module to solve and output a collision-free guide path; then resample the guide path to obtain a number of... The system first establishes the reference pose of each component; then it performs B-spline fitting to initialize the arc length control points and heading control points; subsequently, it enters the Rebound loop and time scaling loop, performing L-BFGS-B backend optimization in each iteration (L-BFGS-B is a bounded constraint quasi-Newton numerical optimizer); trajectory collision detection is performed; local obstacle information is extracted from the Rebound loop, and optimization is performed again after accumulating historical collision constraints; wheel space hard verification is performed, and time scaling is executed if the hardware parameters do not meet the requirements. After all iterations are completed, a final multi-dimensional hard condition verification is performed, which includes verifying the trajectory to be collision-free throughout and the wheel speed exceeding the limit ratio. Wheel acceleration over-limit ratio End point position error End-point heading error Only when all verification conditions are met will the final executable smooth inspection trajectory of the robot be output.

[0206] This embodiment divides the inspection site into two main modules: upper-layer coverage planning and lower-layer trajectory planning. Data connection between the two modules is achieved through unit conversion. The upper layer discretizes the inspection area using a grid, obtains visible polygons through ray casting, and quickly calculates the optimal camera orientation using residual masks and FFT recurrent convolution. A greedy algorithm is used to filter camera point orientations, and the traveling salesman problem is solved to obtain the inspection visit sequence. The lower layer generates a guiding path, parameterizes the trajectory using B-splines, satisfies the differential driving kinematic constraints using trapezoidal integrals, completes gradient calculation using adjoint recursion, implements ESDF-free obstacle avoidance using rebound loops, and combines wheel space constraints, time scaling, and a running state machine to finally output an executable inspection trajectory for the robot.

[0207] Example 2: To verify the overall effectiveness of the proposed automatic inspection planning method for a greedy coverage and ESDF-free differential inspection robot, an 8m×8m simulated inspection scenario containing multiple types of discrete obstacles was constructed to conduct a full-process simulation test. Figure 4 As shown, the simulation simultaneously evaluates the comprehensive performance of the upper-layer sensor greedy coverage point selection, rapid camera orientation calculation, inspection point sequence optimization, and the lower-layer differential-driven robot nonholonomic constraint trajectory generation, local obstacle avoidance without ESDF, and chassis physical constraint adaptation. The simulation scenario's free space is fragmented by multiple obstacles of varying sizes and shapes, with several narrow passageways, representing a typical construction site or warehouse inspection environment. The overall simulation output corresponds to the inspection loop shown in the attached diagram.

[0208] The upper-level module outputs 30 camera deployment points and their corresponding optimal observation orientations. After the TSP completes the access sequence planning, a total of 29 migration trajectories between the points are generated. Combined with the attached figures, we can observe the output characteristics of several algorithms: the inspection path is not simply a series of nearby points, and there is a long-distance migration between some camera points across regions, which confirms the effect of the 2-opt algorithm in optimizing the global length of the TSP path; different trajectory segments show different motion patterns, some segments are smooth turns with large curvature, which are suitable for robot turning and passing, and some segments are about straight-line travel; in the crowded area in the middle of the site where obstacles are highly concentrated, the trajectory does not simply travel along the edge of the obstacle, but actively retreats inward to form a safe gap, which corresponds to the effect of the Rebound loop of this invention to generate collision penalty gradients based on local obstacle information iteratively; the endpoint of each colored trajectory converges to the camera point, and the heading at the end of the trajectory is consistent with the arrow pointing on the camera point in the figure, which reflects the constraint effect of the B-spline control point on the endpoint pose; the simulation title indicates that all 29 trajectory segments are OK, which shows that under the conditions of obstacle segmentation and point dispersion, the SE(2)A* front-end search combined with the back-end trajectory optimization without ESDF can converge to a feasible solution for all point pairings, and there are no planning failure segments.

[0209] After trajectory generation, hardware constraint verification and runtime dynamic obstacle response testing were further conducted. The corresponding complete inspection trajectory was batch-verified for wheel speed, wheel acceleration, endpoint position, and heading error. After wheel space soft penalty and post-processing time scaling, all trajectory segments met the chassis hardware threshold and point-of-sight docking accuracy indicators. During the simulation, the robot simulated sudden situations where new obstacles were added to the work site. While executing tasks along the inspection loop, the robot continuously performed multi-circle collision pre-detection on the trajectory for future time periods. Once a potential collision risk was detected, local replanning was triggered; if replanning failed, the robot entered an emergency stop protection state.

[0210] This embodiment demonstrates that the entire process of coverage selection, point sorting, and executable robot trajectory output can be completed in an integrated manner. In inspection environments with complex obstacle distribution, it can stably output a complete inspection solution that meets coverage requirements, kinematic constraints, hardware limits, and safety obstacle avoidance requirements.

[0211] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for automatic inspection planning of a differential inspection robot based on greedy coverage and without ESDF, characterized in that, include: The rectangular field to be inspected is discretized into a uniform grid map. Ray projection is performed on each candidate camera deployment point to construct the visible polygon of that point, thus obtaining the grid set that each candidate position can cover under different orientations. Construct the current residual mask, discretize the residual mask into angle histograms according to angles, perform cyclic convolution of the angle histograms with the rectangular window functions corresponding to the field of view angles, and calculate the coverage gain value of all candidate orientations at once through fast Fourier transform; Based on the submodality of coverage gain, a greedy strategy is used to select the camera deployment location and corresponding orientation with the largest marginal coverage gain in round by round until the preset coverage threshold is met. For the selected camera deployment points, the inspection visit sequence is determined by solving the traveling salesman problem; For each pair of adjacent camera deployment points in the inspection path, an A* search is performed in the SE(2) state lattice space to generate a collision-free guidance path, and the guidance path is parameterized using uniform cubic B-splines. The arc length control point and the heading control point are used as optimization variables to obtain the initial trajectory. The vehicle pose sequence is recovered from the linear velocity and heading of the trajectory using trapezoidal integrals, ensuring that the differential drive nonholonomic kinematic constraints are strictly satisfied structurally, without treating the planar position coordinates as independent optimization variables; the adjoint recursion algorithm is used to calculate the gradient of the objective function with respect to the linear velocity and heading at the sampling time, and the gradient calculation complexity is linear to the number of trajectory sampling points. In the trajectory optimization iteration, a multi-circle model is used to represent the vehicle body outline. For the sampling point where a collision is detected, a ray is emitted from the preset collision-free reference path to the center of the current collision circle, and only the local obstacle base point, outward normal, and safety distance information are extracted. Within the safety distance range, a C¹ consecutive cubic penalty function is applied to generate a collision elimination gradient along the outward normal direction. Historical collision constraints are accumulated through the Rebound outer loop, and the collision penalty weight is amplified round by round until the trajectory is collision-free throughout. A secondary soft penalty term for exceeding the limit is added to the optimization objective for the speed and acceleration of the left and right wheels; after optimization, if the wheel speed or wheel acceleration still exceeds the physical limit, the speed curve is scaled proportionally by increasing the total trajectory time until the chassis physical constraints are met. During the inspection process, the robot performs collision pre-checks on the planned trajectory for the future preset time period. If a new obstacle is detected that causes a collision, a local replanning or emergency stop state machine is triggered.

2. The method according to claim 1, wherein, The step of constructing the visible polygon of the point includes two processing steps: ray-obstacle line segment intersection processing and visible polygon contour generation. The intersection process of the ray-obstacle line segment is as follows: An observation point is set. A ray is emitted outward from the observation point, with a unit vector in the direction of the ray. The ray parameter equation is denoted as , parameters in the formula Take any obstacle boundary line segment, with its two endpoints as... , Line segment direction vector The parametric equation of a line segment is denoted as , parameters in the formula Define the two-dimensional cross product operation. Let vector Solve the parametric equations of the ray and the line segment simultaneously, and solve for the intersection parameters using cross product elimination: when When the ray is parallel to the boundary line segment of the obstacle, it is determined that the ray is parallel to the boundary line segment of the obstacle; when At the same time satisfy , When determining whether a ray intersects with an obstacle boundary segment, the parameters are selected after a single ray traverses all obstacle boundary segments. The effective intersection point with the smallest value is taken as the nearest obstacle intersection point in the direction of that ray; The visible polygon outline is generated by emitting a set of equally spaced probe rays at the observation point, obtaining the nearest obstacle intersection point corresponding to each probe ray, and connecting all the nearest obstacle intersection points in sequence according to the order of the ray emission angles to form a closed polygon, thus obtaining the visible polygon corresponding to the observation point.

3. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 2, characterized in that, The step of constructing the current residual mask and discretizing the residual mask into an angle histogram by angle includes residual mask construction and angle histogram statistics; The residual mask is constructed as follows: This is the omnidirectional view mask corresponding to the current candidate camera deployment point. Mark whether the grid is inside the omnidirectional view polygon of the candidate camera; The union overlay mask corresponding to all cameras that have been deployed. Check if the marker grid has been covered by deployed cameras; For obstacle masking, The marker grid indicates whether a region belongs to an obstacle; the residual mask expression is: residual mask A value of true indicates that the corresponding grid cell is an uncovered, non-obstacle grid cell that can be observed by the current candidate camera. The angle histogram statistics are obtained by traversing the residual mask. All grid cells with a value of true are selected, and the center coordinates of the grid cells are obtained. Calculate the center of the grid relative to the observation point. azimuth ; azimuth angle Divide the data into corresponding angle bins, and calculate the cumulative sum of the faces of all valid grid cells within each angle bin to obtain an angle histogram. .

4. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 3, is characterized in that, The step of performing a circular convolution between the angle histogram and the rectangular window function corresponding to the field of view, and calculating the coverage gain value of all candidate orientations at once through fast Fourier transform, includes configuring a circular rectangular window and performing an FFT-based circular convolution operation to solve for the coverage gain corresponding to each candidate orientation. Configure a cyclic rectangular window and calculate the total number of angle bins occupied by the field of view (FOV) based on the camera's FOV. ; Set the period equal to the total number of angle discrete values. Circular rectangular window , Expression for a looping rectangular window: In the formula The index of the cyclic window is 1. The index range of the cyclic rectangular window with an index of 1 corresponds to the angle bin range covered by the camera's sector view. Perform a circular convolution operation on the angle histogram. With cyclic rectangular window Perform a circular convolution operation and output the convolution. , Take the first for the camera The residual coverage gain corresponding to each candidate orientation; according to the circular convolution theorem, convolution calculation is performed through Fourier transform: In the formula Represents Fast Fourier Transform, Represents the inverse fast Fourier transform. This represents the element-wise multiplication of a complex array; iterate through the convolution output. Take all values ​​and use the index corresponding to the maximum value to determine the optimal camera orientation.

5. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 4, characterized in that, The step of determining the inspection visit order for the selected camera deployment points by solving the traveling salesman problem includes: The initial access sequence of the open TSP is generated using the greedy nearest neighbor rule, with the specified robot starting point as the first access node of the sequence; at each step, the point with the smallest Euclidean distance from the current node is selected from the set of unvisited camera points as the next access node, until all camera points are included in the access sequence. The 2-opt algorithm is used to iteratively optimize the initial access sequence. During the iteration, disconnection and reconnection operations are performed on two sets of edges in the sequence. The iteration terminates when the total path length no longer decreases, and the open inspection access sequence of the camera points is output.

6. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 5, characterized in that, The step of parameterizing the guidance path using uniform cubic B-splines and using arc length control points and heading control points as optimization variables to obtain the initial trajectory includes constructing a trajectory expression using uniform cubic B-spline basis functions. Four tightly supported basis functions for a uniform cubic B-spline: In the formula These are local parameters within a single segment of the B-spline; Control points by arc length Heading control points As optimization variables, the expressions for the arc length of the trajectory changing with time and the expression for the heading changing with time are: The first, second, and third derivative bases are obtained by analytically differentiating the B-spline basis functions. Combined with the time interval of a single trajectory segment, the linear velocity, angular velocity, linear acceleration, and angular acceleration corresponding to the trajectory sampling time are calculated from the derivative bases.

7. The automatic inspection planning method for a greedy coverage and ESDF-free differential inspection robot according to claim 6, characterized in that, The step of recovering the vehicle pose sequence from the linear velocity and heading of the trajectory using trapezoidal integral includes: Set a fixed sampling time interval , For sampling time index, For the first Linear velocity at the sampling time, For the first Heading angle at sampling time; define heading vector ; Representing the The robot's two-dimensional planar position at the sampling moment; trapezoidal integral iterative formula: From the initial position Iterate every time step, accumulating the position increments to obtain the robot's planar position at all sampling times; differential driving kinematic constraints. , It is achieved by the structural constraints of this integral iterative formula.

8. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 7, characterized in that, The steps of using the adjoint recursion algorithm to calculate the gradient of the objective function with respect to the linear velocity and heading at the sampling time include defining the position adjoint quantity, setting boundary conditions, recursively calculating the adjoint quantity in reverse, and then solving for the gradient at the sampling point from the adjoint quantity. Define the cost function as , Cost function For the Robot planar position at sampling time The gradient; defining the positional adjoint. The location-related quantity is from the sampling time. Start by summing the vectors of gradients at all subsequent positions: ; Set recursive boundary conditions, and denote the index of the last sampling time in the sampling sequence as... Boundary conditions ; Perform recursive calculations backwards from the end of the sequence to the beginning of the sequence. The associated recursive relation is: ; Based on the already obtained covariates , Calculate the cost function for the first Sampling time linear velocity gradient: Similarly, calculate the cost function for the first... Sampling time heading The gradient at the sampling time is calculated; after the gradient calculation is completed, the gradient at the sampling time is scattered and mapped to the corresponding arc length control point and heading control point based on the local tight support characteristics of the B spline.

9. The automatic inspection planning method for a robot based on greedy coverage and ESDF-free differential inspection as described in claim 8, characterized in that, The steps of applying a continuous cubic penalty function C¹ within the safe distance range to generate a collision elimination gradient along the outward normal direction; accumulating historical collision constraints through the Rebound outer loop and amplifying the collision penalty weight round by round include the construction of the cubic penalty function and the iterative processing of the Rebound outer loop. The set of all active collision constraints; Represents the trajectory sampling index. Represents the circle index within the multi-circle model of the vehicle body; For the first Sampling time number The directional spacing corresponding to each circle of the vehicle body. , Let the world coordinates be the center of the vehicle body circle. As the barrier base point, The unit vector normal to the obstacle. To establish a preset safe distance; Collision penalty weights; cubic penalty function: For the preset weight growth coefficient and , For each iteration of the outer loop, before the start of each iteration of the Rebound outer loop, the collision penalty weight is updated by setting the value of the collision penalty weight to the sum of the current weight and the value of the collision penalty weight. of The product of the two is raised to the power of ...

10. The automatic inspection planning method for a greedy coverage and ESDF-free differential inspection robot according to claim 9, characterized in that, The step of adding a secondary soft penalty term for the speed and acceleration of the left and right wheels to the optimization target; after optimization, if the wheel speed or wheel acceleration still exceeds the physical limit, the step of scaling the speed curve proportionally by increasing the total trajectory time includes performing wheel space mapping, secondary soft penalty for exceeding the limit, and time scaling processing. For differential robot wheelbase, Let the linear velocity of the center of mass be denoted as . The angular velocity of the center of mass, For the acceleration along the center of mass, Here is the angular acceleration of the center of mass; the formula for mapping the motion parameters of the center of mass to the motion parameters of the left and right wheels is: For the revolver speed obtained from the mapping Right wheel speed Revolver acceleration Right wheel acceleration Construct two soft penalty terms for exceeding the limit respectively; Time scaling processing calculates the wheel speed over-limit ratio. Wheel acceleration over-limit ratio ;based on and Calculate the scaling factor ; Complete the time-scale scaling of the trajectory to reduce the original total time of the trajectory. With scaling factor Multiply by the product to obtain the scaled total time of the trajectory, thus completing the trajectory time-scale scaling.

Images