Method and system for acquiring dynamic inspection route of unmanned aerial vehicle based on laser point cloud

Abstract

The invention relates to the field of unmanned aerial vehicle route acquisition, in particular to an unmanned aerial vehicle dynamic inspection route acquisition method and system based on laser point cloud. The method comprises the steps of comprehensive processing of three-dimensional point cloud data, marking of equipment inspection point positions to generate waypoints, safe distance collision inspection of the waypoints, position adjustment of the waypoints, planning of inspection paths and the like, through comprehensive processing of the three-dimensional point cloud data, including denoising, classification and thinning, the quality of the point cloud data can be remarkably improved, and by means of the laser point cloud technology, the quality of the three-dimensional point cloud data is improved. The terrain and equipment characteristics of the to-be-inspected area can be accurately captured, so that the accuracy and reliability of inspection are improved. The steps of waypoint safe distance collision inspection and waypoint position adjustment further enhance the autonomous flight ability of the unmanned aerial vehicle, and ensure the safety and stability of the inspection process. When the routing inspection path is planned, an optimal safe path algorithm is adopted, the path between two waypoints is searched, a local route is generated, and finally the local route is connected to form a complete route, so that the high efficiency and integrity of routing inspection are ensured.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) route acquisition, and in particular to a method and system for acquiring a UAV dynamic inspection route based on a laser point cloud. Technical Background

[0002] Autonomous inspection technology based on route planning by drones has been increasingly widely used in the power industry due to its high inspection efficiency and excellent safety. However, the routes currently used by drones mainly rely on manual control of drones to generate points, or are planned through laser point cloud technology. This type of route belongs to the fixed path type. The drone will fly from the first inspection point to the last inspection point in sequence, fully covering and inspecting all preset inspection points of the power grid equipment, which is usually suitable for routine inspection tasks of power grid equipment. However, for the needs of fault inspection and temporary inspection, this type of fixed-path route seems to be inadequate, because these inspection tasks do not need to traverse all points of the equipment, but only need to conduct targeted inspections on specific points of concern.

[0003] Existing route planning schemes for power grid equipment drones mainly include two methods: one is based on manually controlled drones to generate points, and the other is to use laser point cloud technology to plan routes.

[0004] Based on manual control, this method requires the operator to manually control the drone to fly to each key point that needs to be inspected and record the coordinates of these locations. After that, the drone can automatically fly according to these pre-set coordinates. This method is simple and intuitive, and is suitable for situations where the equipment layout is relatively stable and the inspection requirements are clear. However, its disadvantage is that the waypoints need to be manually set before each inspection, which is time-consuming and lacks flexibility when dealing with emergencies or temporary inspection tasks.

[0005] Using laser point cloud technology for route planning, LiDAR technology can generate accurate three-dimensional terrain models, and then design the flight route of the drone based on these data. This method can provide more accurate terrain information, help drones avoid obstacles, and achieve safe and efficient flight. Although this method is more intelligent than manual marking, it still has certain limitations when facing dynamically changing environments or real-time inspection needs at specific points.

[0006] The characteristics of the fixed path type of route, whether it is a route planned by manual dotting or laser point cloud technology, all belong to the fixed path type. This means that once the route is determined, the drone will fly from the first point to the next point step by step until all the preset points are inspected. This fixed path is suitable for tasks that require regular comprehensive inspections, but it is not flexible enough for situations where only a quick response or troubleshooting of a specific area is required. Summary of the invention

[0007] In view of the constraints of current technology, the present invention proposes a method and system for obtaining a dynamic inspection route for UAVs based on laser point clouds. The method relies on the three-dimensional point cloud data of power grid equipment, the location information of UAVs, the attitude information of the gimbal camera, the point cloud obstacle avoidance and the path planning algorithm, and automatically generates the optimal flight path according to the real-time data and current environmental conditions, thus realizing the rapid generation of dynamic inspection routes for specific points of interest. This not only improves the flexibility of UAV inspection operations, but also enhances the ability to respond to emergencies.

[0008] The present invention can be implemented through the following technical solutions:

[0009] A method for acquiring a dynamic inspection route of an unmanned aerial vehicle based on laser point cloud, characterized in that it comprises the following steps:

[0010] S1: Comprehensive processing of 3D point cloud data: Comprehensive processing of the collected 3D point cloud data of the area to be inspected, including denoising, classification and thinning of the point cloud data;

[0011] S2: Mark equipment inspection points to generate waypoints: Divide the point cloud of the area to be inspected into multiple sub-areas. In each sub-area, mark the equipment inspection points to be photographed. At the same time, generate corresponding waypoints based on the photographing distance and the point cloud field of view angle.

[0012] S3: Waypoint safety distance collision check: Select all waypoints in each sub-area and set the safe obstacle avoidance distance; Generate a 3D point cloud 3D grid. The grid with point cloud is the impassable dangerous area. Then expand the dangerous area grid with the safe area to generate a map where the drone can pass;

[0013] S4: Waypoint position adjustment: If there is a dangerous waypoint, adjust the waypoint position and angle with the equipment patrol point as the center and the preset safety distance as the adjustment boundary until the waypoint is safe;

[0014] S5: Planning the inspection route: According to the order of the selected waypoints, the previous and next waypoints use the optimal safe path algorithm to search for the path between the two waypoints and generate a local route; finally, all local area routes are connected to form a complete route.

[0015] The acquisition of the UAV dynamic inspection route based on the laser point cloud is achieved through comprehensive processing of 3D point cloud data, marking inspection points, conducting safety distance collision checks, adjusting waypoint positions and angles, and planning inspection routes. This method not only improves the efficiency and accuracy of inspections, but also reduces personal risks and UAV collision risks during inspections.

[0016] The method for acquiring a dynamic inspection route for an unmanned aerial vehicle based on laser point clouds is characterized in that, in the S1 step, the three-dimensional point cloud data is comprehensively processed: first, the collected three-dimensional point cloud data of the equipment in the inspection area is denoised, and outlier or erroneous data points are removed by statistical analysis or density-based filtering methods, thereby significantly improving the quality of the point cloud data; then, a clustering algorithm, such as DBSCAN or K-means, is used to segment and classify each independent single device in the point cloud data to achieve the individualization of the power equipment; then, data thinning is performed on the point cloud data of each single device, and the voxel grid filtering method is used to reduce the data volume while maintaining the shape characteristics and structural details of the point cloud.

[0017] The method for acquiring a UAV dynamic inspection route based on laser point cloud is characterized in that, in the step S1, the comprehensive processing of three-dimensional point cloud data includes the following steps:

[0018] s11: 3D point cloud data acquisition; perform detailed scans of high-voltage equipment, cameras and other structures within the power grid from multiple angles and positions to ensure that all critical areas are covered; after acquisition, perform data synchronization, correction and feature point matching to ensure accurate alignment of data from different scanning perspectives;

[0019] s12: Point cloud data denoising: Use a denoising algorithm based on a Gaussian mathematical model or point cloud processing software related operations to pre-process the 3D point cloud data;

[0020] s13: Equipment individualization: By using clustering algorithms, such as DBSCAN or K-means, or related operations in point cloud processing software, the point cloud data is effectively cut and classified to construct individual point cloud data of each power equipment;

[0021] s14: point cloud data thinning; the three-dimensional point cloud data of the individualized device obtained through the above point cloud acquisition and processing steps is pre-processed by using a thinning or sampling technique based on voxel grid filtering; this method is achieved by dividing the point cloud space into regular voxels, i.e., a three-dimensional grid, and then reducing the number of points inside each voxel.

[0022] The comprehensive processing of 3D point cloud data in step S1 achieves accurate acquisition and optimized processing of 3D point cloud data of key structures such as high-voltage equipment and cameras in the power grid through acquisition, denoising, individualization and thinning. These processing steps provide accurate and reliable basic data for subsequent equipment inspection point marking, safety distance collision inspection, waypoint position adjustment and inspection path planning.

[0023] The method for acquiring a UAV dynamic inspection route based on laser point cloud is characterized in that, in the step S2, the equipment inspection points are marked to generate waypoints: the UAV waypoint p is based on the equipment inspection point t as the origin, and the shooting distance is determined by manual input; the direction is determined based on the first-person perspective of the point cloud view; the shooting range is determined by the camera lens field of view angle and focal length multiple; the steps of marking the equipment inspection points and generating waypoints are as follows:

[0024] S21: Drag the equipment inspection point to the middle of the view, adjust the point cloud view angle, and mark a point on the point cloud where the equipment inspection point is located as the target point t1 for drone photography;

[0025] S22: automatically create the corresponding waypoint p1 according to the input shooting distance and point cloud view angle;

[0026] S23: Automatically obtain the shooting point cloud range based on the input shooting distance and camera lens focal length;

[0027] S24: Create drone waypoints corresponding to all equipment inspection points.

[0028] Step S2 determines the equipment inspection points through human-computer interaction, and accurately calculates and creates corresponding drone waypoints based on input parameters such as shooting distance, point cloud view angle, and camera lens focal length. These waypoints will serve as flight targets and shooting locations during drone inspections, providing accurate and reliable basic data for subsequent route planning and flight control.

[0029] The method for acquiring a UAV dynamic inspection route based on laser point cloud, in the S3 step, the waypoint safety distance collision check: select all waypoints in each sub-area, set the safe obstacle avoidance distance; generate a 3D grid at a set interval, the grid with point cloud is an impassable dangerous area, and then expand the dangerous area grid with the safe area to generate a map that the UAV can pass, and mark the waypoints in the dangerous area; the steps are as follows:

[0030] S31: Create a three-dimensional grid space, and construct a three-dimensional grid space of nx×ny×nz according to the value range of the valid point cloud data, where x, y, and z are the coordinates of the point cloud in the space;

[0031] S32: Construct an occupied grid map, project the points in the point cloud of the power grid equipment to the grid map in sequence, and when the grid is occupied by a point cloud, the drone cannot pass through;

[0032] S33: Inspection equipment expansion. Considering the size of the drone and the safety distance, the obstacles in the environment, i.e., the inspection equipment, are expanded. The expansion parameter E is introduced to represent the number of circles of obstacle expansion. Its value is shown in the following formula;

[0033] E=ceil(r / R),

[0034] Where r represents the sum of the UAV radius and the minimum safety distance, and R is the grid size, which represents the resolution of the map.

[0035] In the method for acquiring a dynamic inspection route of a UAV based on laser point cloud, in the step S4, the waypoint position is adjusted manually by adjusting the pitch angle and heading angle.

[0036] The method for acquiring a dynamic inspection route of a UAV based on laser point cloud, in the step S5, plans the inspection path: solves the optimal path for the safety inspection of power grid equipment, and uses a solution algorithm based on particle swarm optimization (PSO) algorithm; sets the safety inspection area as a path search space, and sets the space to contain several particles. In order to process the original particles of the PSO algorithm, the random generation algorithm is combined with the solution algorithm to obtain the initialization particles; obtains the global optimal safety inspection path through output iterative calculation; in order to avoid the optimization of the inspection path falling into the local optimum, the individual extreme value and the global extreme value are used to add a weight factor to the PSO algorithm to establish a global optimization PSO solution algorithm:

[0037] The improved particle flight speed is:

[0038]

[0039] In the formula: λ is the weight factor; l is the particle; δ is the flight speed; c1 and c2 are both learning factors; rand() is the random function; is the spatial position of particle l; I is the optimal position searched by a single particle; G is the optimal position searched by all particles in the population. start is the initial weight value; end is the weight value when the iteration ends; ψ is the total number of iterations of PSO operation;

[0040] Each iterative calculation will cause the particle population to be updated, and the spatial position of the updated particle is:

[0041]

[0042] Where: is the updated spatial position of particle l, is the current spatial position of particle l, δ l is the flight speed;

[0043] In the process of solving the optimal power grid equipment safety inspection path, in order to avoid the failure to solve the optimal path information due to the too fast movement speed of the population particles, the particle flight speed limit range is set to δ∈[δ min ,δ max ], where: δ min is the slowest flight speed; δ max For the fastest flight speed;

[0044] The convergence factor is introduced into the PSO algorithm to improve the convergence speed of automatic optimization of the safety inspection path of power grid equipment. The updated particle flight speed is:

[0045]

[0046] Where: ε is the convergence factor;

[0047]

[0048] Where: r is the side length of the search space.

[0049] The method for acquiring the dynamic inspection route of an unmanned aerial vehicle based on laser point cloud, in the automatic optimization of the safety inspection path, divides the entire safety inspection path into multiple segments according to the patrol information collection time of the path points, extracts the central node of each segment, and uses it as the central point of each search space, studies all segmented search spaces, sets the path points contained in each search space to have the same node posture, calculates the distance between adjacent path points in the space, and uses the objective function set by adding the distances of all path points as the data basis for calculating the fitness value.

[0050] The method for acquiring the dynamic inspection route of a UAV based on laser point cloud extracts one path point in each segmented search space, connects the path points to form the optimal safe inspection path, and uses the collision detection method for testing. The path point where the collision problem occurs is extracted, the segmented search space where it is located is reduced, and the position and speed of the PSO particle are repositioned until the collision detection is stopped when the collision problem no longer occurs; in order to meet the kinematic constraints, the optimal path solved is inversely solved to avoid the path generated by the automatic optimization containing path points that the inspection robot cannot reach. If there is a path point that is not in the motion space, the particle position and particle speed are updated, and another particle that meets the kinematic constraints is generated in the search space to adjust the optimized safe inspection path.

[0051] The method for acquiring a UAV dynamic inspection route based on laser point cloud is characterized in that:

[0052] s12: Point cloud data denoising; the specific steps are as follows:

[0053] ① Define the neighborhood of the point: define a neighborhood for each point P in the three-dimensional point cloud, specify a spherical neighborhood with a radius of r, including all points whose distance from point P is less than r;

[0054] ② Calculate the mean and covariance matrix of the neighborhood points: For the points in the neighborhood (x i ,y i ,z i ), calculate the mean vector of all points in the neighborhood, that is, the average position and covariance matrix. The mean vector and covariance matrix describe the spatial distribution of the points; the mean vector μ is a three-dimensional vector, and its calculation formula is as follows:

[0055]

[0056] Where n is the number of points in the neighborhood, μ is the mean vector, Σ is the covariance matrix, (μ x ,μ y ,μ z ) is the component vector of the mean vector, m is the total number of points in the neighborhood, X i is the i-th point in the neighborhood, (x i ,y i ,z i ) is the coordinate of the i-th point in the neighborhood;

[0057] The covariance matrix Σ is a 3×3 matrix that reflects the distribution of points in the neighborhood relative to the mean. Its calculation formula is as follows:

[0058]

[0059]

[0060] ③ Gaussian fitting: For each point in the neighborhood, the Mahalanobis distance between it and the neighborhood mean is calculated to evaluate the compactness of its distribution, and the covariance matrix is ​​used for measurement. The formula is as follows:

[0061]

[0062] Where X is a point in the neighborhood, μ is the mean of all points in the neighborhood, ∑ -1 is the inverse of the covariance matrix;

[0063] ④Noise determination: Based on the Mahalanobis distance D M (X), set a threshold τ, if D M(X) is greater than τ, then point X is considered to be a noise point;

[0064] ⑤ Point cloud data optimization: use Gaussian statistical denoising method, or use Gaussian kernel smoothing technology to further optimize point cloud data; or use denoising operation of point cloud processing software;

[0065] s13: Equipment monomerization; the specific steps are as follows:

[0066] ① Point cloud data standardization: After obtaining the rough individual device point cloud data through the above steps, the next step is to standardize the data to unify the coordinate system and data scale of all individual device point clouds to ensure the effect of the subsequent clustering algorithm;

[0067] ② Select clustering algorithm: Use DBSCAN algorithm to cluster the point cloud of power grid equipment; when applying DBSCAN algorithm, reasonably set the size of the neighborhood, i.e., radius R and the minimum number of points MinPts, to effectively distinguish and identify the point cloud data of various equipment;

[0068] ③ Perform clustering: First, identify all core points whose neighborhood R contains at least MinPts points and shows a high enough density so that they can be considered as part of a cluster; then define boundary points and noise points. Although the number of points in the neighborhood R of boundary points is less than MinPts, they are located in the R neighborhood of at least one core point. The number of points in the R neighborhood of noise points is less than MinPts and they do not belong to the neighborhood of any core point; then, starting from any unprocessed core point, a new cluster is formed by finding all core points that are density-reachable to it; density-reachable means that one can reach another core point from one core point through a series of core points in the neighborhood R; all density-reachable points starting from the core point, including boundary points, are classified into the same cluster; repeat this process until all points are processed and marked as belonging to a cluster or marked as noise;

[0069] ④ Single-unit confirmation: Apply the DBSCAN clustering algorithm to organize the point cloud into independent clusters according to its density. Each cluster corresponds to a single device. After the clustering process is completed, the clustering results are carefully analyzed and verified to ensure that each cluster accurately represents a specific device. If necessary, manual intervention is performed to correct any classification errors caused by the clustering algorithm.

[0070] ⑤ Segmentation and classification of point cloud processing software: In addition to the above clustering algorithms, point cloud processing software CloudCompare or LiPowerline can also be used to perform spatial segmentation and classification of point cloud data to effectively isolate and analyze specific equipment or structural areas.

[0071] ⑥ Denoising of individual device point cloud data: After roughly cutting and classifying the point cloud data and preliminarily obtaining the point cloud data of individual devices, the point cloud data of individual devices can be denoised again;

[0072] s14: Point cloud data thinning; the specific steps are as follows:

[0073] First, determine the size of the voxel. Each voxel is equivalent to a cube in space, and the size is determined by the side length l. For each point in the point cloud, it is assigned to the corresponding voxel according to its coordinates; in each voxel, all points will be merged into a representative point, which is usually the geometric center of all points in the voxel, that is, the average position;

[0074] ,

[0075] Where n is the number of points inside the voxel, (x i ,y i ,z i ) are the coordinates of the point inside the voxel.

[0076] A computer system comprises a storage medium and a processor, wherein the storage medium stores a computer program, and the processor implements the steps of any of the aforementioned methods when executing the computer program.

[0077] In the comprehensive processing of 3D point cloud data, the collected 3D point cloud data of the equipment is first denoised. Outliers or erroneous data points are removed through statistical analysis or density-based filtering methods, thereby significantly improving the quality of the point cloud data. Subsequently, clustering algorithms, such as DBSCAN or K-means, are used to segment and classify each independent unit device in the point cloud data to achieve the individualization of power equipment. Next, data thinning is performed on each unit device, usually using voxel grid filtering to reduce the amount of data to increase the calculation speed while maintaining the shape characteristics and structural details of the point cloud. This series of steps effectively ensures the transformation from raw point cloud data to high-quality equipment point cloud data.

[0078] In area division and waypoint generation, the point cloud data of the entire area to be inspected must first be divided into several smaller sub-areas. This is done to facilitate management and processing, and to make it easier to identify the location of specific components. For each sub-area, the operator needs to manually mark the key component points in the photos to be taken. Then, based on the predetermined shooting distance and the field of view of the point cloud data, waypoints are generated for drone navigation and photography. Specific software tools may be needed in this process to assist in completing the task and ensure the accuracy and completeness of the waypoint settings.

[0079] Once the waypoints are determined, it is necessary to consider how to plan a flight route that is both safe and covers the required checkpoints. This step involves creating a 3D grid where dangerous areas are marked as "not passable" if there is a point cloud within the grid. On this basis, a "safe distance" is further defined and a certain range is extended around the dangerous area as a buffer zone to create a map where the drone can fly safely. This process may involve complex spatial geometry calculations to ensure that the drone can avoid obstacles and maintain safe flight.

[0080] If some waypoints are found to be in unsafe positions, that is, they are too close to obstacles or other dangerous areas, the positions and angles of these waypoints need to be readjusted. When adjusting, the device patrol point is usually used as the reference center point, and the preset safety distance is used as the adjustment boundary to ensure that the adjusted waypoints can not only meet the shooting requirements, but also ensure the safe flight of the drone. This step also requires manual fine-tuning and safety verification to ensure that the final generated flight path is safe and feasible.

[0081] The last step is to select the waypoints corresponding to some equipment points that need attention based on all the waypoints determined previously, and use the point cloud path planning algorithm to calculate the best path connecting adjacent waypoints. These paths are called local routes, and they will be connected in a certain order to form a complete flight route. This step requires not only considering flight efficiency, but also ensuring that the drone can stay in a safe area throughout the flight. The generated flight path will guide the drone to perform automatic inspection tasks and improve the accuracy and efficiency of inspection work.

[0082] The present invention can significantly improve the quality of point cloud data through comprehensive processing of three-dimensional point cloud data, including denoising, classification and thinning, and provide more accurate basic data for subsequent inspection point marking and route planning.

[0083] By using laser point cloud technology, the terrain and equipment features of the area to be inspected can be accurately captured, thereby improving the accuracy and reliability of the inspection.

[0084] The present invention automatically processes point cloud data, marks inspection points, and generates corresponding waypoints, which greatly reduces manual intervention and improves inspection efficiency.

[0085] Waypoint safety distance collision check and waypoint position adjustment steps further enhance the autonomous flight capability of the drone, ensuring the safety and stability of the inspection process.

[0086] By setting a safe obstacle avoidance distance and generating a three-dimensional point cloud 3D grid, the present invention can identify and avoid potential danger areas and ensure the flight safety of the UAV.

[0087] In the case of dangerous waypoints, the present invention can automatically adjust the position and angle of the waypoints to adapt to different flight environments and equipment layouts, thereby enhancing the flexibility and adaptability of inspections.

[0088] When planning the inspection route, the present invention adopts the optimal safe path algorithm to search for the path between two waypoints and generate a local route, which is finally connected to form a complete route, ensuring the efficiency and integrity of the inspection.

[0089] The present invention can also dynamically adjust the route according to the actual situation of the area to be inspected to optimize the inspection efficiency and coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 This paper presents a method for acquiring dynamic inspection routes of UAVs based on laser point clouds.

[0091] Figure 2 It is a comprehensive processing flow for laser point cloud data.

[0092] Figure 3 is the camera lens field of view.

[0093] Figure 4 It is the spatial relationship between the UAV waypoints and the equipment inspection points.

[0094] Figure 5 It is the location of the UAV waypoints and equipment inspection points in the point cloud.

[0095] Figure 6 Capture range previews for point cloud views and drone waypoints.

[0096] Figure 7 This is a schematic diagram of the navigation grid map.

[0097] Figure 8 Adjust the waypoint position parameters.

[0098] Fig. 9 Schematic diagram of PSO search space.

[0099] Fig.10 Schematic diagram of path optimization. DETAILED DESCRIPTION

[0100] like Figure 1 As shown, the present invention provides a method, system and device for obtaining a dynamic inspection route of an unmanned aerial vehicle based on laser point cloud, and the technical solution adopted is:

[0101] S1: Comprehensive processing of 3D point cloud data:

[0102] (1) 3D point cloud data acquisition. For power grid equipment scenarios, a variety of scanning technologies can be used, such as ground static laser scanners, mobile scanning systems or laser scanning technology carried by drones, depth cameras, etc., to perform detailed scans of high-voltage equipment, cameras and other structures in the power grid from multiple angles and positions to ensure that the scan covers all key areas. During the scanning process, the device emits lasers and receives their reflected light to calculate the distance and form point cloud data. After the acquisition is completed, data synchronization, correction and feature point matching are performed to ensure that data from different scanning perspectives are accurately aligned. Through these steps, a high-quality 3D point cloud data of power grid equipment that accurately corresponds to the physical facilities is finally formed.

[0103] (2) Point cloud data denoising. The three-dimensional point cloud data collected in step S1(1) usually presents discrete point cloud characteristics, and due to the limitations of the scanning equipment and environmental factors, these data usually contain various noises and errors. In order to improve the quality of the point cloud, a denoising algorithm based on a Gaussian mathematical model or related operations of point cloud processing software can be used to pre-process the three-dimensional point cloud data. The denoising algorithm based on the Gaussian mathematical model mainly uses the statistical characteristics of the Gaussian distribution (normal distribution) to identify and remove noise points caused by random errors. The specific steps are as follows:

[0104] ① Define the neighborhood of the point: Define a neighborhood for each point P in the point cloud, usually specifying a spherical neighborhood with a radius of r, including all points whose distance from point P is less than r.

[0105] ② Calculate the mean and covariance matrix of the neighborhood points: For the points in the neighborhood (x i ,y i ,z i ), calculate the mean vector (average position) and covariance matrix of all points in the neighborhood. The mean vector and covariance matrix describe the spatial distribution of the points. The mean vector μ is a three-dimensional vector, and its calculation formula is as follows:

[0106]

[0107] Here, n is the number of points in the neighborhood.

[0108] The covariance matrix Σ is a 3×3 matrix that reflects the distribution of points in the neighborhood relative to the mean. Its calculation formula is as follows:

[0109]

[0110] ③ Gaussian fitting: In theory, most points in the neighborhood should be closely distributed around the mean. For each point in the neighborhood, the Mahalanobis distance between it and the neighborhood mean is calculated to evaluate the compactness of its distribution, and the covariance matrix is ​​used to measure it. The formula is as follows:

[0111]

[0112] Where X is a point in the neighborhood, μ is the mean of all points in the neighborhood, ∑ -1 is the inverse of the covariance matrix.

[0113] ④Noise determination: Based on the Mahalanobis distance D M (X), set a threshold τ. If D M If (X) is greater than τ, then point X is considered to be a noise point.

[0114] ⑤ Point cloud data optimization. In addition to the simple Gaussian statistical denoising method, Gaussian kernel smoothing technology can also be used to further optimize point cloud data. Gaussian kernel smoothing is a weighted local averaging technology, where the weight is determined by the Gaussian function, i.e., the closer the point is to the target point, the higher the weight. This method can effectively smooth the point cloud surface, thereby reducing random noise in the data and improving the overall quality of the data.

[0115] ⑥ Denoising operation of point cloud processing software. In addition to the above denoising algorithms, you can also use the ready-made denoising tools provided by the point cloud processing software to remove abnormal points (outliers), such as SOR Filter or NoiseFilter provided by CloudCompare. These tools are specially designed to detect and remove noise and outliers in point cloud data, thereby improving data quality and analysis accuracy, and providing a cleaner and more accurate data set for subsequent data processing and analysis. SOR Filter is based on statistical analysis methods. It identifies possible noise or abnormal points by comparing the distance distribution of each point with its surrounding points. It is suitable for removing irregular points caused by random measurement errors. Noise Filter focuses on the local geometric characteristics of the points, evaluates the geometric consistency of each point with its surrounding points, such as the smoothness of the surface or spatial continuity, and is suitable for processing errors or abnormal points caused by environmental interference or scanning equipment limitations. The combined use of these tools can significantly improve the processing quality and availability of point cloud data.

[0116] (3) Equipment individualization. By using clustering algorithms, such as DBSCAN or K-means, or related operations in point cloud processing software, the point cloud data is effectively cut and classified to construct individual point cloud data of each power equipment. The specific steps are as follows:

[0117] ① Point cloud data standardization. After obtaining the rough individual device point cloud data through the above steps, the next step is to standardize the data to unify the coordinate system and data scale of all individual device point clouds to ensure the effectiveness of the subsequent clustering algorithm.

[0118] ② Select a clustering algorithm. It is crucial to select an appropriate clustering algorithm based on the characteristics of the data. The DBSCAN algorithm is suitable for identifying clusters of arbitrary shapes and has good robustness to noise points, while the K-means algorithm is more suitable for processing large-scale data sets with regular shapes and approximate spherical shapes. Given that power grid equipment usually has complex and irregular shapes, and has diverse sizes and geometric structures, the DBSCAN algorithm has become an ideal choice for point cloud clustering of power grid equipment. When applying the DBSCAN algorithm, the key is to reasonably set the size of the neighborhood, i.e., the radius R and the minimum number of points MinPts, in order to effectively distinguish and identify the point cloud data of various devices.

[0119] ③ Perform clustering. First, identify all core points whose neighborhood R contains at least MinPts points and shows a high enough density so that they can be considered as part of a cluster. Then define boundary points and noise points. Although the number of points in the neighborhood R of the boundary points is less than MinPts, they are located in the R neighborhood of at least one core point. The number of points in the R neighborhood of the noise points is less than MinPts and they do not belong to the neighborhood of any core point. Then, starting from any unprocessed core point, a new cluster is formed by finding all core points that are density-reachable to it. Density-reachable means that one can reach another core point from one core point through a series of core points in the neighborhood R. All density-reachable points starting from the core point (including boundary points) are classified into the same cluster. Repeat this process until all points have been processed and marked as belonging to a cluster or marked as noise.

[0120] ④ Single-unit confirmation. The application of the DBSCAN clustering algorithm can effectively organize the point cloud into independent clusters based on its density, and each cluster corresponds to a single device. After the clustering process is completed, the clustering results need to be carefully analyzed and verified to ensure that each cluster accurately represents a specific device. In addition, manual intervention may be required to correct any classification errors caused by the clustering algorithm to ensure the accuracy and reliability of the data.

[0121] ⑤ Segmentation and classification of point cloud processing software. In addition to the above-mentioned clustering algorithms, point cloud processing software such as CloudCompare or LiPowerline can also be used to perform spatial cutting and classification of point cloud data, effectively isolating and analyzing specific equipment or structural areas. Specifically, plane cutting segments point clouds by setting one or more planes, which is suitable for isolating simple structures. For objects with more complex or irregular shapes, the polygonal cutting tool provides a custom cutting path that can crop point clouds on demand. These cutting methods greatly simplify the process of extracting detailed information on specific equipment or facilities from huge point cloud data, laying the foundation for the subsequent refined processing of individual equipment.

[0122] ⑥ Denoising of individualized device point cloud data. After the rough segmentation and classification of the point cloud data are completed through the above steps and the point cloud data of the individualized device is initially obtained, step 1 (2) can be used to perform denoising on the individualized device point cloud data again, thereby improving the quality of the individualized device point cloud.

[0123] (4) Point cloud data thinning. The three-dimensional point cloud data of the individual devices obtained through the above point cloud acquisition and processing steps usually presents a huge amount of data pressure. In order to reduce the computational burden, the three-dimensional point cloud data can be pre-processed by using a thinning (sampling) technique based on voxel grid filtering. This method is achieved by dividing the point cloud space into regular voxels (three-dimensional grids) and then reducing the number of points inside each voxel. The steps are as follows:

[0124] First, determine the size of the voxel. Each voxel is equivalent to a cube in space, and the size is determined by the side length l. For each point in the point cloud, it is assigned to the corresponding voxel according to its coordinates. In each voxel, all points will be merged into a representative point, which is usually the geometric center of all points in the voxel, that is, the average position.

[0125]

[0126] Where n is the number of points inside the voxel, (x i ,y i ,z i ) is the coordinate of the point in the voxel. Through this method, a large number of points in the original point cloud are effectively reduced, while the simplified point cloud still retains the main geometric characteristics and shape contours of the original point cloud, greatly reducing the computational burden of subsequent processing and significantly improving the processing speed.

[0127] S2: Mark equipment inspection points to generate waypoints:

[0128] like Figure 4 As shown in , the drone waypoint p is based on the equipment inspection point t as the origin, and the shooting distance is determined by manual input; Figure 5 As shown in , the direction is determined by the first-person perspective of the point cloud view; Figure 3 As shown, the shooting range is determined by the camera lens field of view and focal length multiple. The steps to complete marking the device patrol points and automatically generate waypoints are as follows:

[0129] (1) Drag the equipment inspection point to the middle of the view, adjust the point cloud view angle, and mark a point on the point cloud where the equipment inspection point is located as the target point t1 for drone photography;

[0130] (2) Automatically create the corresponding waypoint p1 based on the input shooting distance and point cloud view angle;

[0131] (3) Based on the input shooting distance and camera lens focal length, the shooting point cloud range is automatically obtained, such as Figure 6 shown.

[0132] (4) Create UAV waypoints corresponding to all equipment inspection points through the above steps.

[0133] S3: Waypoint safety distance collision check:

[0134] Select all waypoints in each sub-area and set the safe obstacle avoidance distance. Generate a 3D grid at intervals of 0.1 meters. The grid with point clouds is an impassable dangerous area. Then expand the dangerous area grid with the safe area to generate a map that the drone can pass through and mark the waypoints in the dangerous area. The steps are as follows:

[0135] (1) Create a three-dimensional grid space. According to the value range of valid point cloud data, construct a three-dimensional grid space of nx×ny×nz. x, y, z are the coordinates of the point cloud in space.

[0136] (2) Construct an occupancy grid map. The points in the point cloud of the power grid equipment are projected onto the grid map in sequence. When the grid is occupied by a point cloud, the drone cannot pass through.

[0137] (3) Inspection equipment expansion. Considering the size of the drone and the safety distance, the obstacles in the environment (i.e., inspection equipment) are expanded. The expansion parameter E is introduced to represent the number of circles of obstacle expansion, and its value is shown in the following formula.

[0138]

[0139] Where r is the sum of the drone radius and the minimum safety distance. R is the grid size, which represents the resolution of the map.

[0140] Figure 7 Schematic diagram of a grid map considering obstacle expansion, where the black cells are occupied cells determined based on the point cloud, and the gray cells are occupied cells added by obstacle expansion (E=1).

[0141] S4: Waypoint position adjustment:

[0142] If there is a dangerous waypoint, adjust the waypoint position and angle with the equipment inspection point as the center and the shooting distance as the radius until the waypoint is safe. Manual adjustment is mainly performed through pitch angle and heading angle, such as Figure 8 shown.

[0143] S5: Planning the inspection route:

[0144] According to the order of selected waypoints, the previous and next waypoints use the optimal safe path algorithm to search for the path between the two waypoints, generate a local route, and finally connect all the local area routes to form a complete route.

[0145] In order to solve the optimal path for the safety inspection of power grid equipment, a solution algorithm based on particle swarm optimization (PSO) algorithm is proposed. The safety inspection area is set as a path search space, and a number of particles are set in the space. In order to process the original particles of the PSO algorithm, it is proposed to combine the random generation algorithm with the solution algorithm to obtain the initialization particles; through the output iterative calculation, the global optimal safety inspection path is obtained. In order to avoid the optimization of the inspection path falling into the local optimum, the individual extreme value and the global extreme value are used to adjust the flight distance and flight direction of the original particles of the PSO algorithm.

[0146]

[0147] Where: l is the particle; δ is the flight speed; c1 and c2 are learning factors; rand() is a random function; is the spatial position of particle l; I is the optimal position searched by a single particle; G is the optimal position searched by all particles in the population.

[0148] Each iterative calculation will cause the particle population to be updated, and the spatial position of the updated particle is:

[0149]

[0150] In the process of solving the optimal power grid equipment safety inspection path, in order to avoid the failure to solve the optimal path information due to the too fast movement speed of the population particles, the particle flight speed limit range is set to δ∈[δ min ,δ max ]. Where: min is the slowest flight speed; δ max For the fastest flying speed.

[0151] The particle speed limit range is often 25% to 30% of the particle population search space. Reasonably adjust the range parameters according to the actual situation. In order to avoid the problem of PSO algorithm falling into the local optimum, this paper proposes to add a weight factor to the PSO algorithm and establish a global optimization PSO solution algorithm. The improved particle flight speed is:

[0152]

[0153] Where: λ is the weight factor.

[0154]

[0155] Where: start is the initial weight value; end is the weight value when the iteration ends; ψ is the total number of iterations of PSO operation.

[0156] According to the above formula, the increase of inertia weight particles directly leads to the improvement of the global optimization ability of the safety patrol path, but also causes the reduction of the local optimization ability. The convergence factor is introduced into the PSO algorithm to improve the convergence speed of the automatic optimization of the safety patrol path of power grid equipment. The updated particle flight speed is:

[0157]

[0158] Where: ε is the convergence factor.

[0159]

[0160] Where: r is the side length of the search space.

[0161] The proposed PSO algorithm is applied to the optimization of safety patrol paths. According to the time of patrol information collection of the path points, the entire safety patrol path is divided into multiple segments. The central node of each segment is extracted to serve as the central point of each search space. All segmented search spaces are studied, and the path points contained in each search space are set to have the same node posture. In summary, in the process of searching for the optimal path for safety patrols, only the path point location information needs to be considered. The PSO search space is as follows: Fig. 9 shown.

[0162] Depend on Fig. 9 It can be seen that the distance between adjacent path points in the calculation space and the objective function set by adding the distances of all path points are used as the data basis for calculating the fitness value. One path point is extracted from each segmented search space and connected to form the optimal safe patrol path. In order to ensure the rationality of the solution of the optimal patrol path, the collision detection method is used for testing. The path points with collision problems are extracted, the segmented search space where they are located is reduced, and the position and speed of the PSO particles are relocated until the collision problem no longer occurs and the collision detection is stopped.

[0163] At the same time, in order to meet the kinematic constraints, the optimal path is inversely solved to avoid the path generated by the automatic optimization to contain path points that the patrol robot cannot reach. If there is a path point that is not in the motion space, the particle position and particle velocity need to be updated. Generate another particle that meets the kinematic constraints in the search space to adjust the optimized safe patrol path, such as Fig.10 shown.

Claims

1. A method for acquiring a UAV dynamic inspection route based on laser point cloud, characterized in that: The following steps are involved: S1: Comprehensive processing of 3D point cloud data: Comprehensive processing of the collected 3D point cloud data of the area to be inspected, including denoising, classification and thinning of the point cloud data; S2: Mark equipment inspection points to generate waypoints: Divide the point cloud of the area to be inspected into multiple sub-areas. In each sub-area, mark the equipment inspection points to be photographed. At the same time, generate corresponding waypoints based on the photographing distance and the point cloud field of view angle. S3: Waypoint safety distance collision check: Select all waypoints in each sub-area and set the safe obstacle avoidance distance; Generate a 3D point cloud 3D grid. The grid with point cloud is the impassable dangerous area. Then expand the dangerous area grid with the safe area to generate a map where the drone can pass; S4: Waypoint position adjustment: If there is a dangerous waypoint, adjust the waypoint position and angle with the equipment patrol point as the center and the preset safety distance as the adjustment boundary until the waypoint is safe; S5: Planning the inspection route: According to the order of the selected waypoints, the previous and next waypoints use the optimal safe path algorithm to search for the path between the two waypoints and generate a local route; finally, all local area routes are connected to form a complete route.

2. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1 is characterized in that: In the step S1, the three-dimensional point cloud data is comprehensively processed: first, the three-dimensional point cloud data of the equipment in the inspection area that has been collected is subjected to denoising, and outliers or erroneous data points are removed by statistical analysis or density-based filtering methods, thereby significantly improving the quality of the point cloud data; Subsequently, clustering algorithms, such as DBSCAN or K-means, are used to segment and classify each independent unit device in the point cloud data to achieve the individualization of power equipment; then, data thinning is performed on the point cloud data of each unit device, and the voxel grid filtering method is used to reduce the data volume while maintaining the shape characteristics and structural details of the point cloud.

3. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1 is characterized in that: In the step S1, the comprehensive processing of the three-dimensional point cloud data includes the following steps: s11: 3D point cloud data acquisition; perform detailed scans of high-voltage equipment, cameras and other structures within the power grid from multiple angles and positions to ensure that all critical areas are covered; after acquisition, perform data synchronization, correction and feature point matching to ensure accurate alignment of data from different scanning perspectives; s12: Point cloud data denoising: Use a denoising algorithm based on a Gaussian mathematical model or point cloud processing software related operations to pre-process the 3D point cloud data; s13: Equipment individualization: By using clustering algorithms, such as DBSCAN or K-means, or related operations in point cloud processing software, the point cloud data is effectively cut and classified to construct individual point cloud data of each power equipment; s14: point cloud data thinning; the three-dimensional point cloud data of the individualized device obtained through the above point cloud acquisition and processing steps is pre-processed by using a thinning or sampling technique based on voxel grid filtering; this method is achieved by dividing the point cloud space into regular voxels, i.e., a three-dimensional grid, and then reducing the number of points inside each voxel.

4. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1 is characterized in that: In the above-mentioned step S2, the equipment inspection points are marked to generate waypoints: the drone waypoint p is based on the equipment inspection point t as the origin, and the shooting distance is determined by manual input; the direction is determined based on the first-person perspective of the point cloud view; the shooting range is determined by the camera lens field of view angle and focal length multiple; the steps for marking the equipment inspection points and generating waypoints are as follows: S21: Drag the equipment inspection point to the middle of the view, adjust the point cloud view angle, and mark a point on the point cloud where the equipment inspection point is located as the target point t1 for drone photography; S22: automatically create the corresponding waypoint p1 according to the input shooting distance and point cloud view angle; S23: Automatically obtain the shooting point cloud range based on the input shooting distance and camera lens focal length; S24: Create drone waypoints corresponding to all equipment inspection points.

5. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1, characterized in that: In the above-mentioned S3 step, the waypoint safety distance collision check is as follows: all waypoints are selected in each sub-area, and the safe obstacle avoidance distance is set; a 3D grid is generated at a set interval, and the grid with point cloud is an impassable dangerous area, and then the dangerous area grid is expanded with the safe area to generate a map that the drone can pass through, and the waypoints in the dangerous area are marked; the steps are as follows: S31: Create a three-dimensional grid space, and construct a three-dimensional grid space of nx×ny×nz according to the value range of the valid point cloud data, where x, y, and z are the coordinates of the point cloud in the space; S32: Construct an occupied grid map, project the points in the point cloud of the power grid equipment to the grid map in sequence, and when the grid is occupied by a point cloud, the drone cannot pass through; S33: Inspection equipment expansion. Considering the size of the drone and the safety distance, the obstacles in the environment, i.e., the inspection equipment, are expanded. The expansion parameter E is introduced to represent the number of circles of obstacle expansion. Its value is shown in the following formula; E=ceil(r / R), Where r represents the sum of the UAV radius and the minimum safety distance, and R is the grid size, which represents the resolution of the map.

6. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1, characterized in that: In the step S4, the waypoint position is adjusted by manually adjusting the pitch angle and the heading angle.

7. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 1, characterized in that: In the step S5, the inspection path is planned: the optimal path for the safety inspection of power grid equipment is solved, and a solution algorithm based on particle swarm optimization (PSO algorithm) is used; the safety inspection area is set as a path search space, and a number of particles are set in the space. In order to process the original particles of the PSO algorithm, the random generation algorithm is combined with the solution algorithm to obtain the initialization particles; the global optimal safety inspection path is obtained by outputting iterative calculations; in order to avoid the optimization of the inspection path falling into the local optimum, the individual extreme value and the global extreme value are used to add a weight factor to the PSO algorithm to establish a global optimization PSO solution algorithm: The improved particle flight speed is: In the formula: λ is the weight factor; l is the particle; δ is the flight speed; c1 and c2 are both learning factors; rand() is the random function; is the spatial position of particle l; I is the optimal position searched by a single particle; G is the optimal position searched by all particles in the population, λ start is the initial weight value; end is the weight value when the iteration ends; ψ is the total number of iterations of PSO operation; Each iterative calculation will cause the particle population to be updated, and the spatial position of the updated particle is: Where: is the updated spatial position of particle l, is the current spatial position of particle l, δ l is the flight speed; In the process of solving the optimal power grid equipment safety inspection path, in order to avoid the failure to solve the optimal path information due to the too fast movement speed of the population particles, the particle flight speed limit range is set to δ∈[δ min , δ max ], where: δ min is the slowest flight speed; δ max For the fastest flight speed; The convergence factor is introduced into the PSO algorithm to improve the convergence speed of automatic optimization of the safety inspection path of power grid equipment. The updated particle flight speed is: Where: ε is the convergence factor; Where: r is the side length of the search space.

8. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 6, characterized in that: In the automated optimization of safety patrol paths, the entire safety patrol path is divided into multiple segments according to the time of patrol information collection of the path points. The central node of each segment is extracted to serve as the central point of each search space. All segmented search spaces are studied, and the path points contained in each search space are set to have the same node posture. The distances between adjacent path points in the space are calculated, and the objective function set by adding the distances of all path points is used as the data basis for calculating the fitness value.

9. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 6, characterized in that: One path point is extracted from each segmented search space, and the path points are connected to form the optimal safe patrol path. The collision detection method is used for testing, and the path points with collision problems are extracted. The segmented search space where they are located is narrowed down, and the position and speed of the PSO particles are relocated until the collision detection is stopped when there is no collision problem. In order to meet the kinematic constraints, the inverse solution is performed on the solved optimal path to avoid the path generated by the automatic optimization containing path points that the patrol robot cannot reach. If there is a path point that is not in the motion space, the particle position and particle speed are updated, and another particle that meets the kinematic constraints is generated in the search space to adjust the optimized safe patrol path.

10. The method for acquiring a UAV dynamic inspection route based on laser point cloud according to claim 3, characterized in that: s12: Point cloud data denoising; the specific steps are as follows: ① Define the neighborhood of the point: define a neighborhood for each point P in the three-dimensional point cloud, specify a spherical neighborhood with a radius of r, including all points whose distance from point P is less than r; ② Calculate the mean and covariance matrix of the neighborhood points: For the points in the neighborhood (x i ,y i , z i ), calculate the mean vector of all points in the neighborhood, that is, the average position and covariance matrix. The mean vector and covariance matrix describe the spatial distribution of the points; the mean vector μ is a three-dimensional vector, and its calculation formula is as follows: Where n is the number of points in the neighborhood, μ is the mean vector, ∑ is the covariance matrix, (μ x , μ y , μ z ) is the component vector of the mean vector, m is the total number of points in the neighborhood, X i is the i-th point in the neighborhood, (x i ,y i , z i ) is the coordinate of the i-th point in the neighborhood; The covariance matrix ∑ is a 3×3 matrix that reflects the distribution of points in the neighborhood relative to the mean. Its calculation formula is as follows: ③ Gaussian fitting: For each point in the neighborhood, the Mahalanobis distance between it and the neighborhood mean is calculated to evaluate the compactness of its distribution, and the covariance matrix is ​​used for measurement. The formula is as follows: Where X is a point in the neighborhood, μ is the mean of all points in the neighborhood, ∑ -1 is the inverse of the covariance matrix; ④Noise determination: Based on the Mahalanobis distance D M (X), set a threshold τ, if D M (X) is greater than τ, then point x is considered to be a noise point; ⑤ Point cloud data optimization: use Gaussian statistical denoising method, or use Gaussian kernel smoothing technology to further optimize point cloud data; or use denoising operation of point cloud processing software; s13: Equipment monomerization; the specific steps are as follows: ① Point cloud data standardization: After obtaining the rough individual device point cloud data through the above steps, the next step is to standardize the data to unify the coordinate system and data scale of all individual device point clouds to ensure the effect of the subsequent clustering algorithm; ② Select clustering algorithm: Use DBSCAN algorithm to cluster the point cloud of power grid equipment; when applying DBSCAN algorithm, reasonably set the size of the neighborhood, i.e., radius R and the minimum number of points MinPts, to effectively distinguish and identify the point cloud data of various equipment; ③ Perform clustering: First, identify all core points whose neighborhood R contains at least MinPts points and shows a high enough density so that they can be considered as part of a cluster; then define boundary points and noise points. Although the number of points in the neighborhood R of boundary points is less than MinPts, they are located in the R neighborhood of at least one core point. The number of points in the R neighborhood of noise points is less than MinPts and they do not belong to the neighborhood of any core point; then, starting from any unprocessed core point, a new cluster is formed by finding all core points that are density-reachable to it; density-reachable means that one can reach another core point from one core point through a series of core points in the neighborhood R; all density-reachable points starting from the core point, including boundary points, are classified into the same cluster; repeat this process until all points are processed and marked as belonging to a cluster or marked as noise; ④ Single-unit confirmation: Apply the DBSCAN clustering algorithm to organize the point cloud into independent clusters according to its density. Each cluster corresponds to a single device. After the clustering process is completed, the clustering results are carefully analyzed and verified to ensure that each cluster accurately represents a specific device. If necessary, manual intervention is performed to correct any classification errors caused by the clustering algorithm. ⑤ Segmentation and classification of point cloud processing software: In addition to the above clustering algorithms, point cloud processing software CloudCompare or LiPowerline can also be used to perform spatial segmentation and classification of point cloud data to effectively isolate and analyze specific equipment or structural areas. ⑥ Denoising of individual device point cloud data: After roughly cutting and classifying the point cloud data and preliminarily obtaining the point cloud data of individual devices, the point cloud data of individual devices can be denoised again; s14: Point cloud data thinning; the specific steps are as follows: First, determine the size of the voxel. Each voxel is equivalent to a cube in space, and the size is determined by the side length l. For each point in the point cloud, it is assigned to the corresponding voxel according to its coordinates; in each voxel, all points will be merged into a representative point, which is usually the geometric center of all points in the voxel, that is, the average position; ， Where n is the number of points inside the voxel, (x i ,y i , z i ) are the coordinates of the point inside the voxel.

11. A computer system comprising a storage medium and a processor, wherein the storage medium stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images