Unmanned aerial vehicle flight path planning method and system, medium and terminal

Abstract

The invention discloses an unmanned aerial vehicle flight path planning method and system, a medium and a terminal, relates to the technical field of unmanned aerial vehicle obstacle avoidance, and mainly aims to solve the problems of inaccurate unmanned aerial vehicle flight path planning and high unmanned aerial vehicle collision risk in the prior art. By combining the spatial point cloud data and the image data to construct the three-dimensional space model of the target space region, the problem that the GPS signal is easy to block is avoided; furthermore, a semantic space model of the target space region is obtained by identifying category information and position information of the obstacle, so that the three-dimensional space model not only has geometric information, but also carries the category information of the obstacle, and the accuracy of obstacle avoidance is effectively improved; furthermore, electromagnetic interference prediction is fused into flight path planning, the risk of failure of a flight control system is effectively reduced, and the collision risk of the unmanned aerial vehicle is reduced from multiple dimensions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle obstacle avoidance, in particular to an unmanned aerial vehicle flight path planning method and system, a medium and a terminal. BACKGROUND

[0002] The operation and maintenance safety and inspection efficiency of a substation become a key problem in power system management. The traditional manual inspection method has problems such as high labor intensity, low efficiency, and safety hazards of personnel approaching high-voltage equipment. With the development of unmanned aerial vehicle technology, automatic inspection technology based on unmanned aerial vehicles is gradually applied to substation scenes. Compared with the traditional manual inspection method, it has higher mobility and data acquisition capability.

[0003] At present, the existing unmanned aerial vehicle inspection method is mostly based on GPS navigation and two-dimensional obstacle avoidance algorithm for fault avoidance.

[0004] However, when conducting indoor substation inspection, due to the existence of a large number of three-dimensional obstacles such as power equipment, conductor wiring, and isolation fences inside the substation, the space structure is precise and complex, the GPS signal is easily blocked, resulting in weak signal, and further leading to inaccurate unmanned aerial vehicle flight path planning and high unmanned aerial vehicle collision risk. At the same time, due to the electromagnetic interference generated during the operation of the equipment in the substation, it is easy to interfere with the communication link and navigation system of the unmanned aerial vehicle, causing the flight control system to fail, further increasing the unmanned aerial vehicle collision risk. SUMMARY

[0005] Therefore, the present application provides an unmanned aerial vehicle flight path planning method and system, a medium and a terminal, which mainly aims to improve the problems that due to the existence of a large number of three-dimensional obstacles such as power equipment, conductor wiring, and isolation fences inside the substation, the space structure is precise and complex, the GPS signal is easily blocked, resulting in weak signal, and further leading to inaccurate unmanned aerial vehicle flight path planning and high unmanned aerial vehicle collision risk; and due to the electromagnetic interference generated during the operation of the equipment in the substation, it is easy to interfere with the communication link and navigation system of the unmanned aerial vehicle, causing the flight control system to fail, further increasing the unmanned aerial vehicle collision risk.

[0006] According to one aspect of the present application, an unmanned aerial vehicle flight path planning method is provided, comprising:

[0007] Obtaining space point cloud data and image data of a target space region, and constructing a three-dimensional space model of the target space region based on the space point cloud data and the image data;

[0008] perform category recognition processing and positioning processing on the obstacles contained in the image data based on the obstacle recognition model that has completed model training, to obtain category information and position information of the obstacles, and map the category information and the position information to the three-dimensional space model to obtain a semantic space model of the target space region;

[0009] perform prediction processing on the electromagnetic field intensity space distribution data of the target space region at the current time based on the electromagnetic field intensity data prediction model that has completed model training, to obtain current electromagnetic field intensity space distribution data, and perform screening processing on the current electromagnetic field intensity space distribution data based on a preset electromagnetic field intensity threshold value, to determine an electromagnetic interference risk region, map the electromagnetic interference risk region to the semantic space model to obtain a space risk distribution model of the target space region;

[0010] generate a UAV flight path based on the space risk distribution model, to control a UAV to perform inspection work based on the UAV flight path.

[0011] Preferably, before the obstacle recognition model that has completed model training performs category recognition processing and positioning processing on each obstacle contained in the image data to obtain category information and position information of each obstacle, the method further comprises:

[0012] construct an initial obstacle recognition model based on a convolutional neural network;

[0013] construct an obstacle recognition training data set based on image data of space regions in different work environments, wherein the obstacle recognition training data set contains multiple training image data and corresponding standard image annotation results;

[0014] perform category recognition processing and positioning processing on training obstacles contained in each of the training image data based on the initial obstacle recognition model, to obtain predicted category information and predicted position information of obstacles of each of the training image data;

[0015] integrate the predicted category information and the predicted position information of each of the obstacles to obtain predicted image annotation results of obstacles of each of the training image data;

[0016] calculate a first loss function value between the predicted image annotation results and the standard image annotation results, and perform model training on the initial obstacle recognition model based on the first loss function value to obtain the obstacle recognition model that has completed model training.

[0017] Preferably, before the electromagnetic field intensity data prediction model based on the completed model training predicts the electromagnetic field intensity spatial distribution data of the target space region at the current time, the method further comprises:

[0018] An initial electromagnetic field intensity data prediction model is constructed based on a residual neural network and a long short-term memory network.

[0019] Based on the historical electromagnetic field intensity spatiotemporal distribution data of the target space region, an electromagnetic field intensity prediction training data set is constructed, wherein the electromagnetic field intensity prediction training data set contains the spatial position information of a plurality of point locations in the target space region and corresponding historical electromagnetic field intensity temporal distribution data.

[0020] Based on the initial electromagnetic field intensity data prediction model, the electromagnetic field intensity data of each point location at a preset training time is predicted according to the spatial position information of each point location, to obtain predicted electromagnetic field intensity spatial distribution data at the preset training time.

[0021] The electromagnetic field intensity data at the preset training time is obtained from the historical electromagnetic field intensity temporal distribution data of each point location, to obtain historical electromagnetic field intensity spatial distribution data at the preset training time.

[0022] A second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data is calculated, and the initial electromagnetic field intensity data prediction model is trained based on the second loss function value, to obtain an electromagnetic field intensity data prediction model that has completed model training.

[0023] Preferably, the unmanned aerial vehicle flight path is generated based on the spatial risk distribution model, comprising:

[0024] The target space region is divided to obtain a plurality of three-dimensional grid units.

[0025] The obstacle position point set and the electromagnetic interference risk region point set are extracted from the spatial risk distribution model.

[0026] The three-dimensional grid units that do not have an intersection with the obstacle position point set and the electromagnetic interference risk region point set are marked as passable grid units, to obtain a spatial passable grid map.

[0027] Based on a three-dimensional heuristic search algorithm, the unmanned aerial vehicle flight path is generated according to the spatial passable grid map, wherein the three-dimensional heuristic search algorithm combines the path cost between passable grid units, the heuristic evaluation of flight direction guidance, and the electromagnetic risk penalty factor.

[0028] Preferably, after generating the UAV flight path based on the spatial risk distribution model, the method further comprises:

[0029] acquiring real-time spatial point cloud data and real-time image data of the target space region during the inspection operation of the UAV;

[0030] updating the spatial risk distribution model based on the real-time spatial point cloud data and the real-time image data to obtain an updated spatial risk distribution model;

[0031] updating the space passable grid map based on the updated spatial risk distribution model to obtain an updated space passable grid map;

[0032] detecting whether there is an impassable grid unit in the UAV flight path based on the updated space passable grid map;

[0033] if there is, updating the UAV flight path based on the updated space passable grid map to obtain an updated UAV flight path, and controlling the UAV to complete the remaining inspection operation based on the updated UAV flight path.

[0034] Preferably, the three-dimensional space model of the target space region is constructed based on the spatial point cloud data and the image data, comprising:

[0035] fusing the spatial point cloud data and the image data based on a point cloud image registration algorithm to obtain spatial information data of the target space region;

[0036] filtering and denoising the spatial information data to obtain optimized spatial information data, and constructing the three-dimensional space model of the target space region based on the optimized spatial information data.

[0037] Preferably, the first loss function value is used to represent a total value of a joint loss function, wherein the total value of the joint loss function includes a cross-entropy loss function value and a smoothing loss function value; and the second loss function value is used to represent a mean square error loss function value.

[0038] According to another aspect of the present application, a UAV flight path planning system is provided, comprising:

[0039] a data acquisition module configured to acquire spatial point cloud data and image data of a target space region, and construct a three-dimensional space model of the target space region based on the spatial point cloud data and the image data;

[0040] an obstacle recognition module, configured to perform category recognition processing and positioning processing on obstacles contained in the image data based on an obstacle recognition model that has completed model training, to obtain category information and position information of the obstacles, and to map the category information and the position information to the three-dimensional space model to obtain a semantic space model of the target space region;

[0041] an electromagnetic field prediction module, configured to perform prediction processing on electromagnetic field intensity space distribution data of the target space region at a current time based on an electromagnetic field intensity data prediction model that has completed model training, to obtain current electromagnetic field intensity space distribution data, and to perform screening processing on the current electromagnetic field intensity space distribution data based on a preset electromagnetic field intensity threshold to determine an electromagnetic interference risk region, and to map the electromagnetic interference risk region to the semantic space model to obtain a space risk distribution model of the target space region;

[0042] a flight path planning module, configured to generate a UAV flight path based on the space risk distribution model, so as to control a UAV to perform inspection work based on the UAV flight path.

[0043] Preferably, before the obstacle recognition module, the system further comprises an obstacle recognition model training module, configured to:

[0044] construct an initial obstacle recognition model based on a convolutional neural network;

[0045] construct an obstacle recognition training data set based on image data of space regions in different work environments, wherein the obstacle recognition training data set contains multiple training image data and corresponding standard image annotation results;

[0046] perform category recognition processing and positioning processing on training obstacles contained in each of the training image data based on the initial obstacle recognition model, to obtain predicted category information and predicted position information of obstacles of each of the training image data;

[0047] integrate the predicted category information and the predicted position information of each of the obstacles to obtain predicted image annotation results of obstacles of each of the training image data;

[0048] calculate a first loss function value between the predicted image annotation results and the standard image annotation results, and perform model training on the initial obstacle recognition model based on the first loss function value to obtain an obstacle recognition model that has completed model training.

[0049] Preferably, before the electromagnetic field prediction module, the system further comprises an electromagnetic field intensity data prediction model training module, configured to:

[0050] constructing an initial electromagnetic field intensity data prediction model based on a residual neural network and a long short-term memory network;

[0051] constructing an electromagnetic field intensity prediction training data set based on historical electromagnetic field intensity spatiotemporal distribution data of the target spatial region, wherein the electromagnetic field intensity prediction training data set contains spatial position information of a plurality of point locations within the target spatial region and corresponding historical electromagnetic field intensity temporal distribution data;

[0052] performing prediction processing on electromagnetic field intensity data of each of the point locations at a preset training time based on the spatial position information of each of the point locations, respectively, to obtain predicted electromagnetic field intensity spatial distribution data at the preset training time based on the initial electromagnetic field intensity data prediction model;

[0053] obtaining electromagnetic field intensity data at the preset training time from historical electromagnetic field intensity temporal distribution data of each of the point locations to obtain historical electromagnetic field intensity spatial distribution data at the preset training time;

[0054] calculating a second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data, and performing model training on the initial electromagnetic field intensity data prediction model based on the second loss function value to obtain an electromagnetic field intensity data prediction model that has completed model training.

[0055] Preferably, the flight path planning module is configured to:

[0056] performing division processing on the target spatial region to obtain a plurality of three-dimensional grid cells;

[0057] extracting an obstacle position point set and an electromagnetic interference risk region point set from the spatial risk distribution model;

[0058] marking a three-dimensional grid cell that has no intersection with the obstacle position point set and the electromagnetic interference risk region point set as a passable grid cell to obtain a spatial passable grid map;

[0059] generating a UAV flight path based on the spatial passable grid map based on a three-dimensional heuristic search algorithm, wherein the three-dimensional heuristic search algorithm is a heuristic evaluation that combines a path cost between passable grid cells, a flight direction guide, and an electromagnetic risk penalty factor.

[0060] Preferably, after the flight path planning module, the system further comprises a flight path updating module configured to:

[0061] In the process of controlling the unmanned aerial vehicle to perform the inspection operation, real-time spatial point cloud data and real-time image data of the target space region are collected;

[0062] Based on the real-time spatial point cloud data and the real-time image data, the spatial risk distribution model is updated to obtain an updated spatial risk distribution model;

[0063] Based on the updated spatial risk distribution model, the spatial passable grid map is updated to obtain an updated spatial passable grid map;

[0064] Based on the updated spatial passable grid map, it is detected whether there is an impassable grid unit in the flight path of the unmanned aerial vehicle;

[0065] If there is, based on the updated spatial passable grid map, the flight path of the unmanned aerial vehicle is updated to obtain an updated flight path of the unmanned aerial vehicle, so that the unmanned aerial vehicle completes the remaining inspection operation based on the updated flight path of the unmanned aerial vehicle.

[0066] Preferably, the data acquisition module is configured to:

[0067] Based on a point cloud image registration algorithm, the spatial point cloud data and the image data are fused to obtain spatial information data of the target space region;

[0068] The spatial information data is filtered and denoised to obtain optimized spatial information data, and based on the optimized spatial information data, a three-dimensional space model of the target space region is constructed.

[0069] Preferably, the first loss function value is used to represent a total value of a joint loss function, wherein the total value of the joint loss function includes a cross-entropy loss function value and a smoothing loss function value; and the second loss function value is used to represent a mean square error loss function value.

[0070] According to another aspect of the present application, a storage medium is provided, and at least one executable instruction is stored in the storage medium, which makes a processor execute operations corresponding to the above-mentioned unmanned aerial vehicle flight path planning method.

[0071] According to still another aspect of the present application, a terminal is provided, which includes a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface complete communication with each other through the communication bus;

[0072] The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute operations corresponding to the above-mentioned unmanned aerial vehicle flight path planning method.

[0073] By means of the technical solutions, the embodiments of the present application have at least the following advantages:

[0074] The present application provides a method and system for planning a flight path of a UAV, a medium and a terminal. Firstly, spatial point cloud data and image data of a target space region are acquired, and a three-dimensional space model of the target space region is constructed based on the spatial point cloud data and the image data. Secondly, an obstacle recognition model that has completed model training is used to perform category recognition processing and positioning processing on obstacles contained in the image data, to obtain category information and position information of the obstacles, and the category information and the position information are mapped to the three-dimensional space model to obtain a semantic space model of the target space region. Thirdly, an electromagnetic field intensity data prediction model that has completed model training is used to perform prediction processing on electromagnetic field intensity spatial distribution data of the target space region at a current time, to obtain current electromagnetic field intensity spatial distribution data, and the current electromagnetic field intensity spatial distribution data is filtered based on a preset electromagnetic field intensity threshold to determine an electromagnetic interference risk region, and the electromagnetic interference risk region is mapped to the semantic space model to obtain a spatial risk distribution model of the target space region. Finally, a flight path of a UAV is generated based on the spatial risk distribution model, and the UAV is controlled to perform inspection work based on the flight path. Compared with the prior art, the embodiments of the present application avoid the problem that GPS signals are easily blocked by constructing a three-dimensional space model of a target space region in combination with spatial point cloud data and image data. Further, the category information and the position information of obstacles are identified to obtain a semantic space model of the target space region, so that the three-dimensional space model not only has geometric information but also carries obstacle category information, effectively improving the accuracy of obstacle avoidance. Further, electromagnetic interference prediction is integrated into flight path planning, effectively reducing the risk of failure of the flight control system, and reducing the risk of collision of the UAV from multiple dimensions.

[0075] The above description is only a summary of the technical solutions of the present application. In order to enable one skilled in the art to better understand the technical means of the present application, the content of the specification can be implemented, and in order to enable the above and other purposes, features and advantages of the present application to be more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0076] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the accompanying drawings indicate the same or similar components. In the drawings:

[0077] Figure 1A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0078] Figure 2 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0079] Figure 3 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0080] Figure 4 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0081] Figure 5 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0082] Figure 6 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application.

[0083] Figure 7 A flow chart of a method for planning a flight path of a UAV is shown according to an embodiment of the present application. DETAILED DESCRIPTION

[0084] Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown, it should be understood that the present disclosure can be embodied in many forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0085] It should be understood that the sizes of the various portions shown in the drawings are shown for the purpose of illustration only and are not to scale.

[0086] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the scope of the application or its application or uses.

[0087] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, such techniques, methods, and devices can be considered part of the specification.

[0088] It is to be understood that the same or similar numerals and letters can refer to similar items throughout the drawings and that a description of an item in one drawing does not preclude further discussion of the item in a subsequent drawing.

[0089] The embodiments of the present application can be applied to a computer system / server, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer systems / servers include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems, and the like.

[0090] The computer system / server can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, which perform particular tasks or implement particular abstract data types. The computer system / server can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including memory storage devices.

[0091] The embodiments of the present application provide a method for planning a flight path of a UAV, as shown in Figure 1 The method comprises:

[0092] 101, obtain space point cloud data and image data of a target space region, and construct a three-dimensional space model of the target space region based on the space point cloud data and the image data.

[0093] The target space region can be any region where the UAV needs to perform work, for example, an indoor region of a substation, an outdoor region of a substation, and the like. The space point cloud data can be collected based on a laser radar sensor carried by the UAV, has the advantages of high density and stable depth precision, and can accurately depict the three-dimensional boundary form of entity equipment such as a knife gate support, a transformer pedestal, and an indoor support column. The image data can be collected based on a camera carried by the UAV, and the surface texture, color distribution, and structure contour information of the equipment can be obtained. The three-dimensional space model is used to show the spatial distribution and contour of obstacles in the space region, and the obstacles include equipment (such as a switch cabinet, a knife gate, and a lightning arrester), wires (overhead lines and pipe-through lines), persons (maintenance personnel and security personnel), and other non-structural obstacles (such as scaffolds and temporary piles) that can interfere with flight. In the embodiments of the present application, the current execution end can be a flight path planning unit of a UAV control module, and in the UAV cruising process, the laser radar is controlled to scan and obtain the three-dimensional space point cloud data at a fixed frequency, and the camera is controlled to synchronously collect high-resolution image data.

[0094] It should be noted that compared with the traditional single sensor modeling mode, the embodiment of the application overcomes the problem of insufficient recognition ability of single point cloud on low reflection surface by fusing multi-source data, while retaining the visual details of the image, realizing high-fidelity scene reconstruction, so that the generated three-dimensional space model not only has high-density point cloud resolution, but also contains image texture alignment results consistent with the actual layout relationship, which can truly restore the spatial distribution and contour of obstacles in the space region. Through the high-precision, high-consistency point cloud and image fusion construction mode, it is different from the problem of relying only on SLAM or a single depth camera to generate a sparse model in the prior art.

[0095] 102. Based on the completed model training, the obstacle recognition model is used to perform category recognition processing and positioning processing on the obstacles contained in the image data, to obtain category information and position information of the obstacles, and map the category information and position information to the three-dimensional space model to obtain a semantic space model of the target space region.

[0096] Among them, the obstacles include equipment (such as switch cabinet, knife gap, lightning arrester), wire (overhead line, pipe line), person (maintenance personnel, security personnel) and other non-structural obstacles (such as scaffold, temporary stacking, etc.) that may interfere with flight; the category recognition processing is used to identify the category attributes of the obstacles, for example, equipment, wire, person and other obstacles; the positioning processing is used to determine the position of the obstacles to obtain their spatial coordinates; the semantic space model is used to represent the three-dimensional space model with semantic labels, and the semantic labels can include category information, position information, size parameters and the like of the obstacles. It should be noted that since the person is temporary, it will not be included in the semantic space model.

[0097] In the embodiments of the present application, optionally, the image data is first preprocessed, including image size normalization processing, enhancement processing, etc., to obtain standardized image data. Specifically, to solve the problems of uneven illumination, shielding interference and image resolution difference in the actual shooting images of the substation, the image data obtained in step 101 is uniformly preprocessed, including normalizing the image size to a unified input size, for example, 640x640 pixels, and applying brightness equalization, histogram stretching, Gaussian noise enhancement and other data enhancement strategies to improve the robustness of the image under different environmental conditions and ensure the consistency and generalization ability of the subsequent model input. Further, the standardized image data is input into the obstacle recognition model that has completed model training, the local spatial features of the image are extracted through the convolution layer, and then the different size obstacles are positioned and classified through the multi-scale detection head to output the final class label and the corresponding two-dimensional bounding box of all obstacles in each frame of image. In order to realize the unified representation of the image recognition result and the spatial model, the boundary box position in the two-dimensional image can also be back projected to the point cloud coordinate system. Specifically, through the extrinsic and intrinsic matrices between the image and the point cloud, the pixel coordinates of each obstacle in the image are mapped to the corresponding three-dimensional point cloud area, and the point set in the corresponding spatial range is extracted to calculate the center position, bounding box boundary and orientation parameter of the obstacle. Finally, the class information and position information of the obstacles other than the human are mapped to the three-dimensional spatial model to obtain the semantic spatial model of the target space region.

[0098] 103. Based on the electromagnetic field intensity data prediction model that has completed model training, the electromagnetic field intensity spatial distribution data of the target space region at the current time is predicted and processed to obtain the current electromagnetic field intensity spatial distribution data, and based on the preset electromagnetic field intensity threshold, the current electromagnetic field intensity spatial distribution data is screened and processed to determine the electromagnetic interference risk region. The electromagnetic interference risk region is mapped to the semantic spatial model to obtain the spatial risk distribution model of the target space region.

[0099] The current electromagnetic field intensity spatial distribution data is used to represent the electromagnetic field intensity data of each point in the spatial region at the current time; and the spatial risk distribution model is used to represent a three-dimensional spatial model with a semantic label and an electromagnetic interference risk region marked. In the embodiment of the present application, the current time, which can also be a future time, is first determined based on a path paragraph in the current flight task planning or a time window corresponding to a new region to be entered; further, the time is input into an electromagnetic field intensity data prediction model that has completed model training for prediction processing to obtain the current electromagnetic field intensity spatial distribution data; further, the current electromagnetic field intensity spatial distribution data is point-by-point filtered, and the interference risk degree of each point is determined according to a preset electromagnetic field intensity threshold. Specifically, when the electromagnetic field intensity data of a certain point is higher than the threshold, it is considered that there is a potential signal interference or abnormal operation risk of the equipment, and the point is marked as an electromagnetic interference risk region. In order to ensure the spatial continuity and actual obstacle avoidance feasibility of the marked region, clustering processing can also be performed based on the density and spatial topological relationship of the risk points to merge adjacent high-intensity points into a continuous region, so as to avoid the path oscillation problem caused by local misjudgment; finally, the position information of the electromagnetic interference risk region is mapped into the semantic spatial model obtained in step 102 of the embodiment. Specifically, in the mapping process, the semantic label is kept unchanged, and only a layer of electromagnetic interference risk attribute is newly added to mark the impassable state or restricted risk level of the point or region at the current time. This not only enhances the expression ability of the semantic spatial model to electromagnetic factors, but also provides a time-effective risk constraint input for the subsequent path planning stage.

[0100] 104. Generating a UAV flight path based on the spatial risk distribution model, to control the UAV to perform the inspection operation based on the UAV flight path.

[0101] In the embodiment of the present application, the spatial risk distribution model can extract the identified obstacle position point set and the electromagnetic interference risk region point set. The two point sets together constitute a spatial risk region that needs to be actively avoided in flight, so that the generated UAV flight path avoids obstacles and electromagnetic interference, thereby reducing the risk of UAV collision; further, the UAV flight path is converted into a trajectory control instruction sequence recognizable by the flight control, and the UAV is controlled to perform the inspection task according to the pose information, heading angle and flight speed requirement of each waypoint in the path.

[0102] Compared with the prior art, the embodiment of the application avoids the problem that the GPS signal is easily blocked by constructing a three-dimensional space model of the target space region by combining the space point cloud data and the image data. Further, by identifying the category information and the position information of the obstacles, a semantic space model of the target space region is obtained, so that the three-dimensional space model not only has geometric information, but also carries obstacle category information, effectively improving the accuracy of obstacle avoidance. Further, the electromagnetic interference prediction is integrated into the flight path planning, effectively reducing the risk of failure of the flight control system, and reducing the risk of collision of the unmanned aerial vehicle from multiple dimensions.

[0103] In one embodiment of the application, in order to further limit and illustrate, as shown in Figure 2 Before the embodiment step 102 performs category identification processing and positioning processing on each obstacle contained in the image data based on the completed obstacle identification model, the embodiment method further includes:

[0104] 201, constructing an initial obstacle identification model based on a convolutional neural network.

[0105] In the embodiment of the application, the initial obstacle identification model adopts a convolutional neural network architecture, includes a plurality of convolutional layers and maximum pooling layers arranged alternately to form a feature extraction module for extracting spatial local features of an image. Further, a multi-scale fusion module is arranged after the feature extraction module to improve the detection capability for devices and wires with significant size differences. Further, a fully connected layer is arranged at the end of the network to output a detection result, including the category probability and boundary box regression value of each target in the image, wherein the boundary box parameters adopt a center point coordinate and height coding method to adapt to the scale inconsistency problem.

[0106] 202, constructing an obstacle identification training data set based on image data of the space region of different work environments.

[0107] The obstacle identification training data set includes a plurality of training image data and corresponding standard image annotation results. In the embodiment of the application, by collecting multiple substation sites of different structure types, covering various complex shooting conditions such as daytime, nighttime, backlight, and partial occlusion, the model can have good environmental adaptability. The image content can cover multiple types of objects such as common people (patrol inspectors, operating personnel), equipment (transformers, circuit breakers, busbars), wires (overhead lines, cables), and obstacles (rails, scaffolds, and unreturned objects). The standard image annotation result can be completed by using a professional annotation tool, and records the category label and the corresponding two-dimensional contour boundary (including a polygon or an accurate bounding box) of the target, forming a complete label set.

[0108] 203. Based on the initial obstacle recognition model, perform category recognition processing and location processing on the training obstacles contained in each training image data to obtain predicted category information and predicted location information of the obstacles in each training image data.

[0109] 204. Integrate the predicted category information and predicted position information of each obstacle to obtain the predicted image annotation results of the obstacles in each training image data.

[0110] Based on steps 203 - 204 of the embodiment, predicted image annotation results of obstacles for each training image data may be obtained.

[0111] 205. Calculate a first loss function value between the predicted image annotation result and the standard image annotation result, and perform model training on the initial obstacle recognition model based on the first loss function value to obtain a trained obstacle recognition model.

[0112] Preferably, the first loss function value is used to characterize the total value of the joint loss function, wherein the total value of the joint loss function includes the cross entropy loss function value and the smoothing loss function value.

[0113] In an embodiment of the present application, a first loss function value between the predicted image annotation result and the standard image annotation result is calculated, and the weight parameters of the network are adjusted through a back propagation algorithm until the network converges, thereby obtaining an obstacle recognition model that has completed model training.

[0114] It should be noted that in order to strike a balance between recognition accuracy and boundary precision, a joint loss function can be used, preferably integrating cross-entropy loss (for category label classification) and smooth L1 loss (for bounding box regression), to guide the model to improve classification capabilities while also taking into account positioning accuracy. Furthermore, during training, the Adam optimizer is used for iterative gradient updates, combined with batch normalization and the Dropout mechanism to improve training stability and prevent overfitting. The model inputs each image data in the obstacle recognition training dataset, and undergoes enhancement processing (including random cropping, rotation, color perturbation, etc.) to improve the network's generalization ability in actual usage scenarios. After each round of training, the model's performance on the validation set is evaluated, using mean average precision (mAP) and intersection over union (IoU) as evaluation metrics until convergence on the validation set.

[0115] In an embodiment of the present application, in order to further define and illustrate, Figure 3 As shown, in step 103 of the embodiment, based on the electromagnetic field intensity data prediction model for which model training has been completed, the electromagnetic field intensity spatial distribution data of the target spatial area at the current moment is predicted and processed. Before obtaining the current electromagnetic field intensity spatial distribution data, the embodiment method further includes:

[0116] 301. Construct an initial electromagnetic field intensity data prediction model based on a residual neural network and a long short-term memory network.

[0117] In the embodiments of the present application, the initial electromagnetic field intensity data prediction model adopts a residual neural network and a long short-term memory network, and divides the input data into two parallel processing paths: one path of spatial input extracts spatial features through a multi-layer residual convolution structure, focusing on capturing the spatial difference and structural correlation of electromagnetic interference between different equipment layout areas in the substation; the other path of time series input enters a multi-layer LSTM unit, which is used to model the trend, periodicity and sudden change behavior of electromagnetic interference at a specific location point over time. Finally, the output vectors of the two branches are fused, and the electromagnetic field intensity values at each spatial location at a given future time point are predicted through a fully connected layer regression.

[0118] 302. Construct an electromagnetic field intensity prediction training data set based on the historical electromagnetic field intensity spatio-temporal distribution data of the target spatial area.

[0119] The electromagnetic field intensity prediction training data set includes spatial position information of multiple point locations in the target spatial area and corresponding historical electromagnetic field intensity time series distribution data. The historical electromagnetic field intensity time series distribution data is used to represent the historical electromagnetic field intensity data of a point location over time. It can be understood that the historical electromagnetic field intensity time series distribution data of multiple point locations collectively constitutes the historical electromagnetic field intensity spatio-temporal distribution data of the target spatial area.

[0120] In the embodiments of the present application, data in multiple typical working days and multiple different time periods can be collected, including electromagnetic field intensity values corresponding to each point location, corresponding timestamp information, and three-dimensional spatial coordinates of the point location. While the unmanned aerial vehicle is flying, the electromagnetic sensor measures the electromagnetic field intensity value of the current location in real time and synchronously records the corresponding spatial coordinate information and timestamp information. Preferably, to improve the generalization ability and learning efficiency of the model, all input data is normalized before the network, the spatial coordinates are mapped to a continuous numerical interval in a standard cube, and the timestamp data is input in the form of relative time to express the dynamics of the interference over time.

[0121] It should be noted that the obtained data not only includes instantaneous field strength values, but also covers the continuous change trend of electromagnetic field in the time series

[0122] 303. Based on the initial electromagnetic field intensity data prediction model, the electromagnetic field intensity data of each point location at a preset training time is predicted according to the spatial position information of each point location, and the predicted electromagnetic field intensity spatial distribution data at the preset training time is obtained.

[0123] In the embodiments of the present application, the normalized spatial position information is taken as the model input to predict the predicted electromagnetic field intensity spatial distribution data at the preset training moment.

[0124] 304, the electromagnetic field intensity data at the preset training moment is obtained from the historical electromagnetic field intensity time series distribution data of each point, and the historical electromagnetic field intensity spatial distribution data at the preset training moment is obtained.

[0125] In the embodiments of the present application, the electromagnetic field intensity data at the preset training moment is obtained from the data of the historical electromagnetic field intensity changing with time, and the historical electromagnetic field intensity spatial distribution data at the preset training moment is obtained.

[0126] 305, a second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data is calculated, and the initial electromagnetic field intensity data prediction model is trained based on the second loss function value to obtain the electromagnetic field intensity data prediction model after the model training.

[0127] Preferably, the second loss function value is used to represent the mean square error loss function value.

[0128] In the embodiments of the present application, the second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data is calculated, and is transmitted to each network layer through the back propagation algorithm, and the parameter weight is optimized layer by layer until the loss function reaches the convergence condition on the verification set, and the electromagnetic field intensity data prediction model after the model training is obtained. Preferably, in the training process, the mean square error (MSE) between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data can be used as the loss function, which can not only calculate all the training samples in batches, but also introduce a spatial position weighting strategy, so that the point near the key equipment or the frequently disturbed area is given a higher weight in the training, thereby improving the prediction accuracy of the model in the key area. The mean square error loss function can be expressed as the following formula,

[0129]

[0130] wherein, The second loss function value is represented by N, which represents the number of points in the electromagnetic field intensity prediction training data set, The predicted electromagnetic field intensity data of the i-th point at the preset training moment is represented by E iThe historical electromagnetic field intensity data of the i-th point at the preset training moment is represented. It can be understood that the mean square error loss function quantifies the overall prediction error by calculating the square of the difference between the predicted electromagnetic field intensity data and the historical electromagnetic field intensity data, and averaging all points. In the training process, the model outputs the predicted electromagnetic field intensity data of multiple points at a certain prediction moment each time. The predicted electromagnetic field intensity data is compared with the historical electromagnetic field intensity data in the electromagnetic field intensity prediction training data set point by point. The larger the mean square error loss function value, the greater the deviation between the network prediction and the actual value, and the current parameter combination of the model cannot effectively learn the change rule of electromagnetic interference in the time and space dimensions. On the contrary, when the mean square error loss function value gradually decreases, it indicates that the network gradually enhances its modeling ability for electromagnetic disturbance behavior after adjusting the internal weights. In the training process, the error information is propagated based on the mean square error loss function value, and each parameter node is updated layer by layer from the output layer, so that the results of the mean square error loss function value on the training set and the validation set gradually tend to be stable and converge to the minimum. Unlike the cross-entropy loss commonly used in classification problems, the prediction target in the embodiment of the application is a continuous electromagnetic intensity value, so the mean square error can effectively reflect the order of magnitude error of the predicted value, and is more sensitive to the regression accuracy of the model. This loss function not only helps the model to accurately restore the electromagnetic field intensity of each spatial point, but also provides a reliable numerical basis for subsequent high-risk area identification and path avoidance strategies, thereby enhancing the operational safety and foresight of the entire system in the electromagnetic interference frequent scenario.

[0131] In one embodiment of the application, in order to further limit and illustrate, as shown in Figure 4 The step 104 of generating a UAV flight path based on a spatial risk distribution model in the embodiment includes:

[0132] 401, divide the target space region to obtain a plurality of three-dimensional grid units.

[0133] 402, extract the obstacle position point set and the electromagnetic interference risk area point set from the spatial risk distribution model.

[0134] 403, mark the three-dimensional grid units that do not have intersection with the obstacle position point set and the electromagnetic interference risk area point set as passable grid units, to obtain a spatial passable grid map.

[0135] In the embodiment steps 401-403, first, the target space region is divided into regular three-dimensional grid cells to obtain a plurality of three-dimensional grid cells; further, a set of obstacle position points and a set of electromagnetic interference risk region points are extracted; finally, the three-dimensional grid cells that do not have an intersection with the set of obstacle position points and the set of electromagnetic interference risk region points are marked as passable grid cells; the three-dimensional grid cells containing the obstacle position points and / or the electromagnetic interference risk region points are marked as impassable grid cells, thereby obtaining a space passable grid map. The grid map comprehensively considers the spatial resolution, obstacle edge expansion (Buffer) strategy and flight envelope of the unmanned aerial vehicle when being constructed, to ensure the safety margin and executability in the path planning process.

[0136] 404. Based on the three-dimensional heuristic search algorithm, the unmanned aerial vehicle flight path is generated according to the space passable grid map.

[0137] The three-dimensional heuristic search algorithm is combined with the path cost between passable grid cells, the heuristic evaluation of flight direction guidance and the electromagnetic risk penalty factor. In the embodiment of the present application, the three-dimensional heuristic search algorithm combines the path cost (i.e. the cost function) between passable grid cells and the heuristic evaluation (i.e. the heuristic function) of flight direction guidance, comprehensively evaluates the length, risk exposure degree and turning complexity of the path, and in order to improve the ability to avoid the electromagnetic interference region, a high penalty coefficient is given to the path segment passing through the electromagnetic risk point set in the path evaluation, so as to drive the path search to preferentially avoid the interference region. It should be noted that if the passable region is completely blocked, the minimum risk path is selected according to the risk intensity and the flight control system is prompted to enter the low-speed cautious flight mode.

[0138] In one embodiment of the present application, in order to further limit and illustrate, as shown in Figure 5 The embodiment step 104 generates the unmanned aerial vehicle flight path based on the space risk distribution model. After the unmanned aerial vehicle performs the inspection operation based on the unmanned aerial vehicle flight path, the embodiment method further includes:

[0139] 501. In the process of controlling the unmanned aerial vehicle to perform the inspection operation, real-time space point cloud data and real-time image data of the target space region are collected.

[0140] 502. Based on the real-time space point cloud data and the real-time image data, the space risk distribution model is updated to obtain an updated space risk distribution model.

[0141] 503. Based on the updated space risk distribution model, the space passable grid map is updated to obtain an updated space passable grid map.

[0142] 504、based on the updated space passable grid map, detecting whether there is an impassable grid unit in the UAV flight path.

[0143] 505、if there is, based on the updated space passable grid map, updating the UAV flight path to obtain an updated UAV flight path, so as to control the UAV to complete the remaining inspection operation based on the updated UAV flight path.

[0144] In the embodiment steps 501-505, the UAV continuously collects real-time space point cloud data and real-time image data through the on-board sensors (including laser radar, camera and electromagnetic field intensity sensor) in the process of performing the inspection operation along the flight path generated in the embodiment step 104; further, based on the real-time space point cloud data and the real-time image data, the space risk distribution model is updated to obtain an updated space risk distribution model, and based on the updated space risk distribution model, the space passable grid map is updated to obtain an updated space passable grid map, and based on the updated space passable grid map, whether there is an impassable grid unit in the UAV flight path is detected; further, if there is an impassable grid, it means that a new obstacle (such as a worker mistakenly entering, temporary stacking of objects, etc.) appears in front of the flight path or the prediction result updates to show that this path segment has entered a new risk area where the electromagnetic interference intensity exceeds the threshold value, at this time, the current execution end immediately takes the current pose (position and orientation) as the starting point to re-plan the UAV flight path. Preferably, when re-planning the UAV flight path, the available part of the previous path can be retained to minimize the flight control jitter and control resource consumption caused by path switching.

[0145] In an embodiment of the present application, in order to further limit and illustrate, the three-dimensional space model of the target space region is constructed based on the space point cloud data and the image data in the embodiment step 101, which includes: based on the point cloud image registration algorithm, the space point cloud data and the image data are fused to obtain space information data of the target space region; the space information data is filtered and denoised to obtain optimized space information data, and based on the optimized space information data, a three-dimensional space model of the target space region is constructed.

[0146] The point cloud image registration algorithm can adopt an image-point cloud space conversion model based on external parameter calibration, map each frame of image into a corresponding point cloud view cone, and realize pixel-level registration of the image and the point cloud by using mutual information maximization and gradient consistency constraint, so that the spatial information data of the target space region is obtained. Further, in order to remove outliers and high-frequency noise caused by laser scanning dead angle, light overexposure or sensor error, in the embodiment of the application, the spatial information data can be subjected to voxel filtering and statistical outlier removal operation, and a curvature-constrained bilateral filtering method is introduced to enhance edge continuity, so as to improve the structure definition and stability of the three-dimensional space model.

[0147] It should be noted that by introducing the external parameter registration and information quantity driven fusion strategy, the integrity and accuracy of the model in the spatial geometric expression and the image semantic expression are effectively improved.

[0148] The application provides a method for planning a flight path of a UAV. First, spatial point cloud data and image data of a target space region are obtained, and a three-dimensional space model of the target space region is constructed based on the spatial point cloud data and the image data. Second, an obstacle recognition model that has completed model training is used to perform category recognition processing and positioning processing on obstacles contained in the image data, to obtain category information and position information of the obstacles, and the category information and the position information are mapped to the three-dimensional space model to obtain a semantic space model of the target space region. Third, an electromagnetic field intensity data prediction model that has completed model training is used to perform prediction processing on electromagnetic field intensity spatial distribution data of the target space region at a current time, to obtain current electromagnetic field intensity spatial distribution data, and the current electromagnetic field intensity spatial distribution data is filtered based on a preset electromagnetic field intensity threshold to determine an electromagnetic interference risk region, and the electromagnetic interference risk region is mapped to the semantic space model to obtain a spatial risk distribution model of the target space region. Finally, a flight path of the UAV is generated based on the spatial risk distribution model, and the UAV is controlled to perform inspection work based on the flight path. Compared with the prior art, the embodiment of the application avoids the problem that GPS signals are easily blocked by constructing a three-dimensional space model of a target space region by combining spatial point cloud data and image data. Further, by identifying category information and position information of obstacles, a semantic space model of the target space region is obtained, so that the three-dimensional space model not only has geometric information but also carries obstacle category information, effectively improving the accuracy of obstacle avoidance. Further, the electromagnetic interference prediction is integrated into the flight path planning, effectively reducing the risk of failure of the flight control system, and reducing the risk of collision of the UAV from multiple dimensions.

[0149] Further, as an embodiment of the above Figure 1In the implementation of the method, the embodiment of the present application provides a planning system for a flight path of a UAV, such as Figure 6 As shown, the system comprises:

[0150] a data acquisition module 61, an obstacle identification module 62, an electromagnetic field prediction module 63, and a flight path planning module 64.

[0151] The data acquisition module 61 is configured to acquire point cloud data and image data of a target space region, and construct a three-dimensional space model of the target space region based on the point cloud data and the image data.

[0152] The obstacle identification module 62 is configured to perform category identification processing and positioning processing on obstacles contained in the image data based on an obstacle identification model that has completed model training, to obtain category information and position information of the obstacles, and to map the category information and the position information to the three-dimensional space model to obtain a semantic space model of the target space region.

[0153] The electromagnetic field prediction module 63 is configured to perform prediction processing on electromagnetic field intensity spatial distribution data of the target space region at a current time based on an electromagnetic field intensity data prediction model that has completed model training, to obtain current electromagnetic field intensity spatial distribution data, and to perform screening processing on the current electromagnetic field intensity spatial distribution data based on a preset electromagnetic field intensity threshold value, to determine an electromagnetic interference risk region, and to map the electromagnetic interference risk region to the semantic space model to obtain a space risk distribution model of the target space region.

[0154] The flight path planning module 64 is configured to generate a UAV flight path based on the space risk distribution model, to control a UAV to perform inspection work based on the UAV flight path.

[0155] In a specific application scenario, before the obstacle identification module, the system further comprises an obstacle identification model training module configured to:

[0156] construct an initial obstacle identification model based on a convolutional neural network;

[0157] construct an obstacle identification training data set based on image data of space regions in different work environments, wherein the obstacle identification training data set contains multiple training image data and corresponding standard image annotation results;

[0158] perform category identification processing and positioning processing on training obstacles contained in each of the training image data based on the initial obstacle identification model, to obtain predicted category information and predicted position information of the obstacles of each of the training image data;

[0159] Integrate the predicted category information and the predicted position information of each of the obstacles respectively to obtain a predicted image labeling result of the obstacles of each of the training image data;

[0160] Calculate a first loss function value between the predicted image labeling result and the standard image labeling result, and perform model training on the initial obstacle recognition model based on the first loss function value to obtain an obstacle recognition model that has completed model training.

[0161] In a specific application scenario, before the electromagnetic field prediction module, the system further includes an electromagnetic field intensity data prediction model training module configured to:

[0162] Based on a residual neural network and a long short-term memory network, an initial electromagnetic field intensity data prediction model is constructed.

[0163] Based on historical electromagnetic field intensity spatiotemporal distribution data of the target space region, an electromagnetic field intensity prediction training data set is constructed, wherein the electromagnetic field intensity prediction training data set contains spatial position information of a plurality of point locations within the target space region and corresponding historical electromagnetic field intensity temporal distribution data.

[0164] Based on the initial electromagnetic field intensity data prediction model, electromagnetic field intensity data of each of the point locations at a preset training time is predicted and processed according to the spatial position information of each of the point locations, to obtain predicted electromagnetic field intensity spatial distribution data at the preset training time.

[0165] The electromagnetic field intensity data at the preset training time is obtained from the historical electromagnetic field intensity temporal distribution data of each of the point locations respectively, to obtain historical electromagnetic field intensity spatial distribution data at the preset training time.

[0166] Calculate a second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data, and perform model training on the initial electromagnetic field intensity data prediction model based on the second loss function value to obtain an electromagnetic field intensity data prediction model that has completed model training.

[0167] In a specific application scenario, the flight path planning module is configured to:

[0168] The target space region is divided to obtain a plurality of three-dimensional grid units.

[0169] Obtain an obstacle position point set and an electromagnetic interference risk region point set from the space risk distribution model.

[0170] Mark the three-dimensional grid cells without intersection with the obstacle position point set and the electromagnetic interference risk region point set as passable grid cells, to obtain a spatial passable grid map;

[0171] Based on the three-dimensional heuristic search algorithm, generate a UAV flight path according to the spatial passable grid map, wherein the three-dimensional heuristic search algorithm is a heuristic evaluation combined with the path cost between passable grid cells, flight direction guidance and electromagnetic risk penalty factor.

[0172] In a specific application scenario, the system further comprises a flight path updating module after the flight path planning module, configured to:

[0173] In the process of controlling the UAV to perform the inspection operation, collect real-time spatial point cloud data and real-time image data of the target space region;

[0174] Based on the real-time spatial point cloud data and the real-time image data, update the spatial risk distribution model to obtain an updated spatial risk distribution model;

[0175] Based on the updated spatial risk distribution model, update the spatial passable grid map to obtain an updated spatial passable grid map;

[0176] Based on the updated spatial passable grid map, detect whether there is an impassable grid cell in the UAV flight path;

[0177] If there is, update the UAV flight path based on the updated spatial passable grid map to obtain an updated UAV flight path, so as to control the UAV to complete the remaining inspection operation based on the updated UAV flight path.

[0178] In a specific application scenario, the data acquisition module is configured to:

[0179] Based on a point cloud image registration algorithm, fuse the spatial point cloud data and the image data to obtain spatial information data of the target space region;

[0180] Filter and denoise the spatial information data to obtain optimized spatial information data, and construct a three-dimensional space model of the target space region based on the optimized spatial information data.

[0181] In a specific application scenario, the first loss function value is used to represent the total value of the joint loss function, wherein the total value of the joint loss function includes the cross-entropy loss function value and the smoothness loss function value; the second loss function value is used to represent the mean square error loss function value.

[0182] The application provides a UAV flight path planning system. First, spatial point cloud data and image data of a target space region are acquired, and a three-dimensional space model of the target space region is constructed based on the spatial point cloud data and the image data. Second, an obstacle recognition model that has completed model training is used to perform category recognition processing and positioning processing on obstacles contained in the image data, to obtain category information and position information of the obstacles, and the category information and the position information are mapped to the three-dimensional space model to obtain a semantic space model of the target space region. Third, an electromagnetic field intensity data prediction model that has completed model training is used to perform prediction processing on electromagnetic field intensity spatial distribution data of the target space region at a current time, to obtain current electromagnetic field intensity spatial distribution data, and the current electromagnetic field intensity spatial distribution data is filtered based on a preset electromagnetic field intensity threshold to determine an electromagnetic interference risk region, and the electromagnetic interference risk region is mapped to the semantic space model to obtain a spatial risk distribution model of the target space region. Finally, a UAV flight path is generated based on the spatial risk distribution model, and a UAV is controlled to perform inspection work based on the UAV flight path. Compared with the prior art, the application avoids the problem that GPS signals are easily blocked by constructing a three-dimensional space model of a target space region by combining spatial point cloud data and image data. Further, the category information and the position information of obstacles are identified to obtain a semantic space model of the target space region, so that the three-dimensional space model not only has geometric information, but also carries obstacle category information, effectively improving the accuracy of obstacle avoidance. Further, electromagnetic interference prediction is integrated into flight path planning, effectively reducing the risk of failure of a flight control system, and reducing the risk of UAV collision from multiple dimensions.

[0183] According to an embodiment of the application, a storage medium is provided, and the storage medium stores at least one executable instruction. The computer executable instruction can execute the UAV flight path planning method in any method embodiment described above.

[0184] Based on such understanding, the technical solution of the application can be embodied in the form of a software product, which can be stored in a nonvolatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in various implementation scenarios of the application.

[0185] Figure 7 A structural schematic diagram of a terminal according to an embodiment of the application is shown, and the specific implementation of the terminal is not limited in the specific embodiments of the application.

[0186] As Figure 7As shown, the terminal can include a processor 702, a communications interface 704, a memory 706, and a communications bus 708.

[0187] The processor 702, the communications interface 704, and the memory 706 can communicate with each other through the communications bus 708.

[0188] The communications interface 704 is configured to communicate with network elements such as clients or other servers.

[0189] The processor 702 is configured to execute the program 710, and can execute the related steps in the above-described embodiments of the method for planning a flight path of a UAV.

[0190] Specifically, the program 710 can include program code including computer operation instructions.

[0191] The processor 702 can be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device can be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0192] The memory 706 is configured to store the program 710. The memory 706 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory.

[0193] The program 710 can be specifically configured to cause the processor 702 to perform the following operations:

[0194] Obtain spatial point cloud data and image data of a target space region, and construct a three-dimensional space model of the target space region based on the spatial point cloud data and the image data;

[0195] Perform category recognition processing and positioning processing on obstacles included in the image data based on an obstacle recognition model that has completed model training, to obtain category information and position information of the obstacles, and map the category information and the position information to the three-dimensional space model to obtain a semantic space model of the target space region;

[0196] Based on the electromagnetic field intensity data prediction model of the completed model training, the electromagnetic field intensity spatial distribution data of the target space region at the current time is predicted to obtain current electromagnetic field intensity spatial distribution data, and based on a preset electromagnetic field intensity threshold, the current electromagnetic field intensity spatial distribution data is screened to determine an electromagnetic interference risk region, and the electromagnetic interference risk region is mapped to the semantic space model to obtain a spatial risk distribution model of the target space region.

[0197] Based on the spatial risk distribution model, a UAV flight path is generated to control the UAV to perform inspection operation based on the UAV flight path.

[0198] The storage medium can also include an operating system, a network communication module. The operating system is a program for managing the hardware and software resources of the entity device of the above-mentioned unmanned aerial vehicle flight path planning method, supporting the running of information processing programs and other software and / or programs. The network communication module is used to realize the communication between the components in the storage medium and the communication with other hardware and software in the information processing entity device.

[0199] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0200] The method and system of the present application can be implemented in many ways. For example, the method and system of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-mentioned order of steps for the method is only for illustration, and the steps of the method of the present application are not limited to the above-mentioned order, unless otherwise specifically described. In addition, in some embodiments, the present application can also be implemented as programs recorded in recording media, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers the recording medium for storing the programs for executing the method according to the present application.

[0201] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with a general purpose computing device, which can be centralized on a single computing device or distributed on a network of multiple computing devices, and optionally implemented with program codes executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in an order different from that shown, or made into individual integrated circuit modules, or made into a single integrated circuit module. Thus, the present application is not limited to any particular combination of hardware and software.

[0202] The preferred embodiments of the present application described above are intended to be illustrative only and the present application is not limited to the above described preferred embodiments nor the application described therein. Various modifications made to the preferred embodiments of the application will be apparent to those skilled in the art, and this application includes all such modifications without parting from the spirit and scope of the present application.

Claims

1. A method for planning a flight path of an unmanned aerial vehicle, characterized in that: include: Acquiring spatial point cloud data and image data of a target spatial region, and constructing a three-dimensional spatial model of the target spatial region based on the spatial point cloud data and the image data; Based on the obstacle recognition model that has completed model training, perform category recognition and location processing on the obstacles contained in the image data to obtain category information and location information of the obstacles, and map the category information and location information to the three-dimensional space model to obtain a semantic space model of the target space area; Based on the electromagnetic field intensity data prediction model for which model training has been completed, the electromagnetic field intensity spatial distribution data of the target spatial area at the current moment is predicted and processed to obtain the current electromagnetic field intensity spatial distribution data; and based on a preset electromagnetic field intensity threshold, the current electromagnetic field intensity spatial distribution data is screened and processed to determine the electromagnetic interference risk area, and the electromagnetic interference risk area is mapped to the semantic space model to obtain the spatial risk distribution model of the target spatial area; A drone flight path is generated based on the spatial risk distribution model, so as to control the drone to perform inspection operations based on the drone flight path.

2. The method according to claim 1, characterized in that Before performing category recognition and location processing on each obstacle contained in the image data based on the obstacle recognition model that has completed model training and obtaining category information and location information of each obstacle, the method further includes: Build an initial obstacle recognition model based on convolutional neural network; Based on image data of spatial regions in different working environments, an obstacle recognition training dataset is constructed, wherein the obstacle recognition training dataset includes multiple training image data and corresponding standard image annotation results; Based on the initial obstacle recognition model, performing category recognition processing and location processing on each training obstacle contained in the training image data to obtain predicted category information and predicted location information of the obstacle in each training image data; Integrating the predicted category information and predicted position information of each obstacle to obtain predicted image annotation results of the obstacles in each training image data; A first loss function value between the predicted image annotation result and the standard image annotation result is calculated, and based on the first loss function value, the initial obstacle recognition model is trained to obtain a trained obstacle recognition model.

3. The method according to claim 1, characterized in that Before performing prediction processing on the electromagnetic field intensity spatial distribution data of the target spatial area at the current moment based on the electromagnetic field intensity data prediction model for which model training has been completed to obtain the current electromagnetic field intensity spatial distribution data, the method further comprises: Based on residual neural network and long short-term memory network, the initial electromagnetic field intensity data prediction model is constructed; Based on the historical spatiotemporal distribution data of electromagnetic field intensity in the target spatial area, an electromagnetic field intensity prediction training data set is constructed, wherein the electromagnetic field intensity prediction training data set includes spatial position information of multiple points in the target spatial area and corresponding historical electromagnetic field intensity time series distribution data; Based on the initial electromagnetic field intensity data prediction model, the electromagnetic field intensity data of each of the points at the preset training time is predicted and processed according to the spatial position information of each of the points, so as to obtain the predicted electromagnetic field intensity spatial distribution data at the preset training time; Obtaining the electromagnetic field intensity data at the preset training moment from the historical electromagnetic field intensity time series distribution data of each of the points, and obtaining the historical electromagnetic field intensity spatial distribution data at the preset training moment; A second loss function value between the predicted electromagnetic field intensity spatial distribution data and the historical electromagnetic field intensity spatial distribution data is calculated, and based on the second loss function value, the initial electromagnetic field intensity data prediction model is trained to obtain an electromagnetic field intensity data prediction model that has completed model training.

4. The method according to claim 1, wherein Generating a UAV flight path based on the spatial risk distribution model includes: Dividing the target space area into multiple three-dimensional grid units; Extracting an obstacle location point set and an electromagnetic interference risk area point set from the spatial risk distribution model; Marking three-dimensional grid cells that do not intersect with the obstacle location point set and the electromagnetic interference risk area point set as passable grid cells, and obtaining a spatial passable grid map; Based on a three-dimensional heuristic search algorithm, a UAV flight path is generated according to the spatially traversable grid map, wherein the three-dimensional heuristic search algorithm combines the path cost between traversable grid cells, the heuristic estimation of the flight direction guidance, and the electromagnetic risk penalty factor.

5. The method according to claim 1, wherein After generating a drone flight path based on the spatial risk distribution model and controlling the drone to perform an inspection operation based on the drone flight path, the method further includes: In the process of controlling the UAV to perform the inspection operation, collecting real-time spatial point cloud data and real-time image data of the target spatial area; Based on the real-time spatial point cloud data and the real-time image data, the spatial risk distribution model is updated to obtain an updated spatial risk distribution model; Based on the updated spatial risk distribution model, the spatial passability grid map is updated to obtain an updated spatial passability grid map; Based on the updated spatially passable grid map, detecting whether there are inaccessible grid cells in the flight path of the UAV; If it exists, the UAV flight path is updated based on the updated spatially accessible grid map to obtain an updated UAV flight path, so as to control the UAV to complete the remaining inspection operations based on the updated UAV flight path.

6. The method according to claim 1, characterized in that The constructing of a three-dimensional spatial model of the target spatial area based on the spatial point cloud data and the image data includes: Based on a point cloud image registration algorithm, the spatial point cloud data and the image data are fused to obtain spatial information data of the target spatial area; The spatial information data is filtered and denoised to obtain optimized spatial information data, and a three-dimensional spatial model of the target spatial area is constructed based on the optimized spatial information data.

7. The method according to claim 2 or 3, characterized in that The first loss function value is used to represent the total value of the joint loss function, where the total value of the joint loss function includes the cross entropy loss function value and the smoothing loss function value; the second loss function value is used to represent the mean square error loss function value.

8. A UAV flight path planning system, characterized in that: include: a data acquisition module, configured to acquire spatial point cloud data and image data of a target spatial region, and construct a three-dimensional spatial model of the target spatial region based on the spatial point cloud data and the image data; an obstacle recognition module, configured to perform category recognition and location processing on obstacles contained in the image data based on the obstacle recognition model that has completed model training, obtain category information and location information of the obstacles, and map the category information and location information to the three-dimensional space model to obtain a semantic space model of the target space area; an electromagnetic field prediction module, configured to predict and process the electromagnetic field intensity spatial distribution data of the target spatial area at the current moment based on the electromagnetic field intensity data prediction model for which model training has been completed, thereby obtaining the current electromagnetic field intensity spatial distribution data, and to screen and process the current electromagnetic field intensity spatial distribution data based on a preset electromagnetic field intensity threshold, thereby determining an electromagnetic interference risk area, and mapping the electromagnetic interference risk area to the semantic space model to obtain a spatial risk distribution model of the target spatial area; A flight path planning module is used to generate a UAV flight path based on the spatial risk distribution model, so as to control the UAV to perform inspection operations based on the UAV flight path.

9. A storage medium storing at least one executable instruction, characterized in that: The executable instructions enable the processor to perform operations corresponding to the method for planning a drone flight path as described in any one of claims 1 to 7.

10. A terminal comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, characterized in that the executable instruction enables the processor to perform operations corresponding to the method for planning a flight path of a drone as described in any one of claims 1 to 7.

Images