Autonomous obstacle avoidance method, device and equipment for multi-sensor fusion collaborative edge calculation

Abstract

The invention provides an autonomous obstacle avoidance method, device and equipment for multi-sensor fusion collaborative edge calculation, and relates to the technical field of autonomous obstacle avoidance. The method comprises the following steps: determining an initial inspection route of an inspection unmanned aerial vehicle according to line information of a to-be-inspected power line and unmanned aerial vehicle characteristics of the inspection unmanned aerial vehicle; analyzing first sensor information of each first sensor of the inspection unmanned aerial vehicle through edge calculation to determine a real-time inspection environment of the inspection unmanned aerial vehicle, and analyzing second sensor information of each second sensor through edge calculation to obtain real-time dimension obstacle information of each second sensor; based on the real-time inspection environment and all real-time dimension obstacle information, obtaining real-time obstacle information of the inspection unmanned aerial vehicle; and correcting the initial inspection route according to the real-time obstacle information to obtain an autonomous obstacle avoidance inspection route of the inspection unmanned aerial vehicle. According to the invention, the autonomous obstacle avoidance capability of the inspection unmanned aerial vehicle in a complex environment can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of autonomous obstacle avoidance, and in particular to a multi-sensor fusion collaborative edge computing autonomous obstacle avoidance method, device and equipment. BACKGROUND

[0002] The unmanned aerial vehicle (UAV) is widely used in power line detection due to its high flexibility, convenient operation and rapid arrival at the designated area. It can efficiently and comprehensively patrol the power line and timely find potential problems of the line, such as broken insulator, worn wire, loose line, etc., providing strong guarantee for the stable operation of the power system and greatly improving the efficiency and safety of power line detection.

[0003] The existing UAV power line inspection method generally adopts a UAV carrying sensor inspection scheme. The sensor carried by the UAV can be a laser radar or a visible light camera. The laser radar realizes the measurement of the wire spacing through three-dimensional point cloud modeling, and the visible light camera completes the defect detection of insulator cracks and the like relying on image recognition algorithm. In addition, the laser radar can penetrate part of the vegetation cover to obtain spatial information, and the visual system is good at capturing device surface abnormalities. Both of them can support the basic inspection function of the UAV in sunny weather and standard working conditions.

[0004] However, the existing UAV power line inspection method has perception limitations. For example, the accuracy of the laser radar decreases in strong light or rain and fog weather, and the camera has difficulty in accurately identifying obstacles in insufficient light, which will affect the obstacle avoidance effect of the UAV. SUMMARY

[0005] The embodiments of the present application provide a multi-sensor fusion collaborative edge computing autonomous obstacle avoidance method, device and equipment to solve the problem that the existing UAV power line inspection method is easily affected by weather, which affects the obstacle avoidance effect of the UAV.

[0006] In a first aspect, the embodiments of the present application provide a multi-sensor fusion collaborative edge computing autonomous obstacle avoidance method, comprising:

[0007] determining an initial inspection route of the inspection UAV according to line information of the power line to be inspected and UAV characteristics of the inspection UAV;

[0008] analyzing first sensor information of each first sensor of the inspection UAV through edge computing to determine a real-time inspection environment of the inspection UAV, and analyzing second sensor information of each second sensor through edge computing to obtain real-time dimensional obstacle information of each second sensor;

[0009] obtaining real-time obstacle information of the inspection UAV based on the real-time inspection environment and all real-time dimensional obstacle information.

[0010] The initial inspection route is corrected according to the real-time obstacle information, and an autonomous obstacle-avoiding inspection route of the inspection unmanned aerial vehicle is obtained.

[0011] In a second aspect, an autonomous obstacle-avoiding device based on multi-sensor fusion and cooperative edge computing is provided, and the device comprises:

[0012] A determination module is configured to determine an initial inspection route of the inspection unmanned aerial vehicle according to line information of a power line to be inspected and unmanned aerial vehicle features of the inspection unmanned aerial vehicle;

[0013] An analysis module is configured to analyze first sensor information of each first sensor of the inspection unmanned aerial vehicle through edge computing, to determine a real-time inspection environment of the inspection unmanned aerial vehicle, and to analyze second sensor information of each second sensor through edge computing, to obtain real-time dimensional obstacle information of each second sensor;

[0014] A fusion module is configured to obtain real-time obstacle information of the inspection unmanned aerial vehicle based on the real-time inspection environment and all real-time dimensional obstacle information;

[0015] A correction module is configured to correct the initial inspection route according to the real-time obstacle information, to obtain an autonomous obstacle-avoiding inspection route of the inspection unmanned aerial vehicle.

[0016] In a third aspect, an electronic device is provided, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0017] In the embodiments of the present application, the initial inspection route is planned by combining power line features and unmanned aerial vehicle features, and the environment data and sensor information are analyzed in real time by using edge computing, the obstacles are perceived, and the sensor accuracy is calibrated, so that the autonomous intelligent adjustment of the initial inspection route is realized. By calibrating the sensor information by the environment, the influence of the environment on the sensor information can be reduced, and the autonomous obstacle-avoiding capability of the inspection unmanned aerial vehicle is effectively improved, so that the efficiency and safety of the inspection process are ensured. The scheme provided in the present application can effectively improve the autonomous obstacle-avoiding capability of the unmanned aerial vehicle in a complex environment and the reliability of the inspection. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is an implementation flowchart of the autonomous obstacle-avoiding method based on multi-sensor fusion and cooperative edge computing provided by the embodiments of the present application;

[0019] Figure 2 is an application scenario schematic diagram of the inspection unmanned aerial vehicle of the autonomous obstacle-avoiding method based on multi-sensor fusion and cooperative edge computing provided by the embodiments of the present application;

[0020] Figure 3 is an implementation flowchart of step S140 of the autonomous obstacle avoidance method of the multi-sensor fusion cooperative edge computing provided by the embodiment of the present application;

[0021] Figure 4 is a structural schematic diagram of the autonomous obstacle avoidance device of the multi-sensor fusion cooperative edge computing provided by the embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0024] Referring to Figure 1 , which shows an implementation flowchart of the autonomous obstacle avoidance method of the multi-sensor fusion cooperative edge computing provided by the embodiment of the present application, which is described in detail as follows:

[0025] Step S110, determining an initial inspection route of the inspection unmanned aerial vehicle according to line information of a power line to be inspected and unmanned aerial vehicle features of the inspection unmanned aerial vehicle.

[0026] In some embodiments, the power line to be inspected refers to a power transmission and distribution line system that needs to implement inspection work, usually including overhead lines, cables, towers and auxiliary equipment. Such lines have characteristics such as high voltage level, large span, and complex environment. As shown in Figure 2 , the inspection unmanned aerial vehicle is used to inspect the tower and power line. The line information refers to a data set of the topological structure and technical parameters of the power line, including geographic spatial information, electrical parameters, structural features, etc., wherein the geographic spatial information refers to a spatial position data system of the power line and its surrounding environment, including tower GPS coordinates, conductor sag three-dimensional model, line direction azimuth, terrain features, artificial obstacle distribution, and other spatial topological data information.

[0027] In some embodiments, the unmanned aerial vehicle features are a core performance parameter system that determines the unmanned aerial vehicle operation capability, including power characteristics, load capacity, perception system, communication performance, location of the starting point of the unmanned aerial vehicle, unmanned aerial vehicle wing span, unmanned aerial vehicle volume, and other data. The initial inspection route is a pre-planned flight path generated by multiple constraint conditions, which needs to consider the line direction, safety regulations, operation efficiency, and other line information elements that affect route selection.

[0028] In a possible implementation, the specific processing manner of step S110 is: determining the terrain feature and the line data of the power line to be inspected according to the line information of the power line to be inspected; determining the power data of the inspection unmanned aerial vehicle according to the unmanned aerial vehicle feature of the inspection unmanned aerial vehicle; and determining the initial inspection route of the inspection unmanned aerial vehicle based on the terrain feature, the line data, and the power data of the inspection unmanned aerial vehicle.

[0029] In some embodiments, the terrain feature refers to the landform type and spatial distribution feature of the area where the power line is located, including slope, elevation, ground cover, and the like. The line data refers to the engineering parameters of the power line and the detection time required, and the engineering parameters include information such as conductor electrical characteristics and structural parameters. The initial inspection route refers to a pre-planned flight path generated by comprehensively considering the line feature and the unmanned aerial vehicle capability, which needs to meet the conditions of obstacle avoidance, detection, efficiency, and power constraint.

[0030] Specifically, the initial inspection route can be obtained in the following manner. First, a safe flight corridor is established according to the terrain feature and the wingspan and volume of the inspection unmanned aerial vehicle. Second, a detection priority area is set according to the line data, and the maximum inspectable mileage is calculated through an energy consumption model. Finally, a multi-constraint path optimization algorithm is used to generate an initial inspection route that meets the energy consumption threshold within the corridor.

[0031] In step S120, the first sensor information of each first sensor of the inspection unmanned aerial vehicle is analyzed through edge computing to determine the real-time inspection environment of the inspection unmanned aerial vehicle, and the second sensor information of each second sensor is analyzed through edge computing to obtain the real-time dimensional obstacle information of each second sensor.

[0032] In some embodiments, edge computing is a distributed computing architecture that deploys data processing capabilities on edge devices close to data sources, such as on-board computers of unmanned aerial vehicles, rather than relying on remote cloud servers. Its core advantages lie in low latency, high real-time performance, and bandwidth optimization. The inspection unmanned aerial vehicle is a unmanned aerial vehicle specially used for inspection of power facilities. The inspection unmanned aerial vehicle in the present application integrates multiple sensors and has the capabilities of autonomous flight, data acquisition, and real-time analysis.

[0033] In some embodiments, the first sensor is used to perceive the macro state of the environment and obtain the basic parameters of the unmanned aerial vehicle operation environment to provide input for environment modeling. The first sensor includes temperature sensors, humidity sensors, pressure sensors, wind speed sensors, and the like. The first sensor information is a set of raw environmental parameter data collected by the first sensor, and the first sensor information can form understandable environmental features after edge computing analysis. The real-time inspection environment is a dynamic environment model constructed through multi-source sensor data fusion, which includes temperature, humidity, atmospheric pressure, wind speed, and other key factors affecting the operation of the unmanned aerial vehicle.

[0034] In some embodiments, the second sensor is a high-precision sensor for obstacle detection, and the second sensor focuses more on spatial dimension measurement and can provide accurate data support for obstacle avoidance decision-making. The second sensor can include millimeter wave radar, binocular stereo vision, ultrasonic array, etc. The second sensor information is the raw measurement data output by the second sensor, containing the spatial attributes and dynamic characteristics of the obstacle. After the second sensor information is analyzed by edge computing, real-time dimensional obstacle information of the corresponding dimension can be obtained. The standardized obstacle description generated after the second sensor information is spatio-temporally calibrated and data fused is the real-time dimensional obstacle information, wherein the dimension includes position, size, motion state, etc. The real-time dimensional obstacle information corresponding to the second sensor can be single-dimensional obstacle information or multi-dimensional obstacle information with different confidence levels. For example, the second sensor A can only obtain obstacle position-related information, while the second sensor B can obtain both obstacle position information and obstacle size information. However, the second sensor B is usually used to measure the size of an object, so the size information of the obstacle measured by the second sensor B has a high confidence level, while the obstacle position information is not accurate enough and has a low confidence level.

[0035] In a possible implementation, the specific processing manner of step S120 is: analyzing the first sensor information of each first sensor of the inspection unmanned aerial vehicle through edge computing to obtain real-time sub-inspection environment data of each first sensor information corresponding to the dimension; and fitting the real-time sub-inspection environment data corresponding to each first sensor information and the line information of the power line to be inspected to obtain the real-time inspection environment of the inspection unmanned aerial vehicle.

[0036] In some embodiments, the real-time sub-inspection environment data is structured environment feature data output by a single first sensor after edge computing processing.

[0037] In some embodiments, the step of obtaining the real-time inspection environment by fitting is as follows: through the edge computing node, firstly, spatio-temporal alignment is performed on the multi-source heterogeneous data, that is, the real-time sub-inspection environment data obtained by the first sensor such as temperature and wind speed is spatially registered with the line information, and the data of different first sensor data is time-synchronized. Then, according to a certain fusion algorithm, such as a Bayesian fusion framework, spatial layer fusion, semantic layer mapping, and confidence weighting fusion operations are performed on the real-time sub-inspection environment data and the line information to obtain the real-time inspection environment.

[0038] In a possible implementation, the specific processing manner of step S120 is: determining, according to the sensor characteristics of each second sensor, a corresponding environmental influence data set of each second sensor; and performing the following steps for each second sensor: screening, from the corresponding environmental influence data set of the second sensor, environmental influence information matching the real-time inspection environment; and performing information calibration on the initial second sensor information of the second sensor based on the environmental influence information, to obtain the sensor information of the second sensor.

[0039] In some embodiments, the sensor characteristics are used to describe inherent performance parameters and environmental sensitivity of the sensors. For example, the second sensor A has strong raindrop penetration, the second sensor B has increased measurement error in low visibility, and the second sensor C has large measurement deviation when the temperature exceeds a certain value. The environmental influence data set is a database for recording error characteristics of the sensors under different environmental conditions, and includes quantitative correction parameters. The environmental influence information is the sensor error characteristics and quantitative correction parameters extracted from the database and matching the current environment. For example, the measured value of the sensor A will be 0.5% higher than normal when the temperature exceeds the preset temperature.

[0040] In some embodiments, the initial second sensor information is the raw data output by the second sensor without environmental calibration, and the second sensor information is more accurate information obtained after environmental calibration than the initial second sensor information. The information calibration is a process of error compensation and credibility optimization of the sensor data according to environmental factors.

[0041] In a possible implementation, the specific processing manner of step S120 is: performing the following steps for each second sensor: performing data fitting on the second sensor information of the second sensor at each historical moment, to obtain a historical data fitting result of the second sensor; determining virtual sensor information of the second sensor at the current moment according to the historical data fitting result; judging whether a difference between the second sensor information and the virtual sensor information is within a preset range; if the difference is within the preset range, updating real-time dimensional obstacle information of the second sensor at a previous moment to obtain real-time dimensional obstacle information of the second sensor at the current moment; and if the difference is not within the preset range, analyzing the second sensor information of the second sensor at the current moment through edge computing to obtain real-time dimensional obstacle information of the second sensor.

[0042] In some embodiments, the second sensor information at the historical moment refers to a set of original data collected by the second sensor at a past time point, mainly including time series measurement values. Data fitting is a process of extracting regularity features from historical data through mathematical modeling, for predicting or compensating sensor behavior. The historical data fitting result refers to a quantitative model or statistical feature generated after data fitting, which can reflect the potential regularity of the data collected by the second sensor at the historical moment. It should be noted that the historical moment refers to the moment after the inspection unmanned aerial vehicle starts this inspection.

[0043] In some embodiments, the virtual sensor information is a current moment sensor theoretical output value calculated according to the historical data fitting result, for judging whether the sensor information of the second sensor at the current moment is the same as the trend at the historical moment. The preset range is artificially set according to the characteristics of the sensor.

[0044] It should be noted that the difference value refers to the difference between the second sensor information actually collected at the current moment and the virtual sensor information. When the difference value is within the preset range, the real-time dimensional obstacle information at the current moment can be calculated according to the real-time dimensional obstacle information at the previous moment and the motion data of the inspection unmanned aerial vehicle, such as flight speed and flight direction. When the difference value is not within the preset range, it means that the second sensor has detected a new obstacle or the obstacle previously detected by the second sensor has disappeared or moved, at which time the obstacle needs to be repositioned according to the second sensor information at the current moment to obtain the real-time dimensional obstacle information.

[0045] Through the construction of a dynamic collaborative mechanism of historical model prediction and real-time data verification, the time series modeling of sensor historical data and the real-time analysis of edge computing nodes are optimized, which optimizes the data processing efficiency. In terms of data accuracy, through the dynamic comparison of virtual sensor information and measured data, combined with the adaptive threshold setting of the environment, intelligent identification and rapid response of abnormal data can be realized.

[0046] Step S130, based on the real-time inspection environment and all real-time dimensional obstacle information, the real-time obstacle information of the inspection unmanned aerial vehicle is obtained.

[0047] In some embodiments, the real-time inspection environment refers to a dynamic environment model constructed by real-time analysis of the data of the first sensor through edge computing and combined with line information, for reflecting the comprehensive environmental state of the current operation area of the unmanned aerial vehicle. The real-time dimensional obstacle information is an obstacle feature description generated after edge computing, calibration and multi-dimensional analysis of the data collected by the second sensor, including parameters of different dimensions such as spatial position, size and motion state. The real-time obstacle information is standardized obstacle avoidance decision data generated by fusing the real-time inspection environment and the real-time dimensional obstacle information, including obstacle position, type, dynamic attribute and modified parameters after environmental influence.

[0048] In a possible implementation, the specific processing manner of step S130 is: performing information fusion on the real-time dimensional obstacle information to obtain real-time multi-dimensional obstacle information of the inspection UAV; determining control error information of the inspection UAV according to the real-time inspection environment, and determining real-time obstacle information of the inspection UAV based on the control error information and the real-time multi-dimensional obstacle information.

[0049] In some embodiments, information fusion refers to a process of integrating multi-source heterogeneous data through an algorithm, eliminating contradictions, and extracting unified representation. Real-time multi-dimensional obstacle information refers to standardized obstacle description information formed after fusion of all real-time dimensional obstacle information, including spatial attributes, dynamic attributes, and physical attributes. Control error information refers to inherent deviation between the inspection UAV and the perception system, which changes with different environments, and therefore needs to be determined through the inspection environment, including control deviation of the inspection UAV itself and the impact of control deviation of the inspection UAV itself on the sensor. For example, the inspection UAV will have a certain horizontal deviation in a strong wind environment, therefore, the sensor will also have a certain detection error in the horizontal direction in the strong wind environment, the horizontal deviation of the inspection UAV can be obtained by pre-testing with the same type of inspection UAV, and the detection error of the sensor needs to be calculated considering the position of the sensor installed on the inspection UAV.

[0050] Through information fusion, the characteristics of the obstacle can be comprehensively perceived, and the control error is dynamically calibrated in combination with the real-time inspection environment, which can effectively improve the accuracy and environmental adaptability of obstacle recognition. Through collaborative processing of heterogeneous data of different sensors, the limitations of a single sensor can be eliminated, and the error compensation mechanism for the real-time inspection environment enhances the robustness of the system, which can effectively improve the execution degree of the obstacle avoidance decision basis, that is, the real-time obstacle information.

[0051] Step S140, revising the initial inspection route according to the real-time obstacle information to obtain an autonomous obstacle avoidance inspection route of the inspection UAV.

[0052] In some embodiments, the autonomous obstacle avoidance inspection route is obtained by adjusting the safe trajectory of the initial inspection route according to the obstacle information after the inspection UAV detects the obstacle, and the autonomous obstacle avoidance inspection route needs to realize power line inspection while avoiding obstacles.

[0053] Referring to Figure 3 The specific processing of step S140 can include step S1401 and step S1402, and the specific content is as follows:

[0054] In step S1401, the real-time position of the obstacle is determined according to the real-time obstacle information, and whether the obstacle is located on the initial inspection route is determined according to the real-time position.

[0055] In some embodiments, the real-time position of the obstacle refers to the coordinates of the center position of the obstacle in a three-dimensional space, expressed in a geographic coordinate system.

[0056] It should be noted that when determining whether the obstacle is located on the initial inspection route, not only the center position of the obstacle needs to be considered, but also the volume and shape of the obstacle and other information.

[0057] In a possible implementation, the specific processing manner of step S1401 is as follows: based on the real-time position of the obstacle, real-time coordinate data of the obstacle is determined; the nearest distance between the real-time coordinate data and each coordinate data on the initial inspection route is calculated, and whether the obstacle is located on the initial inspection route is determined according to the nearest distance.

[0058] In some embodiments, the real-time coordinate data is used to represent the real-time position of the obstacle, and a coordinate system needs to be established for the unmanned aerial vehicle body, and the real-time position of the obstacle is reflected in the coordinate system. At this time, the coordinate data corresponding to the obstacle is the real-time coordinate data. The nearest distance is the minimum Euclidean distance between the real-time coordinate data of the obstacle and each coordinate point on the initial inspection route. When calculating, each coordinate point on the initial inspection route also needs to be converted into the coordinate system established by the inspection unmanned aerial vehicle.

[0059] It should be noted that when determining whether the obstacle is located on the initial inspection route, after the minimum Euclidean distance is calculated, the minimum Euclidean distance also needs to be compared with the sum of the maximum radius of the obstacle and the reserved safety distance. If the minimum Euclidean distance is greater than the sum of the maximum radius of the obstacle and the reserved safety distance, it indicates that the obstacle is not located on the initial inspection route. If the minimum Euclidean distance is less than the sum of the maximum radius of the obstacle and the reserved safety distance, it indicates that the obstacle is located on the initial inspection route.

[0060] In step S1402, if the obstacle is located on the initial inspection route, the initial inspection route is modified according to the real-time position of the obstacle and the real-time position of the inspection unmanned aerial vehicle, and an autonomous obstacle-avoiding inspection route of the inspection unmanned aerial vehicle is obtained.

[0061] It should be noted that if the obstacle is not located on the initial inspection route, that is, the inspection unmanned aerial vehicle will not be affected by the obstacle when flying according to the initial inspection route, and at this time, the initial inspection route does not need to be adjusted.

[0062] In some embodiments, the initial inspection route can be corrected by dynamic environment perception and path optimization algorithm. The autonomous obstacle avoidance inspection route is a corrected trajectory generated after the initial inspection route is corrected, and needs to ensure the integrity of the inspection task while avoiding real-time obstacles. When correcting, the minimum fly-around distance, energy optimization and other constraint conditions can be set.

[0063] Through the cooperative processing mechanism of multi-sensor fusion and edge computing, combined with real-time environment calibration and multi-dimensional obstacle information fusion, the autonomous obstacle avoidance capability of the inspection unmanned aerial vehicle in complex environment can be effectively improved. The dynamic environment model is generated by real-time analysis of the first sensor information by edge computing, which can accurately perceive external interference factors such as weather and terrain. Through the environmental influence dataset, the second sensor data is self-calibrated, which can eliminate the measurement deviation of millimeter wave radar and visual sensor caused by rain, fog, strong light and other environmental noise to some extent, improve the obstacle detection accuracy, and through the dynamic verification mechanism of historical data fitting and virtual sensor prediction, the abnormal obstacle can be quickly responded and the data reliability can be optimized. In addition, the multi-dimensional obstacle information and control error compensation are fused to generate an obstacle avoidance path that takes into account the safety distance and inspection efficiency, which can support the unmanned aerial vehicle to correct the inspection route in real time under the sudden obstacle scene such as broken conductor and floating object, and ensure the continuity and reliability of the power line inspection.

[0064] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0065] The following is a device embodiment of the present application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0066] Figure 4 The structure schematic diagram of the autonomous obstacle avoidance device of the multi-sensor fusion cooperative edge computing provided by the embodiments of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the details are as follows:

[0067] As Figure 4 shown, the autonomous obstacle avoidance device of multi-sensor fusion cooperative edge computing 4 comprises:

[0068] The determining module 41 is configured to determine the initial inspection route of the inspection unmanned aerial vehicle according to the line information of the power line to be inspected and the unmanned aerial vehicle characteristics of the inspection unmanned aerial vehicle;

[0069] The analysis module 42 is configured to analyze the first sensor information of each first sensor of the inspection UAV through edge computing, to determine a real-time inspection environment of the inspection UAV, and to analyze the second sensor information of each second sensor through edge computing, to obtain real-time dimensional obstacle information of each second sensor.

[0070] The fusion module 43 is configured to obtain real-time obstacle information of the inspection UAV based on the real-time inspection environment and all real-time dimensional obstacle information.

[0071] The correction module 44 is configured to correct the initial inspection route according to the real-time obstacle information, to obtain an autonomous obstacle-avoiding inspection route of the inspection UAV.

[0072] In a possible implementation, the determination module 41 is specifically configured to: determine a terrain feature and line data of the power line to be inspected according to line information of the power line to be inspected; determine power data of the inspection UAV according to UAV features of the inspection UAV; and determine an initial inspection route of the inspection UAV based on the terrain feature, the line data, and the power data of the inspection UAV.

[0073] In a possible implementation, the analysis module 42 is specifically configured to: analyze the first sensor information of each first sensor of the inspection UAV through edge computing, to obtain real-time sub-inspection environment data of a dimension corresponding to each first sensor information; and fit the real-time sub-inspection environment data corresponding to each first sensor information and the line information of the power line to be inspected, to obtain a real-time inspection environment of the inspection UAV.

[0074] In a possible implementation, the analysis module 42 is further configured to: determine an environment influence data set corresponding to each second sensor according to sensor features of each second sensor; and for each second sensor, perform the following steps: from the environment influence data set corresponding to the second sensor, screen environment influence information matching the real-time inspection environment; and based on the environment influence information, perform information calibration on initial second sensor information of the second sensor, to obtain sensor information of the second sensor.

[0075] In a possible implementation, the analysis module 42 is further configured to: for each second sensor, perform the following steps: perform data fitting on the second sensor information of the second sensor at each historical time to obtain a historical data fitting result of the second sensor; determine virtual sensor information of the second sensor at the current time according to the historical data fitting result; determine whether a difference between the second sensor information and the virtual sensor information is within a preset range; if the difference is within the preset range, update real-time dimension obstacle information of the second sensor at a previous time to obtain real-time dimension obstacle information of the second sensor at the current time; and if the difference is not within the preset range, analyze the second sensor information of the second sensor at the current time in an edge computing manner to obtain real-time dimension obstacle information of the second sensor.

[0076] In a possible implementation, the fusion module 43 is specifically configured to: perform information fusion on the real-time dimension obstacle information to obtain real-time multi-dimensional obstacle information of the inspection unmanned aerial vehicle; determine control error information of the inspection unmanned aerial vehicle according to the real-time inspection environment, and determine real-time obstacle information of the inspection unmanned aerial vehicle based on the control error information and the real-time multi-dimensional obstacle information.

[0077] In a possible implementation, the correction module 44 is specifically configured to: determine a real-time position of the obstacle according to the real-time obstacle information, and determine whether the obstacle is located on the initial inspection route according to the real-time position; if the obstacle is located on the initial inspection route, correct the initial inspection route according to the real-time position of the obstacle and a real-time position of the inspection unmanned aerial vehicle to obtain an autonomous obstacle avoidance inspection route of the inspection unmanned aerial vehicle.

[0078] In a possible implementation, the correction module 44 is further configured to: determine real-time coordinate data of the obstacle based on the real-time position of the obstacle; calculate a nearest distance between the real-time coordinate data and each coordinate data on the initial inspection route, and determine whether the obstacle is located on the initial inspection route according to the nearest distance.

[0079] Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 5 the electronic device 5 of this embodiment includes a processor 50 and a memory 51. The memory 51 stores a computer program 52. The processor 50 implements the steps in each of the method embodiments described above when executing the computer program 52. Alternatively, the processor 50 implements the functions of each module / unit in each of the device embodiments described above when executing the computer program 52.

[0080] For example, the computer program 52 can be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 52 in the electronic device 5.

[0081] The electronic device 5 can include, but is not limited to, the processor 50, the memory 51. Those skilled in the art can understand that, Figure 5 The electronic device 5 is only an example and does not constitute a limitation on the electronic device 5, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the electronic device 5 can also include an input / output device, a network access device, a bus, etc.

[0082] For the convenience and brevity of description, only the above-mentioned division of each functional module / unit is exemplified, and in actual application, the above-mentioned functions can be completed by different functional modules / units according to needs. The above-mentioned modules / units can be realized in the form of hardware, software or a combination of hardware and software.

[0083] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can refer to the related description of other embodiments. If there is no special description and logical conflict, the terms and / or descriptions of different embodiments are consistent and can be mutually referenced, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0084] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An autonomous obstacle avoidance method based on multi-sensor fusion and collaborative edge computing, characterized in that: include: Determining an initial inspection route of the inspection drone based on line information of the power line to be inspected and drone characteristics of the inspection drone; Analyze the first sensor information of each first sensor of the inspection drone through edge computing to determine the real-time inspection environment of the inspection drone, and analyze the second sensor information of each second sensor through edge computing to obtain the real-time dimensional obstacle information of each second sensor; Based on the real-time inspection environment and all real-time dimensional obstacle information, obtain real-time obstacle information of the inspection drone; The initial inspection route is corrected according to the real-time obstacle information to obtain an autonomous obstacle avoidance inspection route for the inspection drone.

2. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The edge computing method is used to analyze each second sensor information separately to obtain the real-time dimensional obstacle information of each second sensor information, including: For each second sensor, perform the following steps: Performing data fitting on the second sensor information of the second sensor at each historical moment to obtain a historical data fitting result of the second sensor; Determining virtual sensor information of the second sensor at a current moment according to the historical data fitting result; Determining whether a difference between the second sensor information and the virtual sensor information is within a preset range; If the difference is within a preset range, the real-time dimensional obstacle information of the second sensor at the previous moment is updated to obtain the real-time dimensional obstacle information of the second sensor at the current moment; If the difference is not within the preset range, the second sensor information of the second sensor at the current moment is analyzed by edge computing to obtain the real-time dimensional obstacle information of the second sensor.

3. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The real-time obstacle information of the inspection drone is obtained based on the real-time inspection environment and all real-time dimensional obstacle information, including: Performing information fusion on the real-time dimensional obstacle information to obtain real-time multi-dimensional obstacle information of the inspection drone; The control error information of the inspection drone is determined according to the real-time inspection environment, and the real-time obstacle information of the inspection drone is determined based on the control error information and the real-time multi-dimensional obstacle information.

4. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The method further comprises: determining an environmental impact data set corresponding to each second sensor according to sensor characteristics of each second sensor; For each second sensor, perform the following steps: Filtering environmental impact information that matches the real-time inspection environment from the environmental impact data set corresponding to the second sensor; Information calibration is performed on initial second sensor information of the second sensor based on the environmental impact information to obtain sensor information of the second sensor.

5. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The analyzing the first sensor information of each first sensor of the inspection drone through edge computing to determine the real-time inspection environment of the inspection drone includes: Analyze the first sensor information of each first sensor of the inspection drone through edge computing to obtain real-time sub-inspection environment data of the dimension corresponding to each first sensor information; The real-time sub-inspection environment data corresponding to each first sensor information and the line information of the power line to be inspected are fitted to obtain the real-time inspection environment of the inspection drone.

6. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The step of correcting the initial inspection route according to the real-time obstacle information to obtain the autonomous obstacle avoidance inspection route of the inspection drone includes: Determining the real-time location of the obstacle based on the real-time obstacle information, and judging whether the obstacle is located on the initial inspection route based on the real-time location; If the obstacle is located on the initial inspection route, the initial inspection route is corrected according to the real-time position of the obstacle and the real-time position of the inspection drone to obtain the autonomous obstacle avoidance inspection route of the inspection drone.

7. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 6 is characterized in that: The determining, based on the real-time position, whether the obstacle is located on the initial inspection route includes: Determining real-time coordinate data of the obstacle based on the real-time position of the obstacle; The shortest distance between the real-time coordinate data and each coordinate data on the initial inspection route is calculated, and based on the shortest distance, whether the obstacle is located on the initial inspection route is determined.

8. The autonomous obstacle avoidance method of multi-sensor fusion collaborative edge computing according to claim 1 is characterized in that: The determining of the initial inspection route of the inspection drone based on the line information of the power line to be inspected and the drone characteristics of the inspection drone includes: Determining the terrain characteristics and line data of the power line to be inspected based on the line information of the power line to be inspected; Determining power data of the inspection drone based on drone characteristics of the inspection drone; An initial inspection route of the inspection drone is determined based on the terrain features, the route data, and the power data of the inspection drone.

9. An autonomous obstacle avoidance device with multi-sensor fusion and collaborative edge computing, characterized in that: include: A determination module, configured to determine an initial inspection route of the inspection drone based on line information of the power line to be inspected and drone characteristics of the inspection drone; An analysis module is configured to analyze first sensor information of each first sensor of the inspection drone through edge computing to determine the real-time inspection environment of the inspection drone, and to analyze second sensor information of each second sensor through edge computing to obtain real-time dimensional obstacle information of each second sensor; A fusion module is used to obtain the real-time obstacle information of the inspection drone based on the real-time inspection environment and all real-time dimensional obstacle information; The correction module is used to correct the initial inspection route according to the real-time obstacle information to obtain the autonomous obstacle avoidance inspection route of the inspection drone.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images