Unmanned Aerial Vehicle (UAV) Line-of-Sight Analysis and Obstacle Avoidance Methods and Related Devices
By analyzing line-of-sight between digital elevation models and digital surface models, and verifying the results through dual-channel methods of laser ranging inverse calculation and large-scale visual models, the problem of predicting and confirming line-of-sight obstruction by UAVs was solved, improving the success rate and reliability of observation missions and reducing endurance consumption and mission delays.
Patent Information
- Application Number
- CN202611101767.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-11-10
AI Technical Summary
Existing drone obstacle avoidance and observation technologies struggle to predict and confirm line-of-sight obstructions before deployment, leading to mission interruptions. Furthermore, the lack of a closed-loop adjustment mechanism impacts mission success rate and reliability.
Line-of-sight analysis is performed using digital elevation model and digital surface model data. Combined with dual-channel verification of laser ranging inverse calculation and visual large model semantic recognition, occlusion prediction and closed-loop relocation are achieved to ensure unobstructed line of sight.
It improves the success rate and reliability of UAV target observation, reduces battery consumption and mission delays, and distinguishes between occlusion and positioning errors through dual-channel verification, enabling rapid elimination of occlusion.
Smart Images

Figure CN123067540A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of unmanned aerial vehicle (UAV) control, and in particular to a UAV line-of-sight analysis and obstacle avoidance method and related apparatus. Background Technology
[0002] Drones are widely used in scenarios such as surveillance, emergency response, and security patrols to conduct close-range observation and continuous evidence collection of specific targets on the ground. In these tasks, drones need to fly to an aerial position where the target can be clearly observed and maintain a stable line of sight to the target during the observation process. The target's geographical location is usually already known when the mission is assigned.
[0003] However, real-world mission environments are often densely built-up urban areas, mountainous terrain with undulating slopes, or areas covered by vegetation, where the line of sight between the drone and the target can easily be obstructed by buildings, terrain, or trees. Once the line of sight is obstructed, the drone cannot obtain a valid image of the target, and the observation mission is interrupted.
[0004] Existing UAV obstacle avoidance and observation position determination technologies struggle to integrate pre-deployment line-of-sight obstruction prediction with on-site obstruction confirmation and adjustment into a cohesive whole. Some obstacle avoidance technologies passively detect obstacles ahead during flight using binocular vision or laser ranging, without assessing whether the UAV's line of sight to a specific target is obstructed. Consequently, they cannot avoid obstructed observation positions before deployment. Even when using maps that include elevation data to select observation positions, the lack of consideration for terrain elevation and the absence of three-dimensional obstruction caused by buildings, vegetation, or other surface attachments mean that the selected positions may still be obstructed on-site. Furthermore, on-site obstruction assessment often relies on simple distance measurements, making it difficult to distinguish between actual obstruction and measurement deviations caused by positioning errors or target movement. There is also a lack of a mechanism for closed-loop adjustment of the observation position after obstruction confirmation. Ultimately, prediction and on-site handling become disconnected, resulting in delayed obstruction detection, blind adjustments to the observation position, and missions often requiring repeated trials, which consumes endurance and delays. Summary of the Invention
[0005] In order to predict line-of-sight obstructions and select observation positions before dispatching drones, and reliably confirm obstructions on-site and adjust observation positions in a closed loop, thereby improving the success rate and reliability of target observation, this application provides a drone line-of-sight analysis and obstacle avoidance method and related device.
[0006] Firstly, this application provides a method for UAV line-of-sight analysis and obstacle avoidance, which adopts the following technical solution:
[0007] A method for line-of-sight analysis and obstacle avoidance for unmanned aerial vehicles (UAVs) includes the following steps:
[0008] S1. Receive the target's location information and load the digital elevation model data and digital surface model data of the surrounding area of the target;
[0009] S2. Based on digital elevation model data and digital surface model data, perform line-of-sight analysis on multiple candidate observation points around the target to predict whether the line of sight from the candidate observation points to the target is blocked, and determine the target observation point and retain alternative observation points from multiple candidate observation points according to the comprehensive score;
[0010] S3. Plan the flight path from the current location of the UAV to the target observation point, and control the UAV to fly along the flight path to the target observation point;
[0011] S4. After the UAV arrives at the target observation point, the line of sight from the UAV to the target is verified by laser ranging inverse calculation and visual big data model semantic recognition. The occlusion verification result is obtained. The laser ranging inverse calculation calculates the inverse position based on the laser ranging result and compares the inverse position with the target position information to verify whether the line of sight is blocked. The visual big data model semantic recognition identifies the occlusion status of the line of sight based on the on-site images collected by the UAV.
[0012] S5. In response to the occlusion verification result indicating that the line of sight is obstructed, perform position relocation of the UAV according to the occlusion verification result, and return to perform occlusion verification after the position relocation is completed, until the occlusion verification result indicates that the line of sight is unobstructed.
[0013] By adopting the above technical solution, before dispatching the UAV, line-of-sight analysis is performed on multiple candidate observation points based on digital elevation model data and digital surface model data to avoid locations where the line of sight will be blocked by terrain and surface attachments from the source. After arriving, the line of sight is verified by a dual-channel approach of geometric comparison calculated by laser ranging and semantic judgment by semantic recognition of visual large model. When occlusion is confirmed, closed-loop relocation and reverification are performed to restore line of sight. The three-way coordination of predictive point selection, on-site dual-channel verification and closed-loop adjustment ensures that occlusion is avoided before dispatch and eliminated on-site, thereby improving the success rate and reliability of target observation.
[0014] Optionally, the laser ranging inverse calculation includes: emitting a laser towards the target using laser ranging and receiving the reflected echo; obtaining the measured distance from the UAV to the target and the horizontal azimuth and elevation angles of the emitted laser based on the reflected echo; transforming the UAV's position from the geodetic coordinate system to the geocentric coordinate system; calculating the target's position relative to the UAV based on the measured distance, horizontal azimuth, and elevation angles; then transforming back to the geodetic coordinate system to obtain the inverse calculated position; determining an obstruction judgment threshold based on the UAV's positioning solution status, the measured distance, and the intensity of the reflected echo; the obstruction judgment threshold increases with the measured distance and is greater when the positioning solution status is a floating solution than when it is a fixed solution; calculating the comprehensive deviation between the inverse calculated position and the target's position information, obtained by weighting the horizontal distance deviation and the altitude deviation; and determining that the line of sight is obstructed when the comprehensive deviation is greater than the obstruction judgment threshold.
[0015] By adopting the above technical solution, the laser ranging result is back-calculated into the target coordinates and compared with the known position. The occlusion is determined geometrically, and the occlusion judgment threshold is adaptive with the positioning accuracy and distance, avoiding misjudgment at long distances or when the positioning is poor.
[0016] Optionally, the visual big model semantic recognition includes: triggering the visual big model semantic recognition when the laser ranging back calculation determines that there is occlusion in the line of sight or the comprehensive deviation falls into the suspicious deviation range, otherwise skipping it; extracting the region of interest containing the back-calculated position from the on-site image based on the back-calculated position; inputting the region of interest into the visual big model to identify structured occlusion information including occlusion status, occlusion type, occlusion orientation, adjustment suggestions and confidence level.
[0017] By adopting the above technical solution, the large visual model is gated by the geometric criteria of laser, and semantic recognition is invoked and the region of interest is focused only when there is suspicion. This reduces the computational cost while obtaining interpretable structured occlusion information.
[0018] Optionally, conflict arbitration is performed on the results of laser ranging back calculation and visual large model semantic recognition: when the laser receives a reflected echo, the occlusion judgment of the laser ranging back calculation shall be used as the standard, and the direction of position repositioning shall be corrected with the adjustment suggestions in the structured occlusion information; when the laser does not receive a reflected echo, the occlusion verification result shall be determined with the occlusion status and confidence level in the structured occlusion information.
[0019] By adopting the above technical solution, arbitration is performed according to the respective reliable scenarios of the two channels, taking into account both the geometric accuracy of the laser and the semantic coverage of the large visual model.
[0020] Optionally, when the overall deviation exceeds the occlusion judgment threshold, laser ranging is first used to back-calculate the reference deviation for landmark reference points with known coordinates around the target. Then, the following distinction is made: if the reference deviation and the overall deviation are consistent in direction and magnitude, it is determined that there is a systematic deviation between the digital elevation model data and the digital surface model data. The coordinate correction amount is calculated based on the reference deviation, and the coordinate registration correction is performed on the above data before reselecting the point; if they are inconsistent, it is determined to be a local occlusion and the location is repositioned.
[0021] By adopting the above technical solutions, we can distinguish between systematic map bias and local temporary occlusion. For the former, we can correct the map and reselect the location; for the latter, we can relocate the location. This avoids blindly exhausting the alternative observation points due to systematic map errors.
[0022] Optionally, common cause failure detection is performed on laser ranging back calculation and visual large model semantic recognition: the signal-to-noise ratio and broadening of the reflected echo, and the clarity and confidence of the on-site image are used as the channel quality of the two channels respectively. When the quality of both channels is lower than their respective lower limits, it is determined to be a common cause failure. The UAV is controlled to hover and perform recovery actions such as adjusting the heading to change the relative light source position or reducing the flight altitude according to the corresponding environment type, and then re-verified. If the common cause failure continues for more than a preset time limit, the result of the target being currently unobservable is output.
[0023] By adopting the above technical solutions, we can identify environmental blindness caused by factors such as dense fog, rain, and backlight that cause simultaneous degradation of both channels, thus avoiding misjudgment or blind flight in such situations, and provide a backup plan by reporting overdue information.
[0024] Optionally, after location relocation, the comprehensive score and digital surface model data are updated based on the occlusion verification results: unrecorded temporary obstacles are marked with obstacle confidence in the digital surface model data, and the comprehensive score and ranking of candidate observation points are recalculated based on the marked data, so that subsequent point selection avoids the temporary obstacle.
[0025] By adopting the above technical solution, the temporary obstacles measured on-site are fed back to the site selection stage, forming a closed-loop optimization from the site to the site selection.
[0026] Optionally, performing line-of-sight analysis and determining candidate observation points includes: performing line-of-sight analysis on preliminary observation points in multiple directions around the target based on digital elevation model data, eliminating directions obstructed by terrain to obtain candidate directions, and then performing line-of-sight analysis on candidate observation points at different distances in the candidate directions based on digital surface model data to obtain line-of-sight capability; wherein the line-of-sight analysis samples the elevation along the line connecting the candidate observation points to the target, and compares the line-of-sight height of each sampling point with the height of surface attachments represented by the digital surface model data after adding a preset safety margin, and when the sum of the height of surface attachments and the safety margin exceeds the line-of-sight height, it is determined that the line of sight at that sampling point is obstructed.
[0027] By adopting the above technical solution, the number of candidate observation points that need to be searched point by point is reduced by using a hierarchical method of topographic coarse screening superimposed with surface fine search, and the impact of elevation data error on occlusion determination is reduced with a safety margin.
[0028] Optionally, when the overall deviation is greater than the occlusion judgment threshold, the visual model can be used to distinguish whether the line of sight is occluded or the target has been moved, based on the direction of change of the measured distance relative to the expected distance and whether the visual big model has identified the target in the region of interest. If the target has been moved, the visual big model can re-retrieve and reposition the target according to the semantic features of the target in the expanded field of view and then reselect the point.
[0029] By adopting the above technical solution, the misjudgment of target movement as occlusion is avoided, and the re-locking of moving targets is supported.
[0030] Optionally, the location repositioning strategy is selected according to the type of obstruction. When there is a building-type obstruction, the vehicle will detour horizontally; when there is a low obstacle, the vehicle will rise and adjust the pan-tilt angle in parallel; and when there is still no line of sight, the vehicle will retreat to the alternative observation point. The movement step size increases as the degree of obstruction of the line of sight increases.
[0031] By adopting the above technical solution, relocalization is driven by semantic occlusion type and adaptive step size, thereby improving the efficiency of occlusion removal.
[0032] Optionally, the overall score is a weighted sum of the line-of-sight score, distance suitability score, pitch angle quality score, and communication signal quality score.
[0033] By adopting the above technical solutions, the advantages and disadvantages of the observation location can be comprehensively evaluated from multiple dimensions.
[0034] Optionally, the observation parameters, including flight altitude, gimbal pitch angle and camera focal length, are calculated for the target observation point. The flight altitude is not lower than the height required to allow the line of sight to pass over the surface attachments, and the camera focal length is such that the target imaging meets the preset ground sampling distance.
[0035] By adopting the above technical solutions, the UAV can obtain target images that meet the resolution requirements with appropriate attitude and focal length.
[0036] Optionally, when planning flight routes, constraints such as meteorological conditions, terrain elevation, communication signal coverage, no-fly zones, and endurance can be taken into account. If no feasible route is found, the constraints can be relaxed in order of priority from low to high until a feasible route is generated.
[0037] By adopting the above technical solution, feasible routes are generated under multiple constraints, and reachability is guaranteed by hierarchical relaxation.
[0038] Optionally, when the position is repositioned to change the flight altitude or horizontal position, the camera focal length is adjusted in conjunction with the position to keep the pixel ratio of the target in the image constant.
[0039] By adopting the above technical solution, the target imaging scale can be kept stable when adjusting the observation position.
[0040] Optionally, during the occlusion verification phase, the aircraft flies at an altitude above ground attachments, and during the monitoring phase after line of sight is achieved, the altitude is lowered to ensure that the ground sampling distance does not exceed a preset upper limit.
[0041] By adopting the above technical solution, both the visibility requirements of the verification phase and the resolution requirements of the monitoring phase are taken into account.
[0042] Optionally, the laser ranging back calculation performs a multi-point scan along the edge direction of the obstructed line of sight, and determines the edge range of the obstruction based on the abrupt change position of the measured distance sequence.
[0043] By employing the above technical solution, the edges of obstructions are delineated to assist in selecting a detour direction.
[0044] Optionally, the results of each occlusion verification can be accumulated into a temporary obstacle record based on location and time, and candidate observation points with temporary obstacles can be eliminated in advance based on this record before point selection.
[0045] By adopting the above technical solutions, we can accumulate historical experience in blocking to avoid temporary obstacles before deployment.
[0046] Optionally, the weight of each score in the overall score can be adaptively adjusted according to the event type corresponding to the target.
[0047] By adopting the above technical solutions, the evaluation of observation locations can be adapted to the different emphases of different event types.
[0048] Secondly, this application provides a UAV line-of-sight analysis and obstacle avoidance device, which adopts the following technical solution:
[0049] A drone line-of-sight analysis and obstacle avoidance device includes: a data loading module configured to receive the target's position information and load digital elevation model data and digital surface model data of the area surrounding the target; a viewpoint optimization module configured to perform line-of-sight analysis on multiple candidate observation points around the target based on the digital elevation model data and digital surface model data to predict whether the line of sight is obstructed, and determine the target observation point and retain alternative observation points according to a comprehensive score; a flight path scheduling module configured to plan a flight path from the drone's current position to the target observation point and control the drone to fly to the target observation point; an obstruction verification module configured to perform obstruction verification on the line of sight from the drone to the target after the drone arrives, using laser ranging back calculation and visual large model semantic recognition; and a relocation module configured to perform position relocation on the drone when there is obstruction in the line of sight and trigger the obstruction verification module for re-verification until the line of sight is clear.
[0050] By adopting the above technical solution, the device uses five modules to collaboratively achieve pre-deployment point selection, on-site dual-channel occlusion verification, and closed-loop repositioning, which corresponds to the above method and achieves the same beneficial effects.
[0051] Thirdly, this application provides a computer-readable storage medium, which adopts the following technical solution:
[0052] A computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the above-described UAV line-of-sight analysis and obstacle avoidance method.
[0053] By adopting the above technical solution, the above method can be stored and distributed in program form, and achieve the same beneficial effect when loaded and executed.
[0054] In summary, this application includes at least one of the following beneficial technical effects:
[0055] First, before dispatching, three-dimensional prediction and point selection are made based on digital elevation model data and digital surface model data. After arrival, the line of sight is verified by laser ranging and visual large model semantic recognition in two channels. After confirming the obstruction, closed-loop repositioning is performed. Prediction, verification and adjustment work together to improve the success rate and reliability of target observation.
[0056] Secondly, compared with single distance measurement, on-site dual-channel verification can distinguish between real occlusion and deviation caused by positioning error or target movement. It also improves verification reliability and avoids blindly exhausting alternative observation points by using conflict arbitration, common cause failure detection and deviation attribution identification.
[0057] Third, the temporary obstacles detected on-site are fed back to the point selection stage, and the system adaptively relocates according to the type and degree of occlusion. Compared with the fixed obstacle avoidance method without feedback, it can eliminate occlusion faster, reduce repeated attempts, and reduce battery consumption. Attached Figure Description
[0058] Figure 1 This is a schematic diagram of the system application environment for the UAV line-of-sight analysis and obstacle avoidance method in one embodiment of this application.
[0059] Figure 2 This is a flowchart of a UAV line-of-sight analysis and obstacle avoidance method in one embodiment of this application.
[0060] Figure 3 This is a schematic diagram of line-of-sight analysis in one embodiment of this application.
[0061] Figure 4 This is a schematic diagram of viewpoint search and comprehensive scoring in one embodiment of this application.
[0062] Figure 5This is a flowchart of on-site occlusion verification and closed-loop adjustment in one embodiment of this application.
[0063] Figure 6 This is a schematic diagram of laser ranging and inverse calculation of occlusion detection in one embodiment of this application.
[0064] Figure 7 This is a schematic diagram of a drone line-of-sight analysis and obstacle avoidance device in one embodiment of this application. Detailed Implementation
[0065] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the scope of the application.
[0066] This application discloses a method for line-of-sight analysis and obstacle avoidance for unmanned aerial vehicles (UAVs). The following section first explains several technical terms involved in the embodiments of this application.
[0067] A digital elevation model (DEM) is a model that represents the elevation information of bare ground.
[0068] A digital surface model (DSM) is a model that, in addition to a digital elevation model (DEM), includes height information of surface attachments such as buildings and vegetation. When combined with a DEM, it can provide more complete occlusion analysis data.
[0069] Visual large models (VLMs) are multimodal artificial intelligence models that can understand both images and text simultaneously.
[0070] The World Geodetic System 1984 (WGS84) is a globally used geographic coordinate system.
[0071] A pan-tilt-zoom (PTZ) camera is a camera that can be controlled to rotate horizontally, tilt, and zoom.
[0072] Line-of-sight analysis is an analytical method that calculates whether the line of sight from the observation point to the target point is obstructed by terrain or obstacles.
[0073] The core idea of this application's embodiments is as follows: Before dispatching a UAV, a line-of-sight analysis is performed on multiple candidate observation points around the target based on digital elevation model data and digital surface model data. This avoids locations where the line of sight might be obstructed by terrain and surface attachments. The target observation point and reserve observation points are then determined from the multiple candidate observation points based on a comprehensive score. After the UAV arrives at the target observation point, the line of sight is verified using two channels: geometric comparison based on laser ranging and semantic judgment based on large-scale visual model semantic recognition. If obstruction is confirmed, the UAV undergoes a closed-loop repositioning based on the verification results and returns for re-verification until the line of sight is clear. The pre-deployment prediction of the observation point, the on-site dual-channel verification, and the closed-loop adjustment after obstruction confirmation work together to minimize obstruction before deployment, reliably confirm it on-site, and actively eliminate it, thereby improving the success rate and reliability of target observation.
[0074] For ease of explanation, the following description uses two operating scenarios. Scenario 1: Monitoring of crowds gathering in a city square: After receiving a report of a sudden crowd gathering in a square, the command center sends the target's location information to the system. Based on this, the system loads digital elevation model data and digital surface model data of the surrounding area and performs subsequent processing. Scenario 2: Emergency response to traffic accidents in mountainous areas: The target area has complex terrain, the target point is located in a valley, and most directions are obscured by mountains and trees. The application environment of the above systems is as follows... Figure 1 As shown.
[0075] The steps are described in detail below. The UAV line-of-sight analysis and obstacle avoidance method of this application includes steps S1 to S5. The overall flow of the above method is as follows: Figure 2 As shown.
[0076] In step S1, the location information of the target is received, and digital elevation model data and digital surface model data of the surrounding area of the target are loaded. Specifically, S1 includes sub-steps S11 to S12.
[0077] S11. Receive emergency information pushed by the command center. The emergency information includes the location information of the target, the event type and priority, represented by the 1984 geodetic coordinates. The event type is, for example, crowd gathering, traffic accident or public security incident. The above emergency information is used as input for subsequent system scheduling and perspective optimization.
[0078] S12. Based on the target's location information, load the digital elevation model data and digital surface model data of the surrounding area of the target. The digital elevation model data provides the elevation of the bare terrain, and the digital surface model data overlays the heights of buildings and vegetation on this basis. The two are combined for line-of-sight analysis in subsequent steps.
[0079] In S2, based on digital elevation model (DEM) and digital surface model (DSM) data, line-of-sight (LOS) analysis is performed on multiple candidate observation points around the target to predict whether the LOS from each candidate observation point to the target is obstructed. The target observation point is then determined from the multiple candidate observation points based on a comprehensive score, and several reserve observation points are retained. Here, candidate observation points refer to the hovering positions of the UAVs deployed around the target according to certain rules and to be evaluated; the target observation point is the candidate observation point with the highest comprehensive score, selected as the primary dispatch point; and the reserve observation points are several candidate observation points with the second highest comprehensive score, retained for sequential testing in case of on-site obstruction. Before dispatch, the LOS of each candidate observation point is evaluated based on elevation data, and points are selected accordingly. This ensures that the target observation point to which the UAV is heading has good LOS before departure, providing a preliminary guarantee for obtaining a clear line of sight upon arrival.
[0080] Specifically, S2 searches for candidate observation points centered on the target point within a preset radius R (default 500 meters) along 16 equally spaced azimuths (every 22.5°). Multiple distance sampling points are set in each azimuth from near to far. For each candidate observation point p, its comprehensive score is calculated.
[0081]
[0082] in, , , , These are the line-of-sight score, distance suitability score, pitch angle quality score, and communication signal quality score for candidate observation point p, respectively. to The corresponding weights are assigned; the candidate observation point with the highest comprehensive score is selected as the target observation point, and several candidate observation points with the second highest comprehensive scores are retained as alternative observation points. Continuing from scenario one, after searching for candidate observation points in 16 directions, the candidate observation point in the northeast direction has the highest comprehensive score and is determined as the target observation point; in scenario two, since the target point is located in a valley and is mostly obscured by mountains and trees, the system finds the only visible direction on the southeast slope in the search of 16 directions, and selects the candidate observation point in this direction as the target observation point.
[0083] In some embodiments, the line-of-sight analysis performed on the candidate observation points and the determination of the candidate observation points include sub-steps S21 to S22.
[0084] S21. Based on digital elevation model data, perform line-of-sight analysis on the initial observation points in multiple directions around the target, eliminate directions blocked by terrain, and obtain candidate directions.
[0085] S22. Based on digital surface model data, perform line-of-sight analysis on candidate observation points at different distances in the candidate orientation to obtain the line-of-sight of each candidate observation point.
[0086] In this process, the line-of-sight analysis samples the elevation along the line connecting the candidate observation point to the target: for sampling point i, the height of the line of sight above the ground at that point is calculated by linear interpolation. And query the height of surface attachments at that point from the digital surface model data. When the height of surface attachments is within the preset safety margin When the sum exceeds the line of sight height, the condition is satisfied. The line of sight at the sampling point was determined to be obstructed. A hierarchical approach was adopted: first, a coarse screening of the terrain based on digital elevation model data, followed by a finer search of surface attachments based on digital surface model data. This reduced the number of candidate observation points requiring point-by-point fine-searching and mitigated the impact of elevation data errors on obstruction determination with a safety margin. The above line-of-sight analysis process is as follows: Figure 3 As shown.
[0087] As another implementation of line-of-sight analysis, it is also possible to not distinguish between the two levels of terrain coarse screening and surface fine search, but to directly perform point-by-point line-of-sight analysis on candidate observation points at all directions and distances within a preset radius based on digital surface model data, and obtain the line-of-sight of each candidate observation point according to the same occlusion criterion; this method also belongs to the above-mentioned line-of-sight analysis based on elevation data.
[0088] The overall score is obtained by weighted summation of the line-of-sight score, distance suitability score, pitch angle quality score, and communication signal quality score. The calculation methods for each sub-score are as follows: The line-of-sight score measures the degree of visibility from the candidate observation point to the target: 1.0 point is awarded for complete visibility; points are deducted proportionally based on the maximum obstruction height if obstruction exists; and 0 points are awarded for complete obstruction. A Gaussian function model can be used to model this score.
[0089]
[0090] in, For maximum shading height, The height of the line of sight above the ground. The attenuation coefficient is used. The distance suitability score is evaluated using a Gaussian function to determine whether the horizontal distance from the candidate observation point to the target is suitable.
[0091]
[0092] in, The horizontal distance from the candidate observation point to the target. The optimal observation distance (default is 100 to 300 meters, adjust according to target type). This represents the distance tolerance (default is 100 meters). The pitch angle quality score is used to evaluate the pitch angle from the candidate observation point to the target. Does it fall within the optimal range, including the optimal pitch angle? The default angle tolerance is -60° to -30°. The default value is 15°. The communication signal quality score is estimated based on a distance attenuation model between the candidate observation point and the nearest communication base station: first, the received signal strength is estimated using a logarithmic distance path loss model.
[0093]
[0094] Then normalize it to a score in the [0,1] interval.
[0095]
[0096] in, For reference distance Signal strength at (default 1 meter) (default -30dBm). The path loss index is set to 3.0 by default in urban areas and 2.0 by default in open areas. The horizontal distance from the candidate observation point to the nearest base station. The minimum available signal strength threshold (default is -90dBm); when the candidate observation point is close to the base station The signal quality score approaches 1.0, and approaches 0 when the signal attenuates to a threshold; below the threshold, it is truncated to 0. If there are signal coverage blind spots in the target area and base station location information cannot be obtained, the communication signal quality score can degenerate into a monotonically decreasing function based on distance.
[0097]
[0098] in, The maximum communication distance is set at 5000 meters by default. Following condition one, the target observation point in the northeast has a completely unobstructed line of sight, a horizontal distance close to the optimal observation distance, and ideal elevation and signal strength. Therefore, its scores for each sub-item are high, and its weighted summation score is the highest among all candidate observation points, thus it is selected as the target observation point. The above viewpoint search and comprehensive scoring process is as follows... Figure 4 As shown.
[0099] In other embodiments, the weights of each sub-item score in the overall score are... to The evaluation of the observation location is adaptively adjusted according to the event type corresponding to the target, so that the emphasis of different event types is adapted to the evaluation of the observation location: for example, for events such as traffic accidents that require rapid on-site location, the weight of line-of-sight score and distance suitability score is increased; for events such as crowd gatherings that require continuous and clear monitoring, the weight of pitch angle quality score is increased; for tasks performed at night or in areas with weak communication signals, the weight of communication signal quality score is increased.
[0100] As an optional implementation, the results of each occlusion verification are accumulated into a temporary obstacle record based on location and time. Candidate observation points with temporary obstacles are pre-selected based on this record before point selection. For example, the location, time, and obstacle type of temporary obstacles confirmed in each on-site occlusion verification are recorded in the temporary obstacle record. In the S2 point selection stage of the subsequent task, the comprehensive score of candidate observation points that fall into the location covered by the temporary obstacle record and are close in time is reduced or they are directly eliminated, so that subsequent dispatches can avoid the temporary obstacle at the source.
[0101] In S3, a flight path is planned from the UAV's current location to the target observation point, and the UAV is controlled to fly along this flight path to the target observation point. The planning of this flight path takes into account various constraints such as weather conditions, terrain elevation, communication signal coverage, no-fly zones, and endurance, and optimizes the altitude profile of the flight path. This automatic flight path planning serves as a prerequisite for obtaining a clear observation perspective, ensuring that the UAV can safely fly to the predetermined target observation point.
[0102] In some embodiments, flight route planning comprehensively considers constraints such as meteorological conditions (e.g., wind speed, wind direction, precipitation), terrain elevation (provided by digital elevation model data and digital surface model data to avoid collisions), communication signal coverage (to ensure stable image transmission and control links), no-fly zone avoidance, and battery endurance. When no feasible route is found, constraints are relaxed sequentially from lowest to highest priority until a feasible route is generated. For example, if a feasible route cannot be obtained under all constraints, lower-priority constraints are relaxed first; if no feasible route is still found, higher-priority constraints are relaxed. Constraints related to flight safety, such as no-fly zone avoidance and terrain collision avoidance, are placed with the highest priority and relaxed last or not relaxed at all. Following scenario one, automatic route planning generates a safe route after considering current meteorological conditions and signal coverage. In scenario two, automatic route planning considers the undulating terrain of mountainous areas and generates a stepped ascent route.
[0103] In other embodiments, observation parameters, including flight altitude, gimbal pitch angle, and camera focal length, are calculated for the target observation point. The flight altitude is not lower than the height required to allow the line of sight to pass over surface attachments, and the camera focal length ensures that the target image meets a preset ground sampling distance. Specifically, the observation parameters are calculated as follows.
[0104] Regarding flight altitude, the ground altitude of candidate viewpoint locations is obtained based on digital surface model data. And sample along the line of sight to obtain the obstacle height of the highest obstacle point relative to the ground. Considering that the line of sight must be higher than the obstacle point, calculate the minimum safe flight altitude.
[0105]
[0106] in, For safety margin (default is 20 meters). The elevation increment per kilometer of horizontal distance (default is 50 meters / kilometer). For horizontal distances, all the above height measurements are in meters; for example, when When the time is meters, the last term in the formula is Rice. In Based on this, and combined with the flight altitude that yields the optimal pitch angle quality score in the comprehensive scoring function, the final flight altitude is determined. .
[0107] Regarding the gimbal pitch angle, the horizontal distance is calculated based on the three-dimensional coordinate relationship between the UAV's position and the target's position. Vertical height difference Gimbal tilt angle
[0108]
[0109] To ensure shooting quality, The observation point should fall within the range of -75° to -15°. If it falls outside this range, a new observation point needs to be selected.
[0110] Regarding camera focal length, it depends on the horizontal distance. Calculate the required focal length based on the required ground sample distance (GSD).
[0111]
[0112] in, For the width of the camera sensor, The image pixel width is used; and based on the calculated focal length value, the closest focal length setting supported by the camera is selected. The above calculations of flight altitude, gimbal pitch angle, and focal length ensure that the target area can be clearly imaged at the target observation point with the optimal gimbal angle and focal length. Continuing from scenario one, the flight altitude, gimbal pitch angle, and focal length are calculated in the same way for the target observation point in the northeast direction, so that the UAV can clearly image the target with a suitable attitude and focal length; in scenario two, for the target observation point on the southeast hillside, due to the different terrain undulations and horizontal distances in the mountainous area, the observation parameters are calculated in the same way to obtain the appropriate parameters.
[0113] In S4, after the UAV arrives at the target observation point, occlusion verification of the UAV's line of sight to the target is performed jointly by laser ranging inverse calculation and visual large model semantic recognition, obtaining the occlusion verification result. Specifically, laser ranging inverse calculation calculates the inverse position based on the laser ranging result and compares the inverse position with the target's position information to verify whether the line of sight is obstructed; visual large model semantic recognition identifies the occlusion status of the line of sight based on the on-site images collected by the UAV. The reason for performing the above dual-channel verification on-site after completing the pre-deployment prediction based on elevation data is that the optimal viewing angle calculated based on digital elevation model data and digital surface model data may be invalidated by temporary obstacles on-site (such as temporarily parked vehicles, newly planted trees, temporary structures, etc., not shown in the map data); laser ranging inverse calculation provides a high-precision geometric verification benchmark, and visual large model semantic recognition provides semantic understanding and adjustment support; the joint verification of both ensures that on-site occlusion can be reliably confirmed. In scenario one, after the drone flew to the target observation point in the northeast direction, it was found that the target was partially obscured by a temporary billboard. Based on this, laser ranging back calculation confirmed the existence of obstruction. In scenario two, after the drone flew to the target observation point on the southeast hillside, laser ranging back calculation confirmed that the line of sight was unobstructed.
[0114] In some embodiments, the laser ranging inverse calculation includes sub-steps S41 to S44.
[0115] S41. The system emits a laser towards the target using laser ranging and receives the reflected echo. Based on the reflected echo, it obtains the measured distance from the UAV to the target. and the horizontal azimuth angle of the laser emission (Clockwise relative to true north) and pitch angle (Horizontal downward is positive).
[0116] S42. Transform the UAV's position from the geodetic coordinate system to the geocentric coordinate system, combine the measured distance, horizontal azimuth angle and elevation angle to calculate the target's position relative to the UAV, and then transform it back to the geodetic coordinate system to obtain the inverse position.
[0117] S43. Determine the occlusion judgment threshold based on the UAV's positioning solution status, measured distance, and intensity of reflected echo; the occlusion judgment threshold increases with the increase of measured distance, and is greater when the positioning solution status is a floating solution than when it is a fixed solution.
[0118] S44. Calculate the comprehensive deviation between the inverse position and the target's position information, which is obtained by weighting the horizontal distance deviation and the height deviation. When the comprehensive deviation is greater than the occlusion judgment threshold, it is determined that there is occlusion in the line of sight.
[0119] The coordinate transformation of S42 specifically includes the following process. First, the UAV's current World Geodetic System 1984 coordinates are transformed... Convert to Earth-centered, Earth-fixed coordinate system:
[0120]
[0121]
[0122]
[0123] in, Let be the radius of curvature of the circle. The above latitude and longitude have the highest eccentricity on Earth. , Substitute all values into radians (if the original coordinates are in degrees, multiply by 1 first). (Convert to radians). Then, convert the laser measurement values... Convert to laser pointing vector in the station center (northeast-sky) coordinate system:
[0124]
[0125] And the pointing vector is transformed from the station-centered coordinate system to the geocentric coordinate system using a rotation matrix:
[0126]
[0127] in, The rotation matrix from the station-centered coordinate system to the geocentric coordinate system is determined by the UAV's latitude and longitude. Next, the laser pointing vector is superimposed onto the UAV's position to obtain the geocentric coordinates of the target point.
[0128]
[0129] Finally, the geocentric coordinates of the target point were converted to World Geodetic System 1984 latitude and longitude: longitude was directly derived from... The latitude is obtained by using an iterative method, let... Iterate to Radius convergence:
[0130]
[0131] After convergence, the height is from The result is obtained. Among them, It is the Earth's semi-major axis (approximately 6,378,137 meters according to the World Geodetic System 1984 model). This is the highest eccentricity of Earth (0.0818191908426 in the World Geodetic System 1984 model), and it can usually be achieved in 3 to 5 iterations. The convergence accuracy in radians; the input and output of the above trigonometric functions are all in radians, and the final latitude and longitude results are multiplied by... Convert back to angle representation.
[0132] In S44, the inverse position will be calculated. relative to the known location of the target Comparison: Horizontal distance deviation The horizontal distance and height deviation between the inverse position and the known target position are calculated using the haversine formula. Overall deviation
[0133]
[0134] in This is the height weighting coefficient (default is 0.1). When the overall deviation is not greater than the occlusion judgment threshold... (Default is 5 meters) When the line of sight is unobstructed, the laser is considered to have successfully reached the target point; when the overall deviation is greater than If the laser rangefinder does not return a signal, it is determined that the line of sight is obstructed. Since the laser hits the obstruction rather than the target when the line of sight is obstructed, the deviation between the calculated pseudo-target coordinates and the real target coordinates is usually greater than 50 meters, thus enabling sensitive detection of obstruction. This method is unaffected by lighting conditions and can work stably in nighttime, backlighting, and other scenarios. The geometric relationship for detecting obstruction using laser ranging is as follows: Figure 6 As shown.
[0135] As an optional implementation, laser ranging inverse calculation also performs multi-point scanning along the edge direction of the obstructed line of sight, and determines the edge range of the obstruction based on the abrupt change position of the measured distance sequence: for example, after determining that the line of sight is obstructed, the laser rangefinder is controlled to measure the distance of multiple sampling points in sequence along the direction of the edge of the obstruction, and a set of measured distance sequences is obtained; when there is a significant abrupt change in the measured distance of adjacent sampling points (from the close distance of hitting the obstruction to the far distance pointing towards the target), the abrupt change position is used as the edge of the obstruction, thereby characterizing the edge range of the obstruction and assisting in the subsequent selection of the detour direction.
[0136] In some embodiments, the above-described visual large model semantic recognition includes sub-steps S45 to S47.
[0137] S45. When laser ranging back calculation determines that there is an obstruction in the line of sight or that the overall deviation falls into the suspicious deviation range, trigger the visual big model semantic recognition; otherwise, skip the semantic recognition.
[0138] S46. Based on the inversely calculated position, extract the region of interest containing the inversely calculated position from the field images collected by the UAV.
[0139] S47. Input the region of interest into the large visual model to identify structured occlusion information, including occlusion status, occlusion type, occlusion location, adjustment suggestions, and confidence level.
[0140] In the structured occlusion information described in S47, the occlusion state includes no occlusion, partial occlusion, and complete occlusion; the occlusion type includes buildings, trees, vehicles, temporary structures, and others; the occlusion orientation is the direction of the occluding object relative to the target; adjustment suggestions include raising, lowering, moving east, moving west, moving north, moving south, adjusting the gimbal angle, and adjusting the focal length; the confidence level ranges from 0 to 1. By using laser geometric criteria to gate the large-scale visual model, semantic recognition is invoked and the region of interest is focused only when there is occlusion or suspicious deviation in the line of sight, thus reducing computational overhead while obtaining interpretable structured occlusion information. Continuing from scenario one, the large-scale visual model analyzes the region of interest containing the inversely calculated position, identifies the occlusion type as a temporary structure (i.e., the aforementioned temporary billboard), and provides an adjustment suggestion to move eastward.
[0141] In one optional implementation, conflict arbitration is performed on the results of laser ranging inverse calculation and visual large model semantic recognition: when the laser rangefinder receives a reflected echo, the occlusion determination from the laser ranging inverse calculation is used as the standard, and the direction of position repositioning is corrected based on the adjustment suggestions in the structured occlusion information; when the laser rangefinder does not receive a reflected echo, the occlusion verification result is determined based on the occlusion state and confidence level in the structured occlusion information. Arbitration is thus performed based on the scenarios where each channel is deemed reliable, utilizing both the geometric accuracy of laser ranging inverse calculation and the semantic coverage of visual large model semantic recognition.
[0142] In some embodiments, when the overall deviation exceeds the occlusion judgment threshold, it is further distinguished whether the deviation originates from a systematic deviation between the digital elevation model data and the digital surface model data, or from local temporary occlusion: First, laser ranging is performed on the landmark reference points with known coordinates around the target to obtain the reference back-calculation deviation of the landmark reference points; if the reference back-calculation deviation is consistent with the above-mentioned overall deviation in direction and magnitude, it is determined that there is a systematic deviation between the digital elevation model data and the digital surface model data, the coordinate correction amount is calculated based on the reference back-calculation deviation, and the above data is re-selected after coordinate registration correction; if the reference back-calculation deviation is inconsistent with the overall deviation, it is determined to be local occlusion, and position relocation is performed. This is because a systematic deviation in map data will cause the back-calculation deviation of each point in the area to show a consistent direction and magnitude, while local temporary occlusion only makes the back-calculation deviation corresponding to the occluded line of sight abnormal; by introducing a landmark reference point with known coordinates for comparison, the two can be distinguished, avoiding blindly exhausting the alternative observation points when there is a systematic deviation in the map.
[0143] In other embodiments, when the overall deviation exceeds the occlusion judgment threshold, it is further distinguished whether the deviation stems from an obstructed view or from a shifted target: the distinction is made based on the direction of change of the measured distance relative to the expected distance, and whether the large visual model identifies the target within the region of interest. If the deviation of the measured distance from the expected distance points to an obstruction, and the large visual model does not identify the target within the region of interest, then the view is determined to be obstructed; if the large visual model identifies target features near the region of interest, and the distance change matches the target movement, then the target is determined to have shifted. When the target is determined to have shifted, the large visual model re-retrieves and repositions the target within the expanded field of view according to its semantic features, and then reselects a point based on the repositioned target location. This avoids misjudging target movement as occlusion and supports re-locking onto moving targets. Continuing with the scenario in Scenario 1 where the target in a crowd gathering may move, the above-mentioned distinction and re-locking enable the system to continue locking onto the target even after it has moved.
[0144] In some embodiments, common-cause failure detection is also performed on laser ranging inverse calculation and visual large model semantic recognition to identify environmental blinding situations such as dense fog, rain, and backlight that cause simultaneous degradation of both channels: the signal-to-noise ratio and broadening of the reflected echo are used as the channel quality of the laser ranging inverse calculation channel, and the clarity of the on-site image and the confidence level of the visual large model output are used as the channel quality of the visual large model semantic recognition channel; when the quality of both channels is lower than their respective lower limits, it is determined to be a common-cause failure, the UAV is controlled to hover, and recovery actions are performed according to the corresponding environmental type—for example, for backlight, the heading is adjusted to change the UAV's orientation relative to the light source, and for dense fog or rain, the flight altitude is reduced—and the verification is repeated after the recovery action is performed; if the common-cause failure continues for more than a preset time limit, the result that the target is currently unobservable is output. This is because environmental factors such as dense fog, rain, and backlight can simultaneously weaken the quality of laser echo and image, causing both channels to fail at the same time. Relying solely on either channel cannot reliably determine the problem. By monitoring the quality of both channels and hovering, recovering according to the environmental type, or reporting after a timeout when both fail due to the same cause, misjudgment or blind flight can be avoided in such situations.
[0145] In other embodiments, the flight altitude of the UAV is differentiated according to its stage: in the occlusion verification stage, it flies at the altitude required to pass over the ground attachments to ensure line of sight; in the monitoring stage after line of sight is achieved, the flight altitude is reduced until the ground sampling distance is no greater than a preset upper limit, thereby balancing the line of sight requirements of the verification stage and the resolution requirements of the monitoring stage.
[0146] In S5, in response to the occlusion verification result indicating that the line of sight is obstructed, the UAV is repositioned according to the occlusion verification result. After the position repositioning is completed, it returns to S4 to perform occlusion verification again, until the occlusion verification result indicates that the line of sight is unobstructed. Here, position repositioning refers to the adjustment of the UAV's observation position (including horizontal position, flight altitude, gimbal angle and focal length, etc.) according to the occlusion verification result. After each position repositioning is completed, it returns to S4 to perform occlusion verification again, and so on, thus forming a closed loop of "on-site occlusion verification → position repositioning → re-verification".
[0147] Specifically, the system simultaneously collects on-site data using a laser rangefinder and a camera. Laser ranging back-calculation serves as the primary channel for occlusion detection, while visual big data model semantic recognition acts as an auxiliary channel. The results from both are combined, and based on the type and severity of occlusion, a relocation strategy is selected, or the system retreats to a backup observation point. The UAV is controlled to adjust its position, altitude, gimbal angle, and focal length, or replans its flight path to the backup observation point. After adjustment, the system is verified again through both laser ranging back-calculation and visual big data model semantic recognition, forming a closed loop. When the overall deviation obtained from laser ranging back-calculation is no greater than the occlusion detection threshold, and visual big data model semantic recognition determines there is no occlusion, it is considered a line-of-sight, a clear observation perspective is obtained, and the closed loop ends. When the number of relocation attempts to the current observation point reaches a preset limit (default 3 times), all backup observation points have failed to be accessed, or the UAV has exceeded airspace or safety restrictions, the closed loop terminates and reports to the command system that the target is currently unobservable and the reason why. To ensure flight safety, there are upper limits on the displacement, altitude change, and gimbal angle change for each relocation. The above-mentioned on-site occlusion verification and closed-loop adjustment process is as follows: Figure 5 As shown.
[0148] Following scenario one, after confirming that the target is partially obscured by a temporary billboard and receiving an adjustment suggestion to move eastward from the large visual model, the UAV performs eastward repositioning and returns to verify the obstruction after repositioning. After repositioning, the overall deviation calculated by laser ranging is reduced to no more than the obstruction judgment threshold, confirming unobstructed line of sight and thus obtaining a clear observation angle of the target. In scenario two, after the UAV flies to the target observation point above the southeastern hillside, laser ranging confirms no obstruction of line of sight, allowing for clear imaging of the accident scene without requiring repositioning.
[0149] In some embodiments, location relocation selects a relocation strategy based on the type of occlusion. For occlusions with clear edges, such as buildings, horizontal detour is performed: based on the edge direction of the occluder obtained from the semantic recognition of the visual large model, the system detours horizontally to a new observation position with a preset offset distance doff (default 10 to 30 meters). After detour, the visibility of the new position is verified by laser ranging, and the gimbal orientation and camera focal length are updated simultaneously. For occlusions such as trees and low obstacles, the flight altitude is increased and the gimbal pitch angle is adjusted in conjunction with the occlusion: the UAV's flight altitude is increased first, and the altitude restrictions of air traffic control are checked and complied with during the increase. At the same time, the gimbal pitch angle is adjusted to compensate for the change in viewing angle caused by the change in altitude, and the focal length is recalculated to ensure clear imaging of the target area. If the above local adjustments fail to achieve visibility after multiple attempts (default 3 times), the system retreats to the alternative observation point: feedback is sent to the point selection stage, and the system replans the flight path to try the alternative observation point with the second highest comprehensive score in turn. When all alternative observation points fail to be tried or exceed the airspace restrictions, the system reports to the command system that the target is currently unobservable and the reason. Furthermore, the step size for location relocation increases with the degree of line-of-sight obstruction, resulting in larger displacements or altitude adjustments per relocation when the obstruction is more severe, thus enabling faster escape from the obstructed area. Continuing from scenario one, based on the eastward movement suggestion given by the visual model for the temporary billboard, the UAV performs location relocation along the eastward direction using a horizontal detour strategy. After detouring, line-of-sight is verified by laser ranging. Thus, location relocation is driven by the semantic level of occlusion type and adaptively adjusts the step size, improving the efficiency of occlusion removal.
[0150] In other embodiments, when the UAV's flight altitude or horizontal position is changed, the camera focal length is adjusted in conjunction with the repositioning to maintain a constant pixel proportion of the target in the image. Specifically, when the repositioning causes a change in the observation distance from the UAV to the target, the focal length is recalculated and adjusted synchronously according to the correspondence between the camera focal length, ground sampling distance, and observation distance: the focal length is increased when the observation distance increases and decreased when the observation distance decreases, so that the pixel size occupied by the target in the acquired image remains essentially unchanged. In this way, while adjusting the observation position to eliminate occlusion, the stability of the target imaging scale is maintained, facilitating continuous target identification and tracking monitoring.
[0151] In some embodiments, after location relocation, the comprehensive score and digital surface model data are updated based on the occlusion verification results. Specifically, in the digital surface model data, temporary obstacles located within the target's surrounding area but not recorded in the digital surface model data are marked with obstacle confidence. That is, temporary obstacles confirmed by on-site occlusion verification that were not originally reflected in the digital surface model data are added to the digital surface model data in the form of obstacle confidence labels. Based on the marked digital surface model data, the comprehensive score of the candidate observation points is recalculated, and the ranking of the candidate observation points is updated accordingly, so that the temporary obstacle is avoided when determining the observation point in the future. In this way, the temporary obstacles measured on-site are fed back to the point selection stage, forming a closed-loop optimization from the on-site to the point selection stage. Compared with a fixed obstacle avoidance method without feedback, this can reduce repeated attempts to avoid the same temporary obstacle.
[0152] Based on the same inventive concept, this application also provides a drone line-of-sight analysis and obstacle avoidance device to implement the above-described drone line-of-sight analysis and obstacle avoidance method. (Refer to...) Figure 7 The device includes a data loading module, a viewpoint optimization module, a flight path scheduling module, an occlusion verification module, and a repositioning module.
[0153] The data loading module is configured to receive the target's location information and load the digital elevation model data and digital surface model data of the area surrounding the target, corresponding to S1 in the above method.
[0154] The viewpoint optimization module is configured to perform line-of-sight analysis on multiple candidate observation points around the target based on digital elevation model data and digital surface model data to predict whether the line of sight is obstructed, and determine the target observation point and retain alternative observation points according to the comprehensive score, corresponding to S2 in the above method; the observation parameters of the target observation point, including flight altitude, gimbal pitch angle and camera focal length, can be calculated by the viewpoint optimization module or incorporated into the flight path scheduling module for calculation.
[0155] The flight path scheduling module is configured to plan the flight path from the UAV's current location to the target observation point and control the UAV to fly to the target observation point, corresponding to S3 in the above method.
[0156] The occlusion verification module is configured to perform occlusion verification on the line of sight from the UAV to the target after the UAV arrives at the target observation point through laser ranging back calculation and visual large model semantic recognition, which corresponds to S4 in the above method.
[0157] The relocation module is configured to relocate the UAV when there is an obstruction in the line of sight and trigger the obstruction verification module to re-verify until the line of sight is clear, corresponding to S5 in the above method.
[0158] The modules described above are interconnected via data flow: the digital elevation model (DEM) and digital surface model (DSM) data loaded by the data loading module serve as input to the viewpoint optimization module; the target observation point and reserved alternative observation points determined by the viewpoint optimization module are output to the flight path scheduling module and the occlusion verification module; the occlusion verification result output by the occlusion verification module drives the relocation module to perform position relocation, and the relocation module then triggers the occlusion verification module for re-verification, thereby forming on-site verification and closed-loop adjustment at the device level, corresponding to the above method. This device, through the coordinated operation of these five modules, achieves pre-deployment point selection, on-site dual-channel occlusion verification, and closed-loop relocation after occlusion confirmation, corresponding to the above method and achieving the same beneficial effects.
[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0160] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for line-of-sight analysis and obstacle avoidance for unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1. Receive the target's location information and load the digital elevation model data and digital surface model data of the area surrounding the target; S2. Based on the digital elevation model data and the digital surface model data, perform line-of-sight analysis on multiple candidate observation points around the target to predict whether the line of sight from the candidate observation points to the target is blocked, and determine the target observation point and retain alternative observation points from the multiple candidate observation points according to the comprehensive score; S3. Plan the flight path from the current position of the UAV to the target observation point, and control the UAV to fly along the flight path to the target observation point; S4. After the UAV arrives at the target observation point, the line of sight from the UAV to the target is verified by laser ranging inverse calculation and visual large model semantic recognition to obtain the occlusion verification result; the laser ranging inverse calculation calculates the inverse position based on the laser ranging result, and compares the inverse position with the position information of the target to verify whether the line of sight is blocked; the visual large model semantic recognition identifies the occlusion state of the line of sight based on the on-site image collected by the UAV. S5. In response to the occlusion verification result indicating that the line of sight is obstructed, perform position relocation of the UAV according to the occlusion verification result, and return to perform the occlusion verification after the position relocation is completed, until the occlusion verification result indicates that the line of sight is unobstructed.
2. The UAV line-of-sight analysis and obstacle avoidance method according to claim 1, characterized in that, The laser ranging inverse calculation includes the following sub-steps: S41. The UAV emits a laser towards the target using laser ranging and receives the reflected echo. Based on the reflected echo, the UAV obtains the measured distance from the target to the target, as well as the horizontal azimuth and pitch angles of the emitted laser. S42. Transform the position of the UAV from the geodetic coordinate system to the station-centered coordinate system via the geocentric coordinate system. Calculate the position of the target relative to the UAV in the station-centered coordinate system by combining the measured distance, the horizontal azimuth angle, and the pitch angle. Then, transform the station-centered coordinate system back to the geodetic coordinate system to obtain the inverse calculated position. S43. Determine an occlusion judgment threshold based on the UAV's positioning solution status, the measured distance, and the intensity of the reflected echo. The occlusion judgment threshold increases with the increase of the measured distance, and the occlusion judgment threshold when the positioning solution status is a floating solution is greater than the occlusion judgment threshold when the positioning solution status is a fixed solution. S44. Calculate the comprehensive deviation between the inversely calculated position and the position information of the target. The comprehensive deviation is obtained by weighting the horizontal distance deviation and the height deviation between the inversely calculated position and the position information of the target. Compare the comprehensive deviation with the occlusion judgment threshold. If the comprehensive deviation is greater than the occlusion judgment threshold, determine that the line of sight is occluded.
3. The UAV line-of-sight analysis and obstacle avoidance method according to claim 2, characterized in that, The semantic recognition of the large visual model includes the following sub-steps: S45. In response to the laser ranging back calculation determining that the line of sight is obstructed, or the comprehensive deviation falls into a preset suspicious deviation range, trigger the visual big model semantic recognition; otherwise, skip the visual big model semantic recognition. S46. Based on the inversely calculated position obtained from the laser ranging, extract the region of interest containing the inversely calculated position from the field image collected by the UAV; S47. Input the region of interest into the large visual model to identify structured occlusion information, which includes occlusion state, occlusion type, occlusion location, adjustment suggestions, and confidence level.
4. The UAV line-of-sight analysis and obstacle avoidance method according to claim 3, characterized in that, S4 further includes performing conflict arbitration on the results of the laser ranging back calculation and the semantic recognition of the visual large model: in response to the laser ranging back calculation receiving the reflected echo, the occlusion determination result of the laser ranging back calculation is used as the occlusion verification result, and the direction of the position relocation is corrected with the adjustment suggestions in the structured occlusion information; in response to the laser ranging back calculation not receiving the reflected echo, the occlusion verification result is determined with the occlusion state in the structured occlusion information and the confidence level.
5. The UAV line-of-sight analysis and obstacle avoidance method according to claim 2, characterized in that, In step S4, in response to the overall deviation being greater than the occlusion judgment threshold, deviation attribution identification is performed before performing the location relocation: the laser ranging back-calculation is performed on the landmark reference points with known coordinates around the target to obtain the reference back-calculation deviation of the landmark reference points; in response to the reference back-calculation deviation being consistent with the overall deviation in direction and magnitude, it is determined that there is a systematic deviation between the digital elevation model data and the digital surface model data in the area surrounding the target, the coordinate correction amount of the area surrounding the target is calculated based on the reference back-calculation deviation, the coordinate registration correction is performed on the digital elevation model data and the digital surface model data using the coordinate correction amount, and step S2 is re-executed based on the corrected digital elevation model data and the digital surface model data; in response to the inconsistency between the reference back-calculation deviation and the overall deviation, it is determined to be local occlusion and the location relocation of the UAV is performed.
6. The UAV line-of-sight analysis and obstacle avoidance method according to claim 3, characterized in that, In step S4, common-cause failure detection is performed on the laser ranging back-calculation and the visual large model semantic recognition: the signal-to-noise ratio and broadening of the reflected echo are obtained as the channel quality of the laser ranging back-calculation, and the sharpness and confidence of the scene image are obtained as the channel quality of the visual large model semantic recognition; in response to the fact that the channel quality of the laser ranging back-calculation and the channel quality of the visual large model semantic recognition are both lower than their respective preset quality lower limits, a common-cause failure is determined; in response to the determination of a common-cause failure, the UAV is controlled to hover at the current position, and a recovery action is performed according to the environment type corresponding to the common-cause failure. The recovery action includes at least one of adjusting the UAV's heading to change the orientation relative to the light source and reducing the UAV's flight altitude. After the recovery action is completed, the occlusion verification is re-executed; in response to the fact that the duration of the common-cause failure exceeds a preset time limit, the result that the target is currently unobservable is output.
7. The UAV line-of-sight analysis and obstacle avoidance method according to claim 1, characterized in that, Following step S5, an update operation is performed on the comprehensive score and the digital surface model data based on the occlusion verification result: in the digital surface model data, temporary obstacles located in the area surrounding the target and not recorded by the digital surface model data are marked with obstacle confidence; based on the marked digital surface model data, the comprehensive score of the candidate observation points is recalculated, and the ranking of the candidate observation points is updated accordingly, so that the temporary obstacles are avoided when determining observation points in the future.
8. The UAV line-of-sight analysis and obstacle avoidance method according to claim 1, characterized in that, The line-of-sight analysis and the determination of candidate observation points in step S2 include the following sub-steps: S21. Based on the digital elevation model data, perform the line-of-sight analysis on the initial observation points in multiple directions around the target, eliminate directions blocked by terrain, and obtain candidate directions; S22. Based on the digital surface model data, perform the line-of-sight analysis on the candidate observation points at different distances in the candidate orientation to obtain the line-of-sight of the candidate observation points; The line-of-sight analysis includes sampling the elevation along the line connecting the candidate observation point to the target, comparing the line-of-sight height of each sampling point on the line with the height of the surface attachments represented by the digital surface model data after adding a preset safety margin, and determining that the line of sight is blocked at the sampling point if the sum of the height of the surface attachments and the safety margin exceeds the line-of-sight height.
9. A UAV line-of-sight analysis and obstacle avoidance device, characterized in that, include: The data loading module is configured to receive the location information of the target and load the digital elevation model data and digital surface model data of the area surrounding the target. The viewpoint optimization module is configured to perform line-of-sight analysis on multiple candidate observation points around the target based on the digital elevation model data and the digital surface model data to predict whether the line of sight from the candidate observation points to the target is blocked, and determine the target observation point and retain alternative observation points from the multiple candidate observation points according to the comprehensive score. The flight path scheduling module is configured to plan a flight path from the current location of the UAV to the target observation point, and control the UAV to fly along the flight path to the target observation point; The occlusion verification module is configured to perform occlusion verification on the line of sight from the UAV to the target after the UAV arrives at the target observation point by combining laser ranging back calculation and visual large model semantic recognition, and obtain the occlusion verification result. The relocation module is configured to respond to the occlusion verification result indicating that the line of sight is obstructed, perform position relocation of the UAV according to the occlusion verification result, and trigger the occlusion verification module to re-execute the occlusion verification after the position relocation is completed, until the occlusion verification result indicates that the line of sight is unobstructed.
10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the UAV line-of-sight analysis and obstacle avoidance method as described in any one of claims 1 to 8.
