An unmanned aerial vehicle autonomous path dynamic programming method and system based on visual obstacle avoidance

Abstract

The application relates to the technical field of intelligent unmanned aerial vehicle manufacturing and flight control, and discloses an unmanned aerial vehicle autonomous path dynamic planning method and system based on visual obstacle avoidance. A depth vision image sequence with a time tag, a measured flight state sequence and control parameters are acquired, a control effective time delay is determined according to a full-link time tag, and a planning time sequence is established; an obstacle occupation slice and a state reachable slice with the same index are generated, and an execution safety slice is formed by combining a UAV body envelope and a UAV body swept envelope; a path segment with an entrance state receiving set, an exit state transmission set and an effective time period is generated according to the execution safety slice, and the path segment is connected into a target path chain; after the flight controller verifies the path version, the measured flight state and the effective time period, the three-dimensional space coordinates, the speed and the heading of the unmanned aerial vehicle are controlled; when the avoidance response state set is empty for the first time, an irreversible conflict time layer is determined, and the target path chain is updated from the safe handover path segment.

Description

Technical Field

[0001] This invention relates to the fields of intelligent unmanned aerial vehicle manufacturing and autonomous flight control technology, and in particular to a method and system for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance. Background Technology

[0002] With the development of intelligent unmanned aerial vehicle (UAV) manufacturing technology, UAVs are increasingly being applied to scenarios such as indoor inspection, warehouse inventory, facility testing, and confined space operations. In these applications, UAVs typically use onboard vision acquisition devices to obtain information about the surrounding environment, determine the position of obstacles in three-dimensional space, generate obstacle avoidance paths based on the UAV's current three-dimensional spatial coordinates and the target's three-dimensional spatial coordinates, and have the flight controller control the UAV's three-dimensional spatial coordinates, speed, and heading according to these paths.

[0003] Existing visual obstacle avoidance trajectory planning technologies typically begin by acquiring environmental images using depth cameras or binocular cameras. Based on these images, a 3D point cloud is generated. The 3D point cloud is then filtered and downsampled, and a grid map, octree map, or local exploration map is constructed. Subsequently, obstacle-occupied areas and passable areas are identified in the constructed map. Candidate paths extending from the UAV's current location to the target location are generated through search or iteration. These candidate paths are then smoothed and sent to the flight controller.

[0004] For example, Chinese invention patent CN112747736B discloses a vision-based indoor drone path planning method. The method uses the drone camera to acquire environmental information and construct a point cloud map. After filtering and downsampling the point cloud, an octree map and an exploration map are constructed. Real-time path planning is performed based on the drone's position, the target position, and the obstacle occupancy status in the map. The planned path is then smoothed so that the drone can fly along the corresponding path.

[0005] The aforementioned existing technologies can generate obstacle avoidance paths for UAVs in three-dimensional space based on visual acquisition results. However, existing planning processes typically focus on determining the spatial accessibility of the path based on the environmental occupancy state at the time of image acquisition, and a time lag still exists between the path planning process and the subsequent flight execution process. Visual images need to undergo exposure, transmission, environmental modeling, path calculation, and control command conversion before flight control commands can be formed. These flight control commands also need to be transmitted to the flight execution components via the flight controller to generate corresponding motion responses. During this period, the UAV continues to move according to the prior control commands, and the three-dimensional spatial coordinates and motion states of dynamic obstacles may also change, causing inconsistencies between the UAV state and obstacle state used during path planning and the state at the time the control commands actually take effect.

[0006] Meanwhile, existing path planning methods typically use the geometric connectivity between waypoints or grid nodes as the basis for path selection. While collision risk can be reduced through path smoothing or setting safe distances for obstacles, there is a lack of unified constraints on the speed, heading, turning ability, climb ability, descent ability, and braking ability of the UAV before and after the control commands take effect, corresponding to the path planning process. As a result, situations may arise where the planned path does not intersect with the area occupied by obstacles in three-dimensional space, but the UAV is limited by its current speed, current heading, and flight performance boundaries, preventing it from reaching the planned spatial position at the corresponding moment, or resulting in discontinuities in three-dimensional spatial coordinates, speed, or heading when switching to an updated path.

[0007] Furthermore, when a dynamic obstacle approaches the current flight path, existing technologies typically trigger replanning based on the distance between the obstacle and the flight path, the predicted collision probability, or whether the current path intersects with the area occupied by the obstacle. Such triggering methods often struggle to combine the duration of the current control commands, the response process of the flight execution components, and the drone's remaining speed, steering, and climb / fall adjustment capabilities to determine whether the current conflict can still be resolved within the flight performance boundaries.

[0008] When the replanning trigger time is delayed, the UAV may not be able to avoid obstacles by adjusting its three-dimensional spatial coordinates, speed, and heading within the remaining response time. When the replanning directly replaces the current execution path with a new path, the initial state of the new path may not match the actual flight state of the UAV, thus affecting the continuity of the UAV's three-dimensional spatial motion control.

[0009] Therefore, the main technical problem that needs to be solved in the existing visual obstacle avoidance trajectory planning technology is: how to match the generated obstacle avoidance trajectory with the flight state that the UAV can achieve at the corresponding execution time when there is a control delay in visual image acquisition, trajectory planning, control command transmission and flight execution response, and the obstacle state and the measured flight state of the UAV are dynamically changing, so as to achieve continuous control of the UAV's three-dimensional spatial coordinates, speed and heading. Summary of the Invention

[0010] This invention aims to solve the problems in existing visual obstacle avoidance trajectory planning, where the time difference between depth visual image acquisition, trajectory planning processing, control command transmission and flight execution response leads to a mismatch between the obstacle avoidance trajectory and the flight state that the UAV can actually achieve at the time of execution, and the problem that the target trajectory chain cannot be continuously connected after the dynamic obstacle state changes. The invention provides a method and system for dynamic planning of autonomous UAV trajectory based on visual obstacle avoidance.

[0011] This invention provides a dynamic planning method for autonomous flight paths of unmanned aerial vehicles (UAVs) based on visual obstacle avoidance. This method is applied to UAVs used in the manufacturing of intelligent unmanned aerial vehicles (UAVs). The UAV includes an AI chip, a flight controller, and flight execution components. The dynamic planning method for autonomous flight paths of UAVs based on visual obstacle avoidance includes:

[0012] Acquire a sequence of depth visual images with exposure timestamps, a sequence of measured flight states with sampling timestamps, the target's three-dimensional spatial coordinates, flight performance boundaries, the UAV's airframe envelope, the current control commands, and the current executed trajectory version. Determine the control activation delay based on the timestamps corresponding to image exposure, AI chip processing, flight controller reception, and flight execution component responses, and establish a planned time sequence based on the control activation delay.

[0013] The three-dimensional spatial coordinates and motion state of obstacles are determined based on the depth visual image sequence with exposure time stamps, and obstacle occupancy slices with the same index as the planned time sequence are generated; the UAV state is recursively deduced based on the measured flight state sequence with sampling time stamps, current control commands, control effective delay and flight performance boundary, and state reachable slices with the same index as the planned time sequence are generated.

[0014] Based on the expansion of the obstacle occupancy slice by the drone's body envelope, states located within the expanded obstacle occupancy slice are excluded from the state reachable slices with the same index as the obstacle occupancy slice, as well as the states where the drone's body sweep envelope intersects with the expanded obstacle occupancy slice, to form an execution safety slice.

[0015] Based on the continuous execution of security slices, a track segment with an inbound state acceptance set, an outbound state transfer set, and a valid time period is generated. The track segments that intersect with the inbound state acceptance set of the next track segment are connected to form the target track chain.

[0016] A new track version is generated based on the current track version. The current track version is recorded as the parent track version of the new track version. The target track chain, the new track version, the parent track version, the entry status acceptance set, and the valid time period are sent to the flight controller. When the parent track version is consistent with the current track version, the current measured flight status is within the entry status acceptance set, and the current time is within the valid time period, the flight controller controls the flight execution components to adjust the UAV's three-dimensional spatial coordinates, speed, and heading.

[0017] An updated obstacle occupancy slice is generated based on the updated depth vision image sequence. A state reachable slice is regenerated based on the updated measured flight state sequence. The planning time of the first reachable state of the entry state acceptance set that does not have collisions and can be continued to the subsequent track segment is determined as the irreversible conflict time layer. A safe handover track segment before the irreversible conflict time layer is selected, and the target track chain is updated from the safe handover track segment.

[0018] The present invention also provides a visual obstacle avoidance-based autonomous trajectory dynamic planning system for UAVs, which is applied to UAVs in the manufacturing of intelligent unmanned aerial vehicles. The visual obstacle avoidance-based autonomous trajectory dynamic planning system for UAVs includes a depth vision acquisition component, a flight status acquisition component, a mission parameter storage component, an AI chip, a flight controller, and a flight execution component.

[0019] The depth vision acquisition component outputs a sequence of depth vision images with exposure timestamps; the flight status acquisition component outputs a sequence of measured flight status images with sampling timestamps; the mission parameter storage component stores the target's 3D spatial coordinates, mission airspace boundaries, flight performance boundaries, and the UAV's airframe envelope; the AI ​​chip determines the control activation delay, establishes a planning time sequence, generates obstacle occupancy slices, state reachability slices, execution safety slices, track fragments, target track chains, and new track versions, determines irreversible conflict time layers, and updates the target track chain from the safe handover of track fragments; the flight controller verifies the parent track version, the currently executed track version, the entry state acceptance set, and the effective time period, and controls the flight execution component to adjust the UAV's 3D spatial coordinates, speed, and heading based on the verified target track chain.

[0020] Compared with existing technologies, the above technical solution has the following advantages:

[0021] 1. This invention determines the control activation delay based on visual image exposure, AI chip processing, control command transmission, and flight execution response, and establishes a planned time sequence based on the control activation delay, so that the obstacle state, the actual flight state of the UAV, and the actual activation time of the control command can be matched under a unified time reference, reducing the obstacle avoidance trajectory deviation caused by the inconsistency between the visual acquisition time and the control execution time.

[0022] 2. This invention generates obstacle occupancy slices and state reachability slices under the same planning time index, and combines them with flight performance boundaries, UAV body envelopes and UAV body sweep envelopes to form execution safety slices. This ensures that the generated obstacle avoidance trajectory not only meets the geometric passability conditions in three-dimensional space, but also meets the constraints of speed, heading, turning, climbing and braking capabilities of the UAV at the corresponding execution time, thereby improving the actual executability of the trajectory.

[0023] 3. This invention configures an ingress state acceptance set, an exit state transfer set, and an effective time period for a track segment, and connects only track segments whose exit state transfer set can be connected to the subsequent ingress state acceptance set as target track chains. This enables the UAV to maintain continuous changes in three-dimensional spatial coordinates, speed, and heading during track switching, reducing track tracking errors and sudden changes in flight attitude caused by mismatch in the initial states of new and old tracks. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the architecture of the UAV autonomous trajectory dynamic planning system based on visual obstacle avoidance according to the present invention.

[0026] Figure 2 This is a flowchart illustrating the dynamic trajectory planning method for UAV autonomous flight path based on visual obstacle avoidance according to the present invention.

[0027] Figure 3 This is a schematic diagram illustrating the process of obstacle occupancy slice, state reachable slice, and execution of safety slice in this invention.

[0028] Figure 4 This is a schematic diagram illustrating the process of incremental update and irreversible conflict handling of the target track chain in this invention.

[0029] Figure 5 This is a schematic diagram of the entire timeline of visual image acquisition, trajectory planning, control command transmission, and flight execution response in this invention.

[0030] Figure 6 This is a schematic diagram illustrating the timing relationship between the flight control commands and the actual flight response of the UAV in this invention.

[0031] Figure 7 This is a schematic diagram illustrating the recursive state reachability slice of the UAV in three-dimensional space according to the present invention.

[0032] Figure 8 This is a schematic diagram illustrating the detection between the sweep envelope of the UAV body and the obstacle-occupied slice of the present invention.

[0033] Figure 9 This is a schematic diagram illustrating the formation of the three-dimensional connected partition, state transition section, and candidate safe corridor of the present invention.

[0034] Figure 10 This is a schematic diagram of the depth visual mapping results in a complex and confined spatial environment according to the present invention.

[0035] Figure 11 This is a schematic diagram comparing the dynamic trajectory of the present invention with the traditional static planning trajectory in a dynamic obstacle scenario.

[0036] Figure 12 This is a schematic diagram illustrating the continuous changes in three-dimensional spatial coordinates and heading during the high-speed avoidance of dynamic obstacles by the UAV of the present invention. Detailed Implementation

[0038] The following is combined Figures 1 to 12This paper provides a detailed description of a method and system for autonomous trajectory dynamic planning of unmanned aerial vehicles (UAVs) based on visual obstacle avoidance. The "autonomous trajectory dynamic planning" in this embodiment does not simply search for a geometric curve in a visual image. Instead, it uses the obstacle state represented by a sequence of depth visual images as environmental input, a sequence of measured flight states with sampling timescales as UAV state input, and the target's three-dimensional spatial coordinates, mission airspace boundaries, and flight performance boundaries as constraint inputs. It calculates the flight state that the UAV can actually reach and safely continue to at each future execution time. The flight controller then converts the flight state into thrust control, roll control, pitch control, and yaw control variables. Therefore, visual processing is used to determine the spatial state of obstacles around the controlled object, trajectory dynamic planning is used to form a control reference, and the final object of action is the UAV's three-dimensional spatial coordinates, velocity, and heading. The term "dynamic" includes at least four aspects: the planning time shifts with the control activation delay, the obstacle motion state is updated with visual observation, the UAV's reachable range is updated with measured flight states, and the target trajectory chain is locally updated with irreversible conflict determination.

[0039] Example 1

[0040] like Figure 1 As shown, the visual obstacle avoidance-based autonomous trajectory dynamic planning system for UAVs in this embodiment is applied to multi-rotor UAVs in the manufacturing of intelligent unmanned aerial vehicles. The system includes a depth vision acquisition component, a flight status acquisition component, a mission parameter storage component, an AI chip, a flight controller, a flight execution component, and a bidirectional planning control interface. The depth vision acquisition component is installed at the front of the UAV fuselage. The flight status acquisition component is connected to the flight controller, the mission parameter storage component is connected to the AI ​​chip, the AI ​​chip exchanges data bidirectionally with the flight controller through the bidirectional planning control interface, and the flight controller is connected to the flight execution component. This forms a closed loop from environmental acquisition, state estimation, trajectory calculation, version verification, physical execution to state feedback, enabling continuous adjustment of the UAV's three-dimensional spatial coordinates, speed, and heading within the same control link.

[0041] The term "closed loop" refers to the AI ​​chip not ending processing after generating the target track chain, but continuing to receive track version acceptance results, planned switchover times, currently executed track versions, and measured flight status after the flight execution components respond, all returned by the flight controller. These feedback results are then used as input for the next planning cycle. The forward data of the bidirectional planning control interface includes the target track chain, new track version, parent track version, entry state acceptance set, effective time period, and generation state fingerprint. The reverse data includes the current control command, currently executed track version, acceptance receipt, planned switchover time, and execution feedback timestamp. This closed loop ensures that the planning and control sides use the same track version and the same state time base, preventing inconsistencies in the state base when the AI ​​chip has generated a new track while the flight controller is still executing the old track.

[0042] The depth vision acquisition component includes a depth camera and an exposure time-stamp generator. The depth camera can be a binocular depth camera, a time-of-flight depth camera, a structured light depth camera, or a combination of a vision camera and a laser rangefinder. This embodiment uses a global shutter binocular depth camera with an image resolution of 1280×720, an acquisition frequency of 60 frames per second, and a depth measurement range of 0.3m to 40m. The exposure time-stamp generator assigns a unique image frame identifier, exposure start time-stamp, and exposure end time-stamp to each depth vision image frame. The image frame identifier is transmitted to the AI ​​chip along with the corresponding image data and is subsequently written into the track data packet generated by the corresponding image frame, thus associating image acquisition, AI chip processing, flight controller reception, and flight execution component response to the same control process.

[0043] The "depth visual image sequence" referred to in this embodiment is a set of multiple depth visual images arranged in the order of exposure, along with a corresponding image frame identifier, exposure start time marker, exposure end time marker, and depth validity marker for each depth visual image. The image frame identifier is used to address the issue of out-of-order image frames during multi-threaded processing and interface transmission; the exposure start time marker is used to determine the moment when the camera begins observing the environment; the exposure end time marker is used to determine the observation reference moment when the depth information of the corresponding image frame is complete; and the depth validity marker is used to indicate the range of pixels in the corresponding image frame that meet the depth confidence requirement. The "exposure time marker" is preferably written by a hardware interrupt triggered by the camera's trigger signal, rather than being added by software after the image data arrives at the AI ​​chip, to avoid the image transmission queuing time being incorrectly included in the environmental observation time.

[0044] The flight status acquisition component includes a sampling timescale generator, which outputs a sequence of measured flight statuses with sampling timescales based on the inertial measurement unit, visual odometry, barometric altimeter, and optional satellite positioning components. Each measured flight status includes at least the UAV's three-dimensional spatial coordinates in the three-dimensional navigation coordinate system, its velocity along the three coordinate axes, roll angle, pitch angle, and yaw angle. In indoor environments where satellite positioning signals are unavailable, the three-dimensional spatial coordinates are obtained by fusing the visual odometry and inertial measurement unit; in outdoor environments, satellite positioning results can be further fused. The exposure timescale generator and the sampling timescale generator preferably use the same hardware clock. When the exposure timescale generator and the sampling timescale generator use different hardware clocks, the flight controller periodically outputs synchronization pulses. The AI ​​chip determines the clock offset based on the recorded timescales of the synchronization pulses in each component and uses the clock offset to correct the exposure timescale, sampling timescale, processing timescale, command reception timescale, and execution feedback timescale.

[0045] The so-called "measured flight state sequence" is a set of state records continuously output by the flight state acquisition component according to the sampling time scale, and is not a nominal state calculated from the target trajectory. When the AI ​​chip does not completely coincide with the flight state sampling time scale at a certain exposure time scale, it interpolates from two adjacent measured flight state records before and after the exposure time scale to obtain the UAV's three-dimensional spatial coordinates, velocity, and attitude that match the corresponding exposure time scale. Linear interpolation is used for the three-dimensional spatial coordinates and velocity, while attitude interpolation that avoids crossing positive and negative boundaries is used for roll angle, pitch angle, and heading angle. Therefore, obstacle coordinate transformation, control activation delay derivation, and trajectory acceptance verification are all based on the measured flight state at the same moment, rather than directly splicing together temporally adjacent but non-corresponding state records.

[0046] The mission parameter storage component stores the target's three-dimensional spatial coordinates, mission airspace boundaries, flight performance boundaries, and the UAV's airframe envelope. The target's three-dimensional spatial coordinates can be written by the mission terminal before takeoff or updated by the host computer during flight. The mission airspace boundaries are represented by a three-dimensional polyhedron, a set of voxels, or a bounded space formed by multiple constrained planes. The flight performance boundaries include at least the maximum horizontal flight speed, maximum climb speed, maximum descent speed, maximum lateral acceleration, maximum braking deceleration, maximum directional angular velocity, maximum roll angle, and maximum pitch angle. The UAV airframe envelope covers the frame, rotors, onboard equipment, and the safety margin required during attitude changes; it can be represented by an ellipsoid, a directional bounding box, or a convex polyhedron. This embodiment uses ellipsoids with half-axis of 0.35m, 0.35m, and 0.18m, and the envelope margin is increased based on rotor downwash and braking distance when load changes exceed a preset load threshold.

[0047] In some implementations, when both the mission terminal and the host computer are able to update the target's 3D spatial coordinates or the mission airspace boundary, the mission parameter storage component configures a parameter version and an effective time stamp for each set of mission parameters, and determines the mission parameter that passes the integrity check and has the latest effective time stamp as the current mission parameter. When the AI ​​chip generates the target track chain, it writes the parameter version corresponding to the current mission parameter into the track data packet to avoid the flight controller still executing the target track chain generated based on the old mission parameters when the mission target has been updated.

[0048] The AI ​​chip uses an embedded processor with parallel matrix operation capabilities and includes a processing time stamp generator, a control delay determination unit, an obstacle state determination unit, a planning slice generation unit, a track segment generation unit, a track version generation unit, an irreversible conflict determination unit, and a track chain update unit.

[0049] The processing time stamp generator records the image reception time stamp of the image frame received by the AI ​​chip and the processing completion time stamp of the corresponding track data packet processing. The control delay determination unit determines the control activation delay based on the exposure time stamp, image reception time stamp, processing completion time stamp, flight controller command reception time stamp, and flight execution component execution feedback time stamp. The obstacle state determination unit generates obstacle states with obstacle identifiers, measured 3D spatial coordinates, motion states, spatial envelopes, and motion branch versions. The planning slice generation unit generates obstacle occupancy slices, state reachable slices, and execution safety slices according to the same planning time index. The track fragment generation unit generates track fragments with ingress state acceptance sets, exit state transfer sets, effective time periods, and fragment versions. The track version generation unit establishes the version relationship between the new track version and the parent track version. The irreversible conflict determination unit determines the planning time when the avoidance response state set is empty for the first time. The track chain update unit determines the safe handover track fragments and implements the connection of the preserved track prefix, repaired track fragments, and candidate preserved track suffixes.

[0050] The aforementioned units are functional units categorized according to data input, processing, and data output. They can be implemented by different computing cores, software threads, or different functional modules within the same processing program in an AI chip, and do not require each functional unit to correspond to an independent physical circuit. The control delay determination unit outputs the planning time base; the obstacle state determination unit outputs the occupancy status of each obstacle at the future planning time; the planning slice generation unit outputs the reachability and safety status of the UAV at the future planning time; the track segment generation unit converts continuous safety states into a sequence of control commands executable by the flight controller; the track version generation unit resolves the sequential relationship between multiple planning results during transmission and execution; the irreversible conflict determination unit determines whether the remaining control capability is still sufficient to complete the avoidance; and the track chain update unit only replaces the track portion affected by the conflict. The aforementioned data dependencies are... Figure 1 and Figure 2 The data is represented by system data stream and method processing stream, respectively.

[0051] The flight controller includes a current execution version register and a track acceptance verifier. The current execution version register stores the track version currently being executed by the UAV. The track acceptance verifier verifies whether the parent track version in the track data packet is equal to the current execution track version in the current execution version register, whether the current measured flight state matches the generated state fingerprint in the track data packet, whether the current measured flight state is within the entry state acceptance set of the first track segment of the target track chain, whether the current moment is within the valid time period of the corresponding track segment, and whether the currently executing track segment can be connected to the first track segment of the target track chain. The flight controller accepts a new track version only if all the above verifications pass.

[0052] The flight execution components include electronic speed controllers, motors, and rotors. The flight controller converts the target's three-dimensional spatial coordinates, target speed, and target heading into thrust control, roll control, pitch control, and yaw control, which in turn drive the flight execution components to change the UAV's three-dimensional spatial coordinates, speed, and heading.

[0053] The current execution version register stores track versions that have been accepted by the flight controller and entered the execution phase, rather than track versions recently generated by the AI ​​chip that have not yet been accepted. The track acceptance verifier performs a final verification before the planning results enter the physical control loop: parent track version verification prevents expired data packets from overwriting the current track; state fingerprint verification determines whether the 3D spatial units, velocity range, heading range, and current execution track version used when the target track chain was generated still match the current measured flight state; entry state acceptance set verification prevents new tracks from starting from states that the drone cannot currently reach; valid time period verification prevents tracks with the correct spatial shape but that have missed the execution time window from being executed; and track segment continuation verification prevents state breaks between the currently executed track segment and the first track segment of the target track chain.

[0054] The electronic speed controller (ESC) in the flight actuator adjusts the speed of each motor according to the output of the flight controller. The motors and rotors generate total thrust and control torque, which in turn changes the translational and attitude motion of the UAV. Therefore, the target track chain is not an abstract path file that directly drives the motors, but a timing control reference that is converted by the flight controller to actually control the UAV's three-dimensional spatial coordinates, speed, and heading.

[0055] The UAVs and planning parameters used in this embodiment are shown in Table 1.

[0056] Table 1 Performance Boundaries and Planning Parameters for Intelligent Unmanned Aerial Vehicles

[0057] Maximum takeoff weight 2.5 kg Determines inertia, braking distance, and thrust conversion. Maximum horizontal flight speed 15.0 m / s Limiting the range of horizontal velocity states Maximum climb rate 5.0 m / s Recursion with a vertical upward orientation Maximum descent speed 4.0 m / s Recursion with a vertical downward orientation Maximum lateral acceleration 6.0 m / s² Limited lateral avoidance capability Maximum braking deceleration 7.0 m / s² Braking capacity within the limited remaining time Maximum angular velocity 120 ° / s Limited heading state recursion The horizontal half-axis of the drone's airframe envelope 0.35 m Used for obstacle occupancy slice expansion The vertical half-axis of the drone's airframe 0.18 m Used for height-direction collision detection Depth vision image acquisition frequency 60 frames / s Determine the visual observation update cycle Flight status sampling frequency 200 Hz Forming a measured flight state sequence Planning time step 0.05 s Determine the time interval between adjacent planning periods Planning time range 3.0 s Determine the time window for future planning

[0058] The parameters in Table 1 are derived from the flight control configuration, payload status, and mission environment constraints of the multi-rotor UAV in this embodiment. The flight performance boundary is written into the mission parameter storage component and reused in state reachability slice recursion, control command sub-sequence verification, and irreversible conflict determination. When the mission payload or flight mode changes, the mission parameter storage component updates the corresponding parameter version, and the AI ​​chip adopts the updated flight performance boundary from the moment the new parameter version takes effect.

[0059] like Figure 2 As shown, after the UAV begins its mission, the depth vision acquisition component continuously outputs a sequence of depth vision images with exposure timestamps, the flight status acquisition component continuously outputs a sequence of measured flight status images with sampling timestamps, the mission parameter storage component provides the AI ​​chip with the target's 3D spatial coordinates, mission airspace boundaries, flight performance boundaries, and the UAV's airframe envelope, and the flight controller provides the AI ​​chip with the current control commands and the current execution track version. The current control commands include thrust control, roll control, pitch control, and yaw control. Based on the aforementioned data, the AI ​​chip performs functions such as determining control activation delay, predicting obstacle states, generating indexed slices, constructing track segments, connecting target track chains, generating track versions, and incremental updates.

[0060] The "current control command" refers to the control quantity that the flight controller has already output to the flight execution components at the start of the current planning cycle and will continue to have an effect during the control activation delay; it is not a new control quantity that the AI ​​chip will generate soon. The "current execution track version" is the version identifier of the target track chain to which the current control command belongs. By simultaneously sending the current control command and the current execution track version to the AI ​​chip, the AI ​​chip can first calculate the UAV's state when the new control result actually takes effect based on the current control command, then start planning the target track chain from the corresponding state, and make the new track version use the current execution track version as its parent track version.

[0061] The control activation delay is not represented solely by image processing time, but rather begins from the moment the depth vision image reflects the environmental state and ends at the moment the control result formed from the corresponding depth vision image actually causes the UAV's dynamic response. Determined according to the following formula:

[0062]

[0063] In the formula, Indicates the delay before control takes effect, in seconds; This indicates the end of the exposure for the corresponding depth visual image frame, in seconds; This indicates the image reception time stamp of the AI ​​chip when it receives the corresponding image frame, in seconds. This indicates the time stamp in seconds indicating when the AI ​​chip completes processing of the corresponding track data packet. This indicates the command reception time stamp for the flight controller to receive the corresponding flight track data packet, in seconds. This indicates the execution feedback timescale, in seconds, indicating the physical response generated by the flight execution component.

[0064] The execution feedback time stamp can be recorded when the change in motor speed reaches 10% of the expected change in the corresponding control command. When using the change in airframe acceleration to record the execution feedback time stamp, the response threshold is determined based on the upper bound of the acceleration noise of the flight status acquisition component in a stationary state and the minimum identifiable response of the flight execution component. If the image frame identifier, exposure time stamp, processing completion time stamp, command reception time stamp, or execution feedback time stamp are reversed, or if the image frame identifiers in the same control process are inconsistent, the control delay determination unit discards the corresponding delay sample. To reduce the impact of thread scheduling jitter and interface queuing fluctuations, the control delay determination unit selects the median of the eight most recent valid delay samples as the control activation delay used in the current planning cycle.

[0065] Figure 5 The end-to-end timing relationship of visual image exposure, image transmission, AI chip processing, track data packet transmission, and flight execution response is shown. In one set of operation records, the duration from the end of exposure to the execution feedback time marker is 82.0 ms, and the end-to-end duration from the start of exposure to the execution feedback time marker is 98.6 ms. Figure 6 The delay relationship between control commands and actual dynamic responses is illustrated. The AI ​​chip aligns the initial state used for trajectory planning with the physical execution time by shifting the state recursion starting point to the actual effective moment of the control command.

[0066] The breakdown of the control activation delay is shown in Table 2.

[0067] Table 2 Breakdown of Control Effective Delay Measurement

[0068] Exposure and readout time From shutter opening to depth image readout completion 16.6 16.0～17.5 AI chip processing time Image received to target track chain generation completed 45 40.0～55.0 Interface transmission time The AI ​​chip sends data to the flight controller for reception. 2 1.5～3.0 Flight execution response time Flight controller receives dynamic response 35 30.0～45.0 From the end of exposure to the execution of response Control Effective Delay 82 71.5～103.0 From exposure to response execution End-to-end time 98.6 87.5～120.5

[0069] The control delay determination unit marks the end of the exposure of the current depth visual image frame. Based on the control effective delay and planning time step Establish a time series for planning. A planning moment Determined according to the following formula: .

[0070] In the formula, Indicates the first Each planning moment is measured in seconds. This indicates the end of the exposure for the current depth vision image frame, in seconds. Indicates the delay before control takes effect, in seconds; Indicates the planning time step, in seconds; This indicates the planning time sequence number starting from zero.

[0071] Let the planning time domain length be Number of floors planned Determined according to the following formula: .

[0072] In the formula, Indicates the number of planned floors; This represents the planning time domain length, in seconds. In this embodiment, the planning time step is 0.05s, the planning time domain length is 3.0s, and the corresponding planning layer number is 60 layers. The planning time step can be set to an integer multiple of the flight control cycle so that the control command subsequence in the track segment is aligned with the control cycle of the flight controller.

[0073] The term "planned time sequence" refers to a set of discrete moments arranged forward according to the planned time step, starting from the moment when the anticipated new control command begins to have a physical effect. The term "same planned time index" refers to segments with the same sequence number in obstacle occupancy, state reachability, and execution safety slices. The data all describe the same absolute future moment. For example, the obstacle with sequence number 10 occupies the slice, and the state of the slice with sequence number 10 both correspond to the end of the exposure. At any given moment, a collision judgment cannot be made between the drone status with sequence number 10 and the obstacle position with sequence number 9 or sequence number 11. The index relationship of the same planning time is the time basis for handling dynamic obstacles and control activation delay in this embodiment.

[0074] The depth vision acquisition component moves with the drone during the exposure. The planning and slicing generation unit generates a camera pose sweep envelope based on the measured flight state corresponding to the exposure start and end timestamps, and transforms the depth points into a 3D navigation coordinate system. The coordinate transformation relationship of each depth point is as follows: .

[0075] In the formula, Indicates the first The three-dimensional coordinates of a depth point in the three-dimensional navigation coordinate system; Represents the rotation matrix from the camera coordinate system to the 3D navigation coordinate system; Indicates the first The three-dimensional coordinates of a depth point in the camera coordinate system; This represents the three-dimensional spatial coordinates of the camera's optical center in the three-dimensional navigation coordinate system. The rotation matrix and the three-dimensional spatial coordinates of the camera's optical center are obtained from the camera's extrinsic parameters and the measured flight state at the corresponding sampling timescale. For attitude changes during exposure, spherical linear interpolation between the exposure start attitude and the exposure end attitude is used to correct the camera pose corresponding to each scan line.

[0076] The term "camera pose sweep envelope" refers to the pose range traversed by the optical center and line of sight of the depth camera as it moves from the initial pose to the final pose during a single exposure. For images read out line by line, different image lines correspond to different acquisition times. For high-speed flying UAVs, directly converting all depth points using a single camera pose would cause obstacle boundaries to be stretched or shifted along the flight direction. Therefore, this embodiment determines the interpolated pose based on the relative time of each scan line within the exposure interval and merges the depth points converted from each interpolated pose, ensuring that the obstacle's initial occupancy unit corresponds to the spatial observation range within the actual exposure time period.

[0077] The AI ​​chip divides the 3D navigation space into voxels, with voxel side lengths selectable from 0.02m to 0.20m, specifically determined based on the minimum size of the obstacle to be identified, depth measurement error, and the available computing resources of the AI ​​chip. This embodiment uses a voxel side length of 0.05m. Depth points falling into the same voxel are merged into one occupied unit; consecutive adjacent occupied units form the initial occupied unit set for the obstacle according to 3D adjacency relationships. For discrete noise points, the AI ​​chip eliminates them based on the number of adjacent voxels and depth confidence; for depth-invalid regions, they are not directly determined as free space, but rather the original occupied state of the corresponding spatial unit is retained or marked as an uncertain occupied unit.

[0078] The spatial envelope of the corresponding obstacle is generated based on the outer boundary of the initial obstacle-occupying unit set. The spatial envelope can be represented by an axis-aligned bounding box, an oriented bounding box, a convex polyhedron, or an ellipsoid. In this embodiment, an oriented bounding box is used to represent dynamic obstacles with a clear heading, and a convex polyhedron is used to represent fixed obstacles with irregular shapes. The planning slice generation unit superimposes the spatial observation uncertainty range generated by the camera pose sweep envelope onto the obstacle spatial envelope, so that the coordinate deviation caused by the UAV's movement during the exposure is covered by the obstacle-occupied area, instead of leaving the corresponding deviation to be compensated in the collision judgment stage.

[0079] The obstacle state determination unit clusters and associates obstacles based on spatial projection overlap rate, centroid displacement, spatial envelope size change, and velocity direction continuity in consecutive image frames. The obstacle state determination unit then clusters obstacles in the previous image frame. The obstacle and the first one in the subsequent image frame The associated cost between obstacles Determined according to the following formula:

[0080] , ;

[0081] In the formula, Indicates the first image frame in the previous image frame The obstacle and the first one in the subsequent image frame The associated cost between obstacles; This represents the overlap rate between two spatial envelope projections; Indicates the normalized centroid distance; This represents the normalized change in size; This indicates the difference in the direction of the normalized velocity; , , and These represent the weights corresponding to the spatial projection overlap rate, normalized centroid distance, normalized size change, and normalized velocity direction difference, respectively.

[0082] In this embodiment, , , and The values ​​are set to 0.35, 0.30, 0.15, and 0.20 respectively. When the association cost is less than 0.55 and is the lowest among the candidate associations, the two obstacle clusters are configured with the same obstacle identifier. When one-to-many or many-to-one candidate associations occur, associations that maintain continuity in centroid position, velocity direction, and spatial envelope are prioritized, and the remaining obstacle clusters are configured with new obstacle identifiers.

[0083] The so-called "obstacle identifier" is a persistent identifier configured by the AI ​​chip for the spatial envelope of objects identified as the same physical obstacle in consecutive image frames. The obstacle identifier is used to combine measurement results from multiple moments into a motion state record, and cannot be determined solely based on the obstacle's sequence number in a single frame. The normalized centroid distance uses the currently allowed maximum correlation distance as the denominator, the normalized size change uses the larger of the two spatial envelopes as the denominator, and the normalized velocity direction difference uses 180° as the denominator. Thus, evaluation terms with different dimensions are transformed into a comparable range, and then correlation costs are formed through corresponding weights.

[0084] When an obstacle with the same obstacle identifier does not appear in subsequent image frames, the obstacle state determination unit projects the spatial envelope of the corresponding obstacle in the previous image frame into the subsequent image frame. Occupied units located between the camera and the predicted spatial envelope, and overlapping with the projection area, are determined as foreground occupant units. The ratio of the number of foreground occupant pixels in the projection area with a depth less than the predicted obstacle depth to the number of effective pixels in the projection area is determined as the occlusion coverage rate.

[0085] In this embodiment, when the occlusion coverage reaches 0.60, the corresponding obstacle is identified as a visual occlusion obstacle, and an occlusion inheritance slice is generated based on the three-dimensional spatial coordinates, velocity range, and heading change range before occlusion. The occlusion inheritance slice continues until an obstacle with the same obstacle identifier is observed again, or the corresponding predicted spatial envelope no longer intersects with all state reachable slices.

[0086] The term "foreground occupancy unit" refers to an occupancy unit located between the camera and the tracked obstacle, capable of obstructing the obstacle's line of sight. The term "occlusion-inherited occupancy slice" refers to the future occupancy range extrapolated from the obstacle's motion state determined before occupancy occurs when the tracked obstacle is temporarily invisible. The aforementioned processing avoids directly interpreting the absence of an observed obstacle as its disappearance. When the predicted spatial envelope still has a possibility of intersecting with any state reachable slice, the occupancy-inherited occupancy slice continues to participate in safety assessment; when the predicted spatial envelope has moved far beyond the UAV's entire reachability range, even if the obstacle has not reappeared, the generation of the corresponding occupancy-inherited occupancy slice ceases to limit the computational load required for occupancy prediction.

[0087] For obstacles associated in at least three consecutive image frames, the obstacle state determination unit calculates the measured three-dimensional spatial coordinates, velocity range, acceleration range, and heading angular velocity range, and establishes constant velocity motion branches, restricted acceleration motion branches, and restricted steering motion branches.

[0088] The constant velocity motion branch is recursively derived according to the following formula: ;

[0089] The constrained acceleration motion branch is recursively derived according to the following formula:

[0090] , ;

[0091] The restricted steering motion branch is recursively derived according to the following formula:

[0092] , ;

[0093] In the formula, Indicates the first The obstacle is in the first... The three-dimensional spatial coordinates of each planning moment; Indicates the first The obstacle is in the first... The velocity vector at each planning moment; Indicates the first The obstacle is in the first... The acceleration vector at each planning moment; Indicates the first The obstacle is in the first... The heading angle at each planned moment; Indicates the first The obstacle is in the first... The heading angular velocity at each planned moment; Represents the rotation matrix about the vertical axis of the three-dimensional navigation coordinate system; Indicates the time step of the plan.

[0094] The constant velocity motion branch represents the motion of an obstacle maintaining its current velocity; the restricted acceleration motion branch represents the motion of an obstacle accelerating or decelerating within the physical boundaries of acceleration; and the restricted turning motion branch represents the motion of an obstacle changing its direction of motion. The velocity range, acceleration range, and heading angular velocity range in each motion branch are jointly determined by the measurement results in consecutive image frames, the corresponding measurement errors, and the permissible motion range for the obstacle type.

[0095] The obstacle state determination unit configures branch versions for each motion branch. When the velocity range, acceleration range, heading angular velocity range, or effective state of a motion branch changes, the corresponding branch version is updated. For each track segment, a set of dependent motion branches used when executing a corresponding safety slice is generated, along with the branch version corresponding to each motion branch in that set.

[0096] The set of dependent motion branches for a given track segment represents the obstacle motion branches used for safety verification of the corresponding track segment. If the subsequently measured 3D spatial coordinates are not within the spatial envelope predicted by a branch version of a motion branch in the set of dependent motion branches, the corresponding motion branch and its corresponding branch version are marked as motion branches to be released, and the corresponding motion branch is not immediately deleted.

[0097] The AI ​​chip retains the obstacle occupancy slice corresponding to the motion branch to be released until the effective period of the track segment that depends on the motion branch to be released ends, or the flight controller returns a completion receipt for the corresponding track segment, before releasing the motion branch to be released and the occupancy area formed by the motion branch to be released. The delayed release process is used to prevent the AI ​​chip from deleting old predictions while the flight controller is still executing track segments that depend on old predictions.

[0098] The obstacle state determination unit predicts the spatial envelope of the same obstacle at each planning time based on each motion branch, and merges all retained motion branches at the same planning time to form a basic obstacle occupancy slice. The obstacle is in the first... Error propagation distance at each planning time point Determined according to the following formula:

[0099]

[0100] In the formula, Indicates the first The obstacle is in the first... Error propagation distance at each planning time; Indicates the first The spatial discrete range of depth points corresponding to each obstacle; This represents the spatial error corresponding to the camera calibration residual; Indicates drones and the first The obstacle is in the first... The relative velocity magnitude at each planning moment; This indicates the delay before control takes effect.

[0101] Let the basic obstacle occupy the slice as The drone's airframe envelope is The error envelope formed by the error expansion distance is The expanded obstacles occupy the slice Determined according to the following formula: .

[0102] In the formula, Indicates the first The obstacle is in the first... After the expansion of a planned time, obstacles occupy the slice; Indicates the first The obstacle is in the first... The basic obstacles at each planning moment occupy the slice; Indicates the envelope of the drone's body; This represents the error envelope formed by the error expansion distance; This refers to Minkowski and...

[0103] For example, if the drone's current horizontal speed is 5.0 m / s, the obstacle's oncoming speed is 1.5 m / s, the relative speed is 6.5 m / s, the control activation delay is 0.082 s, the spatial discrete range of the depth point is 0.05 m, and the spatial error corresponding to the camera calibration residual is 0.03 m, then the error propagation distance is:

[0104]

[0105] After further superimposing the horizontal half-axis of the UAV's body envelope by 0.35m, the minimum exclusion distance in the corresponding direction is 0.963m.

[0106] The term "obstacle occupancy slice" refers to the collection of all possible spaces occupied by obstacles at a given planning time, not a two-dimensional image slice. The basic obstacle occupancy slice is formed by merging the predicted spatial envelopes of all preserved motion branches of each obstacle at the corresponding planning time. The expanded obstacle occupancy slice further incorporates depth measurement discrete errors, camera calibration errors, relative displacements within the control activation delay, and the UAV's body dimensions. Obstacle association, visual occupancy inheritance, motion branch prediction, and error expansion together form the data chain of the obstacle occupancy slice.

[0107] The term "state reachable slice" refers to the set of all flight states that a UAV can reach at a given planned time, starting from its current measured flight state and constrained by control activation delay, current control commands, flight performance boundaries, and mission airspace boundaries. A state reachable slice includes not only three-dimensional spatial coordinates but also velocity and heading. Therefore, two states with the same three-dimensional spatial coordinates but different velocities or headings belong to different state units. The term "interval state unit" refers to a state block jointly defined by an interval of three-dimensional spatial coordinates, a velocity interval, and a heading angle interval. Using interval state units allows for a recursive calculation of a set of similar states, while maintaining coverage of measurement and control errors through upper and lower bounds of the interval.

[0108] During the control hold phase corresponding to the control activation delay, the state recursion uses the thrust, roll, pitch, and yaw control values ​​currently output by the flight controller. The thrust control value is converted into equivalent acceleration in the three-dimensional navigation coordinate system using the UAV's mass and attitude rotation matrix. Let the UAV's mass be... Total thrust is The rotation matrix from the UAV body coordinate system to the three-dimensional navigation coordinate system is: The gravitational acceleration vector is equivalent acceleration Determined according to the following formula: .

[0109] In the formula, This represents the equivalent acceleration vector of the UAV in the three-dimensional navigation coordinate system; Indicates the quality of the drone; This represents the rotation matrix from the UAV's body coordinate system to the three-dimensional navigation coordinate system; This represents the total thrust generated by the flight execution components; This represents the gravitational acceleration vector in the three-dimensional navigation coordinate system.

[0110] After the control activation delay ends, the planning slice generation unit continues recursively using control primitives defined by flight performance boundaries. The finite control primitive library includes hold, acceleration, deceleration, left yaw, right yaw, climb, descent, and combinations of the aforementioned control primitives within permissible limits. The UAV state includes at least three-dimensional spatial coordinates. Velocity vector and heading angle The discrete state recurrence relation is:

[0111] , ,

[0112] In the formula, Indicates the first The three-dimensional spatial coordinates of the drone at each planned moment; Indicates the first The drone velocity vector at each planned moment; This represents the UAV acceleration vector determined by the control primitives and attitude constraints. Indicates the first The drone's heading angle at a planned moment; Indicates that the drone is in The heading angular velocity at each planned moment; This indicates that the flight speed is limited according to the speed range specified in the flight performance boundary. Indicates the time step of the plan.

[0113] Roll and pitch control variables are used to determine the thrust direction, yaw control variable is used to determine the directional angular velocity, and total thrust is used to determine the acceleration along the vertical axis of the UAV. At each planning time, the planning slice generation unit limits the recursive results based on the maximum horizontal flight speed, maximum climb speed, maximum descent speed, maximum lateral acceleration, maximum braking deceleration, maximum directional angular velocity, maximum roll angle, and maximum pitch angle. States exceeding the flight performance boundaries do not proceed to subsequent reachable slices.

[0114] For the planning time layer covered by the control activation delay, the planning slice generation unit only recursively calculates the UAV state according to the current control command and does not allow the use of new control primitives that have not yet had a physical effect. After the control activation delay ends, the planning time layer starts with the set of states formed at the end of the control hold phase, and applies control primitives from a limited control primitive library to each state to generate multiple subsequent UAV states.

[0115] For each state unit, the planning slice generation unit takes the endpoint states of the three-dimensional spatial coordinate interval, velocity interval, and heading angle interval, respectively, and selects control primitives that can generate the maximum lateral displacement, maximum forward displacement, maximum vertical displacement, and maximum heading change for recursion. Then, the interval envelope of all recursion results is taken to form the state unit for the next planning time. Subsequent UAV states that intersect in three-dimensional spatial regions, velocity ranges, and heading ranges are merged into the same state unit. The state reachable slice is composed of all state units corresponding to the same planning time.

[0116] The term "control primitives" refers to the basic control actions and corresponding parameter ranges that a flight controller can stably execute within one or more control cycles. Hold control primitives are used to maintain the current speed and heading; acceleration and deceleration control primitives are used to change the speed along the current heading; left yaw and right yaw control primitives are used to change the heading; climb and descent control primitives are used to change the three-dimensional spatial coordinates of the altitude direction. Before the control activation delay ends, only recursion based on the current control command is allowed, because the target track chain corresponding to the new track version has not yet actually generated control effects; only after the control activation delay ends is it allowed to select new control actions from a limited control primitive library.

[0117] Taking a current horizontal speed of 5.0 m / s and a control activation delay of 0.082 s as an example, under the simplified condition of ignoring acceleration changes during the control activation delay, the UAV will still move approximately 0.41 m along the current speed direction before the new control result takes effect. The corresponding displacement must be included within the reachable range of the initial state formed during the control hold phase; the UAV's three-dimensional spatial coordinates at the exposure moment cannot be directly used as the starting point for the execution of the new control result.

[0118] like Figure 7 As shown, the reachable state slice extends forward along the UAV's current heading, while being constrained by maximum heading angular velocity, lateral acceleration, climb rate, descent rate, and braking capability. The planning slice generation unit first removes state units that exceed the mission airspace boundary from the reachable state slice, and then determines whether the UAV body sweep envelope formed between adjacent planning moments intersects with the expanded obstacle-occupied slice.

[0119] The UAV body sweep envelope can be formed by the union of the UAV body envelopes corresponding to adjacent states and the intermediate attitude interpolation envelope, or by the convex hull of the UAV body envelopes at both ends and the Minkowski sum of the attitude change margin to form an approximate result. In this embodiment, interpolation is performed in five equal parts according to the three-dimensional spatial coordinates and attitude between adjacent states, and the union of the UAV body envelopes corresponding to the five interpolated attitudes is determined as the UAV body sweep envelope.

[0120] like Figure 8As shown, when the UAV body sweep envelope corresponding to a state unit intersects with any expanded obstacle-occupied slice under the same planning time index, the planning slice generation unit removes the corresponding state unit from the state-reachable slice; the remaining state units form the execution safety slice. Since the obstacle-occupied slice and the state-reachable slice use the same planning time index... The collision determination method identifies the spatial relationship between the drone and the obstacle at the same future execution moment, rather than comparing the current obstacle position with the future drone position.

[0121] The term "UAV airframe sweep envelope" refers to the union of all spatial locations traversed by the UAV airframe envelope as it moves continuously from two adjacent flight states, covering the continuous motion space between discrete track sampling points. The term "execution safety slice" is the set of states obtained from the state reachability slice at the corresponding planning time by sequentially removing states exceeding the mission airspace boundary, states where the corresponding UAV airframe sweep envelope intersects with the expanded obstacle-occupied slice, and states that do not meet the flight performance boundaries. Each state in the execution safety slice simultaneously satisfies both the conditions of reachability and that the arrival process does not involve envelope intersection; a state cannot be considered safe simply because its endpoint is located in an idle voxel.

[0122] The track segment generation unit establishes a local track coordinate system during the continuous execution of safety slices. The local track coordinate system uses the direction pointing from the reference state in the previous executed safety slice to the target's three-dimensional spatial coordinates as the forward axis, the opposite direction of gravity as the upward axis, and the right-hand rule to determine the lateral axis. Based on the spatial adjacency and single-step reachability of state units, the track segment generation unit divides each executed safety slice into multiple three-dimensional connected partitions; according to the arrangement order of obstacle markers around each three-dimensional connected partition and their relative orientation to the local track coordinate system, it configures a passage topology marker for the corresponding three-dimensional connected partition.

[0123] The so-called "three-dimensional connectivity partition" is a set of states that can be continuously reached through adjacent state units within the same execution safety slice. Two state units must not only be adjacent in three-dimensional space, but also possess single-step control primitives that satisfy flight performance boundaries to be considered interconnected. The traffic topology identifier does not use fixed text as a unique code, but is generated based on the arrangement order, relative orientation, and connectivity of boundary obstacle identifiers, ensuring that even when obstacles move as a whole but the traffic relationship remains unchanged, the same traffic topology can still be identified.

[0124] A state transition section is defined as the intersection of the set of states in the preceding 3D connected partition that can reach the following 3D connected partition within one planning time step and the following 3D connected partition. When the preceding and following 3D connected partitions have the same traversal topology identifier, the corresponding state transition section is used for the extension of the same candidate safe corridor; when the preceding and following 3D connected partitions have different traversal topology identifiers, but there is a connected section between the two 3D connected partitions that does not go through the expanded obstacle-occupied slice, the corresponding connected section is determined as a topology transformation section. Candidate safe corridors are formed according to the sequence of 3D connected partitions connected by the state transition sections and topology transformation sections, such as... Figure 9 As shown.

[0125] The term "state transition section" is not merely a geometric plane formed by spatial location, but rather a set of states encompassing three-dimensional spatial coordinate ranges, velocity ranges, and heading ranges that can span adjacent planning time points. The term "topology transformation section" is a state transition region that allows the UAV to switch from one obstacle avoidance relationship to another. The term "candidate safe corridor" is a set of three-dimensional connected partitions and corresponding state transition sections connected sequentially according to planning time points, retaining a passable state range within it, and is not equivalent to a single centerline. The candidate safe corridor provides the state space boundary for subsequently generating multiple selectable track segments.

[0126] For each candidate safe corridor, the track segment generation unit determines the intersection of the three-dimensional connected partition at the starting end and the corresponding state reachable slice as the candidate ingress state acceptance set, and generates a control command subsequence from a finite control primitive library. The control command subsequence is generated in the order of combinations of hold, accelerate or decelerate, yaw, climb or descend, and each combination of control primitives must satisfy the flight performance boundary.

[0127] The track segment generation unit applies the same control command sub-sequence to all states in the candidate entry state acceptance set, and recursively derives the corresponding UAV state and UAV body sweep envelope layer by layer. When all recursed states are within the corresponding execution safety slice, and the corresponding UAV body sweep envelope does not intersect with the expanded obstacle-occupied slice, the recursed states are combined into the initial exit state set.

[0128] The intersection of the initial exit state set and the corresponding entry state acceptance set of the next candidate safe corridor is determined as the handover state set. When the initial exit state set contains state units outside the handover state set, it indicates that the same control command subsequence cannot ensure the safe continuation of all entry states to the next candidate safe corridor. The track segment generation unit splits the candidate entry state acceptance set according to the recursive results of each state unit under the control command subsequence, and retains only the split track segments in which all exit state transfer sets are located within the handover state set.

[0129] A flight path segment includes at least an ingress state acceptance set, a control command subsequence, an exit state transfer set, a valid time period, a segment version, a set of dependent motion branches, and corresponding branch versions. The ingress state acceptance set defines the acceptable range of initial states for the flight path segment; the control command subsequence forms a timing control reference for the UAV's three-dimensional spatial coordinates, velocity, and heading; the exit state transfer set defines the state range after the flight path segment execution ends; the valid time period defines the time window during which the corresponding flight path segment can begin and end execution; the segment version distinguishes flight path segments regenerated due to visual observation or state updates; and the set of dependent motion branches and corresponding branch versions determine the obstacle prediction states upon which the safety of the flight path segment is based.

[0130] The track segment generation unit connects multiple track segments whose exit state transfer set is compatible with the next track segment's inlet state acceptance set into candidate target track chains. When multiple candidate target track chains exist, those that cannot guarantee the safety of the entire inlet state set are first eliminated. Then, the number of topology changes, minimum obstacle clearance, control changes, and flight distance are compared sequentially. Candidate target track chains with fewer topology changes, larger minimum obstacle clearance, smaller control changes, and shorter flight distances are preferentially selected.

[0131] The term "target track chain" refers to multiple track segments connected in the order of execution, along with the state transfer relationships between these segments. A target track chain must guarantee at least one inclusion relationship or a pre-defined safe intersection between the exit state transfer set of the preceding track segment and the entry state acceptance set of the following track segment, and ensure that the effective time periods of each track segment are arranged consecutively in the execution order. The number of topology switching times reflects the number of changes in the UAV's detour relationship, the minimum obstacle clearance reflects the safety margin, the control change reflects the smoothness of speed and heading adjustments, and the flight distance reflects the path length to the target's three-dimensional spatial coordinates. The comparison order can be adjusted according to the mission type, but any selected target track chain must first satisfy the conditions of full entry state safety and track segment continuity.

[0132] The track version generation unit generates a new track version for the selected target track chain and writes the currently executing track version into the parent track version field. The track data packet includes at least the new track version, parent track version, generation status fingerprint, target track chain, ingress status acceptance set of the first track segment, valid time period of each track segment, segment version, verification field, and safe track prefix termination time.

[0133] The generated state fingerprint includes the three-dimensional spatial units, velocity range, heading range, and currently executed track version corresponding to the generation of the target track chain. It is used to determine whether the track data packet still corresponds to the current flight state. The term "parent track version" refers to the currently executed track version recognized by the AI ​​chip when generating the target track chain corresponding to the new track version. The target track chain corresponding to the new track version is only allowed to be accepted if the flight controller is still executing the corresponding parent track version.

[0134] The so-called "generated state fingerprint" is a state summary formed by combining the UAV's three-dimensional spatial units, velocity range, heading range, and the currently executed track version used when generating the target track chain. It is used to quickly identify whether the track data packet was generated from a state that matches the current measured flight state. The so-called "safe track prefix termination time" represents the last valid moment in the target track chain corresponding to the currently executed track version where the updated obstacle occupancy slice verification still shows a safe continuous track prefix. Before the flight controller accepts the new track version, the AI ​​chip retains the track segment before the safe track prefix termination time for the flight controller to continue execution.

[0135] After receiving the track data packet, the flight controller performs the following verifications sequentially by the track acceptance verifier:

[0136] First, verify whether the parent track version is consistent with the current execution track version in the current execution version register;

[0137] Second, verify whether the current measured flight status matches the three-dimensional spatial unit, speed range, heading range, and current executed track version in the generated status fingerprint;

[0138] Third, verify whether the current measured flight status is within the entry status acceptance set of the first track segment in the target track chain corresponding to the new track version;

[0139] Fourth, verify whether the current time is within the valid time period of the first track segment;

[0140] Fifth, verify whether the currently executed track segment can be connected to the first track segment of the target track chain corresponding to the new track version.

[0141] After all the aforementioned verifications are passed, the flight controller writes the new track version to the current execution version register, returns an acceptance receipt including the acceptance version and track switch time, and switches to the target track chain corresponding to the new track version at the track switch time. Before receiving the acceptance receipt, the AI ​​chip retains track segments in the target track chain corresponding to the current execution track version that are before the termination time of the safe track prefix, in order to prevent track version rollback caused by communication delays or out-of-order track data packets.

[0142] If any verification fails, the flight controller does not write the new track version to the current execution version register, but instead writes the rejection reason in the returned data. A version mismatch status is returned when the parent track version is inconsistent; a generation state mismatch status is returned when the current measured flight state does not match the generated state fingerprint; an entry state mismatch status is returned when the current measured flight state is not in the entry state acceptance set; a time mismatch status is returned when the current time exceeds the valid time period; and a handover mismatch status is returned when the target track chain corresponding to the current execution track segment and the new track version cannot be continued. The AI ​​chip rereads the current execution track version and measured flight state based on the rejection reason and regenerates the target track chain from the new planned time sequence, without repeatedly sending expired track data packets.

[0143] The flight controller performs position outer loop, velocity inner loop, and attitude control based on the target's three-dimensional spatial coordinates, velocity, and heading output by the target track chain. Position error vector. and velocity error vector Determined according to the following formula: , .

[0144] Target acceleration Determined according to the following formula: .

[0145] In the formula, Represents the position error vector; Represents the velocity error vector; Represents the target's three-dimensional spatial coordinates; Represents the measured three-dimensional spatial coordinates; Indicates the target speed; Indicates the measured speed; Represents the position feedback gain matrix; Represents the velocity feedback gain matrix; This represents the feedforward acceleration provided by the control command subsequence of the track segment; This represents the target acceleration output by the flight controller.

[0146] The flight controller calculates the target thrust, roll angle, pitch angle, and yaw rate based on the target acceleration and heading, and then converts these into control variables for the flight actuators via the attitude controller. The position feedback gain matrix and velocity feedback gain matrix are configured based on the UAV's mass, inertia, flight performance boundaries, and allowable tracking error. The flight controller can also employ linear quadratic control, model predictive control, or other control methods capable of generating flight actuator control variables based on the target's three-dimensional spatial coordinates, velocity, and heading, but the trajectory acceptance verification process remains unchanged.

[0147] During the execution of the target flight path by the UAV, the flight status acquisition component continues to output updated measured flight status sequences, and the depth vision acquisition component continues to output updated depth vision image sequences. The AI ​​chip uses the updated visual data and measured flight status as input for the next planning cycle, regenerates obstacle occupancy slices and state reachability slices, and determines whether the current target flight path still meets the safety conditions at the corresponding planning time.

[0148] When the updated obstacle occupancy slice intersects with the sweep envelope of the UAV in the current target track chain, or when the updated measured flight state deviates from the nominal state range of the current track segment, the irreversible conflict determination unit regenerates the reachable state slice based on the updated measured flight state, the current control command, the control activation delay, and the flight performance boundary.

[0149] For each subsequent planning time step, the irreversible conflict determination unit extracts an initial avoidance response state set from the updated state reachable slice. It then excludes states from this initial avoidance response state set where the corresponding UAV sweep envelope intersects with the obstacle-occupied slice, forming a collision-free state set. For each state in the collision-free state set, finite control primitives are used to recursively iterate over the remaining planning time domain. If the recursive result can enter the entry state acceptance set of any subsequent trajectory segment, the corresponding state is retained. The state set obtained after the aforementioned processing is the avoidance response state set.

[0150] The term "initial avoidance response state set" refers to the set of states within a reachable slice at a given subsequent planning time that can apply at least one avoidance control primitive starting from the corresponding planning time. The term "avoidance response state set" is the set of states obtained after further excluding collision states from the initial avoidance response state set and verifying that the remaining states can still connect to subsequent track segments of the target track chain. A state that can only temporarily avoid an obstacle but subsequently cannot enter any subsequent track segment is not included in the avoidance response state set. Therefore, the avoidance response state set simultaneously reflects both immediate collision-free capability and subsequent track continuity capability.

[0151] The planning moment when the avoidance response state set is first empty is defined as the irreversible conflict time layer. An irreversible conflict time layer does not indicate that the UAV has already collided at that moment, but rather that from that time layer onwards, under the current flight performance boundaries and remaining response time conditions, there is no longer a state where the UAV can both avoid the obstacle and continue to the subsequent target track chain. The irreversible conflict determination unit can thus determine the time position where the UAV's remaining control capability is insufficient before the geometric track actually intersects with the obstacle.

[0152] For example, when the drone's horizontal speed is 5.5 m / s, the maximum braking deceleration is 7.0 m / s², and the control activation delay is 0.082 s, the simplified braking safety distance is... Estimate using the following formula: .

[0153] In the formula, This indicates a simplified braking safety distance, expressed in meters. This indicates the horizontal velocity at the start of the judgment, in meters per second. Indicates the delay before control takes effect, in seconds; This represents the absolute value of the maximum braking deceleration, expressed in meters per second squared.

[0154] Substituting the aforementioned values, the simplified braking safety distance is approximately 2.61m. If the dynamic obstacle distance at a certain planning moment is less than the simplified braking safety distance, and the lateral, vertical, and directional reachable states are all excluded by the corresponding obstacle-occupied slices, then the avoidance response state set may already be empty before the actual spatial collision, and the corresponding planning moment is determined to be an irreversible conflict time layer. The actual determination uses the complete state reachable slice, and does not replace the multi-dimensional state calculation of three-dimensional spatial coordinates, velocity, and heading with the simplified braking safety distance.

[0155] like Figure 4 As shown, before the irreversible conflict time layer, the track chain update unit selects the last track segment whose exit state transfer set intersects with the regenerated state reachable slice as the safe handover track segment. The track chain update unit retains the track segments before the safe handover track segment, forming a retained track prefix; in the current target track chain after the irreversible conflict time layer, it searches for the first track segment whose effective time period has not yet ended and whose ingress state acceptance set intersects with the updated state reachable slice, forming a candidate retained track suffix; a repair track segment is generated between the exit state transfer set of the safe handover track segment and the ingress state acceptance set of the candidate retained track suffix.

[0156] The term "safe handover track segment" refers to the last track segment that is still reachable from the updated measured flight state before the irreversible conflict time layer, and where the exit state transfer set intersects with the regenerated state reachable slice. The term "reserved track prefix" refers to a sequence of consecutive track segments that remain safe after verification of the updated obstacle occupancy slice and state reachable slice before the safe handover track segment. The term "candidate reserved track suffix" refers to a sequence of original track segments that are still within the valid time period after the irreversible conflict time layer, and where the entry state acceptance set can be entered from the updated state reachable slice. The term "repaired track segment" refers to a newly added track segment connecting the exit state transfer set of the safe handover track segment and the entry state acceptance set of the candidate reserved track suffix.

[0157] If the execution time of the repaired track segment differs from the execution time of the replaced track segment, the track chain update unit performs a time shift on the valid time period of the candidate retained track suffix. Let the execution time of the repaired track segment be... The execution time of the replaced track portion is Time shift Determined according to the following formula: .

[0158] In the formula, This indicates the effective time-period translation amount of the candidate retained track suffix; Indicates the execution time for repairing the track segment; This indicates the execution time of the replaced track portion.

[0159] when When the value is positive, the effective time periods of the candidate retained track suffix are shifted backward; when... When the value is negative, the effective time periods of each candidate retained track suffix are shifted forward. After the time shift, the track chain update unit regenerates the corresponding obstacle occupancy slice and execution safety slice according to the shifted absolute planning time, and re-verifies the candidate retained track suffix.

[0160] The re-verification includes at least the following: confirming that the UAV body sweep envelope corresponding to each track segment in the candidate retained track suffix does not intersect with the obstacle-occupied slice at the planned time after translation; confirming that the exit state transfer set of the repaired track segment is compatible with the inlet state acceptance set of the candidate retained track suffix; confirming that the termination time of the effective time period after translation does not exceed the prediction effective time domain of the motion-dependent branch; and confirming that adjacent track segments in the candidate retained track suffix still satisfy the continuation relationship between the exit state transfer set and the inlet state acceptance set.

[0161] Repairing track fragments enables all exit states of the safely handed-over track fragments to reach the entry state acceptance set of the candidate retained track suffix. When the candidate retained track suffix passes re-verification, the track chain update unit sequentially connects the retained track prefix, the repaired track fragment, and the candidate retained track suffix to form the target track chain corresponding to the new track version.

[0162] If the connection fails, the track chain update unit will move the safe handover track segment forward by one unexecuted track segment and move the candidate reserved track suffix backward by one track segment, and then generate a repair track segment again. If a continuable combination still cannot be formed after these adjustments, the emergency handover track segment corresponding to the currently executed track version and the currently executed track segment will be invoked.

[0163] When the updated execution security slice still maintains the current travel topology, the path chain update unit prioritizes generating repair path segments within the current travel topology to reduce unnecessary travel topology switching. When the current travel topology cannot connect to the target's 3D spatial coordinates, the path chain update unit generates a candidate travel topology chain and searches for common path segments in both the current and candidate travel topology chains. When the exit state transfer set of the common path segment can connect to the entry state acceptance sets of the current subsequent path segment and the candidate subsequent path segment, respectively, the corresponding position is determined as the travel topology switching position.

[0164] The term "common flight path segment" refers to a flight path segment in which the current and candidate flight paths have compatible inbound state acceptance sets, compatible outbound state transfer sets, and the same or compatible control command subsequences within the same planning time period. Using the common flight path segment as a flight topology switching point allows the UAV to choose to continue flying along the current detour relationship or switch to a candidate detour relationship after reaching the outbound state transfer set of the common flight path segment, without requiring the UAV to instantaneously switch between two incompatible flight states.

[0165] To handle execution deviations, the AI ​​chip generates an execution state deviation envelope based on the recursive results of the measured flight state and the current flight path segment. The execution state deviation envelope includes at least the ranges of three-dimensional spatial coordinate deviations, velocity deviations, and heading deviations. Let the nominal state be... The actual state is The state deviation is The linearized bias propagation relationship is as follows: .

[0166] In the formula, Indicates the first The state deviation of the drone at each planned moment; Indicates the first The state deviation of the drone at each planned moment; This indicates that the partial derivative of the UAV state recursion function with respect to the UAV state variables is taken, and the result is obtained at the 1st... The Jacobian matrix is ​​obtained by taking values ​​at the nominal state at each planning time step; This represents the disturbance envelope formed by the state measurement error, the upper bound of the wind disturbance, and the control execution error. This refers to Minkowski and...

[0167] The planning slice generation unit superimposes the propagated execution state deviation envelope onto the subsequent state reachable slice, correcting the reachable three-dimensional space region, reachable velocity range, and reachable heading range in the subsequent state reachable slice.

[0168] The so-called "execution state deviation envelope" is the possible deviation range of the measured flight state relative to the nominal state of the current flight path segment, not a single instantaneous error value. The Jacobian matrix is ​​used to approximately describe the impact of the current state deviation on the three-dimensional spatial coordinates, velocity, and heading at the next planning moment after one UAV state recursion; the disturbance envelope is used to cover unmodeled wind disturbances, sensor noise, and motor response errors. Superimposing the propagated execution state deviation envelope onto subsequent state reachable slices is equivalent to expanding the range of states that the UAV may reach in the future, so that subsequent collision judgments are not based solely on the ideal nominal state.

[0169] To address obstacle prediction bias, the obstacle state determination unit extracts the measured spatial envelope of obstacles with obstacle identifiers from the updated depth vision image sequence, and compares the measured spatial envelope of obstacles with the obstacle occupancy slice corresponding to the same planning time to form the obstacle prediction bias envelope.

[0170] The obstacle state determination unit propagates the obstacle prediction deviation envelope to subsequent planning moments according to the corresponding constant velocity motion branch, limited acceleration motion branch, or limited steering motion branch, and expands the subsequent obstacle occupancy slice using the propagated obstacle prediction deviation envelope. The planning slice generation unit determines the planning moment when the corrected state reachable slice intersects with the expanded obstacle occupancy slice, and recalculates the execution safety slice for the corresponding planning moment.

[0171] The so-called "obstacle prediction bias envelope" refers to the range of directions and distances within which the measured spatial envelope of an obstacle obtained from updated visual measurements exceeds the previously predicted spatial envelope. When the updated visual measurement remains within the previously predicted spatial envelope, the prediction bias in the corresponding direction remains within the original prediction error range. When the updated visual measurement exceeds the previously predicted spatial envelope, the excess portion and the corresponding measurement error are added to the obstacle prediction bias envelope and propagated along the corresponding obstacle motion branch to subsequent planning moments. Prediction errors caused by sudden acceleration, deceleration, or turning of obstacles can thus be fed back to subsequent obstacle occupancy slices.

[0172] When a continuity anomaly occurs in a depth vision image sequence, the AI ​​chip performs conservative processing based on image frame identifiers, time-stamp continuity markers, and valid depth markers. If the time-stamp interval between adjacent valid image frames spans at least two planned time points, or if the valid depth marker of the corresponding image frame is invalid, the AI ​​chip reads the observation source bitmap and the most recent valid observation time-stamp corresponding to each idle voxel.

[0173] Voxels determined to be idle solely based on abnormal image frames are converted into indeterminate occupied voxels and incorporated into the obstacle occupancy slice at the corresponding planning time. The planning slice generation unit regenerates the execution safety slice based on the updated obstacle occupancy slice, retaining the safe track prefix that is within the execution safety slice both before and after the conversion.

[0174] If the safe track prefix can still connect to the regenerated execution safety slice, the flight controller continues to execute the safe track prefix, and the AI ​​chip generates subsequent track segments starting from the exit state transfer set of the safe track prefix. If the safe track prefix cannot connect to the regenerated execution safety slice, the flight controller controls the UAV to fly along the safe track prefix to the safe handover track segment, and generates deceleration and hovering control commands based on the exit state transfer set of the safe handover track segment, or enters a fault control process of controlled descent and return to the previous safe space state.

[0175] The term "observation source bitmap" is used to record the depth vision image frame on which a voxel is determined to be in an idle state, and the term "most recent valid observation timescale" is used to record the time when the corresponding voxel was most recently confirmed by valid depth data. When an idle voxel is confirmed by multiple valid images from different perspectives, a single frame anomaly does not immediately change the state of the corresponding voxel; when an idle voxel is confirmed only by the current anomalous image frame, the corresponding voxel is converted into an indeterminate occupied voxel.

[0176] Uncertainly occupied voxels are treated as occupied voxels in collision detection, but the duration of their presence can be shorter than that of fixed obstacle voxels. When subsequent valid depth vision images reconfirm that the corresponding voxel is in an idle state, the AI ​​chip restores it to an idle voxel.

[0177] When generating each normal track segment in the target track chain, the AI ​​chip simultaneously generates an emergency handover track segment. The exit state transfer set of the normal track segment serves as the entry state acceptance set of the emergency handover track segment, and the hovering state with zero speed within the subsequent execution safety slice constitutes the exit state transfer set of the emergency handover track segment.

[0178] The end time of the effective period for a normal flight path segment is determined as the start time of the effective period for an emergency handover flight path segment. Starting from the start time of the effective period for the emergency handover flight path segment, the AI ​​chip verifies layer by layer whether the hovering state is within the corresponding execution safety slice, and determines the last safe planning time before the first unsafe planning time of the hovering state as the end time of the effective emergency period. The start time and the end time of the effective emergency period together constitute the effective emergency period.

[0179] Emergency handover track segments, the entry status acceptance set for emergency handover track segments, the effective emergency time period, and the corresponding track version are pre-cached in the flight controller. If the target track chain verification corresponding to the new track version fails, the bidirectional planning control interface does not receive subsequent track segments within a communication cycle, or the AI ​​chip stops outputting the target track chain, the flight controller continues to execute the track segments in the current track version that are before the termination time of the safe track prefix.

[0180] When the current measured flight state enters the entry state acceptance set of the emergency handover flight path segment, and the current time is within the effective emergency period, the flight controller executes the emergency handover flight path segment, guiding the UAV from the exit state transfer set of the normal flight path segment to the hovering state.

[0181] In other implementations, the exit state transfer set for the emergency handover track segment may also consist of a controlled descent state, a return to the previous safe space state, or a preset hovering state. Regardless of the emergency exit state used, the AI ​​chip should verify all recursive states from the entry state acceptance set to the emergency exit state and the corresponding UAV body sweep envelope based on the subsequent execution of the safety slice.

[0182] The so-called "emergency handover track segment" is a pre-verified and cached backup control command subsequence in the flight controller. It is used to guide the UAV from the exit state transfer set of the normal track segment to a safety-verified emergency state when the target track chain generated by the AI ​​chip cannot enter the physical control loop in time. The emergency handover track segment does not bypass state verification and time verification; it is only allowed to execute if the current measured flight state is within the corresponding entry state acceptance set and the current time is within the effective emergency period. If the aforementioned conditions are not met, the flight controller continues to execute the safety track prefix already verified in the current track version and selects a fault control mode that matches the current measured flight state.

[0183] like Figure 10 As shown, in a complex and confined space, the AI ​​chip constructs a real-time voxel map based on depth points, and simultaneously represents fixed walls, columns, equipment boundaries, and dynamic obstacles in the real-time voxel map. Fixed walls, columns, and equipment boundaries form fixed obstacle-occupying units, while mobile devices and personnel form dynamic obstacle-occupying units with obstacle markers and movement branches.

[0184] like Figure 11As shown, under the same conditions of starting point, target 3D spatial coordinates, and obstacle conditions, traditional static trajectory planning generates a spatially passable path based on the current depth visual image frame, without incorporating control activation delay and future reachability into the same planning time index. In this embodiment, the dynamic trajectory, before a dynamic obstacle crosses the original trajectory, forms a combination of lateral and altitude avoidance based on the future obstacle occupancy slice and the UAV's reachability slice, and maintains the continuity of 3D spatial coordinates, velocity, and heading through the entrance state acceptance set and the exit state transfer set.

[0185] like Figure 12 As shown, the UAV's three-dimensional spatial coordinates and nose orientation change gradually according to the flight path segments during continuous planning time. The flight controller continuously adjusts the target speed, target heading, and target altitude based on the target flight path chain to avoid instantaneous state jumps that do not conform to the maximum heading angular velocity or maximum lateral acceleration when the target flight path chain is updated.

[0186] In some implementations, the AI ​​chip and flight controller terminate the current planning cycle or enter the next processing state according to the following state transition conditions:

[0187] When the UAV's measured 3D spatial coordinates enter the target tolerance area corresponding to the target's 3D spatial coordinates, and the measured speed and measured heading meet the mission termination conditions, the flight controller ends the target track chain execution and enters the mission completion state.

[0188] When the track acceptance verification fails, the flight controller maintains the current track version, and the AI ​​chip regenerates the target track chain based on the rejection reason returned by the flight controller, the current track version, and the updated measured flight status.

[0189] When an irreversible conflict time layer is detected, the track chain update unit determines the safe handover track segment and performs local updates of the preserved track prefix, repaired track segment, and candidate preserved track suffix;

[0190] If a continuous target track chain cannot be formed, or if data transmission is interrupted at the bidirectional planning control interface, the flight controller executes an emergency handover track segment corresponding to the currently executed track version and the currently executed track segment.

[0191] Therefore, normal arrival at the target, track version rejection, irreversible conflict update, and abnormal emergency handover all have clear state transition conditions, and will not cause the target track chain generation and execution process to be in a loop without termination conditions.

[0192] Example 2

[0193] In a unified simulation platform, a multi-rotor UAV is set to fly from three-dimensional spatial coordinates (0m, 0m, 1.5m) to three-dimensional spatial coordinates (20m, 0m, 1.5m), with an initial horizontal velocity of 5.5m / s, a maximum lateral acceleration of 6.0m / s², and a maximum braking deceleration of 7.0m / s². A dynamic obstacle laterally crosses the UAV's initial flight path at a position 10m along the first coordinate axis.

[0194] The initial velocity of the dynamic obstacle is selected within the range of 3.0 m / s to 5.0 m / s, the acceleration is selected within the range of −1.5 m / s² to 1.5 m / s², and the control activation delay is selected within the range of 0.08 s to 0.13 s. The collision criterion is that the minimum distance between the UAV's body envelope and the spatial envelope of the dynamic obstacle is less than 0.85 m.

[0195] The unified simulation platform uses a 0.01s integral step for UAV dynamics, a 0.05s planning time step, and a 20ms flight control cycle. All control tasks use the same UAV dynamics model, depth vision field of view, map resolution, flight performance boundaries, and flight controller parameters, only changing the initial velocity, acceleration, appearance time, and control activation delay of dynamic obstacles.

[0196] Collision-free completion is defined as the UAV entering the target tolerance area corresponding to the target's three-dimensional spatial coordinates within the specified mission time, and the distance between the UAV's body envelope and the obstacle's spatial envelope during the entire flight process is not less than the collision criterion.

[0197] The average minimum obstacle spacing is calculated as follows: determine the minimum distance between the UAV body envelope and the spatial envelope of all obstacles in each mission, and then calculate the arithmetic mean of the minimum distances for all missions.

[0198] A sudden change in heading is defined as the change in target heading angular velocity between two consecutive flight control cycles exceeding a set heading continuity threshold. The heading continuity threshold is determined based on the maximum heading angular velocity in the flight performance boundaries, the flight control cycle, and the allowable heading tracking error.

[0199] The computational cost of partial updates is evaluated using the number of regenerated track segments and the time required for a single partial update. The risk of out-of-order track version execution is evaluated using the number of times a lower-version track data packet arrives after a higher-version track data packet and is executed by the flight controller.

[0200] The root mean square error is used to measure the three-dimensional spatial coordinate tracking error. express: .

[0201] In the formula, This represents the root mean square error in three-dimensional spatial coordinate tracking. Indicates the number of sampling points; Indicates the first Measured three-dimensional spatial coordinates of each sampling point; Indicates the first Reference three-dimensional spatial coordinates of each sampling point; This represents the Euclidean norm.

[0202] Regardless of the alternative method used, obstacle occupancy slices and state reachable slices all use the same planning time index. All track segments are defined by the entry state acceptance set, exit state transfer set, and valid time period to determine the executable state. The flight controller executes the target track chain after completing the verification of the parent track version, the currently executed track version, generating the state fingerprint, the measured flight state, and the valid time period.

Claims

1. A dynamic planning method for autonomous flight paths of unmanned aerial vehicles (UAVs) based on visual obstacle avoidance, characterized in that, include: The AI ​​chip acquires time-stamped depth vision image sequences, measured flight state sequences, target 3D spatial coordinates, flight performance boundaries, UAV body envelope, current control commands, and current executed trajectory versions. Based on the end-link time stamp, the control activation delay is determined and a planned time sequence is established. The three-dimensional spatial coordinates and motion state of obstacles are determined based on image frames, and obstacle occupancy slices with the same index are generated; within the control activation delay, the UAV state is recursively deduced according to the current control command, and then the UAV state is recursively deduced according to the flight performance boundary, generating state reachable slices with the same index. Based on the expansion of the obstacle-occupied slice by the drone's airframe envelope, states located within the expanded obstacle-occupied slice and states where the drone's airframe sweep envelope intersect are excluded from the state-reachable slice, thus forming an execution safety slice. The continuous execution of security slices generates track segments with ingress state acceptance sets, exit state transfer sets, and effective time periods. Connecting the contiguous track segments forms the target track chain. A new track version is generated based on the current track version and the parent track version is recorded. The target track chain and track version are sent to the flight controller. When the parent track version matches, the measured flight status is in the ingress status acceptance set and the time is in the valid time period, the flight controller controls the flight execution components to adjust the UAV's three-dimensional spatial coordinates, speed and heading. Based on the updated data, obstacle occupancy slices and state reachability slices are generated. The planning time of the first reachable state that does not have collision-free and can be connected to subsequent track segments is determined as the irreversible conflict time layer. The previous safe handover track segment is selected and the target track chain is updated from the safe handover track segment.

2. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 1, characterized in that, The determination of the control activation delay includes: Configure image frame identifiers, exposure start timestamps, and exposure end timestamps for each image frame in the depth vision image sequence, and write the image frame identifiers into the track data packets generated based on the corresponding image frames; The system acquires the image reception time stamp of the corresponding image frame received by the AI ​​chip, the processing completion time stamp of the corresponding control command generated, the command reception time stamp of the corresponding control command received by the flight controller, and the execution feedback time stamp of the flight execution component responding to the corresponding control command. Based on the measured flight state corresponding to the exposure start time marker and the exposure end time marker, a camera pose sweep envelope is generated during the image exposure period, and the depth points in the corresponding image frames are converted to the three-dimensional navigation coordinate system to form an initial obstacle occupancy unit set. The spatial envelope of the corresponding obstacle is generated based on the outer boundary of the initial occupied unit set of the obstacle, and the spatial envelope is expanded according to the camera pose sweep envelope. The control activation delay is determined based on the duration between the exposure end time marker and the execution feedback time marker, and the expanded spatial envelope is shifted to the corresponding planning time according to the control activation delay.

3. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 2, characterized in that, The generation of the obstacle-occupied slice includes: Associate the initial occupancy units of obstacles in consecutive image frames, and make the consecutively associated initial occupancy units of obstacles have the same obstacle identifier; If an obstacle with the same obstacle identifier does not appear in a subsequent image frame, the spatial envelope of the obstacle formed by the outer boundary of the initial occupied unit set of the obstacle in the previous image frame is projected to the subsequent image frame. The occupancy unit that overlaps with the projection area of ​​the spatial envelope between the camera and the obstacle is defined as the foreground occupancy unit; When the projection area of ​​the spatial envelope is located within the camera's field of view and is covered by the foreground-occupying unit, the obstacle is identified as a visual occlusion obstacle. Based on the three-dimensional spatial coordinates, velocity range, and heading change range of the visual occupancy obstacle before occupancy, an occupancy inheritance occupancy slice is generated at a subsequent planning time, and the occupancy inheritance occupancy slice is incorporated into the obstacle occupancy slice at the corresponding planning time. The generation of the occlusion inheritance slice is stopped when an obstacle with the same obstacle identifier is detected again, or when the occlusion inheritance slice does not intersect with any of the state reachable slices.

4. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 1, characterized in that, It also includes acquiring the mission airspace boundary, the current control commands including thrust control, roll control, pitch control, and yaw control, and the generation of the state reachable slice includes: During the control holding phase corresponding to the control activation delay, the UAV state is deduced according to the thrust control amount, the roll control amount, the pitch control amount, and the yaw control amount; After the control hold phase, multiple subsequent UAV states are generated from each UAV state based on the speed variation range, climb speed range, descent speed range, lateral acceleration range, and heading angular velocity range defined by the flight performance boundaries. Generate a corresponding drone body sweep envelope based on the drone body envelope between adjacent drone states, and remove subsequent drone states whose drone body sweep envelope exceeds the mission airspace boundary. The states of subsequent UAVs that intersect in three-dimensional space, velocity range, and heading range are merged into a state unit, and the state reachable slice is composed of the state units corresponding to the same planning time.

5. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 4, characterized in that, The process of performing security slicing includes: Each execution security slice is divided into multiple three-dimensional connected partitions; Based on the obstacle-occupied slices that define the lateral, upper, and lower boundaries of each three-dimensional connected partition, configure boundary obstacle identifiers for the obstacles in the corresponding obstacle-occupied slices. Configure access topology identifiers for each three-dimensional connected partition according to the arrangement order of each boundary obstacle identifier in the circumferential direction of the corresponding three-dimensional connected partition; Determine the state transition section between the three-dimensional connected partitions corresponding to adjacent planning moments; When the travel topology identifiers of adjacent three-dimensional connected partitions are the same, or when there is a topology transformation section between adjacent three-dimensional connected partitions that does not intersect with any expanded obstacle-occupied slice, a travel temporal connection is established between the adjacent three-dimensional connected partitions. Candidate safe corridors are formed by three-dimensional connected partitions that are continuously arranged along the passage time sequence.

6. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 5, characterized in that, The generation of the track fragment also includes: The intersection of the three-dimensional connected partition at the starting end of the candidate safety corridor and the corresponding state reachable slice is determined as the entrance state acceptance set; The control command subsequence is applied to each state unit in the ingress state acceptance set, and the state of each state unit after passing through the candidate safety corridor is recursively deduced according to the flight performance boundary. The state units that are located within the candidate safe corridor after recursion and whose corresponding UAV body sweep envelope does not intersect with the expanded obstacle-occupied slice are formed into an initial exit state set. The intersection of the initial exit state set and the entrance state acceptance set corresponding to the next candidate safety corridor is determined as the handover state set; When the initial exit state set contains state units other than the handover state set, the track segment is split according to the state recursion result of the corresponding state unit under the control command subsequence, and the split track segments whose exit state transfer set is entirely located in the handover state set are used to form the target track chain.

7. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 6, characterized in that, The update of the target track chain includes: Record the topology identifiers sequentially according to the three-dimensional connected partitions traversed by the target track chain to form the current topology identifier sequence; When updating the target track chain, track segments with the current transit topology identifier sequence are connected preferentially; When a track segment with the current passable topology identifier sequence cannot be connected to the target three-dimensional space coordinates, a candidate track chain with a candidate passable topology identifier sequence is generated. Among the track segments shared by the target track chain and the candidate track chains, determine the track segments where the exit state transfer set intersects with the subsequent entry state acceptance set in each of the two track chains. The determined track segment is used as the traffic topology switching position from the current traffic topology identifier sequence to the candidate traffic topology identifier sequence.

8. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 3, characterized in that, The generation of the obstacle-occupied slice also includes: Obtain the camera calibration residual, determine the spatial discrete range based on the coordinate distribution of each depth point in the initial occupancy unit set of obstacles with the same obstacle identifier, and determine the measured three-dimensional spatial coordinates of the corresponding obstacle based on the initial occupancy unit set of obstacles with the same obstacle identifier in at least three consecutive image frames. Based on the measured three-dimensional spatial coordinates, establish constant velocity motion branches, restricted acceleration motion branches, and restricted steering motion branches respectively, and configure branch versions for each motion branch; Based on the spatial envelope of the corresponding obstacle predicted by each motion branch at each planning time, the spatial envelopes corresponding to the same planning time are merged, and the spatial envelopes are expanded and merged according to the spatial discrete range, the camera calibration residual, the relative displacement of the UAV and the corresponding obstacle within the control effective delay, and the UAV body envelope. For each track segment, record the set of dependent motion branches used when executing the corresponding safety slice, as well as the branch version corresponding to each motion branch in the set of dependent motion branches; If the subsequently measured 3D spatial coordinates are not located within the spatial envelope predicted by the corresponding branch version of a motion branch in the set of dependent motion branches, the corresponding motion branch and the corresponding branch version will be marked as motion branches to be released. The obstacle-occupied slice corresponding to the motion branch to be released is retained until the effective time period of the track segment dependent on the motion branch to be released ends, or the flight controller returns the execution completion receipt of the corresponding track segment, and then the obstacle-occupied slice corresponding to the motion branch to be released is released.

9. The method for dynamic planning of autonomous flight paths of unmanned aerial vehicles based on visual obstacle avoidance according to claim 4, characterized in that, The determination of the irreversible conflict time layer includes: Based on the updated measured flight state sequence, the current control command, the control activation delay, and the flight performance boundary, the updated state reachability slice is regenerated; For each subsequent planning moment in the target track chain, extract the state units that can be reached within the corresponding remaining time from the updated state reachable slice to form an initial avoidance response state set. The state units where the sweep envelope of the UAV body intersects with the corresponding obstacle-occupied slice are excluded from the initial avoidance response state set to form a collision-free state set. The avoidance response state set is formed by selecting state units from the non-collision state set that can be connected to any subsequent track segment in the target track chain as the entry state acceptance set. The planning time when the avoidance response state set is empty is marked as an irreversible conflict candidate time, and the first irreversible conflict candidate time is determined as the irreversible conflict time layer.

10. A dynamic trajectory planning system for unmanned aerial vehicles (UAVs) based on visual obstacle avoidance, characterized in that, The drones used in the manufacturing of intelligent unmanned aerial vehicles include depth vision acquisition components, flight status acquisition components, mission parameter storage components, AI chips, flight controllers, and flight execution components; The depth vision acquisition component includes an exposure timescale generator, which is used to output a depth vision image sequence with image frame identifiers and exposure timescales; The flight status acquisition component includes a sampling time stamp generator, which is used to output a measured flight status sequence with sampling time stamps. The measured flight status sequence includes the UAV's three-dimensional spatial coordinates, velocity, attitude, and heading. The mission parameter storage component is used to store the target's three-dimensional spatial coordinates, mission airspace boundaries, flight performance boundaries, and UAV body envelope. The flight controller is used to output the current control command, the current executed flight track version, and the command reception timestamp; The flight execution component is used to drive the UAV according to the current control command and output execution feedback timestamps; The AI ​​chip is connected to the depth vision acquisition component, the flight status acquisition component, the task parameter storage component, the flight controller, and the flight execution component, respectively. The AI ​​chip includes a processing time stamp generator, a control delay determination unit, an obstacle status determination unit, a planning slice generation unit, a track fragment generation unit, a track version generation unit, an irreversible conflict determination unit, and a track chain update unit. The processing time stamp generator is used to generate image reception time stamps for the image frames received by the AI ​​chip and processing completion time stamps for the completion of control instruction processing. The control delay determination unit is used to determine the control activation delay based on the exposure time stamp, the processing completion time stamp, the instruction reception time stamp, and the execution feedback time stamp, and to establish a planned time sequence based on the control activation delay; The obstacle state determination unit is used to identify obstacles in the depth vision image sequence, configure obstacle identifiers for each obstacle, and determine the three-dimensional spatial coordinates, motion state, and obstacle state recursion relationship of each obstacle based on adjacent image frames. The planning slice generation unit is used to generate an obstacle occupancy slice with the same index as the planning time sequence according to the obstacle state recursive relationship, expand the obstacle occupancy slice according to the UAV body envelope, recursively calculate the UAV state according to the current control command within the control effective delay, recursively calculate the UAV state according to the flight performance boundary after the control effective delay, generate a state reachable slice with the same index as the planning time sequence, and exclude states located in the expanded obstacle occupancy slice and states whose corresponding UAV body sweep envelope intersects with the expanded obstacle occupancy slice from the state reachable slice with the same index, and generate an execution safety slice; The track segment generation unit is used to determine the entry state acceptance set in the continuous execution safety slice, apply the same control command subsequence to all states in the entry state acceptance set to perform state recursion, and when all states are within the corresponding execution safety slice and the corresponding UAV body sweep envelope does not intersect with the expanded obstacle-occupied slice, the corresponding state recursion results are combined into an exit state transfer set, an effective time period is generated according to the corresponding planning time, the segment version is configured according to the generation order of the track segments, and a track segment with the entry state acceptance set, the exit state transfer set, the effective time period and the segment version is generated, and the track segments are connected to form a target track chain; The track version generation unit is used to generate a new track version based on the currently executed track version, and write the currently executed track version into the parent track version field of the new track version; The irreversible conflict determination unit is used to generate an updated obstacle occupancy slice based on the updated depth visual image sequence, regenerate a state reachable slice based on the updated measured flight state sequence, extract an initial avoidance response state set from the regenerated state reachable slice, exclude states whose corresponding UAV body sweep envelope intersects with the obstacle occupancy slice from the initial avoidance response state set, and filter states that can be continued to the entry state acceptance set of subsequent flight path segments. The first planning moment when the filtered state set is empty is determined as the irreversible conflict time layer. The track chain update unit is used to select the track segment that intersects with the exit state transfer set and the regenerated state reachable slice before the irreversible conflict time layer as the safe handover track segment, and update the target track chain from the safe handover track segment. A bidirectional planning control interface is provided between the AI ​​chip and the flight controller. The bidirectional planning control interface is used to send a track data packet to the flight controller, including the target track chain, the new track version, the parent track version, the entry status acceptance set, and the effective time period, and to return the current control command, the currently executed track version, the acceptance receipt, the track switching time, and the execution feedback time stamp to the AI ​​chip. The flight controller includes a current execution version register and a track acceptance verifier. The current execution version register stores the current execution track version. The track acceptance verifier verifies whether the parent track version is consistent with the current execution track version, whether the current measured flight state matches the corresponding generated state fingerprint, whether the current measured flight state is located in the corresponding entry state acceptance set, whether the current time is located in the corresponding valid time period, and whether the current execution track segment can be connected to the first track segment of the target track chain. After the verification is passed, the flight execution component is controlled to adjust the UAV's three-dimensional spatial coordinates, speed, and heading according to the target track chain.

Images