Multi-source information fusion unmanned aerial vehicle sea target tracking method and system

Abstract

The application provides a multi-source information fusion unmanned aerial vehicle offshore target tracking method and system, relates to the technical field of data processing, and the method comprises the following steps: acquiring image data and extracting the features of a target area; acquiring unmanned aerial vehicle attitude data, radar position detection data and AIS identification data, performing attitude disturbance correction, and aligning the data on the time axis; calculating the visual feature matching offset of adjacent frames, comparing the offset with the radar azimuth change trend; constructing a candidate target space position set, performing distance convergence screening and spatial overlap constraint processing of the geometric projection range of the candidate target space position set; performing motion direction continuity comparison to generate target position data; generating a track control instruction for controlling the flight direction and speed of the unmanned aerial vehicle according to the target position data, and adjusting the subsequent flight path of the unmanned aerial vehicle according to the track control instruction; and the application improves the autonomy and accuracy of unmanned aerial vehicle offshore target tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-source information fusion unmanned aerial vehicle (UAV) sea target tracking method and system. BACKGROUND

[0002] In the existing sea UAV tracking task, a single visual sensor is often used to continuously monitor the target ship, and the position of the target is determined by relying on the appearance features in the image sequence, and the determination result is used as the basis for the UAV to follow the flight path. In such a scheme, the UAV usually determines the position of the target in the vision by color feature matching, contour recognition or template comparison, and drives the heading adjustment with the visual detection result to maintain the continuity of the tracking path. In actual operation, the system relies on continuous frame prediction and image similarity to maintain the judgment of the target motion direction, and lacks correction means for visual drift from other information sources, so that the visual judgment is highly coupled with the actual flight path of the UAV. When the visual judgment deviates, the flight path of the UAV will also deviate accordingly.

[0003] In complex sea conditions, such as sea surface mirror reflection, spray shielding or low-angle strong light irradiation, large-area highlight areas often appear in the visual image, making it difficult to distinguish the appearance features of the target ship from the sea surface reflection texture. In this case, the visual system based on appearance matching is easy to mistake the reflection area as the ship structure, causing continuous deviation of the estimated position of the target in the vision. When the visual deviation is directly used for path adjustment of the UAV, the flight control system will perform path correction according to the wrong target position, which may cause the UAV to deviate from the heading frame by frame, approach the sea surface reflection area, or even lose the tracking object. SUMMARY

[0004] The purpose of the present application is to provide a multi-source information fusion unmanned aerial vehicle sea target tracking method and system, which aims to solve the problems mentioned in the background.

[0005] To solve the above technical problems, the technical solutions of the present application are as follows:

[0006] In a first aspect, a multi-source information fusion unmanned aerial vehicle sea target tracking method is provided, which comprises:

[0007] Obtaining continuous image frames as image data, and extracting the texture distribution, brightness gradient and edge stability of the target area from the image data to generate a first visual feature sequence;

[0008] Obtaining UAV attitude data, radar position detection data and AIS recognition data, and performing attitude disturbance correction to align the data on the time axis to generate first multi-source state data synchronized with the first visual feature sequence;

[0009] According to the first visual feature sequence, the visual feature matching offset of adjacent frames is calculated, and it is compared with the radar azimuth change trend in the first multi-source state data, when the two are inconsistent in the preset time window, the visual drift judgment data is generated;

[0010] According to the visual drift judgment data, the radar position detection data and AIS identification data in the first multi-source state data are called to construct a candidate target spatial position set, and distance convergence screening and spatial overlap constraint processing of geometric projection range are performed on it to generate a position candidate set;

[0011] According to the position candidate set and the target predicted position obtained according to the first visual feature sequence, motion direction continuity comparison is performed, and target position data cross-checked by spatial position is generated;

[0012] According to the target position data cross-checked by spatial position, a flight path control instruction for controlling the flight direction and speed of the unmanned aerial vehicle is generated, and the subsequent flight path of the unmanned aerial vehicle is adjusted according to the flight path control instruction.

[0013] Preferably, according to the first visual feature sequence, the visual feature matching offset of adjacent frames is calculated, and it is compared with the radar azimuth change trend in the first multi-source state data, when the two are inconsistent in the preset time window, the visual drift judgment data is generated, including:

[0014] According to the first visual feature sequence, the texture distribution, brightness gradient and edge stability difference of adjacent frames are extracted to form visual change basic data;

[0015] According to the visual change basic data, continuity analysis processing is performed on texture change, brightness change and edge change to generate visual feature change trend data;

[0016] According to the visual feature change trend data, time alignment and trend direction comparison processing are performed on it and the radar azimuth change trend in the first multi-source state data to generate trend comparison basic data;

[0017] According to the trend comparison basic data, when the trend direction consistency is continuously deviated, the trend amplitude coincidence is continuously deviated, or the trend stability is continuously fluctuated, the visual drift judgment data is generated.

[0018] Preferably, according to the visual drift judgment data, the radar position detection data and AIS identification data in the first multi-source state data are called to construct a candidate target spatial position set, and distance convergence screening and spatial overlap constraint processing of geometric projection range are performed on it to generate a position candidate set, including:

[0019] According to the visual drift judgment data, the position information corresponding to the radar position detection data and the AIS identification data is extracted from the first multi-source state data to form an initial spatial candidate position set;

[0020] According to the initial spatial candidate position set, a multi-round convergence processing is performed according to the distance difference between the initial spatial candidate position set and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced, and a first candidate position set after distance convergence is generated;

[0021] According to the first candidate position set after distance convergence, the first candidate position set after distance convergence is projected to a geometric projection range based on the unmanned aerial vehicle attitude data, and a spatial overlap analysis processing is performed on the landing position to generate projection overlap basic data;

[0022] According to the projection overlap basic data, the candidate positions that do not form effective overlap with the geometric projection range are removed, and the remaining candidate positions constitute a position candidate set.

[0023] Preferably, according to the position candidate set and the target prediction position obtained according to the first visual feature sequence, a motion direction continuity comparison is performed, and target position data subjected to spatial position cross-checking is generated, including:

[0024] According to the first visual feature sequence, the direction change, position change amplitude and direction stability of the target prediction position in consecutive frames are extracted to generate target prediction motion feature data;

[0025] According to the position candidate set, the direction change trend, movement distance change and direction smoothness of the candidate position in consecutive records are extracted to generate candidate spatial motion feature data;

[0026] According to the target prediction motion feature data and the candidate spatial motion feature data, a direction consistency comparison, a change amplitude coincidence comparison and a trend smoothness comparison are performed on the two to generate motion feature comparison basic data containing direction consistency results, amplitude coincidence results and trend smoothness results;

[0027] According to the motion feature comparison basic data, the direction consistency results, the amplitude coincidence results and the trend smoothness results are jointly scored to generate comprehensive coincidence data for representing the matching degree of candidate motion trend;

[0028] According to the comprehensive coincidence data, the position with the maximum comprehensive coincidence data is selected from the position candidate set to generate the target position data subjected to spatial position cross-checking.

[0029] Preferably, according to the visual change basic data, a continuity analysis processing is performed on the texture change, the brightness change and the edge change to generate visual feature change trend data, including:

[0030] According to the visual change basis data, the texture change data, the brightness change data and the edge change data are respectively composed into change sequences in time sequence to form multi-channel change sequence data for continuity analysis;

[0031] According to the multi-channel change sequence data, the change direction, the change amplitude and the change stable interval of each channel change sequence are analyzed and processed in sections to generate continuity analysis result data of multiple channels;

[0032] According to the continuity analysis result data, the change direction consistency, the change rhythm synchronization and the stable interval overlap degree of multiple channels are analyzed and processed comprehensively to generate visual feature change trend data.

[0033] Preferably, according to the initial spatial candidate position set, a plurality of rounds of convergence processing are performed according to the distance difference between the candidate positions and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced, and a first candidate position set after distance convergence is generated, including:

[0034] According to the initial spatial candidate position set, the distance difference between each candidate position and the current position of the unmanned aerial vehicle is calculated to generate distance difference basis data for convergence processing;

[0035] According to the distance difference basis data, a first round of distance screening processing is performed according to a preset distance convergence interval, and candidate positions exceeding the first convergence interval are removed to generate first round screening candidate data, and the distance convergence interval includes a first convergence interval, a second convergence interval and a final convergence interval;

[0036] According to the first round screening candidate data, a step-by-step distance reduction processing is performed according to the second convergence interval, so that the distance range of the candidate positions is gradually narrowed, and second round screening candidate data is generated;

[0037] According to the second round screening candidate data, a final distance screening processing is performed according to the final convergence interval to generate the first candidate position set after distance convergence.

[0038] Preferably, according to the target predicted motion feature data and the candidate spatial motion feature data, the direction consistency comparison, the change amplitude coincidence comparison and the trend smoothness comparison are performed on the two to generate motion feature comparison basis data including direction consistency result, amplitude coincidence result and trend smoothness result, including:

[0039] According to the target predicted motion feature data and the candidate spatial motion feature data, the direction change trend of the two in the continuous period is analyzed and matched to generate direction consistency comparison result data;

[0040] According to the target predicted motion feature data and the candidate space motion feature data, the change step difference of the two in the position change amplitude is subjected to amplitude interval analysis processing to generate change amplitude coincidence comparison result data;

[0041] According to the target predicted motion feature data and the candidate space motion feature data, the trend fluctuation analysis processing is performed on the smoothness of the direction change to generate trend smoothness comparison result data;

[0042] According to the direction consistency comparison result data, the change amplitude coincidence comparison result data and the trend smoothness comparison result data, the three types of comparison results are combined to form motion feature comparison basis data.

[0043] In a second aspect, a multi-source information fusion unmanned aerial vehicle sea target tracking system, the system comprises:

[0044] A visual feature acquisition module is configured to acquire consecutive image frames as image data, and extract texture distribution, brightness gradient and edge stability of a target region from the image data to generate a first visual feature sequence;

[0045] A multi-source state acquisition module is configured to acquire unmanned aerial vehicle attitude data, radar position detection data and AIS identification data, and perform attitude disturbance correction to align the data on a time axis to generate first multi-source state data synchronized with the first visual feature sequence;

[0046] A visual drift determination module is configured to calculate a visual feature matching offset between adjacent frames according to the first visual feature sequence, and compare the offset with a radar direction change trend in the first multi-source state data, and when the two are inconsistent within a preset time window, generate visual drift determination data;

[0047] A candidate position construction module is configured to construct a candidate target space position set according to the visual drift determination data, and call radar position detection data and AIS identification data in the first multi-source state data to perform distance convergence screening and spatial overlap constraint processing of geometric projection range to generate a position candidate set;

[0048] A spatial position verification module is configured to perform motion direction continuity comparison according to the position candidate set and a target predicted position obtained from the first visual feature sequence, and generate target position data verified by spatial position intersection;

[0049] A visual update module is configured to generate a flight path control instruction for controlling the flight direction and speed of the unmanned aerial vehicle according to the target position data verified by spatial position intersection, and adjust the subsequent flight path of the unmanned aerial vehicle according to the flight path control instruction.

[0050] The above-mentioned scheme of the present application at least has the following beneficial effects:

[0051] By extracting the texture distribution, brightness gradient and edge stability of the target region from the continuous image frames, a visual feature sequence containing multi-dimensional representations is formed. Compared with using only color or simple appearance features of templates, the application can more comprehensively reflect the local structure and contrast changes of the target in the complex sea surface background, so that the unmanned aerial vehicle can obtain more stable target visual field reference in the scene of sea surface reflection, spray interference and the like, and provide a more reliable visual basis for subsequent path adjustment.

[0052] By performing attitude disturbance correction and time axis alignment processing on the unmanned aerial vehicle attitude data, radar position detection data and AIS identification data, the data of different sources are organized according to the same time reference, and multi-source state data synchronized with the visual feature sequence is constructed. This method can reduce the influence of unmanned aerial vehicle pitch, roll and other attitude changes on target position judgment, and the target reference information used for path control is no longer dependent on the visual channel, but is based on the coordination and consistency of multi-source data, providing a more stable environment and target reference for subsequent path correction.

[0053] By comparing the visual feature matching offset obtained from the visual feature sequence with the radar azimuth change trend in the multi-source state data, a drift determination result is generated when the two are inconsistent within a preset time window, so that the system can timely identify that the target reference has deviated when the visual reference is affected by strong sea surface reflection or local shielding. This drift determination is no longer limited to internal consistency detection of the image, but uses non-visual channels as a reference, improving the reliability of the reference point in the path control stage, and reducing the path deviation caused by incorrect visual position being directly used for heading adjustment.

[0054] By constructing a candidate target space position set using radar position detection data and AIS identification data when drift is determined to exist, and gradually narrowing the candidate range by combining distance convergence screening and spatial overlap constraints of geometric projection range, the unmanned aerial vehicle can still obtain a target space region matching its flight attitude and detection range when the vision is unreliable. The candidate set provides alternative reference positions from multi-source perception for path planning, which helps to maintain a relatively stable target space guide when the vision is disturbed.

[0055] By comparing the position candidate set with the target predicted position obtained based on the visual feature sequence in terms of motion direction continuity, the target position data subjected to spatial position cross-checking is generated, so that the target position finally used for flight path adjustment not only conforms to the multi-source spatial detection result, but also maintains continuity with the existing motion trend. Furthermore, the flight path control instruction for controlling the flight direction and speed of the unmanned aerial vehicle is generated by taking the checked target position data as input, and the subsequent flight path is adjusted accordingly, so that the unmanned aerial vehicle can complete the flight path correction in the continuous chain of identifying visual drift, reconstructing the target spatial reference and updating the flight direction in the environment of strong reflection on the sea surface, splashes disturbance or attitude fluctuation, thereby improving the stability of the flight path and the reliability of the follow-up control in the sea task. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flow chart of the multi-source information fusion unmanned aerial vehicle sea target tracking method provided by the embodiment of the present application. DETAILED DESCRIPTION

[0057] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0058] As Figure 1 shown, the embodiment of the present application proposes a multi-source information fusion unmanned aerial vehicle sea target tracking method, which comprises:

[0059] acquiring continuous image frames as image data, and extracting texture distribution, brightness gradient and edge stability of the target region from the image data to generate a first visual feature sequence;

[0060] acquiring unmanned aerial vehicle attitude data, radar position detection data and AIS recognition data, and performing attitude disturbance correction to align the data on the time axis to generate first multi-source state data synchronized with the first visual feature sequence;

[0061] calculating the visual feature matching offset of adjacent frames according to the first visual feature sequence, and comparing it with the radar azimuth change trend in the first multi-source state data, when the two are continuously inconsistent within a preset time window, generating visual drift judgment data;

[0062] According to the visual drift judgment data, the radar position detection data and AIS recognition data in the first multi-source state data are called to construct a candidate target spatial position set, and distance convergence screening and spatial overlap constraint processing of geometric projection range are performed on the candidate target spatial position set to generate a position candidate set;

[0063] Perform motion direction continuity comparison according to the position candidate set and the target predicted position obtained according to the first visual feature sequence, and generate target position data that is cross-checked in spatial position;

[0064] According to the target position data cross-checked in spatial position, generate a flight path control instruction for controlling the flight direction and speed of the unmanned aerial vehicle, and adjust the subsequent flight path of the unmanned aerial vehicle according to the flight path control instruction.

[0065] In the embodiments of the present application, by extracting multi-dimensional features such as texture, brightness and edge from consecutive image frames, a visual sequence reflecting the relative position change of the target in the field of view of the unmanned aerial vehicle can be obtained, so that the unmanned aerial vehicle has a basic visual reference point when following the flight path. The visual reference can provide initial direction information for subsequent path adjustment, thereby ensuring that the unmanned aerial vehicle can continuously advance in the target direction under normal lighting conditions.

[0066] By performing attitude disturbance correction and time alignment processing on the attitude data of the unmanned aerial vehicle, the radar position detection data and the AIS recognition data, the multi-source data is kept consistent on the time axis. This operation can eliminate the pose error caused by the unmanned aerial vehicle's own pitch, roll and yaw, and provide a target position information basis that is more consistent with the real space for the flight path planning of the unmanned aerial vehicle, which helps to improve the reliability of the path control command.

[0067] By comparing the visual feature matching offset and the radar bearing change trend, when the visual and non-visual information appear inconsistent relationship in consecutive time periods, the drift of the visual reference point can be identified in time. This drift identification mechanism can serve as an early trigger basis for judging the risk of flight path deviation, so that the unmanned aerial vehicle has the starting condition for path correction when flying in the visual interference area, thereby avoiding the deviation of the flight direction caused by visual deviation.

[0068] When the drift risk occurs, a candidate spatial position set is constructed by calling the radar position and AIS recognition information, and the candidate range is gradually narrowed by using the distance convergence and geometric projection overlap conditions, so that the unmanned aerial vehicle can still obtain the reliable regional position of the target in space under the condition of visual misalignment. This method helps the unmanned aerial vehicle to maintain effective cognition of the target position in complex sea conditions, and provides a stable reference for maintaining the continuity of the flight path.

[0069] By comparing the constrained position candidate and the target predicted position in motion direction and change trend, and generating cross-checked target position data, the unmanned aerial vehicle can select a target position that is more consistent with the actual motion trajectory from the multi-source data as the path guide point. The checking result can reduce the direction deviation caused by visual error in path calculation, and improve the stability of the flight path planning.

[0070] Finally, the flight direction and speed of the track control instruction are generated by using the checked target position, and the subsequent flight path of the unmanned aerial vehicle is adjusted accordingly, so that the unmanned aerial vehicle can still maintain stable propulsion in the target direction under the conditions of disturbed vision, strong sea surface reflection or attitude fluctuation. The method realizes continuous closed loop of path deviation identification, reference point reconstruction and track adjustment through multi-source fusion, so that the unmanned aerial vehicle can maintain reliable flight path control capability in the sea task scene.

[0071] In a preferred embodiment of the present application, consecutive image frames are acquired as image data, and the texture distribution, brightness gradient and edge stability of the target region are extracted from the image data to generate a first visual feature sequence, specifically including:

[0072] Firstly, the optical image acquisition device mounted on the unmanned aerial vehicle continuously acquires sea surface monitoring images at a fixed frame rate, and pre-processes multiple time-adjacent image frames. Through performing brightness equalization, local contrast enhancement and noise suppression in the image preprocessing stage, the interference factors such as sea surface reflection and local flicker in the image are preliminarily alleviated, so as to stably extract visual features subsequently.

[0073] In the pre-processed image, the target region is determined by region detection, and multi-dimensional visual features are further extracted from the region. The texture distribution can be obtained by analyzing the density and directionality of the gray scale change in the region, so that the unmanned aerial vehicle can identify the structure distribution of the target surface; the brightness gradient can be obtained by comparing the change trend of the brightness difference between adjacent pixels in the region, which is used to reflect the brightness contrast relationship between the target boundary and the background; the edge stability can be represented by detecting the consistency of the edge position change in consecutive image frames, which is used to reflect whether the edge of the target in the image remains stable.

[0074] Subsequently, the texture features, brightness features and edge features are combined into a multi-channel visual feature sequence in the time sequence of the image frames, thereby generating the first visual feature sequence. The visual feature sequence not only retains the local structure information of the target in a single frame, but also reflects the change trend of the target in consecutive frames, so that the unmanned aerial vehicle has a basic and continuous visual reference when performing subsequent track adjustment.

[0075] In a preferred embodiment of the present application, the unmanned aerial vehicle attitude data, radar position detection data and AIS recognition data are acquired, and attitude disturbance correction is performed to align the data on the time axis, thereby generating first multi-source state data synchronized with the first visual feature sequence, specifically including:

[0076] Firstly, continuously acquire the attitude data generated by the UAV in real time during flight, including pitch angle, roll angle, yaw angle and other attitude parameters; at the same time, acquire the detection information of the target by the radar, such as distance and azimuth angle, and the information returned by the AIS identification device, such as target identity and corresponding spatial coordinates. The above data are recorded according to the original time stamp generated, so as to perform time axis calibration in the subsequent steps.

[0077] Since the UAV will be affected by wind, wave, air flow disturbance and other factors when flying over the sea, its attitude parameters will change frequently. In order to avoid the deviation of the spatial positioning of the radar and AIS caused by these attitude changes, it is necessary to correct the spatial parameters of the radar and AIS based on real-time attitude data. This correction process can calculate the direction after attitude compensation by analyzing the attitude change trend and combining the relative position relationship of the radar and AIS identification device installed on the UAV, so that the radar and AIS data can reflect the actual spatial direction of the target.

[0078] After completing the attitude disturbance correction, the corrected attitude data, radar data and AIS data need to be aligned on the time axis according to the time stamp. The alignment method is to find the closest time point on the time axis, and generate data consistent with the visual frame time through interpolation method when necessary, so that the data from different sources have corresponding values at the same time point.

[0079] Finally, the aligned attitude data, radar data and AIS data are combined in time sequence to generate the first multi-source state data synchronized with the first visual feature sequence. This synchronized data provides a unified multi-source basis for subsequent judgment of visual drift, construction of spatial candidate position and generation of track control instruction.

[0080] In a preferred embodiment of the present application, according to the target position data verified by spatial position cross-checking, a track control instruction for controlling the flight direction and speed of the UAV is generated, and the subsequent flight path of the UAV is adjusted according to the track control instruction, which specifically includes:

[0081] Firstly, the target position data verified by spatial position cross-checking has been direction consistency compared and trend smoothness verified based on target prediction trend and multi-source spatial candidate trend, so it can be used as a more reliable target spatial position for track adjustment. According to the spatial position, the current position difference, direction difference and speed difference of the UAV relative to the target are determined, so as to calculate the direction in which the UAV should advance and the size of the advancing speed.

[0082] In the calculation of the flight direction, the angle range that the UAV needs to adjust is obtained by comparing the direction of the line connecting the current position of the UAV and the target position after the check. By analyzing the change of the angle in the continuous time period, a smooth reference for controlling the stability of the direction can be obtained, so that the UAV will not produce unnecessary violent maneuvering due to instantaneous direction change. In the calculation of the flight speed, the speed adjustment range is determined according to the distance change trend between the UAV and the target, so that the UAV can maintain continuous approach to the target without sudden speed increase or decrease.

[0083] Subsequently, the direction adjustment amount and the speed adjustment amount obtained by planning are converted into instructions that can directly act on the flight control of the UAV, including parameters such as expected yaw direction and expected forward speed. The flight control executes corresponding power output adjustment according to these parameters, so that the UAV gradually corrects the flight path.

[0084] Finally, the UAV executes direction and speed adjustment according to the generated flight path control instructions, so that it can still steadily advance along the checked target direction in the environment of strong reflection on the sea surface, splash interference or visual deviation, and realize continuous updating and deviation correction of the path.

[0085] In a preferred embodiment of the present application, the visual feature matching offset of adjacent frames is calculated according to the first visual feature sequence, and is compared with the radar azimuth change trend in the first multi-source state data. When the two are inconsistent within a preset time window, visual drift judgment data is generated, including:

[0086] According to the first visual feature sequence, the texture distribution, brightness gradient and edge stability difference of adjacent frames are extracted to form visual change basic data;

[0087] According to the visual change basic data, continuity analysis and processing are performed on texture change, brightness change and edge change to generate visual feature change trend data;

[0088] According to the visual feature change trend data, time alignment and trend direction comparison processing are performed on the visual feature change trend data and the radar azimuth change trend in the first multi-source state data to generate trend comparison basic data;

[0089] According to the trend comparison basic data, when the trend direction consistency continuously deviates, the trend amplitude coincidence continuously deviates, or the trend stability continuously fluctuates, the visual drift judgment data is generated.

[0090] In the embodiment of the present application, by extracting the texture, brightness and edge differences of adjacent frames, the basic data reflecting the visual change characteristics can be formed, so that the unmanned aerial vehicle has a clear time dimension reference for the target shift trend in the image. On this basis, by performing continuity analysis on multiple change characteristics, the trend change of the target at the visual end can be obtained, including the change direction, change amplitude and change stability, which provides necessary information for judging whether the visual reference is reliable. When there is a significant difference between the visual trend and the radar azimuth change trend in a continuous time period, the visual reference can be identified as having shifted in time. The shift determination result is not used for simple image correction, but as a trigger basis for subsequent track deviation risk identification, so that the unmanned aerial vehicle can perceive the instability of the navigation reference point in advance, thereby avoiding the heading adjustment based on the wrong visual position. Through this trend comparison mechanism, the reliability of the reference data in the path adjustment link can be improved in a complex lighting and sea surface reflection environment, and the track deviation caused by visual misjudgment can be reduced.

[0091] In a preferred embodiment of the present application, according to the first visual feature sequence, the texture distribution, brightness gradient and edge stability difference of adjacent frames are extracted to form visual change basic data, specifically including:

[0092] First, in the texture difference extraction aspect, by comparing the detail changes of the gray scale distribution of the target area of adjacent frames, the consistency degree of the structure direction and the improvement or weakening of the texture density, the change amplitude of the texture structure between the two frames is judged. The texture change reflects whether the target surface structure has been displaced or deformed, which helps to judge the local movement trend of the target in the image.

[0093] In the brightness gradient difference extraction aspect, by comparing the brightness change patterns of adjacent frames in the target area, whether the position of the brightness boundary has shifted, whether the brightness transition zone has widened or narrowed, etc. are analyzed to obtain the change of the brightness gradient. The change of the brightness gradient can reflect the stability of the target edge in the sea surface reflection environment.

[0094] In the edge stability difference extraction aspect, by detecting the appearance position, contour continuity and boundary clarity of the target edge in consecutive frames, whether the edge appears to be jittering, missing or deforming is judged. The change of the edge stability can be used to identify the visual shift caused by spray or strong reflection.

[0095] Finally, the texture difference, brightness difference and edge difference are combined in time sequence as visual change basic data for subsequent trend analysis and visual drift judgment.

[0096] In a preferred embodiment of the present application, according to the visual feature change trend data, it is time-aligned and trend direction compared with the radar azimuth change trend in the first multi-source state data to generate trend comparison basis data, specifically including:

[0097] Firstly, the visual feature change trend data generated according to the visual change basis data is obtained, which contains the direction change, change amplitude and change stability of the visual end in the continuous time period. In order to enable the visual trend and the radar azimuth trend to be consistently compared, time axis alignment processing needs to be performed on the two types of data.

[0098] Time alignment is matched by using the time stamp of each data item, and the radar azimuth information closest to the time at the time point corresponding to the visual frame is found in the multi-source state data. In necessary cases, the radar azimuth change trend consistent with the visual time can be calculated by interpolation based on the time difference. In this way, the two types of trend data can be compared under the same time reference.

[0099] After completing the time alignment, the direction change of the visual trend is compared with the azimuth change direction of the radar trend. For example, by comparing the increase and decrease mode of the direction change, the overall offset of the direction change and the continuity of the direction change in the continuous time period, it is judged whether the visual direction change and the radar azimuth change are consistent.

[0100] At the same time, the visual change amplitude is also compared with the amplitude trend of the radar azimuth change, for example, it is judged whether the change size maintains a similar increasing or decreasing trend, and consistency analysis is performed on the stability change of the two.

[0101] Through the above direction comparison, amplitude comparison and stability comparison processing, the trend comparison basis data is finally generated, which provides a reliable basis for subsequent judgment of whether visual drift occurs.

[0102] In a preferred embodiment of the present application, according to the trend comparison basis data, when the trend direction consistency continuously deviates, the trend amplitude consistency continuously deviates, or the trend stability continuously fluctuates, visual drift judgment data is generated, specifically including:

[0103] Firstly, the trend comparison basis data generated in step three is monitored in a continuous time period. The trend comparison basis data reflects the matching of the visual trend and the radar trend in terms of direction, amplitude and stability, so it is necessary to analyze its change in a preset time window in order to identify whether the visual information appears drift.

[0104] In the aspect of direction consistency monitoring, by observing the direction deviation relationship between the visual direction trend and the radar direction trend in continuous time, it is judged whether the two are continuously deviated. For example, when the visual direction continuously deviates from the radar direction trend and presents a continuously deviated and enlarged trend, it is considered that the direction consistency is abnormal.

[0105] In the aspect of change amplitude consistency monitoring, by evaluating the synchronization of the visual change amplitude and the radar change amplitude, for example, judging whether the difference of change step is continuously enlarged and whether the change rhythm gradually loses the corresponding relationship. When the change amplitude appears continuous deviation, it can be considered as a possible drift signal.

[0106] In the aspect of trend stability monitoring, by analyzing whether the visual end change stability interval appears continuous fluctuation, such as stability decline, volatility of direction change obviously enhanced and the like, these can represent that the visual end is affected by sea surface reflection, splash interference or shielding.

[0107] When any of the above deviations continuously exists in the preset time window, visual drift judgment data can be generated to serve as the basis for subsequent construction of alternative spatial reference and start of track adjustment.

[0108] In a preferred embodiment of the present application, according to the visual drift judgment data, the radar position detection data and AIS recognition data in the first multi-source state data are called to construct a candidate target spatial position set, and distance convergence screening and spatial overlap constraint processing of geometric projection range are performed on it to generate a position candidate set, including:

[0109] According to the visual drift judgment data, the position information corresponding to the radar position detection data and the AIS recognition data in the first multi-source state data is extracted to form an initial spatial candidate position set;

[0110] According to the initial spatial candidate position set, multi-round convergence processing is performed according to the distance difference between it and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced to generate a first candidate position set after distance convergence;

[0111] According to the first candidate position set after distance convergence, it is projected to the geometric projection range based on the unmanned aerial vehicle attitude data, and spatial overlap analysis processing is performed on the landing position to generate projection overlap basic data;

[0112] According to the projection overlap basic data, the candidate positions which do not form effective overlap with the geometric projection range are eliminated, and the remaining candidate positions constitute the position candidate set.

[0113] In the embodiment of the present application, when the visual reference is determined to be unreliable, a new candidate target spatial position set can be constructed by extracting spatial information from the radar position and AIS identification content, providing an alternative track reference source for the UAV. Through multi-round distance convergence processing, the candidate space is gradually reduced, which helps to filter out the faraway areas that do not have actual possibilities, so that the UAV pays more attention to the target position range with spatial rationality when planning the path. On the basis of the candidate position after distance convergence, the candidate points are projected to the geometric projection range obtained according to the UAV attitude, and the positions that cannot be projected into the field of view are removed through spatial overlap analysis, so that the final remaining candidate set is closer to the real target area that can be observed under the current flight conditions of the UAV. This process ensures that the UAV still has a stable spatial reference in the case of visual drift, so that the subsequent track planning can be based on a reliable position set, and the heading adjustment does not depend on a single data source, thereby reducing the risk of path deviation caused by field of view interference or attitude change.

[0114] In a preferred embodiment of the present application, according to the visual drift determination data, the position information corresponding to the radar position detection data and the AIS identification data is extracted from the first multi-source state data to form an initial spatial candidate position set, which specifically includes:

[0115] Firstly, after detecting the visual drift determination data at the visual end, the radar position detection data and the AIS identification data corresponding to the visual drift time point are called from the first multi-source state data generated before. The radar data usually includes the azimuth angle and distance information of the target relative to the UAV, and the AIS data includes the latitude and longitude coordinates, heading and speed information broadcasted by the target's own device.

[0116] When processing the radar data, the corrected azimuth angle measured by the radar is determined to be closer to the direction in the actual flight environment by combining the attitude correction result of the UAV. Then, according to the relative relationship between the radar distance information and the current position of the UAV, the possible area of the target in space can be calculated. The possible area can be converted based on the local coordinate system of the UAV's own position.

[0117] When processing the AIS data, the latitude and longitude coordinates of the target are corresponded with the latitude and longitude coordinates of the UAV itself to obtain the spatial position of the target in the geographic coordinate system. Since the sea target may have shaking, heading change and other situations, the possible spatial area range of the target within a short time can be calculated within the allowed time accuracy range according to the heading and speed information provided by the AIS.

[0118] Subsequently, the radar-derived position range and the AIS-derived position range are incorporated into an initial spatial candidate position set. The set contains multiple possible target spatial positions, which are used to replace the target spatial reference in the case of visual misalignment, and provide alternative basis for subsequent path adjustment.

[0119] In a preferred embodiment of the present application, according to the first candidate position set after distance convergence, the first candidate position set is projected into the geometric projection range based on the UAV attitude data, and spatial overlap analysis processing is performed on the landing position to generate projection overlap basis data, specifically including:

[0120] First, the first candidate position set after distance convergence processing is obtained, which retains candidate positions with reasonable relative distances from the UAV. In order to determine whether these candidate positions meet the current observable field of view range of the UAV, the candidate positions need to be projected into the geometric projection space determined by the UAV attitude data.

[0121] The UAV attitude data includes parameters such as pitch angle, roll angle, and yaw angle, which can be used to calculate the current field of view direction and frustum range of the UAV camera. According to this frustum range, the geometric projection range that the UAV can observe in the current attitude can be constructed. This range describes the visible area of the UAV camera in space.

[0122] The geometric relationship between each spatial position in the first candidate position set and the current field of view direction of the UAV is determined, i.e., by analyzing whether the candidate point can form a reasonable projection landing point in the frustum. If the spatial position of the candidate point is within the field of view range, it is considered that the position can be observed by the UAV camera.

[0123] Subsequently, spatial overlap analysis is performed on all candidate positions that can form landing points within the field of view range. Spatial overlap analysis is used to determine whether the candidate position forms an overlapping region with the field of view region predicted by the visual feature in the projection range, thereby determining whether the candidate point meets the spatial position that can be observed by the visual end.

[0124] Finally, all positions with effective projection overlap relationship are generated into projection overlap basis data, which provides the basis for constructing the position candidate set for further screening.

[0125] In a preferred embodiment of the present application, according to the projection overlap basis data, candidate positions that do not form effective overlap with the geometric projection range are removed, and the remaining candidate positions constitute the position candidate set, specifically including:

[0126] First, in the screening process, it is determined whether the candidate position meets one of the following conditions:

[0127] The candidate position can form a stable landing point within the geometric projection range, indicating that the position is within the spatial range allowed by the UAV field of view;

[0128] The candidate position and the target prediction region have obvious overlap on the projection plane, indicating that the position is consistent with the region that the vision end can perceive;

[0129] The candidate position maintains stability matching the field of view range within a continuous time period, indicating that it has the possibility to become the target real position.

[0130] Positions that do not meet the above conditions are excluded from the candidate set, for example: points that deviate significantly from the direction of the UAV field of view, points that cannot form an effective landing point, points with very low or almost no overlap in projection overlap. Through this screening process, a target candidate position set that is more consistent with the real observation conditions of the UAV and has spatial rationality can be obtained.

[0131] Finally, the remaining candidate positions constitute the position candidate set, which will be the core reference for subsequent motion trend comparison and track adjustment.

[0132] In a preferred embodiment of the present application, motion direction continuity comparison is performed according to the position candidate set and the target prediction position obtained from the first visual feature sequence, and spatial position cross-verified target position data is generated, including:

[0133] According to the first visual feature sequence, the direction change, position change amplitude and direction stability of the target prediction position in consecutive frames are extracted to generate target prediction motion feature data;

[0134] According to the position candidate set, the direction change trend, movement distance change and direction smoothness of the candidate position in consecutive records are extracted to generate candidate spatial motion feature data;

[0135] According to the target prediction motion feature data and the candidate spatial motion feature data, direction consistency comparison, change amplitude consistency comparison and trend smoothness comparison are performed on the two to generate motion feature comparison basis data containing direction consistency results, amplitude consistency results and trend smoothness results;

[0136] According to the motion feature comparison basis data, joint scoring processing is performed on the direction consistency results, amplitude consistency results and trend smoothness results to generate comprehensive consistency data for representing the matching degree of candidate motion trend;

[0137] According to the comprehensive consistency data, the position with the maximum comprehensive consistency data is selected from the position candidate set to generate spatial position cross-verified target position data.

[0138] In the embodiment of the present application, by analyzing the direction change, amplitude change and direction stability of the target predicted position, the target motion trend inferred by the unmanned aerial vehicle based on the visual sequence can be obtained; at the same time, by analyzing the direction trend, movement distance change and direction smoothness of the candidate spatial position, the spatial motion trend inferred based on multi-source information can be obtained. By comparing the direction consistency, amplitude coincidence and trend smoothness of the above two trends, the degree of conformity of the target prediction and the spatial reference in the target motion can be judged from multiple dimensions, so as to evaluate the credibility of each candidate position as a track reference point. After obtaining multiple comparison results, by comprehensively processing the comparison results, the spatial position data closer to the real target motion can be obtained, providing a reliable reference position for the unmanned aerial vehicle to adjust the track. Through the motion trend comparison and checking mechanism, the unmanned aerial vehicle no longer relies on the instantaneous position of a single channel in the path updating process, but determines a more reliable track guide point based on the continuous motion law of each channel, which helps to improve the flight path stability under the conditions of reflection, shielding or sea surface disturbance.

[0139] In a preferred embodiment of the present application, according to the first visual feature sequence, the direction change, position change amplitude and direction stability of the target predicted position in the continuous frames are extracted to generate target predicted motion feature data, specifically including:

[0140] Firstly, by analyzing the movement of the texture distribution position, brightness boundary position and edge structure position and other features of the target region in the continuous frames, the reference position point of the target in each frame is determined. The reference position point can be represented as the center position or other stable structure point of the target in the image, which is used to construct the visual position record between the continuous frames.

[0141] In the aspect of direction change extraction, by comparing the movement direction of the reference position points in the adjacent frames in the image coordinates, the direction trend of the reference position points in the continuous frames is analyzed, for example, whether the reference points move left or right in the horizontal direction, whether they move up or down in the vertical direction, and the direction trend formed by these direction changes is recorded.

[0142] In the aspect of position change amplitude extraction, by comparing the position difference of the reference position points in the continuous frames, the displacement amplitude of the target in the image is determined, for example, how many pixels the target moves in the adjacent time period, whether the movement speed accelerates or decelerates, etc. These changes can be used to represent the overall speed of the target motion.

[0143] In the aspect of direction stability extraction, by observing whether the direction changes of multiple frames remain consistent, for example, whether the target continuously moves in a similar direction in the continuous multiple frames, or whether there are direction jumps, direction jitter and other phenomena. The stability description can reflect whether the target prediction result is reliable.

[0144] Finally, the direction change data, the amplitude change data and the direction stability data are combined in time sequence to form the target predicted motion feature data, which provides a visual motion reference for subsequent motion trend comparison.

[0145] In a preferred embodiment of the present application, according to the position candidate set, the direction change trend, the movement distance change and the direction smoothness of the candidate position in the continuous record are extracted to generate the candidate spatial motion feature data, specifically including:

[0146] Firstly, the candidate positions in the position candidate set are arranged in time sequence for analyzing the motion trend of the candidate spatial position under multi-source perception. Each candidate position is derived based on the radar bearing, the AIS heading and the projection overlap result, and thus can reflect the possible movement direction of the target in space.

[0147] In the direction change trend extraction aspect, the direction change trend of the candidate position in the three-dimensional space is determined by comparing the spatial relative position change of the candidate position in the continuous time period, such as whether the target moves forward, sideways or backward relative to the UAV, and whether the direction presents a consistent trend is analyzed.

[0148] In the movement distance change extraction aspect, the distance change of the candidate position in the continuous record is compared, such as whether the distance between the target and the UAV continuously decreases, increases or remains relatively stable, to represent the acceleration, deceleration or holding state of the candidate position in the spatial motion.

[0149] In the direction smoothness extraction aspect, whether the candidate position presents a smooth movement in the time sequence is judged by analyzing the continuity of the direction change curve. For example, if the direction change is stable in multiple continuous time periods, it can be considered that the candidate position has high smoothness, which represents that the motion trend is more consistent with the normal motion characteristics of the target.

[0150] Finally, the direction change trend, the movement distance change and the direction smoothness are combined as the candidate spatial motion feature data according to the record time, which provides a multi-source spatial reference for subsequent trend matching and target position checking.

[0151] In a preferred embodiment of the present application, according to the comprehensive coincidence data, the position with the maximum comprehensive coincidence data is selected from the position candidate set to generate the target position data checked by the spatial position cross-checking, specifically including:

[0152] Firstly, the comprehensive coincidence data is obtained by combining the direction consistency, the amplitude coincidence and the trend smoothness, and is used to measure the fitting degree of the motion trend of a candidate position and the target predicted trend.

[0153] The comprehensive coincidence corresponding to each candidate position is counted, and whether each candidate position has a high matching degree in the three types of alignment indexes is analyzed. For example, if a certain candidate position has consistent direction, amplitude matching and stable trend at the same time, the comprehensive coincidence is higher.

[0154] Subsequently, all candidate positions are sorted in descending order of comprehensive coincidence, and a candidate position with the highest comprehensive coincidence is selected. The position represents the most reliable position under the joint inspection of visual trend and multi-source spatial trend.

[0155] Finally, the candidate position is output as target position data cross-checked by spatial position, which is used to generate subsequent flight control instructions of the unmanned aerial vehicle, so that the unmanned aerial vehicle can perform more stable and reliable path adjustment according to the checked target position.

[0156] In a preferred embodiment of the present application, according to the visual change basic data, continuity analysis and processing are performed on texture change, brightness change and edge change to generate visual feature change trend data, including:

[0157] According to the visual change basic data, the texture change data, the brightness change data and the edge change data are respectively arranged in time sequence to form change sequences, and multi-channel change sequence data for continuity analysis is formed;

[0158] According to the multi-channel change sequence data, the change direction, the change amplitude and the change stable interval of each channel change sequence are analyzed and processed in segments to generate continuity analysis result data of multiple channels;

[0159] According to the continuity analysis result data, comprehensive analysis and processing are performed on the change direction consistency, the change rhythm synchronization and the stable interval overlap degree of multiple channels to generate visual feature change trend data.

[0160] In the embodiment of the present application, by constructing the texture change, brightness change and edge change into multi-channel time sequence data respectively, the unmanned aerial vehicle can obtain the continuous record of the change of the target in the image from the visual end. On this basis, the direction change, amplitude and stable interval of each channel are analyzed segment by segment, and the dynamic behavior characteristics of the target from the visual end can be clearly reflected. Through the comprehensive analysis of the direction consistency, rhythm synchronization and stable interval overlap between multiple channels, more reliable visual feature change trend data can be formed for judging the reliability of the visual reference point. The trend data can be used as a reference basis for the flight path control, so that the unmanned aerial vehicle can identify whether the visual information is affected by the sea surface reflection or the spray disturbance before adjusting the flight direction. Through the multi-channel continuity analysis, the unmanned aerial vehicle can avoid generating incorrect heading instructions due to mistaking the instantaneous visual deviation as the real movement trend during the flight path adjustment stage, thereby maintaining the stability of the path adjustment process.

[0161] In a preferred embodiment of the present application, according to the visual change basic data, the texture change data, the brightness change data and the edge change data are respectively arranged in time sequence to form a multi-channel change sequence data for continuity analysis, which specifically includes:

[0162] Firstly, for each visual feature (texture, brightness and edge), the change result is recorded as an item according to the change amount calculated in the adjacent frames, and arranged in time sequence.

[0163] When constructing the texture change sequence, the change amount of the texture structure difference between the continuous frames is recorded, such as the increase or decrease of the texture density, whether the texture direction is shifted, etc. Through these continuous records, the texture change sequence reflecting the change trend of the target surface structure can be formed.

[0164] When constructing the brightness change sequence, the change amount of the brightness gradient difference is arranged in time sequence, and the enhancement or weakening of the brightness boundary of the target area and the position change of the brightness transition zone are recorded, which are used to represent the brightness stability of the target under the influence of light.

[0165] When constructing the edge change sequence, the features such as the edge position movement, edge continuity and edge definition change are arranged in time sequence, which are used to identify the stability of the visual edge structure.

[0166] Finally, the above three types of features are respectively composed into independent time sequences to form the multi-channel change sequence data, which provides the basis for the subsequent continuity analysis.

[0167] In a preferred embodiment of the present application, the change direction, change amplitude and change stable interval of each channel change sequence are analyzed segment by segment according to the multi-channel change sequence data, and continuous analysis result data of multiple channels is generated, specifically including:

[0168] Firstly, the target of continuity analysis is to identify change patterns and trend behaviors from time series, so as to judge whether the dynamic characteristics of the target presented in the visual end are stable.

[0169] For change direction analysis, by checking the positive and negative trends of each change amount in the sequence, it is judged whether the texture change, brightness gradient change and edge position change of the target in the image are continuously enhanced, continuously weakened or present periodic fluctuations. For example, if the brightness change amount shows an enhancing trend in multiple consecutive frames, it can be considered that the brightness direction is stably changing upward.

[0170] For change amplitude analysis, by comparing the size difference of consecutive change amounts, it is identified whether the change presents a gradually expanding, gradually slowing down or basically consistent trend, such as whether the texture change gradually enhances, whether the edge jitter amplitude becomes larger, etc.

[0171] For change stable interval analysis, by identifying the relatively stable sections in the sequence, such as the consecutive multiple frames of change amounts remaining in a similar range, it can be considered as a stable interval, and vice versa.

[0172] Finally, by segmentally analyzing the change direction, amplitude and stable interval of the three channels (texture, brightness and edge), the corresponding continuity analysis result data of each channel is generated, which provides detailed trend basis for multi-channel comprehensive analysis.

[0173] In a preferred embodiment of the present application, according to the continuity analysis result data, the change direction consistency, change rhythm synchronization and stable interval overlap degree of multiple channels are analyzed comprehensively, and visual feature change trend data is generated, specifically including:

[0174] Firstly, common trends are extracted from three different sources of change sequences to judge the stability degree of visual reference points.

[0175] In terms of change direction consistency analysis, whether the direction trends of the three channels in the same time period are consistent is compared. For example, when the texture change, brightness change and edge change all show an enhancing trend, it can be considered as consistent in direction; if the direction of any channel is completely opposite, it can be considered as inconsistent in direction.

[0176] In the aspect of rhythm synchronization analysis, the rhythm of the three channels is compared to see whether the amplitude increases or decreases at the same time period, whether the speed of the change is similar, etc. When the different channels present similar rhythm, it means that the visual change has synchronization.

[0177] In the aspect of stable interval overlap analysis, the stable intervals of each channel are compared to see whether there is obvious overlap on the time axis, e.g. when the texture, brightness and edge remain stable in the same time period, it means that the visual performance of this time period is more reliable.

[0178] Finally, the evaluation results of the direction consistency, rhythm synchronization and stable interval overlap are comprehensively processed to form complete visual feature change trend data. The trend data is used for subsequent motion trend comparison and visual reference reliability judgment, and provides continuous, stable and interpretable visual basis for the track control.

[0179] In a preferred embodiment of the present application, according to the initial spatial candidate position set, a plurality of rounds of convergence processing are performed according to the distance difference between the candidate positions and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced, and a first candidate position set after distance convergence is generated.

[0180] According to the initial spatial candidate position set, the distance difference between each candidate position and the current position of the unmanned aerial vehicle is calculated to generate distance difference basic data for convergence processing.

[0181] According to the distance difference basic data, a first round of distance screening processing is performed according to a preset distance convergence interval, candidate positions exceeding the first convergence interval are removed, first round screening candidate data is generated, and the distance convergence interval includes a first convergence interval, a second convergence interval and a final convergence interval.

[0182] According to the first round screening candidate data, a step-by-step distance reduction processing is performed according to the second convergence interval, so that the distance range of the candidate positions is gradually narrowed, and second round screening candidate data is generated.

[0183] According to the second round screening candidate data, a final distance screening processing is performed according to the final convergence interval to generate the first candidate position set after distance convergence.

[0184] In the embodiment of the present application, by calculating the distance difference between each candidate position and the current position of the UAV, distance difference basic data for phased convergence can be obtained. The first convergence interval is used for the first round of screening, which can quickly eliminate obviously unsuitable remote areas as a reference, making the candidate range more concentrated. Then the selection range is further narrowed through the second convergence interval, making the candidate area gradually approach the actual possible space area of the target. Finally, the final convergence interval is executed to perform the last level screening on the candidate position, and a high-credibility position set can be generated to provide stable spatial reference for the flight path adjustment. Through this step-by-step convergence strategy, not only the number of candidate positions is reduced, but also the remaining candidate areas are more consistent with the current flight environment and target motion of the UAV. In the flight path control, this kind of convergence processing can effectively reduce the unstable path adjustment caused by the scattered distribution of candidate points, so that the UAV can still rely on stable spatial reference to perform path correction in the case of visual deviation or perception interference.

[0185] In a preferred embodiment of the present application, according to the initial spatial candidate position set, the distance difference between each candidate position and the current position of the UAV is calculated to generate distance difference basic data for convergence processing, specifically including:

[0186] Firstly, each candidate position in the set contains its coordinate information in space, such as distance and direction relative to the UAV, latitude and longitude coordinates, or space projection coordinates, etc.

[0187] The current position of the UAV can be obtained through its navigation (such as GNSS, inertial navigation, etc.). By comparing the spatial differences between the candidate position and the current position of the UAV, the distance difference between the two is calculated. The distance difference here is not represented by a mathematical formula, but by judging the spatial interval between the two points, for example, according to the latitude and longitude difference, the azimuth angle difference, the radar echo distance difference, etc., to obtain a quantitative value representing the spatial distance between the two.

[0188] For each candidate position, a corresponding distance difference record is generated, and these records form the distance difference basic data in the order of candidate positions. The distance difference basic data will be used as the input of the subsequent distance convergence processing to gradually narrow down the position range, so that the remaining candidate positions are more consistent with the actual possible position distribution of the target.

[0189] This step ensures that the initial data entering the distance convergence has a clear spatial reference, which helps to narrow down the candidate range through multiple rounds of screening and improve the reliability of the spatial reference position in the flight path adjustment process.

[0190] In a preferred embodiment of the present application, according to the distance difference basic data, the first round of distance screening processing is performed according to the preset distance convergence interval, and the candidate positions exceeding the first convergence interval are eliminated to generate the first round of screening candidate data, specifically including:

[0191] First, a distance convergence interval is set as a preliminary screening, which is used to determine whether the candidate position is consistent with the expected target distance of the UAV. The interval can be pre-configured according to factors such as task scenarios, radar detection range, AIS positioning accuracy, etc.

[0192] The distance difference base data is read, and it is checked whether each piece of data falls within the first convergence interval. For example, when the distance between a candidate position and the UAV is far beyond the area that the UAV can theoretically observe, it means that the candidate point is not reasonable in the current scenario, and it should be excluded.

[0193] The goal of the first round of screening is to quickly exclude positions that do not obviously meet the target motion conditions, such as distant noise points, false candidate points caused by false positives, or points that do not match the radar detection range at all.

[0194] After screening, all candidate positions that fall within the interval are retained to form the first round of screening candidate data, providing input for the second round of more detailed distance convergence processing.

[0195] In a preferred embodiment of the present application, according to the first round of screening candidate data, a step-by-step distance reduction process is performed according to the second convergence interval, which gradually narrows the distance range of the candidate position, generating the second round of screening candidate data, which specifically includes:

[0196] The second convergence interval can be set to a smaller range than the first convergence interval, which is used to exclude candidate points that have not been excluded in the first round but may still deviate from the target true position. For example, in the first round, the range is generally reasonable, and in the second round, the range is closer to the target true range.

[0197] The process of step-by-step distance reduction includes:

[0198] Compare whether the distance difference of each candidate point significantly deviates from the center range of the second convergence interval;

[0199] Determine whether the distance change of the candidate point in the continuous record presents characteristics consistent with the target motion trend, such as whether it remains stably close or stably far away;

[0200] Exclude points with significant fluctuations in distance changes or obvious inconsistencies with the trend.

[0201] In this way, the candidate position set can be further narrowed, and the spatial distribution of the retained candidate points gradually concentrated, more consistent with the motion behavior of the target in continuous time.

[0202] Finally, the candidate points that meet the requirements of the second convergence interval will form the second round of screening candidate data and enter the final convergence step.

[0203] In a preferred embodiment of the present application, according to the second round of candidate data screening, the final level distance screening process is performed according to the final convergence interval to generate the first candidate position set after distance convergence, which specifically includes:

[0204] The third, and most stringent, distance interval is selected as the final convergence interval. This interval is usually dynamically determined based on the current attitude of the UAV, radar detection accuracy, AIS speed range and other parameters, and represents the core area closest to the possible location of the real target.

[0205] The final screening process will perform the following analysis on each candidate position:

[0206] Check if the distance difference falls within the final convergence interval. If a candidate point passes the first two rounds of screening, but still has a distance difference that is significantly inconsistent with the final interval, it is rejected.

[0207] Check if the continuous distance change shows a smooth or predictable trend, for example, the distance between the target and the UAV should show a change consistent with the direction of target movement.

[0208] Make a rationality judgment in combination with the task scenario, for example, whether the target can jump across such a large distance in a short time.

[0209] Finally, only the candidate points that meet the multiple convergence conditions are retained to form the first candidate position set after distance convergence.

[0210] This set represents the most likely spatial position of the real target under visual drift conditions and will be used for subsequent projection overlap analysis and track control processes.

[0211] In a preferred embodiment of the present application, according to the target predicted motion feature data and the candidate spatial motion feature data, the direction consistency comparison, the change amplitude consistency comparison and the trend smoothness comparison are performed on the two, to generate motion feature comparison basis data containing direction consistency result, amplitude consistency result and trend smoothness result, including:

[0212] According to the target predicted motion feature data and the candidate spatial motion feature data, the direction change trend of the two in the continuous period is analyzed for direction matching, to generate direction consistency comparison result data;

[0213] According to the target predicted motion feature data and the candidate spatial motion feature data, the change step difference in the position change amplitude of the two is analyzed for amplitude interval, to generate change amplitude consistency comparison result data;

[0214] According to the target predicted motion feature data and the candidate spatial motion feature data, the trend fluctuation of the direction change of the two is analyzed for trend smoothness, to generate trend smoothness comparison result data;

[0215] According to the direction consistency comparison result data, the change amplitude coincidence comparison result data and the trend smoothness comparison result data, three types of comparison results are combined to form motion feature comparison basis data.

[0216] In the embodiment of the present application, by analyzing the direction change, amplitude change and direction smoothness of the target prediction position, data reflecting the visual motion trend can be generated; at the same time, by analyzing the direction change trend, movement distance change and direction smoothness of the candidate space position, motion trend data reflecting multi-source space reference can be generated. By comparing the visual trend and the space trend in three dimensions of direction, amplitude and trend smoothness, the consistency degree of the candidate position and the target prediction in the motion law can be evaluated from multiple aspects. After the comparison results are combined, basis data for determining the reliable track reference position can be formed, so that the unmanned aerial vehicle can select a space position that conforms to the actual target motion as the basis for adjusting the heading in the path planning process. Through this trend comparison mechanism, when the unmanned aerial vehicle appears high light interference, local occlusion or target appearance change in the image, it can still adjust the track with a reference point closer to the real motion law, thereby improving the stability and continuity of the flight path.

[0217] In a preferred embodiment of the present application, according to the target prediction motion feature data and the candidate space motion feature data, the direction change trend of the two in the continuous period is analyzed for direction matching, to generate direction consistency comparison result data, which specifically includes:

[0218] Firstly, in the direction change trend analysis, the moving direction of the target prediction position in the continuous frames is compared, such as horizontal, vertical or oblique moving trend, and the direction change of the candidate space position in the continuous records is compared, such as the trend of the target moving forward or laterally offsetting relative to the unmanned aerial vehicle in space.

[0219] In order to determine whether the directions are consistent, the main direction attributes of the two trends are compared, such as whether both are leftward offset, whether both are forward moving, or whether both remain unchanged in a certain direction, etc. When the direction changes of the visual end and the space end at continuous multiple time points all show consistent trends, it is determined that the direction consistency is high; if the directions are opposite or the change directions continuously deviate, the direction consistency is low.

[0220] Finally, the direction consistency comparison result data is generated, which is used to represent the direction matching degree of the visual trend and the candidate space trend, and provides a basis for the subsequent comprehensive coincidence calculation.

[0221] In a preferred embodiment of the present application, according to the target prediction motion feature data and the candidate space motion feature data, the change step difference in the position change amplitude of the two is analyzed for amplitude interval analysis processing, to generate change amplitude coincidence comparison result data, which specifically includes:

[0222] First, the change in position amplitude in the target predicted motion feature data is read, i.e., the change in the moving distance of the target in consecutive frames; then the change in the moving distance in the candidate spatial motion feature data is read, i.e., the change in the spatial position relative to the unmanned aerial vehicle.

[0223] In the amplitude interval analysis, the change amplitude of the visual end is compared with the change amplitude of the spatial end in the same time period. For example, when the target prediction shows that the target is approaching in the image at a relatively stable speed, and the candidate spatial position also presents an approaching trend, it is considered that the change amplitudes of the two are highly consistent.

[0224] Further analysis is made on whether the change step difference is kept within a reasonable range, for example, whether the change amounts of the visual end and the spatial end present the same increasing or decreasing trend; if the change amplitude of the visual end suddenly and sharply increases, while the spatial end remains stable, it indicates that the amplitude relationship of the two is inconsistent.

[0225] Through the amplitude comparison in consecutive time periods, a result indicating the degree of fit of the visual trend and the spatial trend in the change amount, i.e., the change amplitude consistency comparison result data, can be obtained.

[0226] The data can be used for subsequent combination of direction consistency and trend smoothness to comprehensively judge whether the candidate spatial position is more consistent with the real motion law of the target.

[0227] In a preferred embodiment of the present application, according to the target predicted motion feature data and the candidate spatial motion feature data, trend fluctuation analysis processing is performed on the smoothness of the direction change of the two, to generate trend smoothness comparison result data, which specifically includes:

[0228] First, the direction change records of the visual end and the spatial end are read respectively, which reflect whether the motion direction is kept smooth in consecutive time periods.

[0229] The trend fluctuation analysis mainly focuses on the following contents: whether the direction change is continuous and smooth, i.e., whether the visual end or the spatial end presents sudden direction change, direction jump or irregular jitter, etc.

[0230] Whether the direction change presents a regular trend, for example, stable approach, stable deviation, etc., rather than random change.

[0231] Whether the direction fluctuation amplitude exceeds the normal range, if the visual end has a large direction fluctuation at consecutive multiple time points, while the spatial end direction is stable, it indicates that there is a significant inconsistency.

[0232] Detect the fluctuation range of the direction change of both in a preset time window, and compare the smoothness of both. When the direction changes of the vision and the space are both kept in a similar fluctuation trend range, the change is smooth and consistent, and the trend smoothness comparison result is high; if the fluctuation difference between the two is large, the trend smoothness is low.

[0233] Finally, the trend smoothness comparison result data is output, which provides an important dimension for the comprehensive confidence score.

[0234] In a preferred embodiment of the present application, according to the direction consistency comparison result data, the change amplitude coincidence comparison result data, and the trend smoothness comparison result data, the three types of comparison results are combined to form the motion feature comparison basis data, which specifically includes:

[0235] First, the corresponding comparison result data is obtained from the direction consistency, the amplitude coincidence, and the trend smoothness. These data represent the degree of fit between the candidate space position and the target prediction trend in different dimensions.

[0236] In the combination analysis process:

[0237] The confidence of the three types of comparison results is judged, for example, whether the direction consistency meets the high standard, whether the amplitude change is synchronized with the vision end, and whether the trend smoothness has continuity.

[0238] It is judged whether the three dimensions jointly support the trend of a certain candidate position to be closer to the real target motion. If the three comparison results are all high, the candidate position has higher confidence.

[0239] The results of the three dimensions are integrated into a comprehensive matching evaluation according to a certain proportion or rule. The three types of results are combined by setting rules, for example:

[0240] If the three dimensions are all high matching, the overall evaluation is high;

[0241] If a certain dimension is low, the overall evaluation decreases;

[0242] If multiple dimensions are low, it is judged that the trend is not matched.

[0243] Finally, the motion feature comparison basis data is output, which is used for subsequent comprehensive coincidence calculation and provides a basis for further selecting the most reliable candidate target position.

[0244] Embodiments of the present application also provide a multi-source information fusion unmanned aerial vehicle sea target tracking system, which comprises:

[0245] A visual feature acquisition module is configured to acquire consecutive image frames as image data, and extract the texture distribution, luminance gradient, and edge stability of a target region from the image data to generate a first visual feature sequence. A spatial feature acquisition module is configured to acquire a plurality of consecutive spatial frames as spatial data, and extract the direction change, change amplitude, and trend smoothness of the target region from the spatial data to generate a first spatial feature sequence. A feature comparison module is configured to compare the first visual feature sequence and the first spatial feature sequence to obtain a motion feature comparison basis data. A target selection module is configured to select a target position from the motion feature comparison basis data according to a certain rule.

[0246] a multi-source state acquisition module, configured to acquire the UAV attitude data, the radar position detection data and the AIS identification data, and perform attitude disturbance correction, time axis alignment on the data, and generate first multi-source state data synchronized with the first visual feature sequence;

[0247] a visual drift determination module, configured to calculate a visual feature matching offset of adjacent frames according to the first visual feature sequence, and compare the visual feature matching offset with a radar azimuth change trend in the first multi-source state data, and generate visual drift determination data when the two are inconsistent continuously within a preset time window;

[0248] a candidate position construction module, configured to construct a candidate target space position set according to the visual drift determination data, and call the radar position detection data and the AIS identification data in the first multi-source state data, and perform distance convergence screening and spatial overlap constraint processing on a geometric projection range of the candidate target space position set, and generate a position candidate set;

[0249] a spatial position verification module, configured to perform motion direction continuity comparison according to the position candidate set and a target predicted position obtained according to the first visual feature sequence, and generate target position data verified by spatial position intersection;

[0250] a visual update module, configured to generate a flight path control instruction for controlling a flight direction and a speed of the UAV according to the target position data verified by spatial position intersection, and adjust a subsequent flight path of the UAV according to the flight path control instruction.

[0251] It should be noted that the system corresponds to the above method, and all implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0252] Embodiments of the present application also provide a computing device, comprising a processor and a memory storing a computer program, wherein the computer program is executed by the processor to perform the above method. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0253] Embodiments of the present application also provide a computer readable storage medium storing instructions, wherein the instructions are executed on a computer to make the computer perform the above method. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0254] The above is the preferred embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A multi-source information fusion unmanned aerial vehicle sea target tracking method, characterized in that, The method comprises: acquiring continuous image frames as image data, and extracting texture distribution, brightness gradient and edge stability of a target region from the image data to generate a first visual feature sequence; acquiring unmanned aerial vehicle attitude data, radar position detection data and AIS identification data, and performing attitude disturbance correction, time axis alignment of the data, and generation of first multi-source state data synchronized with the first visual feature sequence; calculating visual feature matching offset of adjacent frames from the first visual feature sequence, and comparing the offset with radar azimuth change trend in the first multi-source state data, and generating visual drift determination data when the two are inconsistent within a preset time window; according to the visual drift determination data, constructing a candidate target space position set by calling the radar position detection data and AIS identification data in the first multi-source state data, and performing distance convergence screening and spatial overlap constraint processing of the geometric projection range to generate a position candidate set, specifically comprising: according to the visual drift determination data, extracting position information corresponding to the radar position detection data and AIS identification data from the first multi-source state data to form an initial spatial candidate position set; according to the initial spatial candidate position set, performing multi-round convergence processing according to the distance difference between the initial spatial candidate position set and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced to generate a first candidate position set after distance convergence; according to the first candidate position set after distance convergence, projecting the first candidate position set to a geometric projection range based on the unmanned aerial vehicle attitude data, and performing spatial overlap analysis processing on the landing position to generate projection overlap basic data; according to the projection overlap basic data, eliminating candidate positions that do not form effective overlap with the geometric projection range, so that the remaining candidate positions constitute the position candidate set; performing motion direction continuity comparison according to the position candidate set and the target predicted position obtained according to the first visual feature sequence, and generating target position data after spatial position cross-checking; generating a flight path control instruction for controlling the flight direction and speed of the unmanned aerial vehicle according to the target position data after spatial position cross-checking, and adjusting the subsequent flight path of the unmanned aerial vehicle according to the flight path control instruction. 2.The multi-source information fusion UAV maritime target tracking method according to claim 1, characterized in that, calculating visual feature matching offset of adjacent frames from the first visual feature sequence, and comparing the offset with radar azimuth change trend in the first multi-source state data, and generating visual drift determination data when the two are inconsistent within a preset time window, comprising: extracting texture distribution, brightness gradient and edge stability difference of adjacent frames from the first visual feature sequence to form visual change basic data; performing continuity analysis processing on texture change, brightness change and edge change according to the visual change basic data to generate visual feature change trend data; performing time alignment and trend direction comparison processing of the visual feature change trend data with the radar azimuth change trend in the first multi-source state data to generate trend comparison basic data; generating visual drift determination data when the trend direction consistency is continuously deviated, the trend amplitude coincidence is continuously deviated, or the trend stability is continuously fluctuated according to the trend comparison basic data. 3.The multi-source information fusion unmanned aerial vehicle maritime target tracking method according to claim 1, characterized in that, According to the position candidate set and the target predicted position obtained according to the first visual feature sequence, motion direction continuity comparison is performed, and target position data subjected to spatial position cross-checking is generated, including: According to the first visual feature sequence, the direction change, position change amplitude and direction stability of the target predicted position in consecutive frames are extracted, and target predicted motion feature data is generated; According to the position candidate set, the direction change trend, movement distance change and direction smoothness of the candidate position in consecutive records are extracted, and candidate spatial motion feature data is generated; According to the target predicted motion feature data and the candidate spatial motion feature data, direction consistency comparison, change amplitude coincidence comparison and trend smoothness comparison are performed on the two, and motion feature comparison basis data containing direction consistency results, amplitude coincidence results and trend smoothness results are generated; According to the motion feature comparison basis data, joint scoring processing is performed on the direction consistency results, amplitude coincidence results and trend smoothness results, and comprehensive coincidence data representing the matching degree of the candidate motion trend is generated; According to the comprehensive coincidence data, the position with the maximum comprehensive coincidence data is selected from the position candidate set, and the target position data subjected to spatial position cross-checking is generated. 4.The multi-source information fusion UAV maritime target tracking method according to claim 2, characterized in that, According to the visual change basis data, continuity analysis processing is performed on texture changes, brightness changes and edge changes, and visual feature change trend data is generated, including: According to the visual change basis data, the texture change data, brightness change data and edge change data are respectively arranged in time sequence to form change sequences, and multi-channel change sequence data for continuity analysis is formed; According to the multi-channel change sequence data, the change direction, change amplitude and change stability interval of each channel change sequence are analyzed and processed in sections, and continuity analysis result data of multiple channels is generated; According to the continuity analysis result data, comprehensive analysis processing is performed on the change direction consistency, change rhythm synchronization and stable interval overlap degree of multiple channels, and visual feature change trend data is generated. 5.The multi-source information fusion unmanned aerial vehicle maritime target tracking method according to claim 1, characterized in that, According to the initial spatial candidate position set, multi-round convergence processing is performed according to the distance difference between the initial spatial candidate position set and the current position of the unmanned aerial vehicle, so that the distance range is gradually reduced, and the first candidate position set after distance convergence is generated, including: According to the initial spatial candidate position set, the distance difference between each candidate position and the current position of the unmanned aerial vehicle is calculated, and distance difference basis data for convergence processing is generated; According to the distance difference basis data, first-round distance screening processing is performed according to a preset distance convergence interval, candidate positions exceeding the first convergence interval are removed, and first-round screening candidate data is generated, the distance convergence interval includes a first convergence interval, a second convergence interval and a final convergence interval; According to the first-round screening candidate data, second-round distance reduction processing is performed according to the second convergence interval, so that the distance range of the candidate position is gradually narrowed, and second-round screening candidate data is generated; According to the second-round screening candidate data, final-stage distance screening processing is performed according to the final convergence interval, and the first candidate position set after distance convergence is generated.

6. The multi-source information fusion unmanned aerial vehicle maritime target tracking method according to claim 3, characterized in that, According to the target predicted motion feature data and the candidate space motion feature data, the direction consistency comparison, the change amplitude coincidence comparison and the trend smoothness comparison are performed on the two, and motion feature comparison basis data containing the direction consistency result, the amplitude coincidence result and the trend smoothness result are generated, including: According to the target predicted motion feature data and the candidate space motion feature data, the direction consistency comparison, the change amplitude coincidence comparison and the trend smoothness comparison are performed on the two, and motion feature comparison basis data containing the direction consistency result, the amplitude coincidence result and the trend smoothness result are generated, including: According to the target predicted motion feature data and the candidate space motion feature data, the direction consistency comparison, the change amplitude coincidence comparison and the trend smoothness comparison are performed on the two, and motion feature comparison basis data containing the direction consistency result, the amplitude coincidence result and the trend smoothness result are generated, including: According to the target predicted motion feature data and the candidate space motion feature data, the direction consistency comparison, the change amplitude coincidence comparison and the trend smoothness comparison are performed on the two, and motion feature comparison basis data containing the direction consistency result, the amplitude coincidence result and the trend smoothness result are generated, including: The system is applied to the method of any one of claims 1-6, and the system comprises:

7. A multi-source information fusion unmanned aerial vehicle maritime target tracking system, characterized in that, A visual feature acquisition module is configured to acquire consecutive image frames as image data, and extract texture distribution, brightness gradient and edge stability of a target region from the image data to generate a first visual feature sequence; A multi-source state acquisition module is configured to acquire UAV attitude data, radar position detection data and AIS identification data, and perform attitude disturbance correction and time axis alignment on the data to generate first multi-source state data synchronized with the first visual feature sequence; A visual drift determination module is configured to calculate visual feature matching offset of adjacent frames from the first visual feature sequence, and compare the visual feature matching offset with a radar direction change trend in the first multi-source state data, and generate visual drift determination data when the two are inconsistent within a preset time window; A candidate position construction module is configured to construct a candidate target space position set from the radar position detection data and the AIS identification data in the first multi-source state data according to the visual drift determination data, and perform distance convergence screening and spatial overlap constraint processing of geometric projection range to generate a position candidate set; A spatial position verification module is configured to perform motion direction continuity comparison on the position candidate set and a target predicted position obtained from the first visual feature sequence, and generate target position data verified by spatial position intersection; A visual update module is configured to generate a flight path control instruction for controlling flight direction and speed of a UAV from the target position data verified by spatial position intersection, and adjust a subsequent flight path of the UAV according to the flight path control instruction. The system comprises:

8. A computing device, comprising: One or more processors; A storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6. ​ 9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to implement the method in any one of claims 1 to 6.

Images