Vision-based multi-unmanned aerial vehicle cooperative target positioning method

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative target positioning method based on vision, and belongs to the technical field of unmanned aerial vehicle target positioning. The method comprises the following steps: determining an operation area of field multi-unmanned aerial vehicle cooperative target positioning, and constructing a temporary calibration analysis space in a specified range by taking a target positioning center as a reference; according to the method, the calibration correction signal is generated through calculation of the total nonlinear drift distance, a complete closed loop is formed through cooperative positioning recheck verification after sensor adjustment, whole-process management and control from initial calibration to environment adaptation to positioning verification are achieved, the uniformity of initial calibration references of heterogeneous sensors is guaranteed, and the calibration accuracy of the heterogeneous sensors is improved. The multi-unmanned aerial vehicle cooperative positioning method has the advantages that the multi-unmanned aerial vehicle cooperative positioning robustness and reliability are greatly improved, the continuity and accuracy of the positioning task in the field severe environment are ensured, and the sudden failure of the positioning system caused by calibration deviation or environment drift is avoided.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target localization technology, specifically a vision-based multi-UAV collaborative target localization method. Background Technology

[0002] Against the backdrop of the rapid development of UAV technology, multi-UAV collaborative target positioning has become a core supporting technology in fields such as military reconnaissance, emergency rescue, agricultural monitoring, and traffic inspection. Traditional single-UAV positioning relies on GNSS (such as GPS), which is easily affected by complex environments such as urban canyons, indoor spaces, or electromagnetic shielding, leading to signal loss and decreased accuracy. While multi-UAV collaboration can improve positioning robustness through multi-sensor data fusion and information interaction, it faces challenges such as communication delays, data redundancy, complex collaborative strategies, and limited computing resources.

[0003] In conjunction with the above, it should be noted that the Chinese patent application number CN2022102273578 discloses a vision-based multi-UAV cooperative target localization method, which can adopt different target localization methods according to different target states, thereby achieving rapid and accurate target localization; during cooperative localization, by determining the coarse spatial coordinates of the target in real time, the UAVs involved in cooperative localization can quickly and accurately approach and capture the target, achieving rapid and accurate target localization.

[0004] However, during continuous use, multiple drones may be damaged and need replacement. The replaced drones may have heterogeneous sensors, such as drone A using an RGB camera, drone B using an infrared camera, and drone C using a fisheye camera. The thermal imaging distortion of the infrared camera, the barrel distortion model of the fisheye camera, and the pinhole model of the RGB camera may differ. Alternatively, there may be differences in sensor models, chips, and the adaptation and tuning of the sensors. This can cause the unified parameter matrix of the initial joint calibration to drift nonlinearly with changes in the environment. This results in a relative offset of the calibration reference between multiple drones, which can lead to sudden failure of the positioning system during multi-drone collaborative target positioning.

[0005] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a vision-based multi-UAV cooperative target localization method to solve the problems mentioned above.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a vision-based multi-UAV cooperative target localization method, comprising the following steps:

[0008] Step 1: Determine the operational area for multi-UAV collaborative target localization in the field, construct a temporary calibration and analysis space within a specified range based on the target localization center, collect basic parameters of the UAV itself and initial interference data of the field environment, and upload them to the central processing unit.

[0009] Step 2: Construct a differentiation framework based on the timeline to perform a linkage analysis on the basic parameters of the UAV body and the unified visual calibration benchmark, and summarize the data according to the UAV number to obtain the initial value of the basic deviation of the heterogeneous sensor calibration and the signal to be corrected.

[0010] Step 3: Based on the timeline, construct the environmental interference weights of various outdoor environments on various sensors of the UAV in the temporary calibration analysis space. Based on the initial interference data of the outdoor environment and the normal working environment standard values ​​of each sensor of the current UAV, analyze the environmental deviation value together. Combined with the participation of the environmental interference weights, obtain the environmental interference influence coefficient.

[0011] Step 4: Extract each deviation value from the initial value of the heterogeneous sensor calibration reference deviation and multiply it by the corresponding environmental interference influence coefficient to obtain the additional drift increment and total nonlinear drift of the UAV sensor due to environmental interference. Use this to analyze the reverse adjustment parameters and generate the calibration correction signal.

[0012] Step 5: Based on the calibration correction signal, perform multiple sensor adjustments on the newly added UAV with the signal to be corrected, and re-verify the corrected cooperative target positioning array.

[0013] Furthermore, using the target positioning center as a reference, a work area with a radius of several meters is delineated and marked as a temporary calibration analysis space. Based on the pre-deployed environmental perception terminals and parameter acquisition gateways around the positioning target within the temporary calibration analysis space, for each UAV participating in the collaborative target positioning array, its visual sensor and related body parameters are acquired through the parameter acquisition gateway and marked as the basic parameters of the UAV itself. The environmental perception terminals collect field environmental data within the current temporary calibration analysis space and mark it as initial field environmental interference data. The basic parameters of the UAV itself and the initial field environmental interference data are uploaded in real time to the central processing unit of the operation and maintenance temporary calibration analysis space.

[0014] Furthermore, in the temporary calibration analysis space, a timeline is established with the test command generation node as the cutoff boundary. The timeline before the cutoff boundary is marked as the standard timestamp, and the timeline after the cutoff boundary extending to the current cutoff time is marked as the timestamp to be corrected.

[0015] Extract the initial joint calibration unified parameter matrix preset before multi-UAV collaborative positioning within the standard timestamp. The initial joint calibration unified parameter matrix represents the unified visual calibration benchmark that all UAVs must follow.

[0016] Furthermore, based on the basic parameters of the UAV body within the time stamp to be corrected, the actual working parameters of the current sensors of each newly added UAV are marked as the actual parameters of each UAV's sensors. The actual parameters of each UAV's sensors are compared and analyzed with the corresponding key values ​​in the initial joint calibration unified parameter matrix. The difference between the actual parameters of each UAV's sensors and the unified visual calibration benchmark is analyzed and marked as the initial deviation value. The data is summarized according to the UAV number assigned when each UAV is incorporated into the cooperative target positioning array and marked as the initial value of the heterogeneous sensor calibration basic deviation.

[0017] Furthermore, based on the initial joint calibration unified parameter matrix of the standard timestamp, the maximum and minimum differences of the values ​​of all UAV intrinsic standard values, extrinsic standard angles, and distortion correction standard models are obtained and marked as standard parameter differences. The average of all standard parameter differences of the same type is calculated and marked as a floating coefficient. The floating coefficient is used as the maximum and minimum values ​​of the corresponding values ​​in the intrinsic standard values, extrinsic standard angles, and distortion correction standard models to construct a floating range and mark it as the deviation acceptable threshold range. By comparing the initial value of the heterogeneous sensor calibration basic deviation with the deviation acceptable threshold range, UAVs whose initial value of the heterogeneous sensor calibration basic deviation exceeds the deviation acceptable threshold range are marked, and a correction signal is generated based on the UAV number.

[0018] Furthermore, in the temporary calibration analysis space, based on the sensitivity of all UAV sensors to various environmental factors in the current field in the standard timestamp, they are marked as standard sensitivity values, and the sensitivity of various UAV sensors of different models added in the timestamp to be corrected to various environmental factors in the current field is marked as sensitivity values ​​to be corrected.

[0019] Retrieve the pre-stored sensor environment sensitivity characteristic table, compare the standard sensitivity value and the sensitivity value to be corrected with the recorded value of the corresponding sensor in the sensor environment sensitivity characteristic table to obtain the difference, and obtain the standard sensitivity multi-value difference of the standard sensitivity value and the sensitivity multi-value difference of the sensitivity value to be corrected.

[0020] Furthermore, using the timestamps corresponding to the standard sensitivity value and the sensitivity value to be corrected as the X-axis and the units of the sensor environmental sensitivity characteristic table as the Y-axis, an intuitive curve table is constructed. The average difference of all recorded values ​​in the sensor environmental sensitivity characteristic table is set as the measurement threshold. The measurement threshold, the difference of the standard sensitivity multi-value, and the difference of the sensitivity multi-value to be corrected are plotted on the intuitive curve table using a point-connection method. The average difference between the standard sensitivity multi-value and the measurement threshold is obtained and marked as the standard mean, and the average difference between the difference of the sensitivity multi-value to be corrected and the measurement threshold is marked as the correction mean. The difference between the standard mean and the correction mean for each corresponding data is analyzed and marked as the environmental interference weight.

[0021] Furthermore, by comparing the initial interference data of the field environment with the normal operating environment standard values ​​of each sensor of multiple UAVs in the current temporary calibration analysis space, the deviation values ​​between the current environmental factors and the normal environmental factors are analyzed and marked as environmental deviation values. The value obtained by combining the environmental interference weight with the environmental deviation value is marked as the comprehensive impact value of environmental interference on each UAV sensor. The comprehensive impact values ​​are summarized by UAV number and marked as environmental interference impact coefficients.

[0022] Furthermore, based on the UAV's serial number, the initial value of the corresponding heterogeneous sensor calibration reference deviation is retrieved. Each deviation value in the initial value of the heterogeneous sensor calibration reference deviation is extracted and multiplied by the corresponding environmental interference influence coefficient to obtain the additional drift increment of the UAV sensor due to environmental interference. Then, the initial deviation value and the drift increment are added together to obtain the total nonlinear drift of each UAV sensor.

[0023] Based on the total nonlinear drift, the reverse adjustment parameters are analyzed and organized according to the UAV number and sensor type. These parameters are then marked as nonlinear drift correction values. Based on the nonlinear drift correction values, control signals are generated for each UAV and marked as calibration correction signals. These calibration correction signals are then sent to the corresponding UAV's sensor control module through the parameter acquisition gateway.

[0024] Furthermore, after receiving the calibration correction signal, the overall monitoring system makes physical or software adjustments to the UAV sensors of the signal to be corrected according to the parameters covered in the calibration correction signal. The corrected UAV cooperative target positioning array is used to perform cooperative positioning of the preset test target in the temporary calibration analysis space. The positioning data of all UAVs is aggregated to the central processing unit and marked as the corrected cooperative positioning data.

[0025] The true coordinates of the test target are retrieved from the database of the temporary calibration analysis space and marked as the actual coordinates of the target. The corrected collaborative positioning data is compared with the actual coordinates of the target to generate qualified and unqualified signals.

[0026] The beneficial effects of this invention are:

[0027] 1. This invention constructs a temporary calibration analysis space, combines it with a timeline differentiation framework, accurately extracts the initial joint calibration unified parameter matrix before multi-UAV collaborative positioning, and compares it with the actual parameters of each UAV's sensors within the time stamp to be corrected. This generates initial deviation values ​​and a summary of initial deviation values ​​for heterogeneous sensor calibration. Simultaneously, it filters out UAVs that need correction by using deviation acceptable threshold ranges and generates correction signals. This achieves a shift from passively accepting positioning failures caused by calibration deviations to actively locating the source of initial deviations in heterogeneous sensors. It ensures a unified initial calibration benchmark for multi-UAV collaborative positioning, avoids inaccurate initial calibration caused by differences in sensor models, chips, or calibration, effectively solves the problem of initial calibration benchmark offset for heterogeneous sensors in multi-UAVs, and overcomes the limitation of traditional methods in locating initial calibration deviations of heterogeneous sensors.

[0028] 2. This invention constructs an environmental interference weight, calculates the environmental deviation between the initial interference data in the field and the standard value of the normal operating environment of the sensor, and then obtains the environmental interference influence coefficient. Through the linkage calculation of the initial deviation value and the environmental interference influence coefficient, the total nonlinear drift is obtained and a calibration correction signal is generated, achieving precise correction of additional environmental drift. This transforms the unquantifiable and ambiguous influence of field environmental interference into calculable and correctable specific parameters, dynamically offsetting the drift accumulation caused by harsh environments such as temperature, vibration, and lighting. This significantly improves the collaborative positioning adaptability of multiple UAVs in complex field environments, accurately quantifies the impact of field environmental interference on the nonlinear drift of heterogeneous sensors, breaks through the bottleneck of traditional methods that rely solely on static calibration and cannot cope with dynamic interference in the field environment, ensures the continuity and accuracy of positioning tasks in harsh field environments, and avoids sudden failures of the positioning system due to calibration deviations or environmental drift. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the method of the present invention;

[0031] Figure 2 This is a flowchart of the system of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example 1: Please refer to Figure 1 - Figure 2 As shown, this embodiment is a vision-based multi-UAV cooperative target localization method, including the following steps:

[0034] Step 1: Determine the operational area for multi-UAV collaborative target positioning in the field. Based on the fact that during the operation of the original multi-UAVs, there is damage and replacement or additional UAVs are temporarily added to the collaborative target positioning array, the overall monitoring system used to control the multi-UAV collaborative target positioning issues a test command. Taking the target positioning center as the reference, a 50-meter radius operational area is delineated and marked as a temporary calibration analysis space.

[0035] Based on the pre-deployed environmental perception terminals and parameter acquisition gateways within the temporary calibration and analysis space surrounding the positioning target, the parameter acquisition gateway is used to collect data from UAVs connected to the cooperative target positioning array. For each UAV participating in the cooperative target positioning array, the parameter acquisition gateway acquires its visual sensor and related body parameters, marking them as the basic parameters of the UAV body. The basic parameters of the UAV body include the type of each UAV's visual sensor, sensor internal parameters, sensor external parameters, and sensor distortion correction parameters at the factory. Among them, the UAV visual sensor type is represented as RGB camera A, infrared camera B, and fisheye camera C; sensor internal parameters are represented as focal length and principal point coordinates; sensor external parameters are represented as the relative attitude angle between the sensor body and the fuselage.

[0036] The environmental sensing terminal collects field environmental data within the current temporary calibration and analysis space, marking it as initial field environmental interference data. This initial field environmental interference data is represented by ambient temperature, air humidity, ground vibration intensity, and natural light intensity. The basic parameters of the UAV itself and the initial field environmental interference data are uploaded in real time to the central processing unit of the temporary calibration and analysis space to complete the collection and storage of core basic data. The data is then marked and stored in a cloud-based historical database for subsequent data analysis of multi-UAV collaborative target positioning. This process obtains the initial characteristic information of heterogeneous sensors from multiple UAVs and the basic interference status of the field environment, providing raw data support for subsequent analysis of heterogeneous sensor calibration deviations and the impact of the environment on drift.

[0037] Step 2: In the temporary calibration analysis space, establish a timeline with the test command generation node as the cutoff boundary. Mark the timeline before the cutoff boundary as the standard timestamp, and mark the timeline after the cutoff boundary extending to the current cutoff time as the timestamp to be corrected.

[0038] Extract the initial joint calibration unified parameter matrix preset before multi-UAV cooperative positioning within the standard timestamp. The initial joint calibration unified parameter matrix represents the unified visual calibration benchmark that all UAVs must follow. The unified visual calibration benchmark includes unified intrinsic standard values, extrinsic standard angles, and distortion correction standard models.

[0039] Based on the basic parameters of the UAV body within the timestamp to be corrected, the actual working parameters of the current sensors of each newly added UAV are marked as the actual parameters of each UAV sensor. The actual parameters of each UAV sensor are represented as the actual internal standard values, external standard angles, and distortion correction standard models of UAVs A, B, and C, as well as the parts where there are data differences, such as the actual focal length of the RGB camera of UAV A, the actual thermal imaging distortion parameters of the infrared camera of UAV B, and the actual cylindrical distortion model parameters of the fisheye camera of UAV C.

[0040] The actual parameters of each UAV's sensors are compared and analyzed with the corresponding key values ​​in the initial joint calibration unified parameter matrix. The differences between the actual parameters of each UAV's sensors and the unified visual calibration benchmark are analyzed and marked as initial deviation values. For example, the actual focal length of UAV A is 0.2 mm larger than the unified visual calibration benchmark focal length, the actual thermal imaging distortion angle of UAV B is 0.1 degrees larger than the unified visual calibration benchmark, and the actual barrel distortion coefficient of UAV C is 0.3 degrees larger than the unified visual calibration benchmark. The data differences between the actual parameters of each UAV's sensors and the corresponding key values ​​in the initial joint calibration unified parameter matrix are marked as initial deviation values. The data is summarized according to the UAV number assigned when each UAV is incorporated into the cooperative target positioning array and marked as the initial value of the heterogeneous sensor calibration basic deviation.

[0041] Based on the initial joint calibration unified parameter matrix using standard timestamps, the maximum and minimum differences of various values ​​in the standard values ​​of all UAV intrinsic parameters, standard angles of extrinsic parameters, and distortion correction standard models are obtained and marked as standard parameter differences. The average of all standard parameter differences of the same type is calculated and marked as a floating coefficient. The floating coefficient is used as the maximum and minimum values ​​of the corresponding values ​​in the standard values ​​of intrinsic parameters, standard angles of extrinsic parameters, and distortion correction standard models to construct a floating range, which is marked as the deviation acceptable threshold range. By comparing the initial value of the heterogeneous sensor calibration base deviation with the deviation acceptable threshold range, UAVs whose initial value of the heterogeneous sensor calibration base deviation exceeds the deviation acceptable threshold range are marked. A correction signal is generated based on the UAV number. Through parameter comparison and analysis, the degree of deviation between the multi-UAV heterogeneous sensors and the unified visual calibration benchmark in the initial state is clarified, the initial source of nonlinear drift is located, and a deviation basis is provided for subsequent calculation of the total drift amount in combination with environmental interference.

[0042] Step 3: In the temporary calibration analysis space, based on the sensitivity of all UAV sensors to various environmental factors in the current field in the standard timestamp, mark them as standard sensitivity values. The sensitivity includes temperature sensitivity, humidity sensitivity, vibration sensitivity, and light sensitivity. Mark the sensitivity of various UAV sensors of different models added in the timestamp to be corrected to various environmental factors in the current field as sensitivity values ​​to be corrected. Retrieve the pre-stored sensor environmental sensitivity characteristic table, which records the sensitivity of different types of sensors to various environmental factors. Compare the standard sensitivity value and the sensitivity value to be corrected with the recorded values ​​of the corresponding sensors in the sensor environmental sensitivity characteristic table to obtain the difference. Obtain the standard sensitivity multi-value difference of the standard sensitivity value and the sensitivity multi-value difference to be corrected of the sensitivity value to be corrected.

[0043] Using the timestamps corresponding to the standard sensitivity value and the sensitivity value to be corrected as the X-axis, and with the starting point of the time stamp to be corrected and the starting point of the standard time stamp cut-off aligned vertically to establish time synchronization, and ensuring that the two cut-off time segments maintain a uniform length, an intuitive curve table is constructed using the units of the sensor environmental sensitivity characteristic table as the Y-axis. The average difference between all recorded values ​​in the sensor environmental sensitivity characteristic table is set as the measurement threshold. Using a point-connection method, the measurement threshold, the difference between the standard sensitivity multi-values, and the difference between the difference between the sensitivity multi-values ​​to be corrected are plotted on the intuitive curve table as corresponding intuitive floating curves. The average difference between the difference between the standard sensitivity multi-values ​​and the measurement threshold is obtained and marked as the standard mean, and the average difference between the difference between the difference between the sensitivity multi-values ​​to be corrected and the measurement threshold is marked as the correction mean. The differences between the standard mean and the correction mean for each corresponding data are analyzed and marked as environmental interference weights. For example, the interference weight of temperature on an infrared camera is 0.3, the interference weight of vibration on a fisheye camera is 0.4, and the interference weight of illumination on an RGB camera is 0.3. The units are normalized and are not limited to these.

[0044] Compare the initial interference data of the field environment with the normal operating environment standard values ​​of each sensor of multiple UAVs in the current temporary calibration analysis space. Normal operating environment standard values ​​include, for example, the normal operating temperature for infrared cameras is 15-25℃, the normal operating vibration intensity for fisheye cameras is ≤3, and the normal operating light intensity for RGB cameras is 500-1000 lux. Analyze the deviation values ​​between the current environmental factors and the normal environmental factors and mark them as environmental deviation values. For example, if the current temperature is 30℃, it is 5℃ higher than the upper limit of the normal temperature for infrared cameras; if the current vibration intensity is 5, it is 2 levels higher than the normal vibration intensity for fisheye cameras. Combine this with the environmental interference weight and the ambient temperature... The value obtained by multiplying the environmental deviation values ​​is marked as the comprehensive impact value of environmental interference on each UAV sensor. For example, the infrared camera of UAV B has a temperature deviation of 5℃, which, when multiplied by the interference weight of 0.3, results in a comprehensive impact value of 0.32; the fisheye camera of UAV C has a vibration deviation of level 2, which, when multiplied by the interference weight of 0.4, results in a comprehensive impact value of 0.38. The comprehensive impact values ​​are summarized by UAV number and marked as environmental interference impact coefficients to quantify the degree of interference of harsh outdoor environments on heterogeneous sensors, clarify the specific impact of environmental factors on nonlinear drift, and provide interference basis for subsequent calculation of the total drift correction amount of the joint deviation initial value.

[0045] Example 2:

[0046] Step 4: Retrieve the initial value of the heterogeneous sensor calibration reference deviation according to the UAV's serial number. Extract each deviation value from the initial value of the heterogeneous sensor calibration reference deviation and multiply it by the corresponding environmental interference influence coefficient to obtain the additional drift increment of the UAV sensor caused by environmental interference. For example, the focal length deviation of UAV A is 0.2 mm and the distortion angle deviation of UAV B is 0.1 degree; the corresponding environmental interference influence coefficients are 0.25 for UAV A and 0.32 for UAV B. In summary, the focal length drift increment of UAV A is 0.2 × 0.25 = 0.05 mm and the distortion angle drift increment of UAV B is 0.1 × 0.32 = 0.032 degrees.

[0047] Then add the initial deviation value to the drift increment to obtain the total nonlinear drift of each UAV sensor. For example, the total focal length drift of UAV A is 0.2 + 0.05 = 0.25 mm, and the total distortion angle drift of UAV B is 0.1 + 0.032 = 0.132 degrees.

[0048] Based on the total nonlinear drift, analyze the reverse adjustment parameters, which are the values ​​that need to be adjusted back to the unified visual calibration reference. For example, for drone A, the focal length needs to be reduced by 0.25 mm, for drone B, the distortion angle needs to be reduced by 0.132 degrees, and for drone C, the barrel distortion coefficient needs to be reduced by 0.414. Organize the reverse adjustment parameters according to the drone number and sensor type, and mark them as nonlinear drift correction amounts.

[0049] Based on the nonlinear drift correction, control signals are generated for each UAV and marked as calibration correction signals. For example, the RGB camera focal length is adjusted to XX millimeters to UAV A, and the infrared camera distortion angle is corrected to XX degrees to UAV B. The calibration correction signals are then sent to the corresponding UAV's sensor control module through the parameter acquisition gateway. Through logical conversion of core data, the total nonlinear drift of heterogeneous sensors in the field environment is quantified, and precise parameter adjustment commands are generated. This directly solves the problem of relative offset of calibration references between multiple UAVs and provides a correction basis for subsequent positioning accuracy verification.

[0050] Step 5: After receiving the calibration correction signal, the overall monitoring system makes physical or software adjustments to the drone sensors to be corrected according to the parameters covered in the calibration correction signal. For example, drone A adjusts the focal length of the RGB camera to the correction value through the sensor control chip, drone B corrects the thermal imaging distortion parameters of the infrared camera through the algorithm, and drone C updates the barrel distortion model of the fisheye camera.

[0051] After correction, control multiple UAVs in the temporary calibration analysis space to perform collaborative positioning of the preset test target in the temporary calibration analysis space. Collect target images and calculate positioning coordinates through the vision sensors of each UAV. The positioning data of all UAVs are aggregated to the central processing unit and marked as the corrected collaborative positioning data.

[0052] The actual coordinates of the test target are retrieved from the database of the temporary calibration analysis space and marked as the actual coordinates of the target. The corrected collaborative positioning data is compared with the actual coordinates of the target, and the average error of multi-UAV collaborative positioning is calculated, such as the distance deviation between the average value of the multi-UAV positioning coordinates and the actual coordinates. The average error is compared with a preset qualified threshold, which can be 1 meter.

[0053] If the average error is less than or equal to the preset qualified threshold, a positioning qualified signal is generated, and multiple UAVs exit the temporary calibration phase and enter the formal field collaborative target positioning.

[0054] If the average error is greater than the acceptable threshold, an unacceptable signal is generated, and the process returns to step two. The basic parameters of the UAV body are re-analyzed, and the initial value of the new heterogeneous sensor calibration reference deviation is calculated. The subsequent steps are repeated until the positioning error is acceptable.

[0055] By performing correction operations and verifying positioning accuracy, it is confirmed whether the nonlinear drift defect of heterogeneous sensors has been resolved, ensuring the accuracy of collaborative positioning of multiple UAVs in harsh field environments, avoiding sudden failure of the positioning system, and ensuring the reliability of subsequent formal positioning missions.

[0056] Combining Examples 1 and 2, a calibration correction signal is generated by calculating the total nonlinear drift. This signal is then verified through collaborative positioning after sensor adjustment, forming a complete closed loop. This overcomes the limitations of traditional methods that rely on single-dimensional correction, enabling full-process control from initial calibration to environmental adaptation to positioning verification. This ensures the uniformity of the initial calibration benchmark for heterogeneous sensors and dynamically offsets the impact of environmental interference in the field. It significantly improves the robustness and reliability of multi-UAV collaborative positioning, ensuring the continuity and accuracy of positioning tasks in harsh field environments and preventing sudden failures of the positioning system due to calibration deviations or environmental drift.

[0057] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0058] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0059] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A vision-based multi-UAV cooperative target localization method, characterized in that, Includes the following steps: Step 1: Determine the operational area for multi-UAV collaborative target localization in the field, construct a temporary calibration and analysis space within a specified range based on the target localization center, collect basic parameters of the UAV itself and initial interference data of the field environment, and upload them to the central processing unit. Step 2: Construct a differentiation framework based on the timeline to perform a linkage analysis on the basic parameters of the UAV body and the unified visual calibration benchmark, and summarize the data according to the UAV number to obtain the initial value of the basic deviation of the heterogeneous sensor calibration and the signal to be corrected. Step 3: Based on the timeline, construct the environmental interference weights of various outdoor environments on various sensors of the UAV in the temporary calibration analysis space. Based on the initial interference data of the outdoor environment and the normal working environment standard values ​​of each sensor of the current UAV, analyze the environmental deviation value together. Combined with the participation of the environmental interference weights, obtain the environmental interference influence coefficient. Step 4: Extract each deviation value from the initial value of the heterogeneous sensor calibration reference deviation and multiply it by the corresponding environmental interference influence coefficient to obtain the additional drift increment and total nonlinear drift of the UAV sensor due to environmental interference. Use this to analyze the reverse adjustment parameters and generate the calibration correction signal. Step 5: Based on the calibration correction signal, perform multiple sensor adjustments on the newly added UAV with the signal to be corrected, and re-verify the corrected cooperative target positioning array.

2. The vision-based multi-UAV cooperative target localization method according to claim 1, characterized in that, Using the target positioning center as a reference, a work area with a radius of several meters is delineated and marked as a temporary calibration analysis space. Based on the pre-deployed environmental perception terminals and parameter acquisition gateways around the positioning target within the temporary calibration analysis space, for each UAV participating in the collaborative target positioning array, its visual sensor and related body parameters are acquired through the parameter acquisition gateway and marked as the basic parameters of the UAV itself. The environmental perception terminals collect the field environment data within the current temporary calibration analysis space and mark it as the initial interference data of the field environment. The basic parameters of the UAV itself and the initial interference data of the field environment are uploaded to the central processing unit of the operation and maintenance temporary calibration analysis space in real time.

3. The vision-based multi-UAV cooperative target localization method according to claim 1, characterized in that, In the temporary calibration analysis space, a timeline is established with the test command generation node as the cutoff boundary. The timeline before the cutoff boundary is marked as the standard timestamp, and the timeline after the cutoff boundary extending to the current cutoff time is marked as the timestamp to be corrected. Extract the initial joint calibration unified parameter matrix preset before multi-UAV collaborative positioning within the standard timestamp. The initial joint calibration unified parameter matrix represents the unified visual calibration benchmark that all UAVs must follow.

4. The vision-based multi-UAV cooperative target localization method according to claim 3, characterized in that, Based on the basic parameters of the UAV body within the time stamp to be corrected, the actual working parameters of the current sensors of each newly added UAV are marked as the actual parameters of each UAV's sensors. The actual parameters of each UAV's sensors are compared and analyzed with the corresponding key values ​​in the initial joint calibration unified parameter matrix. The difference between the actual parameters of each UAV's sensors and the unified visual calibration benchmark is analyzed and marked as the initial deviation value. The data is summarized according to the UAV number assigned when each UAV is incorporated into the cooperative target positioning array and marked as the initial value of the heterogeneous sensor calibration basic deviation.

5. The vision-based multi-UAV cooperative target localization method according to claim 4, characterized in that, Based on the initial joint calibration unified parameter matrix of standard timestamps, the maximum and minimum differences of the standard values ​​of all UAV intrinsic parameters, standard angles of extrinsic parameters, and distortion correction standard models are obtained and marked as standard parameter differences. The average of all standard parameter differences of the same type is calculated and marked as a floating coefficient. The floating coefficient is used as the maximum and minimum values ​​of the corresponding values ​​in the standard values ​​of intrinsic parameters, standard angles of extrinsic parameters, and distortion correction standard models to construct a floating range and mark it as the deviation acceptable threshold range. By comparing the initial value of the heterogeneous sensor calibration basic deviation with the deviation acceptable threshold range, UAVs whose initial value of the heterogeneous sensor calibration basic deviation exceeds the deviation acceptable threshold range are marked, and a correction signal is generated based on the UAV number.

6. The vision-based multi-UAV cooperative target localization method according to claim 1, characterized in that, In the temporary calibration analysis space, based on the sensitivity of all UAV sensors to various environmental factors in the current field in the standard timestamp, they are marked as standard sensitivity values, and the sensitivity of various UAV sensors of different models added in the timestamp to be corrected to various environmental factors in the current field is marked as sensitivity values ​​to be corrected. Retrieve the pre-stored sensor environment sensitivity characteristic table, compare the standard sensitivity value and the sensitivity value to be corrected with the recorded value of the corresponding sensor in the sensor environment sensitivity characteristic table to obtain the difference, and obtain the standard sensitivity multi-value difference of the standard sensitivity value and the sensitivity multi-value difference of the sensitivity value to be corrected.

7. A vision-based multi-UAV cooperative target localization method according to claim 6, characterized in that, Using the timestamps corresponding to the standard sensitivity value and the sensitivity value to be corrected as the X-axis and the units of the sensor environmental sensitivity characteristic table as the Y-axis, an intuitive curve table is constructed. The average difference of all recorded values ​​in the sensor environmental sensitivity characteristic table is set as the measurement threshold. The measurement threshold, the difference of the standard sensitivity multi-value, and the difference of the sensitivity multi-value to be corrected are plotted on the intuitive curve table using a point-connection method. The average difference between the standard sensitivity multi-value and the measurement threshold is obtained and marked as the standard mean, and the average difference between the difference of the sensitivity multi-value to be corrected and the measurement threshold is marked as the correction mean. The difference between the standard mean and the correction mean for each corresponding data is analyzed and marked as the environmental interference weight.

8. The vision-based multi-UAV cooperative target localization method according to claim 7, characterized in that, By comparing the initial interference data of the field environment with the normal operating environment standard values ​​of each sensor of multiple UAVs in the current temporary calibration analysis space, the deviation values ​​between the current environmental factors and the normal environmental factors are analyzed and marked as environmental deviation values. The value obtained by combining the environmental interference weight with the environmental deviation value is marked as the comprehensive impact value of environmental interference on each UAV sensor. The comprehensive impact values ​​are summarized by UAV number and marked as environmental interference impact coefficients.

9. A vision-based multi-UAV cooperative target localization method according to claim 1, characterized in that, According to the UAV's serial number, retrieve the initial value of the calibration reference deviation of the corresponding heterogeneous sensor, extract each deviation value in the initial value of the calibration reference deviation of the heterogeneous sensor, multiply it by the corresponding environmental interference influence coefficient, and obtain the drift increment of the UAV sensor due to environmental interference. Then add the initial deviation value and the drift increment to obtain the total nonlinear drift of each UAV sensor. Based on the total nonlinear drift, the reverse adjustment parameters are analyzed and organized according to the UAV number and sensor type. These parameters are then marked as nonlinear drift correction values. Based on the nonlinear drift correction values, control signals are generated for each UAV and marked as calibration correction signals. These calibration correction signals are then sent to the corresponding UAV's sensor control module through the parameter acquisition gateway.

10. A vision-based multi-UAV cooperative target localization method according to claim 1, characterized in that, After receiving the calibration correction signal, the overall monitoring system makes physical or software adjustments to the UAV sensors to be corrected according to the parameters covered in the calibration correction signal. The corrected UAV cooperative target positioning array is used to perform cooperative positioning of the preset test target in the temporary calibration analysis space. The positioning data of all UAVs is aggregated to the central processing unit and marked as the corrected cooperative positioning data. The true coordinates of the test target are retrieved from the database of the temporary calibration analysis space and marked as the actual coordinates of the target. The corrected collaborative positioning data is compared with the actual coordinates of the target to generate qualified and unqualified signals.

Images