Unmanned aerial vehicle automatic return method and system based on inertial navigation and vision system

Abstract

The application provides a kind of unmanned plane automatic return method and system based on inertial navigation and vision system, it is related to unmanned plane technical field, when unmanned plane triggers return instruction, the flight state data of unmanned plane output by inertial navigation system is acquired, including flight attitude, speed and current position information, simultaneously, the flight environment image data collected by vision acquisition system is acquired, including terrain, landmark and sky background feature image, the cooperative correlation mechanism of flight state data and flight environment image data is established, the mapping relationship is generated to identify return reference mark and position information, the relative position relationship is calculated in combination with the current position of unmanned plane, the initial return path scheme is planned, and then flight control instruction is generated according to flight attitude and speed information, to control unmanned plane to execute return operation, improve the precision and reliability of unmanned plane automatic return.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically, to a method and system for automatic return-to-home of UAVs based on inertial navigation and vision systems. Background Technology

[0002] With the continuous expansion of drone applications, the importance of automatic return-to-home functionality is becoming increasingly prominent when drones perform various tasks. Traditional automatic return-to-home methods for drones mainly rely on a single navigation system, commonly based on the Global Positioning System (GPS). GPS navigation determines the drone's position by receiving satellite signals and then plans the return route. However, GPS signals have many limitations. In complex environments, such as urban areas with tall buildings, dense forests, or mountainous areas, GPS signals are easily blocked or interfered with, leading to inaccurate positioning or even signal loss. This prevents the drone from accurately performing its return-to-home mission, increasing the risk of the drone being lost or colliding.

[0003] In addition, some return-to-home methods that rely solely on inertial navigation systems also have shortcomings. Inertial navigation systems calculate the position and attitude of the UAV by measuring its acceleration and angular velocity, but during long-term operation, the cumulative sensor errors can lead to a gradual decrease in positioning accuracy, making it difficult to meet the requirements for high-precision return-to-home.

[0004] While visual navigation systems can acquire rich environmental information, when used alone, they do not provide a comprehensive understanding of the drone's flight status and cannot accurately combine the drone's real-time dynamics to plan the optimal return path. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an automatic return-to-home method for unmanned aerial vehicles (UAVs) based on inertial navigation and vision systems, the method comprising:

[0006] When the drone triggers the return-to-home command, the drone's flight status data is obtained in real time from the inertial navigation system on the drone, and the flight environment image data is obtained in real time from the visual acquisition system on the drone. The drone's flight status data includes the drone's flight attitude information, flight speed information and current position information, and the flight environment image data includes terrain feature images, landmark feature images and sky background feature images around the drone's flight path.

[0007] A collaborative association mechanism is established between the UAV flight status data and the flight environment image data. Through this collaborative association mechanism, the correspondence between each parameter in the UAV flight status data and each feature in the flight environment image data is mined to generate a collaborative association mapping relationship.

[0008] Based on the collaborative association mapping relationship, feature recognition processing is performed on the flight environment image data to identify the return reference marker for return positioning from the flight environment image data, and the return reference marker location information is obtained. The return reference marker location information includes the coordinates of the return reference marker in the image coordinate system and the geographic coordinate system position corresponding to the coordinates.

[0009] By combining the current position information in the UAV flight status data with the position information of the return-to-home reference marker, the relative positional relationship between the UAV and the return-to-home reference marker is calculated. Based on the relative positional relationship, the return-to-home path of the UAV from its current position to the location of the return-to-home reference marker is planned to obtain the initial return-to-home path scheme.

[0010] Based on the initial return path scheme, and combined with the flight attitude and speed information in the UAV flight status data, flight control commands are generated to control the flight of the UAV. The flight control commands are then sent to the UAV's flight control system to control the UAV to perform the return operation according to the flight control commands.

[0011] Furthermore, embodiments of the present invention also provide an automatic return-to-home system for unmanned aerial vehicles (UAVs) based on inertial navigation and vision systems, characterized in that it includes:

[0012] A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to execute the above-described automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems by executing the machine-executable instructions.

[0013] In another aspect, embodiments of the present invention also provide a computer program product, the computer program product including machine-executable instructions, the machine-executable instructions being stored in a computer-readable storage medium, a processor of a computer device reading the machine-executable instructions from the computer-readable storage medium, the processor executing the machine-executable instructions, causing the computer device to execute the above-described method for automatic return-to-home of a UAV based on an inertial navigation and vision system.

[0014] Based on the above, a collaborative correlation mechanism was established by comprehensively utilizing the real-time flight status data output by the inertial navigation system and the real-time flight environment image data acquired by the visual acquisition system. This mechanism can deeply explore the correspondence between various parameters of the UAV's flight status and various features of the flight environment image, generating a precise collaborative correlation mapping relationship. Based on this collaborative correlation mapping relationship, the return-to-home reference marker used for return-to-home positioning is accurately identified from the flight environment image data, and its position information is obtained. Combined with the UAV's current position information, the relative positional relationship between the UAV and the return-to-home reference marker can be accurately calculated, thereby planning a scientific and reasonable initial return-to-home path. Then, flight control commands are generated based on the UAV's flight attitude and speed information, effectively controlling the UAV to execute the return-to-home operation according to the planned path. This overcomes the limitations of a single navigation system, significantly improves the accuracy and reliability of the UAV's automatic return-to-home, reduces the risk of return-to-home failure in complex environments, and ensures the safe and efficient operation of the UAV. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the execution flow of the automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of exemplary hardware and software components of an unmanned aerial vehicle (UAV) automatic return-to-home system based on inertial navigation and vision systems provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an embodiment of the automatic return-to-home method for unmanned aerial vehicles (UAVs) based on inertial navigation and vision systems. The following is a detailed description of this automatic return-to-home method for UAVs based on inertial navigation and vision systems.

[0018] Step S110: When the UAV triggers the return-to-home command, acquire the UAV flight status data output in real time by the inertial navigation system on the UAV, and at the same time acquire the flight environment image data collected in real time by the visual acquisition system on the UAV. The UAV flight status data includes the UAV's flight attitude information, flight speed information and current position information, and the flight environment image data includes terrain feature images, landmark feature images and sky background feature images around the UAV's flight path.

[0019] In this embodiment, taking a drone used for urban mapping as an example, after the drone completes a mapping task for a certain area, its internal return-to-home trigger mechanism is activated. For example, a signal indicating the completion of the task is transmitted to the drone's control system, which then sends data acquisition commands to the inertial navigation system and the visual acquisition system. The inertial navigation system, for example, uses a microelectromechanical system (MEMS) inertial navigation module from a certain brand. This inertial navigation module outputs the drone's flight attitude information in real time, including attitude angle information such as pitch angle, roll angle, and yaw angle; flight speed information, including the drone's velocity components in three-dimensional space, such as the velocity along different axes of the geographic coordinate system; and current position information, such as the three-dimensional coordinates in the geographic coordinate system obtained based on the fusion positioning of the Global Navigation Satellite System (GNSS) and the Inertial Measurement Unit (IMU).

[0020] Meanwhile, the drone is equipped with a visual acquisition system, such as a high-resolution camera, which collects flight environment image data in real time. This includes terrain feature images around the drone's flight path, such as images of buildings and roads near the surveying area; landmark feature images, such as images of tall buildings and sculptures near the surveying area; and sky background feature images, i.e., images of the sky above the drone, which contain information such as cloud cover and sky color. The above data is collected at a certain frequency to ensure that the drone's status and flight environment information can be obtained in real time.

[0021] Step S120: Establish a collaborative association mechanism between the UAV flight status data and the flight environment image data. Through this collaborative association mechanism, mine the correspondence between each parameter in the UAV flight status data and each feature in the flight environment image data, and generate a collaborative association mapping relationship.

[0022] Step S121: Analyze the parameter dimensions of the UAV flight status data, extract the attitude parameters corresponding to the flight attitude information, the speed parameters corresponding to the flight speed information, and the position parameters corresponding to the current position information from the UAV flight status data, and determine the numerical type and data acquisition frequency of each parameter.

[0023] In this step, the flight status data of the aforementioned urban mapping UAV is first analyzed in terms of its parameter dimensions. Attitude parameters such as pitch, roll, and yaw are extracted from the flight attitude information output by the inertial navigation system; velocity components along each axis of the geographic coordinate system are extracted from the flight speed information as velocity parameters; and three-dimensional coordinates in the geographic coordinate system are extracted from the current position information as position parameters. Then, the numerical type of each parameter is determined. For example, the numerical type of attitude parameters is angle values, the numerical type of velocity parameters is values ​​per unit of length per unit of time, and the numerical type of position parameters is geographic coordinate values. Simultaneously, the data acquisition frequency of each parameter is determined. This data acquisition frequency is consistent with the output frequency of the inertial navigation system. For example, if the output frequency of the inertial navigation system is a fixed value, the data acquisition frequency of each parameter will be the same.

[0024] Step S122: Analyze the feature dimensions of the flight environment image data, extract features from the flight environment image data to obtain the terrain texture features corresponding to the terrain feature image, the landmark outline features corresponding to the landmark feature image, and the sky brightness features corresponding to the sky background feature image, and determine the feature type and image acquisition frequency of each feature.

[0025] In this step, the feature dimensions of the flight environment image data from the aforementioned urban mapping UAV are analyzed. Image feature extraction algorithms are used to process the terrain feature image, extracting terrain texture features such as the texture direction and roughness of the terrain surface; processing the landmark feature image, extracting landmark contour features such as the outer contour shape and vertex coordinates of the landmark; and processing the sky background feature image, extracting sky brightness features such as the distribution of brightness values ​​in different areas of the sky. Then, the feature type of each feature is determined; for example, the feature type of terrain texture features is texture feature, the feature type of landmark contour features is contour feature, and the feature type of sky brightness features is brightness feature. Simultaneously, the image acquisition frequency for each feature is determined, and this image acquisition frequency is consistent with the image acquisition frequency of the visual acquisition system. For example, if the image acquisition frequency of the visual acquisition system is a fixed value, the image acquisition frequency of each feature will be the same.

[0026] Step S123: Synchronize the data acquisition frequency of the UAV flight status data with the image acquisition frequency of the flight environment image data. Based on the time-synchronized UAV flight status data and the time-synchronized flight environment image data, construct a collaborative association model. This collaborative association model uses each parameter in the time-synchronized UAV flight status data as input variables and each feature in the time-synchronized flight environment image data as output variables.

[0027] In this step, the acquisition frequencies of the flight status data and flight environment image data of the aforementioned urban mapping UAV are first synchronized. For example, if the acquisition frequency of the UAV flight status data is higher than that of the flight environment image data, the flight status data is downsampled to match the image acquisition frequency; if the acquisition frequency of the flight status data is lower than that of the image acquisition frequency, the flight status data is interpolated to match the image acquisition frequency. After time synchronization, a collaborative correlation model is constructed. This collaborative correlation model can employ a neural network model, such as a multilayer perceptron (MLP) model. Parameters from the time-synchronized UAV flight status data, such as attitude, velocity, and position parameters, are used as input variables of the model; features from the time-synchronized flight environment image data, such as terrain texture features, landmark contour features, and sky brightness features, are used as output variables of the model. The model structure can include an input layer, hidden layers, and an output layer. The number of neurons in the input layer matches the dimension of the input variables, the number of neurons in the output layer matches the dimension of the output variables, and the number of neurons in the hidden layers can be adjusted experimentally.

[0028] Step S124: Train the collaborative association model by adjusting the association weight parameters in the collaborative association model so that the deviation between the features output by the collaborative association model and the actual features in the time-synchronized flight environment image data is within a preset range, thereby obtaining the trained collaborative association model.

[0029] Step S1241: Collect multiple sets of historical time-synchronized UAV flight status data and time-synchronized flight environment image data, and divide them into training datasets and validation datasets accordingly.

[0030] In this step, multiple sets of historical data from different mission scenarios are collected for the aforementioned urban mapping drones, including time-synchronized drone flight status data and time-synchronized flight environment image data. Then, the data is divided into training and validation datasets according to a certain ratio, for example, an 8:2 ratio, with 80% of the data used as the training dataset and 20% as the validation dataset.

[0031] Step S1242: Normalize the UAV flight status data in the training dataset, input the normalized UAV flight status data into the initially constructed collaborative association model, and obtain the flight environment image feature prediction results output by the collaborative association model.

[0032] In this step, the UAV flight status data in the training dataset is processed using a normalization method, such as min-max normalization, to map the data to the interval [0,1]. The normalized UAV flight status data is then input into the initially constructed collaborative association model. The model processes the input data based on its internal network structure and initial association weight parameters, outputting flight environment image feature prediction results. These prediction results include predicted values ​​for terrain texture features, landmark outline features, and sky brightness features.

[0033] Step S1243: Extract the actual features of the corresponding flight environment image data in the training dataset, and calculate the error value between the feature prediction result and the actual features.

[0034] In this step, actual features of the flight environment image data corresponding to the input flight state data are extracted from the training dataset, such as actual terrain texture features, landmark outline features, and sky brightness features. Then, an error calculation method, such as the mean squared error (MSE) method, is used to calculate the error value between the feature prediction result and the actual features. This error value is used to measure the accuracy of the model prediction.

[0035] Step S1244: Based on the error value, the association weight parameters in the collaborative association model are adjusted using the gradient descent method to gradually reduce the error value. After each adjustment, the adjusted association weight parameters are substituted into the collaborative association model, and the error value between the feature prediction result output by the collaborative association model and the actual feature is recalculated.

[0036] In this step, based on the calculated error value, the gradient descent method is used to adjust the association weight parameters in the collaborative association model. The gradient descent method updates the weight parameters along the negative gradient of the error function, gradually reducing the error value. After each adjustment, the adjusted weight parameters are substituted into the collaborative association model, and the flight state data from the training dataset is re-inputted. The error value between the model's output feature prediction and the actual features is calculated to observe the change in the error value.

[0037] Step S1245: Repeat the above adjustment process until the error value is less than the preset training error threshold. At this point, stop adjusting the association weight parameters in the collaborative association model to obtain the preliminary trained collaborative association model.

[0038] In this step, the adjustment process in step S1244 is repeated, that is, the weight parameters are continuously adjusted according to the error value, and the error value is recalculated until the error value is less than the preset training error threshold. When the error value meets the requirements, the adjustment of the weight parameters is stopped, and the resulting collaborative association model is the preliminary trained collaborative association model.

[0039] Step S1246: Input the normalized UAV flight status data in the validation dataset into the pre-trained collaborative association model to obtain the feature prediction results of the validation stage, extract the actual features of the corresponding flight environment image data in the validation dataset, and calculate the error value of the validation stage.

[0040] In this step, the UAV flight status data in the validation dataset is also normalized and then input into the pre-trained collaborative association model to obtain the feature prediction results for the validation stage. Simultaneously, the actual features of the corresponding flight environment image data are extracted from the validation dataset, and the error value for the validation stage is calculated using the same error calculation method as in step S1243.

[0041] Step S1247: When the error value in the verification stage is less than the preset verification error threshold, the pre-trained co-association model is determined to meet the training requirements and is taken as the co-association model after training is completed; when the error value in the verification stage is greater than or equal to the preset verification error threshold, return to the training dataset training step, increase the size of the training dataset or adjust the learning rate of the gradient descent method, and retrain the co-association model until the error value in the verification stage is less than the verification error threshold, and the co-association model after training is completed is obtained.

[0042] In this step, the error value in the validation phase is compared with a preset validation error threshold. If the error value in the validation phase is less than the validation error threshold, it means that the pre-trained co-association model meets the training requirements, and it is used as the trained co-association model. If the error value in the validation phase is greater than or equal to the validation error threshold, the process returns to step S1241, increasing the size of the training dataset, for example, by collecting more historical data and adding it to the training dataset, or by adjusting the learning rate of the gradient descent method, for example, by reducing the learning rate to make the model training more stable. Then, the co-association model is retrained until the error value in the validation phase is less than the validation error threshold, and the trained co-association model is obtained.

[0043] Step S125: Input the real-time acquired UAV flight status data into the trained collaborative association model to obtain the flight environment image feature prediction result corresponding to the UAV flight status data. At the same time, extract the actual features in the real-time flight environment image data and calculate the matching degree between the feature prediction result and the actual features.

[0044] In this step, for the aforementioned urban mapping UAV, real-time acquired UAV flight status data is input into the trained collaborative association model. The model outputs the flight environment image feature prediction results corresponding to the flight status data. Simultaneously, actual features are extracted from the real-time flight environment image data, such as actual terrain texture features, landmark outline features, and sky brightness features. Then, a matching degree calculation method, such as the cosine similarity method, is used to calculate the matching degree between the feature prediction results and the actual features. This matching degree is used to measure the similarity between the features predicted by the model and the actual features.

[0045] Step S126: Adjust the association weight parameters in the collaborative association model according to the matching degree so that the matching degree between the feature prediction result and the actual feature meets the preset requirements. Based on the adjusted association weight parameters, generate a collaborative association mapping relationship between UAV flight status data and flight environment image data.

[0046] In this step, the association weight parameters in the collaborative association model are adjusted based on the calculated matching degree. For example, if the matching degree is lower than a preset requirement, the gradient descent method is used to adjust the weight parameters until the matching degree between the feature prediction result and the actual feature meets the preset requirement. Once the matching degree meets the requirement, a collaborative association mapping relationship between the UAV flight status data and the flight environment image data is generated based on the adjusted association weight parameters. This collaborative association mapping relationship can be represented as a functional relationship, meaning that for a given UAV flight status data, the features of the corresponding flight environment image data can be obtained through this mapping relationship.

[0047] Step S130: Based on the collaborative association mapping relationship, perform feature recognition processing on the flight environment image data, identify the return reference marker for return positioning from the flight environment image data, and obtain the return reference marker location information. The return reference marker location information includes the coordinates of the return reference marker in the image coordinate system and the geographic coordinate system position corresponding to the coordinates.

[0048] Step S131: Based on the collaborative association mapping relationship, determine the return reference marker feature type that should be included in the flight environment image data corresponding to the current flight status data of the UAV. The return reference marker feature type includes at least one of the preset landmark outline feature type, terrain texture feature type and sky background feature type.

[0049] In this step, for the aforementioned urban mapping UAV, based on the collaborative association mapping relationship generated in step S126 and combined with the UAV's current flight status data, such as its current position, speed, and attitude, the return-to-home reference marker feature type that should be included in the flight environment image data corresponding to this flight status data is determined. For example, when the UAV's current position is close to a landmark building, based on the collaborative association mapping relationship, it is determined that the corresponding flight environment image data should include the landmark outline feature type of that landmark building, and may also include the terrain texture feature type of the surrounding terrain and the sky background feature type.

[0050] Step S132: Perform regional processing on the flight environment image data, dividing the flight environment image data into multiple image sub-regions. Each image sub-region corresponds to a specific spatial range, which is determined based on the UAV's flight altitude and the field of view of the visual acquisition system.

[0051] In this step, the flight environment image data of the aforementioned urban mapping UAV is processed by an image region segmentation algorithm. First, based on the UAV's flight altitude and the field of view of the visual acquisition system, the spatial range corresponding to each image sub-region is calculated. For example, if the field of view of the visual acquisition system is the horizontal angle α and the vertical angle β, and the UAV's flight altitude is h, then the projection range of each image sub-region on the ground can be calculated using trigonometric functions. Then, based on this spatial range, the flight environment image data is divided into multiple image sub-regions. The size and shape of each image sub-region can be determined based on the calculation results, for example, dividing it into multiple rectangular image sub-regions.

[0052] Step S133: Extract features from each image sub-region, extracting terrain texture features, landmark outline features, and sky background features from each image sub-region to obtain a multi-dimensional feature set corresponding to each image sub-region.

[0053] In this step, for each image sub-region, image feature extraction algorithms are used to extract terrain texture features, landmark contour features, and sky background features. For example, for terrain texture features, the gray-level co-occurrence matrix method can be used to extract features such as contrast and correlation; for landmark contour features, edge detection algorithms can be used to extract edge points of the contours, and then features such as the length and curvature of the contours can be calculated; for sky background features, brightness statistics methods can be used to extract features such as the average brightness and brightness variance of the sky. These features are then integrated to obtain a multi-dimensional feature set corresponding to each image sub-region. This multi-dimensional feature set contains multiple feature parameters for terrain texture features, landmark contour features, and sky background features.

[0054] Step S134: Compare the multi-dimensional feature set corresponding to each image sub-region with the features in the preset return reference sign feature library. The return reference sign feature library stores standard features corresponding to various return reference signs, including standard landmark outline features, standard terrain texture features and standard sky background features.

[0055] In this step, a pre-defined return-to-home reference sign feature library stores standard features corresponding to various return-to-home reference signs, such as standard landmark outline features of a landmark building, standard terrain texture features of the surrounding terrain, and standard sky background features. The multi-dimensional feature set corresponding to each image sub-region is compared with the standard features in the feature library, and a similarity calculation method, such as the Euclidean distance method, is used to calculate the similarity between the multi-dimensional feature set and the standard features.

[0056] Step S135: Based on the comparison results, select image sub-regions whose similarity to the standard features meets the preset conditions, and identify the image sub-regions as suspected reference label regions.

[0057] In this step, based on the comparison results in step S134, image sub-regions whose similarity to the multi-dimensional feature set and standard features meets preset conditions are selected. For example, when the similarity is greater than a preset similarity threshold, the image sub-region is identified as a suspected reference marker region, which may contain return-to-home reference markers used for return-to-home positioning.

[0058] Step S136: Extract detailed features from the suspected reference sign region, extracting edge features, color distribution features, and shape features from the suspected reference sign region to obtain a set of detailed features of the suspected reference sign region.

[0059] Step S1361: Perform image enhancement processing on the suspected reference sign region, and use an edge detection algorithm to extract the edges of the enhanced suspected reference sign region image to obtain a set of edge pixels of the suspected reference sign region. Based on the set of edge pixels, construct an edge contour curve, which is used to describe the outer contour shape of the suspected reference sign region.

[0060] In this step, for the suspected reference marker region, image enhancement processing is first performed, such as using histogram equalization to enhance image contrast and make details in the image clearer. Then, an edge detection algorithm, such as the Canny edge detection algorithm, is used to extract edges from the enhanced suspected reference marker region image, obtaining a set of edge pixels for the suspected reference marker region. Based on this set of edge pixels, a curve fitting method, such as the Bezier curve fitting method, is used to construct an edge contour curve, which can describe the outer contour shape of the suspected reference marker region.

[0061] Step S1362: Smooth the edge contour curve, and calculate the length, curvature and number of inflection points of the edge contour based on the smoothed edge contour curve as the edge features of the suspected reference identification area.

[0062] In this step, the constructed edge contour curve is smoothed, for example, by using a Gaussian filter to smooth the curve and remove noise points. Then, based on the smoothed edge contour curve, the length of the edge contour is calculated, for example, by calculating the sum of the distances between points on the curve; the curvature is calculated, for example, by calculating the curvature values ​​of the curve at each point and then taking the average or maximum value as the curvature feature; the number of inflection points is calculated, for example, by detecting points of concavity and convexity change in the curve. The features calculated above are used as the edge features of the suspected reference marker region.

[0063] Step S1363: Perform color space conversion on the enhanced suspected reference marker area image, converting it from RGB color space to HSV color space, and extract the hue, saturation, and brightness parameters from the HSV color space to obtain the color parameter set of the suspected reference marker area.

[0064] In this step, the enhanced image of the suspected reference marker region undergoes color space conversion, changing it from the RGB color space to the HSV color space. Within the HSV color space, hue, saturation, and brightness parameters are extracted. For example, by traversing each pixel in the image, the hue, saturation, and brightness values ​​of each pixel are obtained. These values ​​are then statistically analyzed to obtain a set of color parameters for the suspected reference marker region. This set of color parameters contains the hue, saturation, and brightness values ​​of different pixels.

[0065] Step S1364: Based on the set of color parameters, statistically analyze the percentage data in the suspected reference mark area to form the color distribution characteristics of the suspected reference mark area. The percentage data includes the percentage of pixels corresponding to different hues, the percentage of pixels corresponding to different saturations, and the percentage of pixels corresponding to different brightness.

[0066] In this step, based on the color parameter set obtained in step S1363, the proportion data of the suspected reference marker area is statistically analyzed. For example, the proportion of pixels corresponding to different hues is calculated, that is, the proportion of pixels of each hue to the total number of pixels is calculated; the proportion of pixels corresponding to different saturations is calculated, that is, the proportion of pixels of each saturation to the total number of pixels is calculated; the proportion of pixels corresponding to different brightness is calculated, that is, the proportion of pixels of each brightness to the total number of pixels is calculated. The above proportion data constitutes the color distribution characteristics of the suspected reference marker area.

[0067] Step S1365: Perform shape analysis on the enhanced suspected reference sign region image, and use the shape descriptor algorithm to calculate the circularity, rectangularity, and aspect ratio of the suspected reference sign region to describe the overall shape features of the suspected reference sign region; at the same time, extract the internal structural features of the suspected reference sign region, and combine the internal structural features with the overall shape features to form the shape features of the suspected reference sign region.

[0068] In this step, shape analysis is performed on the enhanced image of the suspected reference sign region. A shape descriptor algorithm, such as the Hu moment algorithm, is used to calculate the circularity, rectangularity, and aspect ratio of the suspected reference sign region. Circularity measures how closely the region's shape resembles a circle, rectangularity measures how closely it resembles a rectangle, and aspect ratio measures the ratio of the region's width to its height. Simultaneously, internal structural features are extracted from the suspected reference sign region, such as the number and size of holes within the region. These internal structural features are combined with the overall shape features to constitute the shape characteristics of the suspected reference sign region.

[0069] Step S1366: Integrate the extracted edge features, color distribution features and shape features to form a set of detailed features for the suspected reference marker region. Each set of detailed features includes an edge feature parameter group, a color distribution feature parameter group and a shape feature parameter group.

[0070] In this step, the edge features obtained in step S1362, the color distribution features obtained in step S1364, and the shape features obtained in step S1365 are integrated to form a set of detailed features for the suspected reference marker region. Each set of detailed features includes a set of edge feature parameters, such as the length, curvature, and number of inflection points of the edge contour; a set of color distribution feature parameters, such as the percentage of pixels corresponding to different hues, saturations, and brightness; and a set of shape feature parameters, such as roundness, rectangularity, aspect ratio, and internal structural features.

[0071] Step S137: Perform a secondary comparison between the detailed feature set and the detailed features of the corresponding standard features in the return reference sign feature library, calculate the similarity of the detailed features, and when the similarity exceeds a preset threshold, determine the corresponding suspected reference sign area as the target area containing the return reference sign.

[0072] In this step, the detailed feature set obtained in step S1366 is compared a second time with the detailed features of the corresponding standard features in the return-to-home reference marker feature library. The similarity calculation method used in step S134 is employed to calculate the similarity of the detailed features. When the similarity exceeds a preset threshold, the corresponding suspected reference marker area is determined as the target area containing the return-to-home reference marker; this target area is the area where the reference marker for UAV return-to-home positioning is located.

[0073] Step S138: Extract the coordinates of the target area in the image coordinate system of the flight environment image data, and combine the intrinsic and extrinsic parameters of the UAV vision acquisition system to convert the coordinates in the image coordinate system into coordinates in the geographic coordinate system, while recording the UAV flight status data corresponding to the target area.

[0074] In this step, the coordinates of the target area in the image coordinate system of the flight environment image data are extracted, such as the coordinates of the top-left and bottom-right corners of the area. Then, combining the intrinsic parameters of the UAV vision acquisition system, such as the camera's focal length and principal point coordinates, and the extrinsic parameters, such as the camera's position and attitude, a coordinate transformation algorithm is used to convert the coordinates in the image coordinate system to coordinates in the geographic coordinate system. Simultaneously, the UAV flight status data corresponding to the target area, such as the UAV's position, speed, and attitude, are recorded for subsequent calculations of relative positional relationships.

[0075] Step S139: Based on the coordinates of the target area in the image coordinate system, the coordinates in the geographic coordinate system, and the corresponding UAV flight status data, generate return-to-home reference marker location information. The return-to-home reference marker location information also includes the size information of the target area in the flight environment image data. The size information is used to assist in determining the distance between the UAV and the return-to-home reference marker.

[0076] In this step, based on the coordinates of the target area in the image coordinate system, the coordinates in the geographic coordinate system, and the corresponding UAV flight status data obtained in step S138, return-to-home reference marker location information is generated. This location information includes the coordinates of the target area in the image coordinate system, the coordinates in the geographic coordinate system, and also the size information of the target area in the flight environment image data, such as the width and height of the area. The size information can be converted into the actual length by calculating the number of pixels of the target area in the image coordinate system, combined with the pixel size of the visual acquisition system and the flight altitude. This size information is used to help determine the distance between the UAV and the return-to-home reference marker. For example, based on the size information and the field of view of the visual acquisition system, the approximate distance between the UAV and the return-to-home reference marker can be estimated.

[0077] Step S140: Combine the current position information in the UAV flight status data with the position information of the return-to-home reference marker, calculate the relative positional relationship between the UAV and the return-to-home reference marker, and plan the return-to-home path of the UAV from the current position to the location of the return-to-home reference marker based on the relative positional relationship to obtain the initial return-to-home path scheme.

[0078] Step S141: Extract the current location information from the UAV flight status data. The current location information includes the three-dimensional coordinates of the UAV in the geographic coordinate system. Extract the three-dimensional coordinates of the return-home reference marker in the geographic coordinate system from the return-home reference marker location information.

[0079] In this step, for the aforementioned urban mapping UAV, the current location information is extracted from the UAV's flight status data. This current location information includes the UAV's three-dimensional coordinates in the geographic coordinate system, such as (longitude 1, latitude 1, altitude 1). Simultaneously, the three-dimensional coordinates of the return-to-home reference marker in the geographic coordinate system are extracted from the return-to-home reference marker location information, such as (longitude 2, latitude 2, altitude 2).

[0080] Step S142: Convert the three-dimensional coordinates of the current location of the UAV in the geographic coordinate system and the three-dimensional coordinates of the return reference mark location in the geographic coordinate system into three-dimensional coordinates in the same plane coordinate system. The origin of the plane coordinate system is the return starting point, and the coordinate unit in the plane coordinate system is the length unit.

[0081] In this step, a coordinate transformation algorithm is used to convert the 3D coordinates of the UAV's current geographic location and the 3D coordinates of the return-to-home reference location into 3D coordinates in the same planar coordinate system. First, the origin of the planar coordinate system is determined as the return-to-home starting point, i.e., the position where the UAV triggers the return-to-home command. Then, the latitude and longitude coordinates in the geographic coordinate system are converted to x and y coordinates in the planar coordinate system, for example, using the Mercator projection method or the Gauss-Kruger projection method. The altitude coordinate remains unchanged, and the coordinate unit in the planar coordinate system is a length unit, such as a meter.

[0082] Step S143: Calculate the difference between the three-dimensional coordinates of the current position of the UAV in the plane coordinate system and the three-dimensional coordinates of the position of the return-to-home reference marker in the plane coordinate system. This yields the relative distances between the UAV and the return-to-home reference marker in the longitude, latitude, and altitude directions. The units of these relative distances are all length units, which constitute the basic parameters of the relative positional relationship between the UAV and the return-to-home reference marker.

[0083] In this step, the difference between the UAV's current 3D coordinates (x1, y1, z1) in a planar coordinate system and the return-to-home reference marker's 3D coordinates (x2, y2, z2) in a planar coordinate system is calculated. The relative distance in the longitude direction is x2-x1, the relative distance in the latitude direction is y2-y1, and the relative distance in the altitude direction is z2-z1. These relative distances are all in units of length, such as meters. These relative distances constitute the basic parameters of the relative positional relationship between the UAV and the return-to-home reference marker, which are used for subsequent path planning.

[0084] Step S144: Combining the UAV's flight speed information, analyze the flight time required for the UAV to reach the return-to-home reference marker position under the current flight attitude. At the same time, combine the terrain undulations corresponding to the terrain features in the flight environment image data to determine whether there are terrain obstacles under the current relative position.

[0085] In this step, by combining the drone's flight speed information, such as its current speed and direction, we analyze the flight time required for the drone to reach the return-to-home reference marker position in its current flight attitude. Flight time can be obtained by dividing the relative distance by the flight speed. Simultaneously, by combining the terrain features in the flight environment image data, such as changes in terrain height and the location of obstacles, we determine whether terrain obstacles exist in the current relative position. For example, by comparing the terrain height with the drone's flight altitude, we can determine whether terrain obstacles exist.

[0086] Step S145: When there are terrain obstacles, adjust the flight altitude parameters based on the relative positional relationship between the UAV and the return-to-home reference marker to keep the UAV's flight path at a preset safe distance from the terrain obstacles. At the same time, by adjusting the flight altitude parameters, ensure that the field of view of the visual acquisition system of the UAV always covers the area where the return-to-home reference marker is located during the flight, thereby continuously acquiring flight environment image data containing the return-to-home reference marker.

[0087] In this step, when a terrain obstacle is detected, the flight altitude parameters are adjusted based on the relative positional relationship between the UAV and the return-to-home (RTW) reference marker. For example, if the height of the terrain obstacle is higher than the UAV's current flight altitude, the flight altitude parameters are increased to maintain a preset safe distance between the UAV's flight path and the terrain obstacle, such as a certain length. Simultaneously, by adjusting the flight altitude parameters, the field of view of the UAV's visual acquisition system is ensured to consistently cover the area where the RTF reference marker is located during flight. For example, a suitable flight altitude is calculated based on the field of view of the visual acquisition system and the position of the RTF reference marker to ensure that the field of view covers the area, thereby continuously acquiring flight environment image data containing the RTF reference marker.

[0088] Step S146: Based on the adjusted flight altitude parameters and the relative positional relationship between the UAV and the return-to-home reference marker, construct a return-to-home path planning model. The return-to-home path planning model takes the current position of the UAV, the position of the return-to-home reference marker, the flight speed, and terrain obstacle information as inputs, and outputs the coordinates of the path nodes of the return-to-home path.

[0089] In this step, a return path planning model is constructed based on the adjusted flight altitude parameters and the relative positional relationship between the UAV and the return-to-home reference marker. This model can employ path planning algorithms such as A* or RRT (Fast Search Random Tree). The inputs to the return-to-home path planning model are the UAV's current position, the return-to-home reference marker's position, flight speed, and terrain obstacle information, including the obstacle's location and altitude. The output of the model is the coordinates of the path nodes, which constitute the basic framework of the return path.

[0090] Step S147: Generate multiple candidate return routes through the return route planning model. Each candidate return route contains multiple consecutive path nodes. Each path node corresponds to a location coordinate in the geographic coordinate system. At the same time, record the path length and estimated flight time of each candidate return route.

[0091] In this step, multiple candidate return routes are generated using a return route planning model. For example, when using the A* algorithm, different heuristic functions or search parameters are set to generate multiple different candidate return routes. Each candidate return route contains multiple consecutive path nodes, each path node corresponding to a location coordinate in a geographic coordinate system, such as (longitude, latitude, altitude). Simultaneously, the path length of each candidate return route is recorded, for example, by calculating the sum of the distances between path nodes; the estimated flight time is also recorded, for example, by dividing the path length by the flight speed.

[0092] Step S148: Conduct a feasibility assessment for each candidate return path and obtain the feasibility assessment results. The assessment indicators include whether the candidate return path avoids terrain obstacles, whether the UAV can continuously obtain return reference signs during the path flight, and whether the estimated flight time of the path is within a reasonable range.

[0093] Step S1481: Obtain the terrain features corresponding to each image sub-region in the flight environment image data, and combine them with the UAV's flight altitude parameters to determine whether there is a situation where the spatial position of each path node in each candidate return path has a terrain altitude that exceeds the UAV's flight altitude. If so, the candidate return path has not avoided terrain obstacles; if not, the candidate return path has avoided terrain obstacles.

[0094] In this step, terrain features corresponding to each image sub-region in the flight environment image data are acquired, such as terrain height information. Combined with the UAV's flight altitude parameters, it is determined whether the spatial location corresponding to each path node in each candidate return path has a terrain height exceeding the UAV's flight altitude. For example, for each path node, its corresponding terrain height is acquired. If the terrain height is greater than the UAV's flight altitude, it indicates that there is a terrain obstacle at that path node, and the candidate return path does not avoid the terrain obstacle; if the terrain height is less than or equal to the UAV's flight altitude, it indicates that there is no terrain obstacle at that path node, and the candidate return path avoids the terrain obstacle.

[0095] Step S1482: For each candidate return path, calculate the expected flight time of the UAV at each path node based on the path node sequence and the UAV's flight speed. Combine the field of view and image acquisition frequency of the visual acquisition system to determine whether the UAV can acquire flight environment image data containing the return reference marker at each path node within the expected flight time. The determination method is: calculate the proportion of the return reference marker in the field of view of the visual acquisition system. When the proportion exceeds the preset proportion threshold, it is determined that the flight environment image data containing the return reference marker can be acquired.

[0096] In this step, for each candidate return-to-home path, the estimated flight time of the UAV at each path node is calculated based on the path node sequence and the UAV's flight speed. For example, the arrival time at each path node is calculated based on the distance between path nodes and the flight speed. Then, combining the field of view and image acquisition frequency of the visual acquisition system, it is determined whether the UAV can acquire flight environment image data containing the return-to-home reference marker at each path node within the estimated flight time. The specific determination method is as follows: calculate the proportion of the return-to-home reference marker in the field of view of the visual acquisition system, for example, by calculating the proportion of the number of pixels of the return-to-home reference marker in the image to the number of pixels in the corresponding field of view. When the proportion exceeds a preset proportion threshold, it is determined that flight environment image data containing the return-to-home reference marker can be acquired; when the proportion is lower than or equal to the preset proportion threshold, it is determined that flight environment image data containing the return-to-home reference marker cannot be acquired.

[0097] Step S1483: If flight environment image data containing return-to-home reference markers can be collected at all path nodes, then the candidate return-to-home path meets the requirement of continuously acquiring return-to-home reference markers. If flight environment image data containing return-to-home reference markers cannot be collected at at least one path node, then the candidate return-to-home path does not meet the requirement of continuously acquiring return-to-home reference markers.

[0098] In this step, based on the judgment result in step S1482, it is determined whether each candidate return path meets the requirement of continuously acquiring return reference identifiers. If flight environment image data containing return reference identifiers can be collected at all path nodes, then the candidate return path meets the requirement of continuously acquiring return reference identifiers; if at least one path node cannot collect flight environment image data containing return reference identifiers, then the candidate return path does not meet the requirement of continuously acquiring return reference identifiers.

[0099] Step S1484: Calculate the total path length of each candidate return path, and combine it with the average flight speed of the UAV to calculate the estimated total flight time of each candidate return path. Compare the estimated total flight time with the preset maximum allowed flight time.

[0100] In this step, the total path length of each candidate return path is calculated, for example, by summing the distances between path nodes. Combined with the UAV's average flight speed, the estimated total flight time for each candidate return path is calculated; the estimated total flight time equals the total path length divided by the average flight speed. Then, this estimated total flight time is compared with the preset maximum allowable flight time.

[0101] Step S1485: If the estimated total flight time is less than or equal to the preset maximum allowable flight time, then the estimated flight time of the candidate return route is within a reasonable range; if the estimated total flight time is greater than the preset maximum allowable flight time, then the estimated flight time of the candidate return route is not within a reasonable range.

[0102] In this step, based on the comparison results in step S1484, it is determined whether the estimated flight time of each candidate return route is within a reasonable range. If the estimated total flight time is less than or equal to the preset maximum allowable flight time, then the estimated flight time of the candidate return route is within a reasonable range; if the estimated total flight time is greater than the preset maximum allowable flight time, then the estimated flight time of the candidate return route is not within a reasonable range.

[0103] Step S1486: For each candidate return route, count the number of evaluation indicators that are met. When all three evaluation indicators are met, the candidate return route passes the feasibility assessment. When any one evaluation indicator is missing, the candidate return route fails the feasibility assessment.

[0104] In this step, for each candidate return route, the number of evaluation indicators that are met is counted. Evaluation indicators include whether the candidate return route avoids terrain obstacles, whether it meets the requirement of continuously acquiring return reference signs, and whether the estimated flight time is within a reasonable range. When all three evaluation indicators are met, the candidate return route passes the feasibility assessment; when any one evaluation indicator is missing, the candidate return route fails the feasibility assessment.

[0105] Step S1487: Analyze the candidate return routes that fail the feasibility assessment. If the failure is due to not avoiding terrain obstacles, adjust the coordinates of some path nodes in the candidate return route, increase the flight altitude parameters, and regenerate the candidate return route.

[0106] In this step, candidate return paths that failed the feasibility assessment are analyzed. If the failure was due to failure to avoid terrain obstacles, the coordinates of some path nodes in the candidate return path are adjusted, for example, by moving the path nodes above or to the side of the terrain obstacles, while increasing the flight altitude parameter to enable the UAV to avoid the terrain obstacles. Then, candidate return paths are regenerated using the same method as in step S147.

[0107] Step S1488: If the evaluation fails due to the inability to continuously acquire the return-to-home reference marker, adjust the distribution density of the path nodes, increase the number of path nodes, and make the distance between adjacent path nodes smaller than the ground distance corresponding to the field of view of the visual acquisition system, so that the UAV can continuously acquire image data containing the return-to-home reference marker during flight and regenerate the candidate return-to-home path.

[0108] In this step, if the evaluation fails due to the inability to continuously acquire the return-to-home reference marker, the distribution density of the path nodes is adjusted by increasing the number of path nodes. For example, the distance between adjacent path nodes is reduced so that the distance between adjacent path nodes is less than the ground distance corresponding to the field of view of the visual acquisition system. In this way, during the flight of the UAV, the field of view of the visual acquisition system can cover the area where the return-to-home reference marker is located, thereby continuously acquiring image data containing the return-to-home reference marker. Then, candidate return-to-home paths are regenerated, using the same method as in step S147.

[0109] Step S1489: If the evaluation fails due to the estimated flight time being outside the reasonable range, optimize the flight routes between path nodes, delete redundant path nodes, regenerate candidate return routes, and conduct a feasibility evaluation on the regenerated candidate return routes again until a candidate return route that passes the feasibility evaluation is obtained.

[0110] In this step, if the evaluation fails due to the estimated flight time being outside the reasonable range, the flight routes between path nodes are optimized. For example, shorter paths are used to connect path nodes, and redundant path nodes are removed to reduce the total path length. Then, candidate return routes are regenerated using the same method as in step S147. The regenerated candidate return routes are then subjected to another feasibility evaluation using the same method as in step S148, until a candidate return route that passes the feasibility evaluation is obtained.

[0111] Step S149: Based on the feasibility assessment results, select candidate return routes that meet all assessment indicators, and choose the route with the shortest path length and the shortest expected flight time from the candidate return routes that meet the conditions as the initial return route scheme.

[0112] In this step, based on the feasibility assessment results in step S148, candidate return paths that meet all assessment indicators are selected. Then, from these candidate return paths that meet the conditions, the path with the shortest path length and the shortest expected flight time is selected as the initial return path scheme. For example, by comparing the path length and expected flight time of each candidate return path, the path with the shortest path length and the shortest expected flight time is selected, and this path is the initial return path scheme for the UAV.

[0113] Step S150: Based on the initial return path scheme, and combined with the flight attitude information and flight speed information in the UAV flight status data, generate flight control commands for controlling the UAV flight, send the flight control commands to the UAV flight control system, and control the UAV to perform the return operation according to the flight control commands.

[0114] Step S151: Analyze the initial return path scheme, extract the coordinates of all path nodes in the initial return path scheme, sort the path nodes according to the flight order to obtain the path node sequence, and each path node corresponds to a target location.

[0115] In this step, the initial return path scheme of the aforementioned urban mapping UAV is analyzed to extract the coordinates of all path nodes, such as the geographic coordinates (longitude, latitude, and altitude) of each path node. Then, the path nodes are sorted according to the flight order to obtain a path node sequence. Each path node corresponds to a target location, and the UAV needs to fly towards these target locations in sequence.

[0116] Step S152: Calculate the flight direction between two adjacent path nodes based on the path node sequence. The flight direction is determined based on the coordinates of the previous path node and the coordinates of the next path node. At the same time, calculate the distance between two adjacent path nodes.

[0117] In this step, the flight direction between two adjacent path nodes is calculated based on the path node sequence. For example, given the coordinates (x1, y1, z1) of the preceding path node and the coordinates (x2, y2, z2) of the following path node, the flight direction can be obtained by calculating the direction of the vector (x2-x1, y2-y1, z2-z1), which can be represented by angular parameters such as azimuth and pitch angles. Simultaneously, the distance between two adjacent path nodes is calculated, for example, by calculating the magnitude of the vector; the unit of distance is length.

[0118] Step S153: Combine the current flight speed information in the UAV flight status data to calculate the flight time required for the UAV to fly from the current path node to the next path node. Based on the flight time and the distance between adjacent path nodes, adjust the UAV's flight speed so that the time for the UAV to reach the next path node is consistent with the preset time.

[0119] In this step, the flight time required for the drone to travel from the current path node to the next path node is calculated by combining the current flight speed information from the drone's flight status data, such as the current flight speed magnitude. The flight time is equal to the distance between adjacent path nodes divided by the current flight speed. Then, the drone's flight speed is adjusted based on this flight time and a preset time. For example, if the flight time is greater than the preset time, the flight speed is increased; if the flight time is less than the preset time, the flight speed is decreased, so that the time for the drone to reach the next path node matches the preset time.

[0120] Step S154: Based on the current flight attitude information of the UAV and the flight direction corresponding to the next path node, calculate the attitude angle that the UAV needs to adjust. The attitude angle includes pitch angle, roll angle and yaw angle. By adjusting the attitude angle, the flight direction of the UAV is made consistent with the flight direction corresponding to the next path node.

[0121] In this step, based on the UAV's current flight attitude information, such as the current pitch, roll, and yaw angles, and the flight direction corresponding to the next path node, the attitude angles that the UAV needs to adjust are calculated. For example, by calculating the angle difference between the current flight direction and the target flight direction, the pitch, roll, and yaw angles that need to be adjusted are obtained. By adjusting these attitude angles, the UAV's flight direction is made consistent with the flight direction corresponding to the next path node.

[0122] Step S155: Based on the adjusted flight speed and attitude angle, generate corresponding speed control parameters and attitude control parameters. The speed control parameters are used to adjust the magnitude and rate of change of the UAV's flight speed, and the attitude control parameters are used to adjust the rate of change of the UAV's pitch angle, roll angle, and yaw angle.

[0123] In this step, corresponding speed control parameters and attitude control parameters are generated based on the adjusted flight speed and attitude angles. The speed control parameters are used to adjust the magnitude and rate of change of the UAV's flight speed, such as the adjustment amount of speed and the magnitude of acceleration; the attitude control parameters are used to adjust the rate of change of the UAV's pitch, roll, and yaw angles, such as the amount of angle change and the magnitude of angular velocity.

[0124] Step S156: Integrate the speed control parameters and attitude control parameters according to a preset instruction format to generate flight control instructions. Each flight control instruction corresponds to the flight control requirements of a path node segment, where the path node segment is the flight path between two adjacent path nodes.

[0125] In this step, the speed control parameters and attitude control parameters are integrated according to a preset command format to generate flight control commands. The preset command format may include the command type, parameter value range, parameter order, etc. Each flight control command corresponds to the flight control requirements of a path node segment, that is, the control requirements of the flight segment between two adjacent path nodes, such as the control requirements of the flight segment from path node A to path node B.

[0126] Step S157: Send the flight control commands to the UAV's flight control system in the order of the path node sequence, so that the flight control system can parse the speed control parameters and attitude control parameters in the flight control commands, adjust the UAV's power system output according to the speed control parameters, and adjust the UAV's control surface angle according to the attitude control parameters, thereby controlling the UAV to fly towards the next path node according to the flight control commands.

[0127] In this step, flight control commands are sent to the UAV's flight control system in the order of the path node sequence. After parsing the speed control parameters and attitude control parameters in the flight control commands, the flight control system adjusts the UAV's power system output according to the speed control parameters, such as adjusting the motor speed to change the magnitude and rate of change of flight speed; and adjusts the UAV's control surface angles according to the attitude control parameters, such as adjusting the angles of the elevator, ailerons, and rudder to change the rate of change of pitch, roll, and yaw angles, thereby controlling the UAV to fly towards the next path node according to the flight control commands.

[0128] Step S158: During the flight of the UAV, based on the UAV flight status data output by the inertial navigation system and the flight environment image data collected by the visual acquisition system, determine whether the UAV has deviated from the initial return path plan.

[0129] For example, step S1581: During the flight of the UAV in accordance with the flight control command, the current flight status data of the UAV output by the inertial navigation system is acquired in real time at preset time intervals. The current flight status data of the UAV includes the current three-dimensional position coordinates, flight attitude angle and flight speed of the UAV.

[0130] In this step, as the UAV flies according to flight control commands, the current flight status data of the UAV output by the inertial navigation system is acquired in real time at preset time intervals, such as every 0.1 seconds. This current flight status data includes the UAV's current three-dimensional position coordinates, such as (longitude, latitude, altitude); flight attitude angles, such as pitch angle, roll angle, and yaw angle; and flight speed, such as velocity components along each axis of the geographic coordinate system.

[0131] Step S1582: Acquire the current flight environment image data collected by the visual acquisition system in real time at the same time interval, extract features from the current flight environment image data to obtain the current terrain texture features, current landmark outline features and current sky background features.

[0132] In this step, the current flight environment image data collected by the visual acquisition system is acquired in real time at the same time interval as in step S1581. Then, feature extraction is performed on the current flight environment image data, using the same feature extraction method as in step S133, to obtain the current terrain texture features, current landmark outline features, and current sky background features.

[0133] Step S1583: Extract the target path node coordinates corresponding to the current flight time of the UAV in the initial return path plan. The target path node coordinates are the planar coordinates of the path node that the UAV should reach in the current flight time.

[0134] In this step, the coordinates of the target path nodes corresponding to the current flight time of the UAV are extracted from the initial return path plan. For example, based on the UAV's flight speed and the sequence of path nodes in the initial return path plan, the path node that the UAV should reach at the current flight time is calculated, and then the planar coordinates of that path node are extracted.

[0135] Step S1584: Calculate the distance difference between the current three-dimensional position coordinates of the UAV and the coordinates of the target path node. The distance difference is the Euclidean distance in the plane coordinate system. Compare the distance difference with the preset position deviation threshold. If the distance difference is less than the position deviation threshold, it is preliminarily determined that the UAV has not deviated from the initial return path plan.

[0136] In this step, the distance difference between the UAV's current 3D position coordinates and the target path node coordinates is calculated. This distance difference is the Euclidean distance in a planar coordinate system, for example, by calculating the square root of the sum of the squares of the coordinate differences. This distance difference is compared with a preset position deviation threshold. If the distance difference is less than the position deviation threshold, it is preliminarily determined that the UAV has not deviated from the initial return path plan; if the distance difference is greater than or equal to the position deviation threshold, further judgment is required.

[0137] Step S1585: If the distance difference is greater than or equal to the position deviation threshold, then further combine the features in the current flight environment image data to make a judgment, and compare the current terrain texture features, current landmark outline features and current sky background features with the preset features at the corresponding path nodes in the initial return path plan.

[0138] In this step, if the distance difference is greater than or equal to the position deviation threshold, the characteristics in the current flight environment image data are further combined for judgment. The current terrain texture features, current landmark outline features, and current sky background features are compared with the preset features at the corresponding path nodes in the initial return path plan. The similarity calculation method is the same as in step S134 to calculate the similarity between the current features and the preset features.

[0139] Step S1586: Calculate the similarity between the current feature and the preset feature. If the similarity exceeds the preset feature similarity threshold, the deviation is corrected by adjusting the flight control command. If the similarity does not exceed the preset feature similarity threshold, the return route is replanned.

[0140] In this step, the similarity between the current feature and the preset feature is calculated and then compared with the preset feature similarity threshold. If the similarity exceeds the preset feature similarity threshold, it means that although the UAV's position is deviated, the characteristics of the flight environment are consistent with the preset features, and the deviation can be corrected by adjusting the flight control commands. If the similarity does not exceed the preset feature similarity threshold, it means that the UAV's flight environment is inconsistent with the preset environment, and the return path needs to be replanned.

[0141] Step S1587: Based on the current flight attitude angle of the UAV, calculate the attitude angle difference between the current flight attitude angle and the target flight attitude angle at the corresponding path node in the initial return path plan, and compare the attitude angle difference with the preset attitude deviation threshold.

[0142] In this step, the difference between the current flight attitude angle of the UAV and the target flight attitude angle at the corresponding path node in the initial return path plan is calculated, taking into account the current flight attitude angle of the UAV. For example, the differences in pitch angle, roll angle, and yaw angle are calculated. This attitude angle difference is then compared with a preset attitude deviation threshold.

[0143] Step S1588: If the attitude angle difference is less than the attitude deviation threshold, and the distance difference is less than the position deviation threshold and the similarity between the current feature and the preset feature exceeds the feature similarity threshold, then it is determined that the UAV has not deviated from the initial return path plan and continues to fly according to the current flight control command; if the attitude angle difference is greater than or equal to the attitude deviation threshold, or the distance difference is greater than the position deviation threshold and the similarity between the current feature and the preset feature does not exceed the feature similarity threshold, then it is determined that the UAV has deviated from the initial return path plan and it is necessary to generate new flight control commands or replan the return path.

[0144] In this step, based on the comparison results of steps S1584, S1586, and S1587, it is determined whether the UAV has deviated from the initial return-to-home path. If the attitude angle difference is less than the attitude deviation threshold, and the distance difference is less than the position deviation threshold, and the similarity between the current feature and the preset feature exceeds the feature similarity threshold, then it is determined that the UAV has not deviated from the initial return-to-home path and continues to fly according to the current flight control command. If the attitude angle difference is greater than or equal to the attitude deviation threshold, or the distance difference is greater than the position deviation threshold and the similarity between the current feature and the preset feature does not exceed the feature similarity threshold, then it is determined that the UAV has deviated from the initial return-to-home path, and new flight control commands need to be generated or the return-to-home path needs to be replanned.

[0145] Step S1589: Record the results of each judgment, including position deviation value, feature similarity value and attitude deviation value, and construct a UAV flight deviation record set. Analyze the deviation trend of the UAV based on the UAV flight deviation record set. If the deviation trend is expanding, shorten the adjustment time interval of flight control commands. If the deviation trend is shrinking, maintain the current adjustment frequency of flight control commands.

[0146] In this step, the results of each judgment are recorded, including position deviation, feature similarity, and attitude deviation. These results are then compiled into a UAV flight deviation record set. Based on this set, the deviation trend of the UAV is analyzed. For example, by observing the changes in position deviation, feature similarity, and attitude deviation, it is determined whether the deviation trend is increasing or decreasing. If the deviation trend is increasing, the adjustment time interval of flight control commands is shortened, for example, from 0.1 seconds to 0.05 seconds; if the deviation trend is decreasing, the current adjustment frequency of flight control commands is maintained.

[0147] Step S159: When the UAV is found to have deviated from the initial return path, the relative position between the current position of the UAV and the next path node is recalculated. Based on the recalculated relative position, the speed control parameters and attitude control parameters are adjusted, new flight control commands are generated, and the new flight control commands are sent to the flight control system. The flight control system executes the new flight control commands to correct the UAV's flight path and bring the UAV back to the flight trajectory corresponding to the initial return path.

[0148] In this step, when the UAV deviates from the initial return-to-home path, the relative position between the UAV's current position and the next path node is recalculated, using the same method as in step S143. Based on the recalculated relative position, the speed control parameters and attitude control parameters are adjusted, using the same method as in steps S153 and S154. Then, new flight control commands are generated, using the same method as in steps S155 and S156. The new flight control commands are sent to the flight control system, which executes them, adjusting the UAV's power system output and control surface angles to correct the UAV's flight path and return it to the flight trajectory corresponding to the initial return-to-home path.

[0149] Based on the same inventive concept, please refer to Figure 2 The diagram shows a schematic block diagram of an automatic return-to-home system 100 based on an inertial navigation and vision system, provided in an embodiment of this application, for executing the above-described automatic return-to-home method for UAVs based on inertial navigation and vision systems. The automatic return-to-home system 100 based on an inertial navigation and vision system may include a communication unit 110, a machine-readable storage medium 120, and a processor 130.

[0150] In this embodiment, both the machine-readable storage medium 120 and the processor 130 are located within the UAV automatic return-to-home system 100 based on inertial navigation and vision systems, and are separately configured. However, it should be understood that the machine-readable storage medium 120 may also be independent of the UAV automatic return-to-home system 100 based on inertial navigation and vision systems, and may be accessed by the processor 130 via a bus interface. Alternatively, the machine-readable storage medium 120 may also be integrated into the processor 130 and may communicate and interact with external systems through the communication unit 110.

[0151] The processor 130 is the control center of the inertial navigation and vision-based UAV automatic return-to-home system 100. It connects various parts of the system via various interfaces and lines, and performs overall monitoring of the system by running or executing software programs and / or modules stored in the machine-readable storage medium 120, and by calling data stored in the machine-readable storage medium 120. Optionally, the processor 130 may include one or more processing cores; for example, it may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, and the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. The machine-readable storage medium 120 is used to store machine-executable instructions for executing the scheme of this application, and the processor 130 is used to execute the machine-executable instructions stored in the machine-readable storage medium 120 to realize the UAV automatic return method based on inertial navigation and vision system provided in the aforementioned method embodiment.

[0152] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for automatic return-to-home of a UAV based on inertial navigation and vision systems, characterized in that, The method includes: When the drone triggers the return-to-home command, the drone's flight status data is obtained in real time from the inertial navigation system on the drone, and the flight environment image data is obtained in real time from the visual acquisition system on the drone. The drone's flight status data includes the drone's flight attitude information, flight speed information and current position information, and the flight environment image data includes terrain feature images, landmark feature images and sky background feature images around the drone's flight path. The data acquisition frequency of the UAV flight status data and the image acquisition frequency of the flight environment image data are synchronized. Based on the time-synchronized UAV flight status data and the time-synchronized flight environment image data, a collaborative association model is constructed. This collaborative association model takes each parameter in the time-synchronized UAV flight status data as input variables and each feature in the time-synchronized flight environment image data as output variables. The collaborative association model is trained by adjusting the association weight parameters in the collaborative association model so that the deviation between the features output by the collaborative association model and the actual features in the time-synchronized flight environment image data is within a preset range, thus obtaining the trained collaborative association model. The real-time acquired UAV flight status data is input into the trained collaborative association model to obtain the flight environment image feature prediction result corresponding to the UAV flight status data. At the same time, the actual features in the real-time flight environment image data are extracted, and the matching degree between the feature prediction result and the actual features is calculated. The association weight parameters in the collaborative association model are adjusted according to the matching degree to ensure that the matching degree between the feature prediction results and the actual features meets the preset requirements. Based on the adjusted association weight parameters, a collaborative association mapping relationship between UAV flight status data and flight environment image data is generated. Based on the collaborative association mapping relationship, feature recognition processing is performed on the flight environment image data to identify the return reference marker for return positioning from the flight environment image data, and the return reference marker location information is obtained. The return reference marker location information includes the coordinates of the return reference marker in the image coordinate system and the geographic coordinate system position corresponding to the coordinates. By combining the current position information in the UAV flight status data with the position information of the return-to-home reference marker, the relative positional relationship between the UAV and the return-to-home reference marker is calculated. Based on the relative positional relationship, the return-to-home path of the UAV from its current position to the location of the return-to-home reference marker is planned to obtain the initial return-to-home path scheme. Based on the initial return path scheme, and combined with the flight attitude and speed information in the UAV flight status data, flight control commands are generated to control the flight of the UAV. The flight control commands are then sent to the UAV's flight control system to control the UAV to perform the return operation according to the flight control commands.

2. The automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems according to claim 1, characterized in that, Also includes: The parameter dimensions of the UAV flight status data are analyzed, and the attitude parameters corresponding to the flight attitude information, the speed parameters corresponding to the flight speed information, and the position parameters corresponding to the current position information are extracted from the UAV flight status data. The numerical type and data acquisition frequency of each parameter are determined. The feature dimensions of the flight environment image data are analyzed, and feature extraction is performed on the flight environment image data to obtain the terrain texture features corresponding to the terrain feature image, the landmark outline features corresponding to the landmark feature image, and the sky brightness features corresponding to the sky background feature image. The feature type and image acquisition frequency of each feature are then determined.

3. The automatic return-to-home method for UAVs based on inertial navigation and vision systems according to claim 2, characterized in that, The training of the collaborative association model, by adjusting the association weight parameters in the collaborative association model to ensure that the deviation between the features output by the collaborative association model and the actual features in the time-synchronized flight environment image data is within a preset range, yields a trained collaborative association model, including: Collect multiple sets of historical time-synchronized UAV flight status data and time-synchronized flight environment image data, and divide them into training datasets and validation datasets accordingly. The UAV flight status data in the training dataset is normalized, and the normalized UAV flight status data is input into the initially constructed collaborative association model to obtain the flight environment image feature prediction results output by the collaborative association model. Extract the actual features of the corresponding flight environment image data in the training dataset, and calculate the error value between the feature prediction result and the actual features; Based on the error value, the association weight parameters in the collaborative association model are adjusted using the gradient descent method to gradually reduce the error value. After each adjustment, the adjusted association weight parameters are substituted into the collaborative association model, and the error value between the feature prediction result output by the collaborative association model and the actual feature is recalculated. Repeat the above adjustment process until the error value is less than the preset training error threshold. At this point, stop adjusting the association weight parameters in the collaborative association model to obtain the preliminary trained collaborative association model. The normalized UAV flight status data in the validation dataset is input into the pre-trained collaborative association model to obtain the feature prediction results of the validation stage. The actual features of the corresponding flight environment image data in the validation dataset are extracted, and the error value of the validation stage is calculated. When the error value during the verification phase is less than the preset verification error threshold, the pre-trained collaborative association model is determined to meet the training requirements and is regarded as the completed collaborative association model. When the error value in the validation phase is greater than or equal to the preset validation error threshold, return to the training dataset training step, increase the size of the training dataset or adjust the learning rate of the gradient descent method, and retrain the co-association model until the error value in the validation phase is less than the validation error threshold, and obtain the trained co-association model.

4. The automatic return-to-home method for UAVs based on inertial navigation and vision systems according to claim 1, characterized in that, Based on the collaborative association mapping relationship, feature recognition processing is performed on the flight environment image data to identify the return-to-home reference marker for return-to-home positioning from the flight environment image data, and the location information of the return-to-home reference marker is obtained, including: Based on the aforementioned collaborative association mapping relationship, the return-home reference marker feature type that should be included in the flight environment image data corresponding to the current flight status data of the UAV is determined. The return-home reference marker feature type includes at least one of the preset landmark outline feature type, terrain texture feature type, and sky background feature type. The flight environment image data is processed by dividing it into multiple image sub-regions. Each image sub-region corresponds to a specific spatial range, which is determined based on the UAV's flight altitude and the field of view of the visual acquisition system. Feature extraction is performed on each image sub-region, extracting terrain texture features, landmark outline features, and sky background features from each image sub-region to obtain a multi-dimensional feature set corresponding to each image sub-region; The multi-dimensional feature set corresponding to each image sub-region is compared with the features in the preset return reference sign feature library. The return reference sign feature library stores standard features corresponding to various return reference signs, including standard landmark outline features, standard terrain texture features and standard sky background features. Based on the comparison results, image sub-regions whose similarity to standard features meets the preset conditions are selected, and these image sub-regions are identified as suspected reference label regions. Detail feature extraction is performed on the suspected reference sign region, extracting edge features, color distribution features, and shape features in the suspected reference sign region to obtain a set of detailed features of the suspected reference sign region; The detailed feature set is compared with the detailed features of the corresponding standard features in the return reference sign feature library. The similarity of the detailed features is calculated. When the similarity exceeds a preset threshold, the corresponding suspected reference sign area is determined as the target area containing the return reference sign. Extract the coordinates of the target area in the image coordinate system of the flight environment image data, and combine the intrinsic and extrinsic parameters of the UAV vision acquisition system to convert the coordinates in the image coordinate system into coordinates in the geographic coordinate system, while recording the UAV flight status data corresponding to the target area. Based on the coordinates of the target area in the image coordinate system, the coordinates in the geographic coordinate system, and the corresponding UAV flight status data, return-to-home reference marker location information is generated. This return-to-home reference marker location information also includes the size information of the target area in the flight environment image data. The size information is used to assist in determining the distance between the UAV and the return-to-home reference marker.

5. The automatic return-to-home method for UAVs based on inertial navigation and vision systems according to claim 4, characterized in that, The step involves extracting detailed features from the suspected reference sign region, including edge features, color distribution features, and shape features, to obtain a set of detailed features for the suspected reference sign region, including: The suspected reference sign region is subjected to image enhancement processing, and an edge detection algorithm is used to extract the edges of the enhanced suspected reference sign region image to obtain the edge pixel set of the suspected reference sign region. Based on the edge pixel set, an edge contour curve is constructed, which is used to describe the outer contour shape of the suspected reference sign region. The edge contour curve is smoothed, and the length, curvature and number of inflection points of the edge contour are calculated based on the smoothed edge contour curve as edge features of the suspected reference identification area. The enhanced image of the suspected reference marker region is converted from RGB color space to HSV color space. The hue, saturation and brightness parameters in the HSV color space are extracted to obtain the color parameter set of the suspected reference marker region. Based on the set of color parameters, the percentage data of the suspected reference mark area is statistically analyzed to form the color distribution characteristics of the suspected reference mark area. The percentage data includes the percentage of the number of pixels corresponding to different hues, the percentage of the number of pixels corresponding to different saturations, and the percentage of the number of pixels corresponding to different brightness. Shape analysis is performed on the enhanced image of the suspected reference sign region. The shape descriptor algorithm is used to calculate the circularity, rectangularity, and aspect ratio of the suspected reference sign region to describe the overall shape characteristics of the suspected reference sign region. Simultaneously, the internal structural features of the suspected reference mark area are extracted, and the internal structural features are combined with the overall shape features to form the shape features of the suspected reference mark area. The extracted edge features, color distribution features, and shape features are integrated to form a set of detailed features for the suspected reference marker region. Each set of detailed features contains a set of edge feature parameters, a set of color distribution feature parameters, and a set of shape feature parameters.

6. The automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems according to claim 1, characterized in that, The process involves combining the current position information from the UAV's flight status data with the position information of the return-to-home reference marker to calculate the relative positional relationship between the UAV and the return-to-home reference marker. Based on this relative positional relationship, a return-to-home path is planned for the UAV from its current position to the location of the return-to-home reference marker, resulting in an initial return-to-home path scheme, including: Extract the current location information from the UAV flight status data, which includes the three-dimensional coordinates of the UAV in the geographic coordinate system; and extract the three-dimensional coordinates of the return-home reference marker in the geographic coordinate system from the return-home reference marker location information. The three-dimensional coordinates of the current location of the UAV in the geographic coordinate system and the three-dimensional coordinates of the return reference mark location in the geographic coordinate system are converted into three-dimensional coordinates in the same plane coordinate system. The origin of this plane coordinate system is the return point, and the coordinate units in the plane coordinate system are the length units. The difference between the three-dimensional coordinates of the current position of the UAV in the plane coordinate system and the three-dimensional coordinates of the position of the return-to-home reference marker in the plane coordinate system is calculated to obtain the relative distance between the UAV and the return-to-home reference marker in the longitude, latitude and altitude directions. The units of the relative distances are all length units, which constitute the basic parameters of the relative positional relationship between the UAV and the return-to-home reference marker. By combining the drone's flight speed information, we analyze the flight time required for the drone to reach the return-to-home reference marker position under the current flight attitude. At the same time, by combining the terrain features in the flight environment image data and the corresponding terrain undulations, we determine whether there are terrain obstacles under the current relative position. When terrain obstacles exist, the flight altitude parameters are adjusted based on the relative positional relationship between the UAV and the return-to-home reference marker to ensure that the UAV's flight path maintains a preset safe distance from the terrain obstacles. At the same time, by adjusting the flight altitude parameters, the field of view of the UAV's visual acquisition system always covers the area where the return-to-home reference marker is located during the flight, thereby continuously acquiring flight environment image data containing the return-to-home reference marker. Based on the adjusted flight altitude parameters and the relative positional relationship between the UAV and the return-to-home reference marker, a return-to-home path planning model is constructed. This model takes the current position of the UAV, the position of the return-to-home reference marker, the flight speed, and terrain obstacle information as inputs, and outputs the coordinates of the path nodes of the return-to-home path. Multiple candidate return routes are generated through the return route planning model. Each candidate return route contains multiple consecutive path nodes, and each path node corresponds to a location coordinate in the geographic coordinate system. The path length and estimated flight time of each candidate return route are also recorded. A feasibility assessment was conducted on each candidate return route to obtain the feasibility assessment results. The assessment indicators included whether the candidate return route avoided terrain obstacles, whether the UAV could continuously obtain return reference signs during the flight of the route, and whether the estimated flight time of the route was within a reasonable range. Based on the feasibility assessment results, candidate return routes that meet all assessment indicators are selected, and the route with the shortest path length and the shortest expected flight time is chosen as the initial return route scheme from among the candidate return routes that meet the conditions.

7. The automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems according to claim 6, characterized in that, The feasibility assessment of each candidate return route, and the resulting feasibility assessment results, include: The terrain features corresponding to each image sub-region in the flight environment image data are obtained. Combined with the flight altitude parameters of the UAV, it is determined whether there is a terrain height exceeding the flight altitude of the UAV at the spatial location of each path node in each candidate return path. If there is, the candidate return path has not avoided terrain obstacles; if not, the candidate return path has avoided terrain obstacles. For each candidate return path, the estimated flight time of the UAV at each path node is calculated based on the path node sequence and the UAV's flight speed. Combining the field of view and image acquisition frequency of the visual acquisition system, it is determined whether the UAV can acquire flight environment image data containing the return reference marker at each path node within the estimated flight time. The determination method is as follows: calculate the proportion of the return reference marker in the field of view of the visual acquisition system. When the proportion exceeds the preset proportion threshold, it is determined that the flight environment image data containing the return reference marker can be acquired. If flight environment image data containing return-to-home reference markers can be collected at all path nodes, then the candidate return-to-home path meets the requirement of continuously acquiring return-to-home reference markers. If flight environment image data containing return-to-home reference markers cannot be collected at at least one path node, then the candidate return-to-home path does not meet the requirement of continuously acquiring return-to-home reference markers. Calculate the total path length of each candidate return path, and combine it with the average flight speed of the UAV to calculate the estimated total flight time of each candidate return path. Then compare the estimated total flight time with the preset maximum allowable flight time. If the estimated total flight time is less than or equal to the preset maximum allowable flight time, then the estimated flight time of the candidate return route is within a reasonable range; if the estimated total flight time is greater than the preset maximum allowable flight time, then the estimated flight time of the candidate return route is not within a reasonable range. For each candidate return route, the number of evaluation indicators that are met is counted. When all three evaluation indicators are met, the candidate return route passes the feasibility assessment. When any one evaluation indicator is missing, the candidate return route fails the feasibility assessment. Analyze the candidate return routes that fail the feasibility assessment. If the failure is due to not avoiding terrain obstacles, adjust the coordinates of some path nodes in the candidate return route, increase the flight altitude parameters, and regenerate the candidate return route. If the evaluation fails due to the inability to continuously acquire return-to-home reference markers, the distribution density of path nodes will be adjusted, the number of path nodes will be increased, and the distance between adjacent path nodes will be less than the ground distance corresponding to the field of view of the visual acquisition system. This will enable the UAV to continuously acquire image data containing return-to-home reference markers during flight and regenerate candidate return-to-home paths. If the assessment fails due to the estimated flight time being outside the reasonable range, the flight routes between path nodes are optimized, redundant path nodes are deleted, and candidate return routes are regenerated. The regenerated candidate return routes are then reassessed for feasibility until a candidate return route that passes the feasibility assessment is obtained.

8. The automatic return-to-home method for unmanned aerial vehicles based on inertial navigation and vision systems according to claim 1, characterized in that, The step of generating flight control commands for controlling the UAV's flight based on the initial return path scheme and combining the flight attitude and speed information in the UAV's flight status data, and sending the flight control commands to the UAV's flight control system to control the UAV to perform the return operation according to the flight control commands, includes: The initial return path scheme is analyzed, and the coordinates of all path nodes in the initial return path scheme are extracted. The path nodes are sorted according to the flight order to obtain a path node sequence, and each path node corresponds to a target position. Based on the path node sequence, calculate the flight direction between two adjacent path nodes. This flight direction is determined based on the coordinates of the previous path node and the coordinates of the next path node. At the same time, calculate the distance between two adjacent path nodes. By combining the current flight speed information in the drone's flight status data, the flight time required for the drone to fly from the current path node to the next path node is calculated. Based on this flight time and the distance between adjacent path nodes, the drone's flight speed is adjusted so that the time for the drone to reach the next path node is consistent with the preset time. Based on the current flight attitude information of the UAV and the flight direction corresponding to the next path node, the attitude angle that the UAV needs to adjust is calculated. This attitude angle includes pitch angle, roll angle and yaw angle. By adjusting this attitude angle, the flight direction of the UAV is made consistent with the flight direction corresponding to the next path node. Based on the adjusted flight speed and attitude angle, corresponding speed control parameters and attitude control parameters are generated. The speed control parameters are used to adjust the magnitude and rate of change of the UAV's flight speed, and the attitude control parameters are used to adjust the rate of change of the UAV's pitch angle, roll angle, and yaw angle. The speed control parameters and attitude control parameters are integrated according to a preset instruction format to generate flight control instructions. Each flight control instruction corresponds to the flight control requirements of a path node segment, and the path node segment is the flight path between two adjacent path nodes. Flight control commands are sent to the UAV's flight control system in the order of the path node sequence. The flight control system then parses the speed control parameters and attitude control parameters in the flight control commands, adjusts the UAV's power system output according to the speed control parameters, and adjusts the UAV's control surface angles according to the attitude control parameters, thereby controlling the UAV to fly towards the next path node according to the flight control commands. During the flight of the UAV, the flight status data of the UAV output by the inertial navigation system and the flight environment image data collected by the visual acquisition system are used to determine whether the UAV has deviated from the initial return path plan. When the drone is found to have deviated from the initial return-to-home path, the relative position between the drone's current position and the next path node is recalculated. Based on the recalculated relative position, the speed control parameters and attitude control parameters are adjusted, new flight control commands are generated, and the new flight control commands are sent to the flight control system. The flight control system executes the new flight control commands to correct the drone's flight path and bring the drone back to the flight trajectory corresponding to the initial return-to-home path.

9. An automatic return-to-home system for unmanned aerial vehicles (UAVs) based on inertial navigation and vision systems, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the UAV automatic return-to-home method based on an inertial navigation and vision system as described in any one of claims 1 to 8 by executing the machine-executable instructions.

10. A computer program product, characterized in that, The computer program product includes machine-executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the machine-executable instructions from the computer-readable storage medium and executes the machine-executable instructions, causing the computer device to perform the automatic return-to-home method for unmanned aerial vehicles based on an inertial navigation and vision system as described in any one of claims 1 to 8.

Images