Unmanned aerial vehicle high-precision automatic driving and nest returning charging control system

Abstract

The invention relates to the technical field of unmanned aerial vehicle high-precision automatic driving nest-returning charging, and discloses an unmanned aerial vehicle high-precision automatic driving and nest-returning charging control system, an unmanned aerial vehicle end assembly associates a semantic segmentation result with map points generated by an SLAM module, constructs a 3D map containing semantic information, performs pose estimation by using the 3D map, and calculates the position of the unmanned aerial vehicle. A dynamic cost map is constructed by fusing visual features and semantic information, global path search is performed by adopting an improved A * algorithm, an optimized trajectory is generated through B spline smoothing, and finally, automatic navigation of the unmanned aerial vehicle is controlled by combining a rule engine and a deep learning model. And full-process control of the unmanned aerial vehicle for automatically returning to the nest for charging is realized through data interaction with the nest end assembly. Based on the semantic vision technology, high-precision automatic driving of the unmanned aerial vehicle is truly achieved, automatic nest returning charging is synchronously achieved, the working efficiency of the unmanned aerial vehicle is improved, and the unmanned aerial vehicle is charged without manual intervention.

Description

Technical Field

[0001] The present invention relates to the technical field of high-precision automatic driving and return-to-nest charging for unmanned aerial vehicles (UAVs), and in particular to a high-precision automatic driving and return-to-nest charging control system for UAVs. Background Art

[0002] In the modern technological landscape, autonomous drone technology has become a highly anticipated field, leading the development of automated and intelligent aircraft. Deep learning, a key branch of artificial intelligence, is playing a crucial role in enabling autonomous drones. The rise of autonomous drones is due to a convergence of factors. First, the rapid development of drone technology has enabled their increasing application in diverse fields, including military, civilian, agricultural, surveillance, and search and rescue. Second, the rapid development of deep learning technology has opened up new opportunities for the application of artificial intelligence, enabling drones to better perceive, understand, and make decisions, thereby achieving a higher level of autonomous flight.

[0003] The development of deep learning and the improvement of computing power have made large-scale data processing and advanced image analysis more feasible, providing a solid foundation for the development of high-precision autonomous drones.

[0004] Considering that drones need to quickly replenish energy after completing a flight mission for a period of time, and the use of lithium batteries is the currently commonly used energy supply method, replenishing the power of such drones generally relies on manual intervention for charging. This method is inefficient and difficult to popularize in remote and uninhabited areas. Summary of the Invention

[0005] The present invention provides a high-precision automatic driving and return to the nest charging control system for unmanned aerial vehicles (UAVs), aiming to achieve the technical goals of high-precision automatic driving of UAVs and automatic return to the nest charging.

[0006] A high-precision autonomous driving and return-to-nest charging control system for unmanned aerial vehicles (UAVs), comprising a UAV-side component and a nest-side component. The UAV-side component associates semantic segmentation results with map points generated by a SLAM module to construct a 3D map containing semantic information. The 3D map is used for pose estimation, a dynamic cost map is constructed by fusing visual features with semantic information, and an improved A* algorithm is used for global path search. An optimized trajectory is generated through B-spline smoothing. Finally, the automatic navigation of the UAV is controlled by combining a rule engine and a deep learning model. Data interaction with the nest-side component is used to achieve full-process control of the UAV's automatic return to the nest for charging.

[0007] Preferably, the power management module transmits the monitored battery data to the power algorithm processing module, which predicts the battery life through an algorithm. When the predicted battery life is less than a predetermined threshold, the low battery data is sent to the decision system algorithm module;

[0008] The decision system algorithm module sends instructions and instruction circulation evidence information to the drone-side communication module to obtain the nest location data. The drone-side communication module obtains the nest location data from the nest-side positioning module. The nest-side positioning communication module sends the nest location to the drone-side communication module. The drone-side communication module returns the nest location data to the decision system algorithm module. The decision system algorithm module decides to let the drone return to the nest location for charging based on the nest location data sent by the drone-side communication module.

[0009] Preferably, the method for the decision system algorithm module to generate instruction circulation evidence information is:

[0010] The first circulation node parameter is generated based on the identifier of the recognition decision system algorithm, and the second circulation node parameter is generated based on the identifier of the drone-side communication module. The circulation node parameter generation process is as follows: non-character symbols in the identifier are removed, all characters in the identifier are converted into decimal code values ​​in sequence, the decimal code values ​​are converted into binary numbers, the binary numbers are sequentially concatenated to obtain a binary string, and the hash value of the binary string in the finite domain space is calculated and recorded as the circulation node parameter.

[0011] Using the second circulation node parameter as the power exponent of the first circulation node parameter, performing a power operation to obtain a fixed circulation certificate, converting the fixed circulation certificate into a binary string according to the program step of converting an identifier into a binary string, and mapping the binary string into a fixed-length binary string I;

[0012] Identify the text format date of the creation instruction, remove the separating hyphen in the date, convert the text format date into a binary string according to the program steps of converting the identifier into a binary string, and map the binary string into a fixed-length binary string II, where the binary string II has the same length as the binary string I;

[0013] Perform an XOR operation on binary string I and binary string II to obtain the instruction circulation evidence information.

[0014] The beneficial effects of the present invention are as follows:

[0015] Relying on semantic vision technology, the present invention can realize true high-precision autonomous driving of drones. Based on the high-precision autonomous driving technical solution of drones, the drone can simultaneously automatically return to the nest for charging, thereby improving the working efficiency of the drone, reducing labor costs, and eliminating the need for human intervention to charge the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the system architecture diagram of the UAV's high-precision autonomous driving and return-to-nest charging control system;

[0017] Figure 2 This is a structural diagram of the intermediate and deep feature aggregation layers of the deep feature aggregation network of the present invention. DETAILED DESCRIPTION

[0018] like Figure 1 As shown in the figure, the UAV high-precision automatic driving and return to the nest charging control system consists of UAV-side components and nest-side components;

[0019] The drone-side components consist of sensor hardware, image perception algorithm module, image processor module, path planning optimization algorithm, decision system algorithm module, central processing unit module, communication module, battery, power management module and power management algorithm module;

[0020] The machine nest end components consist of a machine nest power module, a positioning communication module, a machine nest motor drive module, and a machine nest wireless charging module;

[0021] The working procedures of the UAV high-precision automatic driving and return to the nest charging control system are as follows:

[0022] Procedure 1:

[0023] The sensor hardware on the drone side is used to sense environmental conditions and transmit the collected road / object / obstacle data to the image perception algorithm module. The image perception algorithm module transmits the road / object / obstacle data to the image processor module.

[0024] Program 2:

[0025] The image perception algorithm module performs semantic segmentation on the data, and the synchronous image processor module extracts visual features from the data. Specifically, the following procedures are executed:

[0026] Procedure 2-1:

[0027] Use a deep feature aggregation network to perform real-time semantic segmentation on road / object / obstacle images and identify semantic labels for background, road, object, and obstacle categories;

[0028] The deep feature aggregation network includes a basic feature representation layer, an intermediate feature aggregation layer, a deep feature aggregation layer, and a feature aggregation module. The three-level feature representation can fine-tune feature information to compensate for feature information loss caused by upsampling and downsampling. The feature aggregation module optimizes information from different feature extraction layers, giving the model better feature learning capabilities.

[0029] The basic feature representation layer is a shallow multi-scale feature extraction process based on the CNN framework. It operates through the intermediate feature aggregation layer, the deep feature aggregation layer, and the feature aggregation layer module jump connection, and adds batch normalization operations to reduce the impact of network parameter offset caused by backpropagation. Its calculation formula is:

[0030] O x = f ( F x ;θ ) (1)

[0031] Where, O x Is the output of the basic feature representation layer;

[0032] f is the convolution, pooling and activation operations;

[0033] F x are feature maps of different resolutions;

[0034] θ is the model parameter learned by f;

[0035] like Figure 2 As shown in the figure, the intermediate feature aggregation layer is used to extract the intermediate features of the basic feature representation layer, and then pass them to the deep feature aggregation layer to complete feature aggregation at the same time. Specifically:

[0036] The basic feature representation layer includes three downsampling and one upsampling operations. The intermediate feature aggregation layer will perform multiple convolution operations on the basic layer features obtained by each sampling. The intermediate feature aggregation layer includes five convolution operations, covering all upsampling in the basic feature representation layer and the downsampling process of the next layer. After the second convolution operation, the information obtained in the basic feature representation layer is supplemented by cropping and channel splicing to enhance the model's feature expression capabilities.

[0037] The calculation process of the intermediate feature aggregation layer is:

[0038] f2=M1( f1;θ1) (2)

[0039] f3=M2(f2;θ2)⊙F1 (3)

[0040] f4=M3( f3;θ3)⊙F2 (4)

[0041] f5=M4( f4;θ4)⊙F3 (5)

[0042] f6=M5(f5;θ5)⊙F4 (6)

[0043] In the above formula, ⊙ is the multiplication of corresponding elements;

[0044] M is the middle layer convolution;

[0045] θ is the convolution kernel parameter;

[0046] F is the post-aggregation feature;

[0047] In order to reduce the loss of feature information caused by upsampling and downsampling operations, the intermediate feature aggregation layer uses clipping and cascading operations to migrate the aggregated features f3, f4, f5, and f6 to the upsampled and downsampled features B1, B2, B3, and B4 of the basic feature representation layer;

[0048] The deep feature aggregation layer is used to mine deep feature information and extract the inherent features and topological structure information contained in the image. It is located at the bottom layer of the model. The deep feature aggregation layer includes four aggregation operations. Except for the first aggregation operation, other aggregation operations will output the aggregated feature AF. Its calculation process is:

[0049] J1= f1⊕ f2 (7)

[0050] J2= J1⊕ f3 (8)

[0051] J3= J2⊕ f4 (9)

[0052] J4= J3⊕ f5 (10)

[0053] Where ⊕ is the channel splicing operation, J is the deep aggregation feature;

[0054] The feature aggregation module is used to fuse the features of the intermediate aggregation layer and the deep aggregation layer, and to perform weighted adjustment on the features to enrich the feature information. The feature aggregation module takes f and J as input and obtains the feature f through convolution operation. c and J c ;

[0055] In order to supplement the feature dimension and enrich the semantic feature information, the feature aggregation module obtains the aggregated feature A1 by channel splicing and adding the corresponding elements, extracts the feature A2 from A1 by convolution operation, and obtains the aggregated feature A3 by convolution and SoftMax function operation, so that A3 has the information content of f and J. According to the feature information contained in A3, the feature aggregation module multiplies f by the corresponding elements. c and J c After weighting, feature fusion is performed through original addition and 1×1 convolution to obtain the aggregated feature AF. The calculation process of A1 and A3 in the feature aggregation module is as follows:

[0056] A1=(f c (f;θ6)+J c (J;θ7))‖J c ‖ f c (11)

[0057] A3=exp(A2(A1; θ8))÷∑exp(A2(A1; θ8)) (12)

[0058] Where, ‖ is the feature channel cascade;

[0059] θ6 is J c The convolution parameters of

[0060] θ7 is f c The convolution parameters of

[0061] θ8 is the convolution parameter of A2;

[0062] After getting A3, first assign f c and J c The weights are then added to the weighted features to adjust the number of channels required for the convolution operation so that the number of channels of AF is the same as that of the middle layer;

[0063] A3 weights the basic feature layer and the deep feature aggregation layer as follows:

[0064] A4=A3⊙f c + A3⊙J c (13)

[0065] A5=f A4 ( A4; θ A4 ) (14)

[0066] Where, f A4 It is the convolution operation between A4 and A5;

[0067] θ A4 f A4 The convolution kernel parameters;

[0068] The process of semantic segmentation in a deep feature aggregation network is divided into two stages: the feature encoder obtains feature maps of different resolutions through convolution and pooling operations on the basic feature representation layer; the feature decoder restores the feature map resolution size through upsampling operations;

[0069] Extract visual features from the data to obtain discriminative information such as edges, textures, shapes, colors, and local key points, and transmit the visual features to the path planning optimization algorithm module;

[0070] Procedure 2-2:

[0071] The semantic segmentation results (class labels such as background, road, object, and obstacle) are associated with the map points (feature points / voxels / object instances) generated by the SLAM module to construct a 3D map containing semantic information, and the 3D map with semantic information is used for pose estimation.

[0072] Procedure 2-3:

[0073] Transmit the 3D map with semantic information and real-time positioning data to the path planning optimization algorithm module, and simultaneously transmit it to the decision system algorithm module;

[0074] Program 3:

[0075] The path planning module constructs a dynamic cost map by fusing geometric (visual features) and semantic information (such as traffic rules and scene categories), and uses an improved A* algorithm (such as dynamic heuristic weights) to perform global path search. After generating an optimized trajectory through B-spline smoothing, the spatiotemporal trajectory information is transmitted in real time to the decision system algorithm module, which is supported by the central processing unit module.

[0076] Program 4:

[0077] The decision-making system algorithm module combines the rule engine and deep learning model to achieve real-time decision-making in complex scenarios;

[0078] Program 5:

[0079] The power management module transmits the monitored battery data to the power algorithm processing module;

[0080] Program 6:

[0081] The power algorithm processing module predicts the battery life through an algorithm. When the battery is low on power, it sends the low power data to the decision system algorithm module.

[0082] Procedure 7:

[0083] The decision system algorithm module sends instructions and instruction circulation evidence information to the UAV communication module to obtain the nest location data. The UAV communication module obtains the nest location data from the nest positioning module. The nest positioning communication module sends the nest location to the UAV communication module.

[0084] The specific implementation procedures for the decision system algorithm module to generate instruction circulation evidence information are as follows:

[0085] Procedure 7-1:

[0086] Identifies the unique identifier UUID of the decision system algorithm module itself d-m ;

[0087] The unique identifier UUID in text string format d-m Remove non-character symbols in ;

[0088] Use the character encoding standard ASCII to encode the UUID in sequence d-m Convert all characters in to decimal code values;

[0089] Convert all decimal code values ​​to binary numbers in sequence. The specific process is: repeatedly divide the decimal code value by 2, record the remainder each time, and stop when the remainder is 0 or 1. Arrange all the remainders in reverse order to get the binary number;

[0090] Concatenate all binary numbers in sequence to get the binary string b d-m ;

[0091] Based on hash function H: {0,1} * →Z p , calculate the binary string b d-m The hash value h d-m :

[0092] h d-m =H(b d-m );

[0093] Among them, {0,1} * represents the set of all finite length strings consisting of 0 and 1, p is a prime number, Z p It is a finite field consisting of p integers from 0 to p-1;

[0094] Procedure 7-2:

[0095] Get the unique identifier UUID of the drone communication module c-m ;

[0096] The unique identifier UUID in text string format c-m Remove non-character symbols in ;

[0097] Based on the identifier UUID d-m Convert to binary string b d-m The procedure steps are to convert the UUID c-m Convert to binary string b c-m ;

[0098] Based on hash function H: {0,1} * →Z p , calculate the binary string b c-m The hash value h c-m :

[0099] h c-m =H(bc-m );

[0100] Procedure 7-3:

[0101] Calculate fixed circulation certificate h d-m→c-m =h d-m ^h c-m ;

[0102] Based on the identifier UUID d-m Convert to binary string b d-m The procedure steps are as follows: d-m→c-m Convert to binary string b d-m→c-m ;

[0103] Based on hash function H1: {0,1} * →{0,1} L , calculate the binary string b d-m→c-m A fixed-length L-bit binary string b' d-m→c-m =H1(b d-m→c-m );

[0104] Among them, {0,1} L Represents a set of fixed-length L strings composed of binary characters;

[0105] Procedure 7-4:

[0106] Identify the text format of the creation instruction date T d-c ;

[0107] Set the date T d-c Remove the separating hyphens in ;

[0108] Based on the identifier UUID d-m Convert to binary string b d-m The procedure steps are as follows: d-c Convert to binary string b d-c ;

[0109] Based on hash function H1: {0,1} * →{0,1} L , calculate the binary string b d-c A fixed-length L-bit binary string b' d-c =H1(b d-c );

[0110] Procedure 7-5:

[0111] The binary string b' d-m→c-m With the binary string b' d-c Perform XOR operation to obtain instruction circulation evidence information F d-m→p-c =b'd-m→c-m ⊕b' d-c ;

[0112] Procedure 8:

[0113] The positioning communication module at the machine nest sends a command to the motor drive module at the machine nest to open the door of the machine nest in advance, and simultaneously sends a command to the wireless charging module at the machine nest to start wireless charging in advance;

[0114] Procedure 9:

[0115] The UAV communication module returns the nest location data to the decision system algorithm module;

[0116] Procedure 10:

[0117] The decision-making system algorithm module determines whether the drone should return to the nest location for charging based on the nest location data sent by the drone communication module.

Claims

1. A high-precision automatic driving and return to the nest charging control system for UAVs, characterized by: The system consists of a drone-side component and a nest-side component. The drone-side component associates the semantic segmentation results with the map points generated by the SLAM module to construct a 3D map containing semantic information. The 3D map is used for pose estimation, and a dynamic cost map is constructed by fusing visual features with semantic information. An improved A* algorithm is used for global path search, and an optimized trajectory is generated through B-spline smoothing. Finally, the automatic navigation of the drone is controlled by combining a rule engine and a deep learning model, and the full process control of the drone's automatic return to the nest for charging is achieved through data interaction with the nest-side component.

2. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 1, characterized in that: The drone-side components consist of sensor hardware, image perception algorithm module, image processor module, path planning optimization algorithm, decision system algorithm module, central processing unit module, communication module, battery, power management module and power management algorithm module.

3. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 2, characterized in that: The machine nest end components consist of a machine nest power module, a positioning communication module, a machine nest motor drive module and a machine nest wireless charging module.

4. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 3, characterized in that: The power management module transmits the monitored battery data to the power algorithm processing module, which predicts the battery life through an algorithm. When the predicted battery life is less than a predetermined threshold, the low battery data is sent to the decision system algorithm module; The decision system algorithm module sends instructions and instruction circulation evidence information to the drone-side communication module to obtain the nest location data. The drone-side communication module obtains the nest location data from the nest-side positioning module. The nest-side positioning communication module sends the nest location to the drone-side communication module. The drone-side communication module returns the nest location data to the decision system algorithm module. The decision system algorithm module decides to let the drone return to the nest location for charging based on the nest location data sent by the drone-side communication module.

5. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 4, characterized in that: The method by which the decision system algorithm module generates instruction circulation evidence information is as follows: The first circulation node parameter is generated based on the identifier of the recognition decision system algorithm, and the second circulation node parameter is generated based on the identifier of the drone-side communication module. The circulation node parameter generation process is as follows: non-character symbols in the identifier are removed, all characters in the identifier are converted into decimal code values ​​in sequence, the decimal code values ​​are converted into binary numbers, the binary numbers are sequentially concatenated to obtain a binary string, and the hash value of the binary string in the finite domain space is calculated and recorded as the circulation node parameter. Using the second circulation node parameter as the power exponent of the first circulation node parameter, performing a power operation to obtain a fixed circulation certificate, converting the fixed circulation certificate into a binary string according to the program step of converting an identifier into a binary string, and mapping the binary string into a fixed-length binary string I; Identify the text format date of the creation instruction, remove the separating hyphen in the date, convert the text format date into a binary string according to the program steps of converting the identifier into a binary string, and map the binary string into a fixed-length binary string II, where the binary string II has the same length as the binary string I; Perform an XOR operation on binary string I and binary string II to obtain the instruction circulation evidence information.

6. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 4, characterized in that: The image perception algorithm module transmits the information to the image processor module, which extracts visual features from the data to obtain discriminative information such as edges, textures, shapes, colors, and local key points, and transmits the visual features to the path planning optimization algorithm module.

7. A high-precision automatic driving and return-to-nest charging control system for UAVs according to claim 4, characterized in that: The decision system algorithm module is supported by the central processing unit module.

8. The high-precision automatic driving and return-to-nest charging control system for unmanned aerial vehicles according to claim 4 is characterized in that: The positioning communication module at the machine nest end sends an instruction to the motor drive module at the machine nest end to open the machine nest door in advance, and simultaneously sends an instruction to the wireless charging module at the machine nest end to start wireless charging in advance.

Images