A multi-UAV control system based on air gestures

Abstract

The present invention discloses a multi-UAV control system based on air gestures, which includes a GPS positioning unit, a two-dimensional plane display unit, an obstacle existence judgment unit, an obstacle type identification unit, a formation reconstruction information sending unit, a formation reconstruction adjustment unit, an information instruction control unit, a dynamic node position prediction unit, a gesture wearable device unit, a gesture formation control unit, a gesture type action judgment unit, a feature vector detection unit and a gesture feature prediction unit. When the obstacle existence judgment unit detects that the UAV moves from the current position to the next position, the main UAV judges whether there is an obstacle based on its own video terminal. When the formation reconstruction adjustment unit detects that the obstacle in front of the UAV is a dynamic obstacle, it controls the UAV and the UAV formation to reorganize the flight. The gesture formation control unit controls the main UAV to send a gesture signal through the wearable device, effectively controlling the UAV to reorganize the formation to bypass the obstacle.

Description

Technical Field

[0001] The present invention relates to the field of drone control technology, and in particular to a multi-drone control system based on air gestures. Background Art

[0002] Drones are popular among the public because they can operate in harsh environments and are easy to operate, which has led to a wide range of market development. However, there are still many problems in the application of drones. For example, when a drone is commanded to avoid obstacles, since drones are usually in the air, the signal reception is slower than on the ground. Therefore, when avoiding obstacles in the air, drones need a certain reaction ability to avoid them. If they cannot avoid them in time, the expensive drones will cause explosion hazards.

[0003] Drone performances usually control the drone's formation transformation matrix method, which requires good control of the drone. There are many ways to control drones, such as through mobile terminals, remote controls, etc. However, controlling drones through mobile terminals will result in low sensitivity and poor signal reception. At the same time, when controlling drones through remote controls, there will be a certain distance limit, and when the remote control is low in power, the drone cannot be controlled to move, resulting in the drone being unable to effectively avoid obstacles.

[0004] When the drone team is performing obstacle avoidance, if the obstacle detected is a dynamic obstacle, the position of the dynamic obstacle's action curve will affect the drone's performance according to the pre-set route. The dynamic position of the drone needs to be adjusted at all times to prevent some drones from leaving the team and failing to complete the performance.

[0005] Therefore, a multi-UAV control system based on air gestures is needed to solve the above problems. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-UAV control system based on air gestures to solve the problems raised in the above background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a multi-UAV control system based on air gestures, the system comprising a GPS positioning unit, a two-dimensional plane display unit, an obstacle existence judgment unit, an obstacle type recognition unit, a formation reconstruction information sending unit, a formation reconstruction adjustment unit, an information instruction control unit, a dynamic node position prediction unit, a gesture wearable device unit, a gesture formation control unit, a gesture type action judgment unit, a feature vector detection unit, a gesture feature prediction unit and a general control terminal;

[0008] The GPS positioning unit is used to locate the location of the drone in real time, and send the positioning information to the two-dimensional plane display unit through the information receiving unit;

[0009] The two-dimensional plane display unit is used to specifically display the distance information of more drones according to the position coordinates of the drones, so as to understand the detailed information between the drones themselves and the drone formations, so that the understanding of the drone information is more detailed and comprehensive;

[0010] The obstacle existence judgment unit is used to detect when the drone moves from the current position to the next position. The drone determines whether there is an obstacle based on its own video terminal, thereby affecting the normal state of the drone flying at high altitude;

[0011] The obstacle type identification unit is used to detect the type of obstacle currently existing and determine whether it is a static obstacle or a dynamic obstacle, so as to analyze the influence range of different types of obstacles;

[0012] The formation reconstruction information sending unit is used to control the operation of the drone and the drone formation when it detects that the obstacle in front of the drone is in a static state;

[0013] The formation reconstruction adjustment unit is used to control the drones and the drone formation to regroup and fly when it detects that the obstacle in front of the drone is a dynamic obstacle;

[0014] The information instruction control unit is used to control the flight of the UAV according to the received instructions, so that the UAV can complete the performance task;

[0015] The dynamic node position prediction unit is used to predict the unknown moving node position of the obstacle based on the moving nodes of the dynamic obstacle, so as to effectively avoid the collision between the drone and the dynamic obstacle based on the known moving nodes of the obstacle, which may cause the drone to fail to complete the task in time;

[0016] The prediction factor influence unit is used to reasonably predict the position of the next node of the dynamic obstacle according to the factors affecting the obstacle;

[0017] The gesture wearable device unit is used to effectively control the movement of the drone by wearing a control device on the hand, so that the drone can be effectively controlled by gestures in an emergency, preventing the drone from being in a dangerous state, and at the same time increasing the accuracy of the drone being controlled; the control device is installed with a monitoring receiving terminal and a vibration sensor, the vibration sensor is installed above the finger of the wearable device, and can effectively detect the degree of finger shaking, and the monitoring receiving terminal is used to identify the gesture formed by the hand movement and compare it with the gesture action stored in the gesture database terminal, so as to accurately and effectively control the drone;

[0018] The gesture formation control unit is used to send gesture signals to the main drone through the wearable device control, effectively controlling the drones to re-form around obstacles;

[0019] The gesture type action judgment unit is used to record gesture actions through the monitoring receiving terminal, and classify the actions into static gestures and dynamic gestures, so that different actions can be accurately identified, and the accuracy of controlling the drone can be improved through different actions;

[0020] The feature vector detection unit is used to identify different gestures according to the feature vector formed by the degree of bending between each finger and the palm;

[0021] The gesture feature prediction unit is used to predict the complete gesture according to the semi-finished gesture action that has been formed but not formed, so as to speed up the action of controlling the drone;

[0022] The main control end is used to send the transmitted signal to the main drone.

[0023] Furthermore, the steps to control the drone to move are:

[0024] Z1: Obtain the position signal of the total UAV at time d, detect the position signal of the total UAV from time d to time d+1, and monitor whether there are obstacles;

[0025] Z2: Determine the shape of the obstacle. If it is a static obstacle, proceed to step 3; if it is a dynamic obstacle, proceed to step 4.

[0026] Z3: According to the position of the static obstacle, determine whether the static obstacle will affect the movement of the UAV formation. If so, manually control the main UAV to move in another direction; if not, control the main UAV to perform a formation performance;

[0027] Z4: Determine whether the dynamic obstacle has the boundary conditions to collide with the drone formation, and publish the information to the general control drone to reconstruct the remaining drone formation; according to the moving curve of the dynamic obstacle, bypass the curve to form a new drone formation rectangle. When feedback is given to bypassing the obstacle, the flight speed of the moving drones is adjusted to return to the original drone formation rectangle position.

[0028] Furthermore, when the monitoring receiving terminal detects that the distance between the obstacle and the UAV approaches the obstacle at time d and time d+1 gradually decreases, it indicates that the current obstacle is a static obstacle. When the distance between the obstacle and the UAV approaches the obstacle at time d and time d+1 remains unchanged or increases, it indicates that the current obstacle is a dynamic obstacle, where d and d+1 are time.

[0029] When a static obstacle is detected, the coordinates of the activity range of the total drone and the drone formation are set to r1*r2. When the coordinates of the static obstacle are detected, [r1*r2], so it is necessary to control the main drone and the main drone to control the drone formation to move until A [r1*r2], A, r1, r2 refer to coordinates;

[0030] When a dynamic obstacle is detected, according to the deviation of wind speed, the coordinates of the dynamic obstacle detected in the time period from d to d+n are W={w1,w2,w3...w n}, n is the time period, according to the BP neural network prediction W i The location information of i refers to the time period i. The prediction steps are:

[0031] Z01: According to the existing number of samples n, W={w1,w2,w3...w n}, determine the number of samples that need to be predicted is k;

[0032] Z02: Determine the number of input layer factors, output layer factors and hidden layer factors;

[0033] Z03: Initialize the threshold and weight factors between the input layer and the hidden layer;

[0034] Z04: Use the trained network to make predictions and get the prediction results in the output layer;

[0035] Taking the matrix of the drone group that will not be disturbed by dynamic obstacles as the core matrix, the influence degree Q of the reconstruction of the partial drone team is determined according to the matrix coordinates of the original drone group and the coordinates of the predicted dynamic obstacles.

[0036] ;

[0037] Q0 refers to the initial impact level, It refers to the impact degree when one more drone is added or the moving distance increases by 1km. G refers to the distance between the drone and the obstacle. 预 Refers to the safe distance between the drone and obstacles, X G It refers to the number of drones affected by obstacles, and X0 refers to the standard number of drones affected by obstacles.

[0038] According to the formation reconstruction adjustment unit, when the distance between the dynamic obstacle and the original drone is greater than the preset distance through the predicted position result analysis, it means that some drones that have moved their positions speed up to their original positions, where the distance is calculated by Euclidean distance. The original position set of some drones that need to move but have not moved is H={h1,h2,h3...h k}, the position of the drone after moving is , k refers to the kth time period;

[0039] ;

[0040] In the fixed time period from k to k+1, the required increase in the moving speed of the drone is , where: q refers to the time when the drones just start to move, k refers to the final time when some drones catch up with the drones that have not been reconstructed in advance, V 原 It refers to the original moving speed of the UAV, and k+1 refers to the k+1th time period.

[0041] The steps for gesture-controlled drone movement are:

[0042] Z1: Detect the shaking frequency of each finger of the user, take the data information formed when the finger starts to move as the starting point of the feature vector, and the data information formed when the finger stops moving as the end point of the feature vector, and form a vector , record the curve formed by the finger feature vector and take a photo;

[0043] Z2: Record the position information of each finger in each frame of the video and the resulting shaking arc, record the start and end time of the shaking, and classify the arc curves formed by the shaking to form different gestures;

[0044] Z3: Compare the formed gesture with the gestures in the information library and distinguish between interference gestures and complete gestures.

[0045] Furthermore, in the steps Z1-Z2, the gesture-formed action is the bending degree between each finger and the palm, specifically referring to the curve and curvature formed by the first bone and the second bone on the finger and the connecting line between the wrist and the palm. Different gesture actions can be formed according to the above-formed conditions, and different gesture types are formed according to different gesture actions, and the gesture type is determined by the following steps:

[0046] According to the action formed by the current gesture, the initial time of starting to form the gesture is Y. When it is detected that the gesture has stopped within Y+i, the action is confirmed as a static gesture. When the gesture action does not stop at any time and the frequency formed is higher than the preset frequency, it means that the current gesture is a dynamic gesture, where Y+i refers to the i seconds formed more than the initial time.

[0047] Furthermore, in step Z3, the feature vector formed by the finger With the standard eigenvector For comparison, the standard eigenvector The data information formed when the finger starts to move is used as the starting point of the feature vector, and the data information formed when the finger stops moving is used as the end point of the feature vector;

[0048] Calculate the eigenvector With the standard eigenvector The cosine value between ;

[0049] ;

[0050] express and The angle between is the modulus of the eigenvector A, is the modulus of the eigenvector B, It refers to the eigenvector and the eigenvector The similarity between

[0051] when When , it represents the eigenvector and the eigenvector A high similarity means that the action formed by the gesture is the same as the action formed by the standard, which means that the information database contains the same gesture and the gesture can control the drone;

[0052] when When , it represents the eigenvector and the eigenvector The similarity is low, which means that the action formed by the gesture is different from the standard action, which means that the gesture is not included in the information database and the gesture is not necessary and cannot be used for the drone to operate.

[0053] When the drone is controlled by gestures, if some gestures are detected to be highly similar to the standard gestures, the gesture feature prediction unit predicts the remaining gestures based on the existing similar gestures and sends the k1 and k2 tags to the control end. The k1 and k2 tags control the drone to perform actions and form interference gestures respectively. The error rate formed by the gesture operation of the drone is P. When P>P 预 When P <P 预 When , it means that the interference between k1 label and k2 label is weak, P 预 Refers to the preset error rate.

[0054] The interfering gestures are ranked from largest to smallest. When an interfering gesture with a higher ranking appears on the main control terminal, any of the pop-up interfering gesture labels cannot be selected until the completeness of the gesture is equal to the standard completeness.

[0055] The interfering gesture includes a gesture curve formed by the user's gesture shaking. The gesture curve formed by the user's gesture shaking is compared with the standard gesture to prevent the curve formed by the hand shaking from interfering with the overall drone movement.

[0056] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0057] 1. The type of obstacle is determined by the moving path of the obstacle identified at the main drone end within the time period, and the drone is effectively controlled to pass through the obstacle according to the identified type, ensuring that the drone will not collide with the obstacle and ensure the safe passage of the drone. At the same time, the BP neural network is used to predict the dynamic obstacle position in an unknown time period based on the existing training set, so that the main drone can effectively avoid collision with obstacles. The drone formation readjusts the drone team to keep the drone away from the obstacle, and sends a gesture command to the main drone, requiring the obstacle to quickly return to its original position after leaving the drone, so as to ensure the normal performance of the drone;

[0058] 2. The gesture features can quickly identify gesture commands and effectively control the overall drone action, so that the remaining drones can complete the remaining commands according to the overall drone action. At the same time, using gestures to control the operation of drones can reduce the burden of carrying equipment to control drones, and will not cause power outages or inaccurate signal reception. According to the user's gestures, it is necessary to determine what type of gesture is required, detect the curve formed by gesture jitter, and avoid controlling the overall drone due to the incorrectly formed curve, which causes the drone to generate erroneous commands, thereby enhancing the effectiveness of drone control. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0060] Figure 1 It is a schematic diagram of the module composition of a multi-UAV control system based on air gestures of the present invention;

[0061] Figure 2 It is a schematic diagram of the steps of a multi-UAV control system based on air gestures of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1-Figure 2 The present invention provides a technical solution: a multi-UAV control system based on air gestures, the system comprising a GPS positioning unit, a two-dimensional plane display unit, an obstacle existence judgment unit, an obstacle type recognition unit, a formation reconstruction information sending unit, a formation reconstruction adjustment unit, an information instruction control unit, a dynamic node position prediction unit, a gesture wearable device unit, a gesture formation control unit, a gesture type action judgment unit, a feature vector detection unit, a gesture feature prediction unit and a general control terminal;

[0064] The GPS positioning unit is used to locate the location of the drone in real time, and send the positioning information to the two-dimensional plane display unit through the information receiving unit;

[0065] The two-dimensional plane display unit is used to specifically display the distance information of more drones according to the position coordinates of the drones, so as to understand the detailed information between the drones themselves and the drone formations, so that the understanding of the drone information is more detailed and comprehensive;

[0066] The obstacle existence judgment unit is used to detect when the drone moves from the current position to the next position. The drone determines whether there is an obstacle based on its own video terminal, thereby affecting the normal state of the drone flying at high altitude;

[0067] The obstacle type identification unit is used to detect the type of obstacle currently existing and determine whether it is a static obstacle or a dynamic obstacle, so as to analyze the influence range of different types of obstacles;

[0068] The formation reconstruction information sending unit is used to control the operation of the drone and the drone formation when it detects that the obstacle in front of the drone is in a static state;

[0069] The formation reconstruction adjustment unit is used to control the drones and the drone formation to regroup and fly when it detects that the obstacle in front of the drone is a dynamic obstacle;

[0070] The information instruction control unit is used to control the flight of the UAV according to the received instructions, so that the UAV can complete the performance task;

[0071] The dynamic node position prediction unit is used to predict the unknown moving node position of the obstacle based on the moving nodes of the dynamic obstacle, so as to effectively avoid the collision between the drone and the dynamic obstacle based on the known moving nodes of the obstacle, which may cause the drone to fail to complete the task in time;

[0072] The prediction factor influence unit is used to reasonably predict the position of the next node of the dynamic obstacle according to the factors affecting the obstacle;

[0073] The gesture wearable device unit is used to effectively control the movement of the drone by wearing a control device on the hand, so that the drone can be effectively controlled by gestures in an emergency, preventing the drone from being in a dangerous state, and at the same time increasing the accuracy of the drone being controlled; the control device is installed with a monitoring receiving terminal and a vibration sensor, the vibration sensor is installed above the finger of the wearable device, and can effectively detect the degree of finger shaking, and the monitoring receiving terminal is used to identify the gesture formed by the hand movement and compare it with the gesture action stored in the gesture database terminal, so as to accurately and effectively control the drone;

[0074] The gesture formation control unit is used to send gesture signals to the main drone through the wearable device control, effectively controlling the drones to re-form around obstacles;

[0075] The gesture type action judgment unit is used to record gesture actions through the monitoring receiving terminal, and classify the actions into static gestures and dynamic gestures, so that different actions can be accurately identified, and the accuracy of controlling the drone can be improved through different actions;

[0076] The feature vector detection unit is used to identify different gestures according to the feature vector formed by the degree of bending between each finger and the palm;

[0077] The gesture feature prediction unit is used to predict the complete gesture according to the semi-finished gesture action that has been formed but not formed, so as to speed up the action of controlling the drone;

[0078] The main control end is used to send the transmitted signal to the main drone.

[0079] Furthermore, the steps to control the drone to move are:

[0080] Z1: Obtain the position signal of the total UAV at time d, detect the position signal of the total UAV from time d to time d+1, and monitor whether there are obstacles;

[0081] Z2: Determine the shape of the obstacle. If it is a static obstacle, proceed to step 3; if it is a dynamic obstacle, proceed to step 4.

[0082] Z3: According to the position of the static obstacle, determine whether the static obstacle will affect the movement of the UAV formation. If so, manually control the main UAV to move in another direction; if not, control the main UAV to perform a formation performance;

[0083] Z4: Determine whether the dynamic obstacle has the boundary conditions to collide with the drone formation, and publish the information to the general control drone to reconstruct the remaining drone formation; according to the moving curve of the dynamic obstacle, bypass the curve to form a new drone formation rectangle, and when feedback is given to bypassing the obstacle, adjust the flight speed of the moving drones to return to the original drone formation rectangle position;

[0084] Among them, dynamic obstacles usually refer to inanimate objects, such as balloons, etc.

[0085] When some drones are rearranged to move forward, they need to be rearranged to bypass the obstacle area, but the drones that are controlled by the main drone and are not affected by obstacles also need to move towards the destination. Therefore, some drones affected by obstacles should quickly rush to the previous area to complete the overall drone performance.

[0086] Furthermore, when the monitoring receiving terminal detects that the distance between the obstacle and the UAV approaches the obstacle at time d and time d+1 gradually decreases, it indicates that the current obstacle is a static obstacle. When the distance between the obstacle and the UAV approaches the obstacle at time d and time d+1 remains unchanged or increases, it indicates that the current obstacle is a dynamic obstacle, where d and d+1 are time.

[0087] Among them, the drone itself is moving in one direction. If the obstacle is a static obstacle, the distance between the obstacle and the drone will gradually decrease; if the obstacle is a dynamic obstacle, the obstacle and the drone are both moving. If the speed of the obstacle is the same as that of the drone, the distance between the obstacle and the drone will basically not change. If the speed of the obstacle is lower than that of the drone, the distance between the obstacle and the drone will increase, and the premise is that the directions of the drone and the obstacle are moving in a corresponding direction. Because the moving direction of the dynamic obstacle is not controllable, it is assumed here that the drone and the obstacle move close to each other.

[0088] When a static obstacle is detected, the coordinates of the activity range of the total drone and the drone formation are set to r1*r2. When the coordinates of the static obstacle are detected, [r1*r2], so it is necessary to control the main drone and the main drone to control the drone formation to move until A [r1*r2], A, r1, r2 refer to coordinates;

[0089] Among them, r1*r2 refers to the range of the drone during the aerial performance. [r1*r2], indicating that the static obstacle is within the performance range of the drone. Since the impact range of static obstacles is smaller than that of dynamic obstacles, it is only necessary to control the drone to leave the current area until the drone is no longer affected by the static obstacle.

[0090] When a dynamic obstacle is detected, according to the deviation of wind speed, the coordinates of the dynamic obstacle detected in the time period from d to d+n are W={w1,w2,w3...w n}, n is the time period, according to the BP neural network prediction W i The location information of i refers to the time period i. The prediction steps are:

[0091] Z01: According to the existing number of samples n, W={w1,w2,w3...w n}, determine the number of samples that need to be predicted is k;

[0092] Z02: Determine the number of input layer factors, output layer factors and hidden layer factors;

[0093] Z03: Initialize the threshold and weight factors between the input layer and the hidden layer;

[0094] Z04: Use the trained network to make predictions and get the prediction results in the output layer;

[0095] In this process, the input layer factors of the BP neural network determined are wind speed and drone coordinate position, and the output layer refers to the coordinate value of the dynamic obstacle. The weight between the input layer and the hidden layer is set to a matrix type. The coordinate change of the dynamic obstacle to be predicted is determined according to the preset number of samples, and the change value is sent to the user for feedback;

[0096] Taking the matrix of the drone group that will not be disturbed by dynamic obstacles as the core matrix, the influence degree Q of the reconstruction of the partial drone team is determined according to the matrix coordinates of the original drone group and the coordinates of the predicted dynamic obstacles.

[0097] ;

[0098] Q0 refers to the initial impact level, It refers to the impact degree when one more drone is added or the moving distance increases by 1km. G refers to the distance between the drone and the obstacle. 预 Refers to the safe distance between the drone and obstacles, X G It refers to the number of drones affected by obstacles, and X0 refers to the standard number of drones affected by obstacles;

[0099] The matrix of the drone group that will not be disturbed by dynamic obstacles is used as the core matrix so that the drones affected by obstacles can fly around the core matrix to prevent all drones from being affected by fluctuations. The influence degree Q of the drone team reconstruction is calculated to highlight the influence degree of the drone reconstruction. The influence degree is related to the number of drones moving and the distance from the original core matrix. The greater the displayed distance and the more drones are affected, the higher the influence degree.

[0100] According to the formation reconstruction adjustment unit, when the distance between the dynamic obstacle and the original drone is greater than the preset distance through the predicted position result analysis, it means that some drones that have moved their positions speed up to their original positions, where the distance is calculated by Euclidean distance. The original position set of some drones that need to move but have not moved is H={h1,h2,h3...h k}, the position of the drone after moving is , k refers to the kth time period;

[0101] ;

[0102] In the fixed time period from k to k+1, the required increase in the moving speed of the drone is , where: q refers to the time when the drones just start to move, k refers to the final time when some drones catch up with the drones that have not been reconstructed in advance, V 原 refers to the original moving speed of the UAV, and k+1 refers to the k+1th time period;

[0103] During the calculation, V 原 , ,in It is to calculate the time from the drone affected by the obstacle to the original position of the drone core. 原 and The difference can be used to calculate the speed that needs to be controlled after the drone is not affected by obstacles, so that the drone can quickly rush to the core matrix and complete the performance within the specified time period.

[0104] The steps for gesture-controlled drone movement are:

[0105] Z1: Detect the shaking frequency of each finger of the user, take the data information formed when the finger starts to move as the starting point of the feature vector, and the data information formed when the finger stops moving as the end point of the feature vector, and form a vector , record the curve formed by the finger feature vector and take a photo;

[0106] Z2: Record the position information of each finger in each frame of the video and the resulting shaking arc, record the start and end time of the shaking, and classify the arc curves formed by the shaking to form different gestures;

[0107] Z3: Compare the formed gesture with the gestures in the information library and distinguish between interference gestures and complete gestures.

[0108] Furthermore, in the steps Z1-Z2, the gesture-formed action is the bending degree between each finger and the palm, specifically referring to the curve and curvature formed by the first bone and the second bone on the finger and the connecting line between the wrist and the palm. Different gesture actions can be formed according to the above-formed conditions, and different gesture types are formed according to different gesture actions, and the gesture type is determined by the following steps:

[0109] According to the action formed by the current gesture, the initial time of starting to form the gesture is Y. When it is detected that the gesture has stopped within Y+i, the action is confirmed as a static gesture. When the gesture action does not stop at any time and the frequency formed is higher than the preset frequency, it means that the current gesture is a dynamic gesture, where Y+i refers to the i seconds formed more than the initial time.

[0110] Furthermore, in step Z3, the feature vector formed by the finger With the standard eigenvector For comparison, the standard eigenvector The data information formed when the finger starts to move is used as the starting point of the feature vector, and the data information formed when the finger stops moving is used as the end point of the feature vector;

[0111] Calculate the eigenvector With the standard eigenvector The cosine value between ;

[0112] ;

[0113] express and The angle between is the modulus of the eigenvector A, is the modulus of the eigenvector B, It refers to the eigenvector and the eigenvector The similarity between

[0114] when When , it represents the eigenvector and the eigenvector A high similarity means that the action formed by the gesture is the same as the action formed by the standard, which means that the information database contains the same gesture and the gesture can control the drone;

[0115] when When , it represents the eigenvector and the eigenvector The similarity is low, which means that the action formed by the gesture is different from the standard action, indicating that the gesture is not included in the information database and the gesture is not necessary and cannot be used for the drone to operate;

[0116] When the drone is controlled by gestures, if some gestures are detected to be highly similar to the standard gestures, the gesture feature prediction unit predicts the remaining gestures based on the existing similar gestures and sends the k1 and k2 tags to the control end. The k1 and k2 tags control the drone to perform actions and form interference gestures respectively. The error rate formed by the gesture operation of the drone is P. When P>P 预 When P <P 预 When , it means that the interference between k1 label and k2 label is weak, P 预 It refers to the preset error rate;

[0117] The error rate set above is when a user controls the drone through gestures, and the similarity of the first half of the detected gestures is high, resulting in an error. The drone can be controlled to fly normally according to the ranking of the errors.

[0118] When multiple gesture devices send control commands to the master drone, the master drone only recognizes gesture commands of the current model. Other gesture devices cannot obtain control authority of the master drone and cannot have a corresponding impact on the master drone.

[0119] The interfering gestures are ranked from largest to smallest. When an interfering gesture with a higher ranking appears on the main control terminal, any of the pop-up interfering gesture labels cannot be selected until the completeness of the gesture is equal to the standard completeness.

[0120] The interfering gesture includes a gesture curve formed by the user's gesture shaking. The gesture curve formed by the user's gesture shaking is compared with the standard gesture to prevent the curve formed by the hand shaking from interfering with the overall drone movement.

[0121] Example 1: When multiple drones control a drone formation to perform a performance in the air, the master drone detects video information and finds that an obstacle is included in a predetermined performance range. When it detects that the obstacle has not moved within a specified time and the distance between the drone and the obstacle gradually decreases, it indicates that the current obstacle is a static obstacle. A command is issued to the master drone through gestures to control the master drone and the drone formation to move to other locations until the drone is not affected by the obstacle.

[0122] Example 2: When multiple drones control a drone formation to perform a performance in the air, the main drone detects video information that a dynamic obstacle is included in the predetermined performance range, and the rectangular array of the original drone formation is , when there is a dynamic obstacle affecting the flight of the drone, and the specific coordinates of the impact measured by the BP neural network are the positions of the remaining two rows of the rectangular array, the upper array flies to the backup performance position at a speed of 150m / min per hour, and the remaining drones follow the second row by adjusting the array. The total drone receives the information that the dynamic obstacle is away from the array and feeds it back to the remaining drones, so that the remaining drones can adjust their speed to catch up with the current drone array. It is detected that the current position of the drones in the first and second rows is (4800,5600), and the position of the remaining drones is (3200,4000). It is detected that the time taken for the first and second rows to move from the original position to the current position is 20min, and the flight speed of the original drone is therefore:

[0123] ;

[0124] Therefore, some drones need to increase their speed by 37m / min to catch up with the core drone array.

[0125] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0126] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-UAV control system based on air gestures, characterized by: The system includes a GPS positioning unit, a two-dimensional plane display unit, an obstacle existence judgment unit, an obstacle type recognition unit, a formation reconstruction information sending unit, a formation reconstruction adjustment unit, an information instruction control unit, a dynamic node position prediction unit, a gesture wearable device unit, a gesture formation control unit, a gesture type action judgment unit, a feature vector detection unit, a gesture feature prediction unit and a general control terminal; The GPS positioning unit is used to locate the location of the drone in real time, and send the positioning information to the two-dimensional plane display unit through the information receiving unit; The two-dimensional plane display unit is used to specifically display the distance information of more drones according to the position coordinates of the drone; The obstacle existence judgment unit is used to detect whether there is an obstacle when the drone moves from the current position to the next position, and the drone determines whether there is an obstacle based on its own video terminal; The obstacle type identification unit is used to detect the type of obstacle currently existing and determine whether it is a static obstacle or a dynamic obstacle; The formation reconstruction information sending unit is used to control the operation of the drone and the drone formation when it detects that the obstacle in front of the drone is in a static state; The formation reconstruction adjustment unit is used to control the drones and the drone formation to regroup and fly when it detects that the obstacle in front of the drone is a dynamic obstacle; The information command control unit is used to control the flight of the UAV according to the received command; The dynamic node position prediction unit is used to predict the unknown moving node position of the obstacle based on the nodes that have moved of the dynamic obstacle; The prediction factor influence unit is used to reasonably predict the position of the next node of the dynamic obstacle according to the factors affecting the obstacle; The gesture wearable device unit is used to effectively control the motion of the drone by wearing a control device on the hand; the control device is equipped with a monitoring receiving terminal and a vibration sensor, the vibration sensor is installed above the finger of the wearable device, and can effectively detect the degree of finger shaking, and the monitoring receiving terminal is used to identify the gesture formed by the hand movement and compare it with the gesture action stored in the gesture database terminal; The gesture formation control unit is used to send gesture signals to the main drone through the wearable device control, effectively controlling the drones to re-form around obstacles; The gesture type action judgment unit is used to record the gesture actions through the monitoring receiving terminal, and classify the actions into static gestures and dynamic gestures, and can improve the accuracy of controlling the drone through different actions; The feature vector detection unit is used to identify different gestures according to the feature vector formed by the degree of bending between each finger and the palm; The gesture feature prediction unit is used to predict a complete gesture according to an existing but unformed semi-finished gesture; The main control end is used to send the transmitted signal to the main drone.

2. The multi-UAV control system based on air gestures according to claim 1, characterized in that: The steps to control the drone to move are: Z1: Obtain the position signal of the total UAV at time d, detect the position signal of the total UAV from time d to time d+1, and monitor whether there are obstacles; Z2: Determine the shape of the obstacle. If it is a static obstacle, proceed to step 3; if it is a dynamic obstacle, proceed to step 4. Z3: According to the position of the static obstacle, determine whether the static obstacle will affect the movement of the UAV formation. If so, manually control the main UAV to move in another direction; if not, control the main UAV to perform a formation performance; Z4: Determine whether the dynamic obstacle has the boundary conditions to collide with the drone formation, and publish the information to the general control drone to reconstruct the remaining drone formation; according to the moving curve of the dynamic obstacle, bypass the curve to form a new drone formation rectangle. When feedback is given to bypassing the obstacle, the flight speed of the moving drones is adjusted to return to the original drone formation rectangle position.

3. The multi-UAV control system based on air gestures according to claim 2, characterized in that: When the monitoring receiving terminal detects that the obstacle moves at time d and time d+1, the distance gradually decreases compared with the drone approaching the obstacle, indicating that the current obstacle is a static obstacle. When the obstacle moves at time d and time d+1, the distance remains unchanged or increases compared with the drone approaching the obstacle, indicating that the current obstacle is a dynamic obstacle, where d and d+1 are time; When a static obstacle is detected, the coordinates of the activity range of the total drone and the drone formation are set to r1*r2. When the coordinates of the static obstacle are detected, [r1*r2], so it is necessary to control the main drone and the main drone to control the drone formation to move until A [r1*r2], A, r1, r2 refer to coordinates; When a dynamic obstacle is detected, according to the deviation of wind speed, the coordinates of the dynamic obstacle detected in the time period from d to d+n are W={w1,w2,w3...w n }, n is the time period, according to the BP neural network prediction W i The location information of i refers to the time period i, and the prediction steps are: Z01: According to the existing number of samples n, W={w1,w2,w3...w n }, determine the number of samples that need to be predicted is k; Z02: Determine the number of input layer factors, output layer factors and hidden layer factors; Z03: Initialize the threshold and weight factors between the input layer and the hidden layer; Z04 uses the trained network to make predictions and obtains the prediction results at the output layer; Taking the matrix of the drone group that will not be disturbed by dynamic obstacles as the core matrix, the influence degree Q of the reconstruction of the partial drone team is determined according to the matrix coordinates of the original drone group and the coordinates of the predicted dynamic obstacles. ； Q0 refers to the initial impact level, It refers to the impact degree when one more drone is added or the moving distance increases by 1km. G refers to the distance between the drone and the obstacle. 预 Refers to the safe distance between the drone and obstacles, X G It refers to the number of drones affected by obstacles, and X0 refers to the standard number of drones affected by obstacles; According to the formation reconstruction adjustment unit, when the distance between the dynamic obstacle and the original drone is greater than the preset distance through the predicted position result analysis, it means that some drones that have moved their positions speed up to their original positions, where the distance is calculated by Euclidean distance. The original position set of some drones that need to move but have not moved is H={h1,h2,h3...h k }, the position of the drone after moving is , k refers to the kth time period; ； In the fixed time period from k to k+1, the required increase in the moving speed of the drone is , where: q refers to the time when the drones just start to move, k refers to the final time when some drones catch up with the drones that have not been reconstructed in advance, V 原 It refers to the original moving speed of the UAV, and k+1 refers to the k+1th time period.

4. The multi-UAV control system based on air gestures according to claim 1, characterized in that: The steps for gesture-controlled drone movement are: Z1: Detect the shaking frequency of each finger of the user, take the data information formed when the finger starts to move as the starting point of the feature vector, and the data information formed when the finger stops moving as the end point of the feature vector, and form a vector , record the curve formed by the finger feature vector and take a photo; Z2: Record the position information of each finger in each frame of the video and the resulting shaking arc, record the start and end time of the shaking, and classify the arc curves formed by the shaking to form different gestures; Z3: Compare the formed gesture with the gestures in the information library and distinguish between interference gestures and complete gestures.

5. The multi-UAV control system based on air gestures according to claim 2, characterized in that: In steps Z1-Z2, the gesture movement is the bending degree between each finger and the palm, specifically referring to the curve and curvature formed by the first bone and the second bone on the finger and the connecting line between the wrist and the palm. Different gesture movements can be formed according to the above-mentioned conditions.

6. The multi-UAV control system based on air gestures according to claim 4, characterized in that: In step Z3, the feature vector formed by the finger is With the standard eigenvector For comparison, the standard eigenvector The data information formed when the finger starts to move is used as the starting point of the feature vector, and the data information formed when the finger stops moving is used as the end point of the feature vector; Calculate the eigenvector With the standard eigenvector The cosine value between ; ； express and The angle between is the modulus of the eigenvector A, is the modulus of the eigenvector B, It refers to the eigenvector and the eigenvector The similarity between when When , it represents the eigenvector and the eigenvector A high similarity means that the action formed by the gesture is the same as the action formed by the standard, which means that the information database contains the same gesture and the gesture can control the drone; when When , it represents the eigenvector and the eigenvector The similarity is low, which means that the action formed by the gesture is different from the standard action, which means that the gesture is not included in the information database and the gesture is not necessary and cannot be used for the drone to operate.

7. The multi-UAV control system based on air gestures according to claim 1, characterized in that: When the drone is controlled by gestures, if some gestures are detected to be highly similar to the standard gestures, the gesture feature prediction unit predicts the remaining gestures based on the existing similar gestures, and sends the k1 and k2 tags to the general control end. The k1 and k2 tags control the drone to perform actions and form interference gestures respectively. The error rate formed by the gesture operation of the drone is P. When P>P 预 When P <P 预 When , it means that the interference between k1 label and k2 label is weak, P 预 It refers to the preset error rate, k1 and k2 are labels; The interfering gestures are ranked from largest to smallest. When an interfering gesture with a higher ranking appears on the main control terminal, any of the pop-up interfering gesture labels cannot be selected until the completeness of the gesture is equal to the standard completeness.

8. The multi-UAV control system based on air gestures according to claim 7, characterized in that: The interfering gesture includes a gesture curve formed by the user's gesture shaking. The gesture curve formed by the user's gesture shaking is compared with the standard gesture to prevent the curve formed by the hand shaking from interfering with the overall drone movement.

Images