A kind of intelligent voice control system and method for unmanned aerial vehicle flight based on big data

Abstract

The present invention discloses an intelligent voice control system and method for unmanned aerial vehicle flight based on big data, and belongs to the field of intelligent voice control technology. The present invention comprises: S10: using a first voice model, a second voice model and a third voice model to process the real-time voice instructions received by a voice receiving device to obtain a plurality of task instructions; S20: the unmanned aerial vehicle control terminal controls the motion state of the unmanned aerial vehicle according to the target control instruction; S30: selecting whether to change the target control instruction received by the unmanned aerial vehicle control terminal, and determining the change result. The present invention trains the trained traditional acoustic model through a phoneme sequence to ensure that each voice model obtained after training can accurately identify and accurately correct the voice instructions issued by the user, so as to avoid the inability to control the unmanned aerial vehicle due to the difference between the voice instructions input by the user and the control instructions input by the unmanned aerial vehicle control system, thereby improving the use effect of the system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice control technology, and in particular to an intelligent voice control system and method for unmanned aerial vehicle flight based on big data. Background Art

[0002] With the continuous development of science and technology, drones have gradually become an important aviation test platform and an important means of academic research. However, the traditional operation control method has a great contrast with its flexible airborne payload characteristics, and there is a contradiction in operation efficiency and operation success rate. At the same time, it limits the use of drones in many application scenarios. The drone voice control system based on speech recognition can solve the above problems because of its fast, convenient and high accuracy.

[0003] When the existing technology uses voice recognition technology to control the drone by voice, it is necessary to accurately match the command keywords before they can be converted into control commands for issuance. This requires the user to be proficient in the voice control commands of the drone, which reduces the use effect. At the same time, the existing technology cannot perform real-time analysis on the early warning response of the drone, resulting in the drone being unable to take corresponding measures in advance when a collision occurs. In addition, the existing technology cannot effectively analyze the feasibility of the control commands identified by the voice recognition technology based on the voice commands input by the user. Summary of the invention

[0004] The purpose of the present invention is to provide a big data-based intelligent voice control system and method for unmanned aerial vehicle flight to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for intelligent voice control of unmanned aerial vehicle flight based on big data, the method comprising:

[0006] S10: receiving historical voice commands issued by the user through a voice receiving device to obtain a voice command error correction set, training a traditional acoustic model using the keyword and non-keyword phoneme sequences analyzed through the voice command error correction set to obtain a first voice model, a second voice model, and a third voice model, and processing the real-time voice commands received by the voice receiving device using the first voice model, the second voice model, and the third voice model to obtain a plurality of task commands;

[0007] S20: Analyze the feasibility of each task instruction by simulating the motion scene of the drone using the surrounding images collected by the camera device, determine the target control instruction of the drone based on the analysis result, and the drone control terminal controls the motion state of the drone according to the target control instruction;

[0008] S30: When the UAV control terminal controls the motion state of the UAV according to the target control instruction, the real-time motion state of the UAV is obtained through the UAV state receiving device, and the early warning response of the UAV is analyzed. Based on the analysis result, it is selected whether to change the target control instruction received by the UAV control terminal, and the change result is determined;

[0009] S40: The drone control terminal adjusts the motion state of the drone according to the change result, and modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state adjustment and the surrounding images collected by the camera device installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and displays the modified three-dimensional motion scene through the display software loaded in the user's smart terminal.

[0010] Further, the S10 includes:

[0011] S101: Acquire historical voice commands issued by the user, compare the acquired historical voice commands with standard control commands of the drone, store inconsistent historical voice commands in a set, obtain a voice command error correction set, and mark the keywords in the voice command error correction set according to the keywords in the standard control commands of the drone;

[0012] S102: non-keywords with synonymous features are obtained from the voice command error correction set, CTC is used as a loss function to train the traditional acoustic model, and the phoneme sequences of non-keywords with synonymous features are obtained according to the historical voice commands issued by the user, and the phoneme sequences of non-keywords with synonymous features are used as input, and the standard phoneme sequences corresponding to the non-keywords with synonymous features are used as output, and the trained traditional acoustic model is trained again to obtain a first voice model;

[0013] According to the historical voice commands issued by the user, the phoneme sequence of each marked keyword is obtained, the phoneme sequence of each marked keyword is used as input, the standard phoneme sequence corresponding to the marked keyword is used as output, and the trained traditional acoustic model is retrained to obtain a second voice model;

[0014] In the voice command error correction set, non-keywords with non-synonymous features are obtained, and according to the historical voice commands issued by the user, the phoneme sequences of non-keywords with non-synonymous features are obtained, and the phoneme sequences of non-keywords with non-synonymous features are used as input, and the standard phoneme sequences corresponding to the non-keywords with non-synonymous features are used as output, and the trained traditional acoustic model is trained twice to obtain a third voice model; the trained traditional acoustic model is trained by the phoneme sequence to ensure that each voice model obtained after training can accurately identify and accurately correct the voice commands issued by the user, so as to avoid the voice commands input by the user being different from the control commands input by the drone control system and being unable to control the drone, thereby improving the use effect;

[0015] S103: Split the voice command issued by the user at time t, and input the non-keywords with synonymous features, keywords, and non-keywords with non-synonymous features obtained by the splitting into the first voice model, the second voice model, and the third voice model respectively, and combine and arrange the output words and output phrases of the first voice model, the second voice model, and the third voice model according to the composition logic of the standard control command to obtain a plurality of task instructions.

[0016] Further, the S20 includes:

[0017] S201: installing a plurality of camera devices on the drone, collecting images around the drone through the camera devices, performing a three-dimensional simulation of the motion scene of the drone based on the collected images, constructing a three-dimensional space coordinate system in the three-dimensional motion scene with the position of the drone at time t as the coordinate origin, obtaining the position coordinates of each associated obstacle in the three-dimensional motion scene, calculating the distance value between the drone and each associated obstacle according to a three-dimensional distance calculation formula, determining the calculated minimum distance value d, and the associated obstacle corresponding to the minimum distance value d is the nearest associated obstacle;

[0018] S202: The issuing time T′ of the previous voice command issued by the user at time t is obtained, and the flight acceleration of the UAV in the time period [T′, t] is analyzed according to the previous voice command of the voice command issued by the user at time t. If there is no speed description word in the previous voice command of the voice command issued by the user at time t, and the speed description words include addition, subtraction, rapid addition, rapid subtraction, rapid addition and rapid subtraction, then a=0; if there is a speed description word in the previous voice command of the voice command issued by the user at time t, then a=u i , i = 1, 2, 3, 4, 5, 6, indicating the number corresponding to each speed description word, u iIt indicates the corresponding flight acceleration of the drone when there is a speed description word numbered i in the voice command. When i=1, it indicates that the speed description word is subtraction, when i=2, it indicates that the speed description word is rapid subtraction, when i=3, it indicates that the speed description word is rapid subtraction, when i=4, it indicates that the speed description word is addition, when i=5, it indicates that the speed description word is rapid addition, and when i=6, it indicates that the speed description word is rapid addition. By matching and analyzing each speed description word with the running acceleration of the drone, the running state change of the drone can be quickly analyzed through the voice command issued by the user, thereby improving the determination rate of the target control command;

[0019] S203: Calculate the time required for the UAV to move from the coordinate origin to the position of the nearest associated obstacle according to T=d / [V0+a(tT′)], where V0 represents the flight speed of the UAV at time T′, and a represents the flight acceleration of the UAV in the time period [T′, t] determined according to the previous voice command issued by the user at time t;

[0020] When T < the emergency braking time of the UAV when the flight speed is V0+a(tT′), the feasibility of the task instruction without speed descriptor E0=0, and a mathematical model is constructed to predict the feasibility of each task instruction with speed descriptor. The specific mathematical model is:

[0021]

[0022] Where V′ represents the average speed of the nearest associated obstacle in the time period [T′, t], and u ti Indicates the flight acceleration of the drone when the user's voice command at time t contains a speed description word numbered i;

[0023] When T≥the emergency braking time of the UAV at the flight speed of V0+a(tT′), according to The feasibility of task instructions without speed descriptors was calculated;

[0024] according to Calculate the feasibility of task instructions with speed descriptors;

[0025] E i and E0 are put into the set M, and the maximum value in the set M is searched. If the maximum value found = E0, then the task instruction without speed description word is the target control instruction. If the maximum value found = E k , then there is a task instruction with a speed description word numbered k as the target control instruction, k = 1, 2, 3, 4, 5, 6;

[0026] S204: The UAV control terminal controls the motion state of the UAV according to the target control instruction.

[0027] Further, the S30 includes:

[0028] S301: The time x at which the drone control terminal controls the motion state of the drone according to the target control command is determined, and the position coordinates (X) of the drone at the time x+y*g are received by the drone state receiving device. x+y*g ,Y x+y*g ) and flight speed V x+y*g To obtain, g represents the acquisition time interval of the UAV motion state, y = 1, 2, ..., n, represents the number corresponding to each acquisition time of the UAV motion state, and n represents the total number;

[0029] S302: The camera installed on the drone is used to obtain the position coordinates (X′ x+y*g ,Y′ x+y*g ) is obtained, and the distance value D of the drone to the nearest associated obstacle at the time x+y*g is calculated according to the three-dimensional distance calculation formula. x+y*g Calculate according to c x+(y+1)*g =(D x+(y+1)*g -D x+y*g ) / g calculates the movement speed of the nearest associated obstacle at the time x+(y+1)*g, and according to γ x+(y+1)*g =[(c x+(y+1)*g -V x+(y+1)*g )*g+D x+y*g ]-V x+(y+1)*g *f x+(y+1)*g Predict the early warning response judgment index of the UAV at the moment x+(y+1)*g; By analyzing the real-time motion of the nearest associated obstacle and the UAV, combined with the emergency braking distance corresponding to any motion of the UAV analyzed, the early warning response judgment index of the UAV is predicted, which improves the prediction accuracy, and can adjust the motion state of the UAV before the UAV collides with the nearest associated obstacle, thereby improving the flight safety of the UAV;

[0030] Among them, V x+(y+1)*g represents the flight speed of the drone at the time x+(y+1)*g, f x+(y+1)*g Indicates that the UAV is flying at a speed of V x+(y+1)*g Emergency braking time:

[0031] When γ x+(y+1)*g ≤0, the target control command received by the UAV control terminal is changed. x+(y+1)*g When > 0, the target control command received by the UAV control terminal will not be changed;

[0032] S303: Determine the change result of the target control instruction received by the UAV control terminal.

[0033] Further, the S303 includes:

[0034] According to p = [(c x+(y+1)*g ) 2 -(V x+(y+1)*g ) 2 ] / (2*|γ x+(y+1)*g |) Calculate the target flight acceleration of the UAV, and adjust the keywords of the target control command according to the target flight acceleration of the UAV. The specific adjustment method is:

[0035] Let p = u i The corresponding number is determined, and the determined number is q, q = 1, 2, 3, 4, 5, 6, then the speed description word corresponding to the number q is the adjustment word of the target control instruction keyword;

[0036] Based on the adjustment results, the modified target control instruction is obtained, where p represents the target flight acceleration of the UAV.

[0037] An intelligent voice control system for UAV flight based on big data, the system comprising a voice model building module, a UAV motion state control module, an early warning response situation analysis module and a three-dimensional motion scene display module;

[0038] The speech model building module is used to train the traditional acoustic model using the phoneme sequences of keywords and non-keywords analyzed through the speech instruction error correction set to obtain a plurality of speech models, and use the obtained plurality of speech models to process the real-time speech instructions received by the speech receiving device to obtain a plurality of task instructions;

[0039] The UAV motion state control module is used to simulate the motion scene of the UAV through the surrounding images collected by the camera device, analyze the feasibility of each task instruction, and the UAV control terminal controls the motion state of the UAV according to the analyzed target control instruction;

[0040] The warning response situation analysis module is used to obtain the real-time motion state of the drone through the drone state receiving device, analyze the warning response situation of the drone, and based on the analysis result, select whether to change the target control instruction received by the drone control terminal, and determine the change result;

[0041] The three-dimensional motion scene display module is used to modify the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state is adjusted, and the surrounding images collected by the camera device installed on the drone, and display the modified three-dimensional motion scene through the display software loaded in the user's smart terminal.

[0042] Furthermore, the speech model construction module includes a speech instruction error correction set marking unit, a speech model construction unit and a task instruction output unit;

[0043] The voice command error correction set marking unit compares the user's historical voice commands with the standard control commands of the drone to obtain a voice command error correction set, and marks the keywords in the voice command error correction set according to the keywords in each standard control command of the drone;

[0044] The speech model construction unit uses CTC as a loss function to train the traditional acoustic model, and according to the historical speech instructions issued by the user, obtains the phoneme sequence of each non-keyword with synonymous features, the phoneme sequence of each marked keyword, and the phoneme sequence of each non-keyword with non-synonymous features, takes the obtained phoneme sequences as input, and takes the standard phoneme sequence of each non-keyword with synonymous features, the standard phoneme sequence of each marked keyword, and the standard phoneme sequence of each non-keyword with non-synonymous features as output, to obtain the first speech model, the second speech model, and the third speech model respectively;

[0045] The task instruction output unit splits the voice instructions issued by the user, and inputs the non-keywords with synonymous features, keywords, and non-keywords with non-synonymous features obtained by the splitting into the first voice model, the second voice model, and the third voice model respectively, and combines and arranges the output words and output terms of each voice model according to the composition logic of the standard control instructions to obtain a plurality of task instructions.

[0046] Further, the UAV motion state control module includes a nearest associated obstacle determination unit, a flight acceleration analysis unit, a voice command feasibility prediction unit and a UAV motion state control unit;

[0047] The nearest associated obstacle determination unit performs a three-dimensional simulation of the motion scene of the drone using surrounding images collected by the camera device, constructs a three-dimensional space coordinate system in the three-dimensional motion scene, calculates the distance value between the drone and each associated obstacle, and determines the nearest associated obstacle based on the calculation result;

[0048] The flight acceleration analysis unit analyzes the flight acceleration of the drone according to the last voice command of the voice command issued by the user;

[0049] The voice command feasibility prediction unit calculates the time required for the UAV to move from the coordinate origin to the position of the nearest associated obstacle according to the constructed mathematical formula, compares the calculation result with the emergency braking time of the UAV at the corresponding flight speed, and predicts the feasibility of each task instruction through the constructed mathematical models according to the comparison result, and determines the target control instruction of the UAV based on the prediction result;

[0050] The UAV motion state control unit controls the motion state of the UAV through the target control instruction received by the UAV control terminal.

[0051] Furthermore, the warning response situation analysis module includes a UAV motion state acquisition unit, a warning response judgment index prediction unit and a change result determination unit;

[0052] The UAV motion state acquisition unit acquires the real-time position coordinates and real-time flight speed of the UAV after the motion state is controlled through the UAV state receiving device;

[0053] The early warning response judgment index prediction unit calculates the real-time movement speed of the nearest associated obstacle according to the real-time distance value between the UAV and the nearest associated obstacle, predicts the real-time early warning response judgment index of the UAV based on the calculation result, and selects whether to change the target control instruction received by the UAV control terminal based on the prediction result;

[0054] The change result determination unit changes the target control instruction received by the drone control terminal.

[0055] Furthermore, the three-dimensional motion scene display module modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state is adjusted and the surrounding images collected by the camera device installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and the modified three-dimensional motion scene is displayed through the display software loaded in the user's smart terminal.

[0056] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0057] 1. The present invention trains the trained traditional acoustic model through the phoneme sequence to ensure that each speech model obtained after training can accurately identify and accurately correct the voice commands issued by the user, thereby avoiding the situation where the voice commands input by the user are different from the control commands input by the drone control system and the drone cannot be controlled. This process does not require the user to be proficient in the control commands written by the drone control system, and the present invention can also control the operating status of the drone according to the synonyms of the written control commands, thereby improving the use effect of the system.

[0058] 2. The present invention matches and analyzes each speed descriptor with the operating acceleration of the drone, and can quickly analyze the change in the operating state of the drone through the voice commands issued by the user. Based on the analysis results, the feasibility of each task instruction is predicted, thereby improving the determination rate of the target control instructions.

[0059] 3. The present invention analyzes the real-time motion conditions of the nearest associated obstacle and the UAV, combines the analyzed emergency braking distance corresponding to the UAV in any motion condition, and predicts the early warning response judgment index of the UAV, thereby improving the prediction accuracy. It can also adjust the motion state of the UAV before the UAV collides with the nearest associated obstacle, thereby improving the flight safety of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 It is a schematic diagram of the workflow of an intelligent voice control system and method for unmanned aerial vehicle flight based on big data of the present invention;

[0062] Figure 2 It is a schematic structural diagram of the working principle of an intelligent voice control system and method for unmanned aerial vehicle flight based on big data of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] See also Figure 1 and Figure 2 The present invention provides a technical solution: a method for intelligent voice control of UAV flight based on big data, the method comprising:

[0065] S10: receiving historical voice commands issued by the user through a voice receiving device to obtain a voice command error correction set, training a traditional acoustic model using the keyword and non-keyword phoneme sequences analyzed through the voice command error correction set to obtain a first voice model, a second voice model, and a third voice model, and processing the real-time voice commands received by the voice receiving device using the first voice model, the second voice model, and the third voice model to obtain a plurality of task commands;

[0066] S10 includes:

[0067] S101: Obtain the historical voice commands issued by the user, compare the obtained historical voice commands with the standard control commands of the drone. The standard control commands refer to the control commands written in the drone control system. Store the historical voice commands with inconsistent comparison results in a set to obtain a voice command error correction set. Mark the keywords in the voice command error correction set according to the keywords in each standard control command of the drone. The keywords include takeoff, landing, forward, backward, front turn, rear turn, left front, right front, left rear, right rear, left turn, right turn, ascent, descent, hover, acceleration, deceleration, etc.;

[0068] S102: Obtain the non-keywords with synonymous features in the voice command error correction set. Non-keywords refer to other words or phrases in the voice command except for the keywords. If a word or phrase has a synonym, it is considered that the word or phrase has synonymous features. Use CTC as the loss function to train the traditional acoustic model (this training method is an existing technology. The traditional acoustic model is a mixture Gaussian model - hidden Markov model. CTC enables the RNN to directly learn sequence data without pre-labeling the mapping relationship between the input sequence and the output sequence in the training data, making the RNN model achieve better results in sequence learning tasks such as speech recognition. The RNN model is the recurrent neural network). According to the historical voice commands issued by the user, obtain the phoneme sequences of each non-keyword with synonymous features. Use the phoneme sequences of each non-keyword with synonymous features as the input, and use the standard phoneme sequences corresponding to the non-keywords with synonymous features as the output. The phoneme sequence refers to the sequence of phonetic notations of a word or phrase, and the standard phoneme sequence refers to the sequence of phonetic notations of each word or phrase in each standard control command. Perform secondary training on the trained traditional acoustic model to obtain the first voice model. For example, the phoneme sequence of the keyword "forward" is: q-i-a-n-j-i-n;

[0069] According to the historical voice commands issued by the user, obtain the phoneme sequences of each marked keyword. Use the phoneme sequences of each marked keyword as the input, and use the standard phoneme sequences corresponding to the marked keywords as the output. Perform secondary training on the trained traditional acoustic model to obtain the second voice model;

[0070] Obtain the non-keywords with non-synonymous features in the voice command error correction set. According to the historical voice commands issued by the user, obtain the phoneme sequences of each non-keyword with non-synonymous features. Use the phoneme sequences of each non-keyword with non-synonymous features as the input, and use the standard phoneme sequences corresponding to the non-keywords with non-synonymous features as the output. Perform secondary training on the trained traditional acoustic model to obtain the third voice model;

[0071] S103: split the voice command issued by the user at time t, input the non-keywords, keywords and non-keywords with synonymous features obtained by the splitting into the first voice model, the second voice model and the third voice model respectively, and arrange the output words and output words of the first voice model, the second voice model and the third voice model according to the composition logic of the standard control command to obtain a number of task instructions. For example, one of the standard control instructions of the drone is: fly forward, and the voice command "fly to the left rear" issued by the user is split, "towards" is input into the first voice model, "left rear" is input into the second voice model, and "fly" and "direction" are input into the third voice model. The output word of the first voice model is "toward", the output words of the second voice model are "back" and "left", and the output word is "left rear", and the output words of the third voice model are "fly" and "direction", then according to the composition logic of the standard control instruction, they are arranged and combined to obtain the task instructions "fly backward", "fly to the left" and "fly to the left rear";

[0072] S20: Analyze the feasibility of each task instruction by simulating the motion scene of the drone using the surrounding images collected by the camera device, determine the target control instruction of the drone based on the analysis result, and the drone control terminal controls the motion state of the drone according to the target control instruction;

[0073] S201: installing a plurality of camera devices on the drone, the camera devices including cameras, collecting images around the drone through the camera devices, performing a three-dimensional simulation of the motion scene of the drone based on the collected images, constructing a three-dimensional space coordinate system with the position of the drone at time t as the coordinate origin in the three-dimensional motion scene, obtaining the position coordinates of each associated obstacle in the three-dimensional motion scene, the associated obstacle refers to an object located on the preset flight route of the drone and capable of forcing the drone to deviate from the preset route, calculating the distance value between the drone and each associated obstacle according to a three-dimensional distance calculation formula, determining the calculated minimum distance value d, and the associated obstacle corresponding to the minimum distance value d is the nearest associated obstacle;

[0074] S202: The issuing time T′ of the last voice command issued by the user at time t is obtained. The user has not issued any voice command in the time period (T′, t). According to the last voice command of the voice command issued by the user at time t, the flight acceleration of the UAV in the time period [T′, t] is analyzed. If there is no speed description word in the last voice command of the voice command issued by the user at time t, and the speed description words include addition, subtraction, rapid addition, rapid subtraction, rapid addition and rapid subtraction, then a=0. If there is a speed description word in the last voice command of the voice command issued by the user at time t, then a=u i, i = 1, 2, 3, 4, 5, 6, indicating the number corresponding to each speed description word, u i Indicates the flight acceleration of the drone when there is a speed descriptor numbered i in the voice command. When i=1, it means the speed descriptor is subtraction; when i=2, it means the speed descriptor is rapid subtraction; when i=3, it means the speed descriptor is rapid subtraction; when i=4, it means the speed descriptor is addition; when i=5, it means the speed descriptor is rapid addition; when i=6, it means the speed descriptor is rapid addition;

[0075] S203: Calculate the time required for the UAV to move from the coordinate origin to the position of the nearest associated obstacle according to T=d / [V0+a(tT′)], where V0 represents the flight speed of the UAV at time T′, and a represents the flight acceleration of the UAV in the time period [T′, t] determined according to the previous voice command issued by the user at time t;

[0076] When T < the emergency braking time of the UAV when the flight speed is V0+a(tT′), the feasibility of the task instruction without speed descriptor E0=0, and a mathematical model is constructed to predict the feasibility of each task instruction with speed descriptor. The specific mathematical model is:

[0077]

[0078] Where V′ represents the average speed of the nearest associated obstacle in the time period [T′, t], and u ti It indicates the flight acceleration of the drone when the speed description word numbered i exists in the voice command issued by the user at time t. The syntax of the IF function is IF (logical expression, result when the value is true, result when the value is false);

[0079] When T≥the emergency braking time of the UAV at the flight speed of V0+a(tT′), according to The feasibility of task instructions without speed descriptors was calculated;

[0080] according to Calculate the feasibility of task instructions with speed descriptors;

[0081] E i and E0 are put into the set M, and the maximum value in the set M is searched. If the maximum value found = E0, then the task instruction without speed description word is the target control instruction. If the maximum value found = E k , then there is a task instruction with a speed description word numbered k as the target control instruction, k = 1, 2, 3, 4, 5, 6;

[0082] S204: The UAV control terminal controls the motion state of the UAV according to the target control instruction;

[0083] S30: When the UAV control terminal controls the motion state of the UAV according to the target control instruction, the real-time motion state of the UAV is obtained through the UAV state receiving device, and the early warning response of the UAV is analyzed. Based on the analysis result, it is selected whether to change the target control instruction received by the UAV control terminal, and the change result is determined;

[0084] S30 includes:

[0085] S301: The time x at which the drone control terminal controls the motion state of the drone according to the target control command is determined, and the position coordinates (X) of the drone at the time x+y*g are received by the drone state receiving device. x+y*g ,Y x+y*g ) and flight speed V x+y*g To obtain, g represents the acquisition time interval of the UAV motion state, y = 1, 2, ..., n, represents the number corresponding to each acquisition time of the UAV motion state, and n represents the total number;

[0086] S302: The camera installed on the drone is used to obtain the position coordinates (X′ x+y*g ,Y′ x+y*g ) is obtained, and the distance value D of the drone to the nearest associated obstacle at the time x+y*g is calculated according to the three-dimensional distance calculation formula. x+y*g Calculate according to c x+(y+1)*g =(D x+(y+1)*g -D x+y*g ) / g calculates the movement speed of the nearest associated obstacle at the time x+(y+1)*g, and according to γ x+(y+1)*g =[(c x+(y+1)*g -V x+(y+1)*g )*g+D x+y*g ]-V x+(y+1)*g *f x+(y+1)*g Predict the early warning response judgment index of the UAV at time x+(y+1)*g;

[0087] Among them, V x+(y+1)*g represents the flight speed of the drone at the time x+(y+1)*g, f x+(y+1)*g Indicates that the UAV is flying at a speed of V x+(y+1)*g Emergency braking time:

[0088] When γ x+(y+1)*g ≤0, the target control command received by the UAV control terminal is changed. x+(y+1)*gWhen > 0, the target control command received by the UAV control terminal will not be changed;

[0089] S303: Determine the result of changing the target control instruction received by the UAV control terminal. The specific method is: according to p=[(c x+(y+1)*g ) 2 -(V x+(y+1)*g ) 2 ] / (2*|γ x+(y+1)*g |) Calculate the target flight acceleration of the UAV, and adjust the keywords of the target control command according to the target flight acceleration of the UAV. The specific adjustment method is:

[0090] Let p = u i The corresponding number is determined, and the determined number is q, q = 1, 2, 3, 4, 5, 6, then the speed description word corresponding to the number q is the adjustment word of the target control instruction keyword;

[0091] Based on the adjustment result, the modified target control instruction is obtained, where p represents the target flight acceleration of the UAV;

[0092] S40: The drone control terminal adjusts the motion state of the drone according to the change result, and modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state adjustment and the surrounding images collected by the camera installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and displays the modified three-dimensional motion scene through the display software loaded in the user's smart terminal. The smart terminal includes a mobile phone, a tablet, etc., and the display software refers to software that can display a three-dimensional motion scene.

[0093] An intelligent voice control system for UAV flight based on big data, the system includes a voice model building module, a UAV motion state control module, an early warning response situation analysis module and a three-dimensional motion scene display module;

[0094] The voice model building module is used to train the traditional acoustic model using the phoneme sequences of keywords and non-keywords analyzed through the voice command error correction set to obtain a number of voice models, and use the obtained several voice models to process the real-time voice commands received by the voice receiving device to obtain a number of task commands;

[0095] The speech model building module includes a speech instruction error correction set marking unit, a speech model building unit and a task instruction output unit;

[0096] The voice command error correction set marking unit compares the user's historical voice commands with the standard control commands of the drone to obtain a voice command error correction set, and marks the keywords in the voice command error correction set according to the keywords in each standard control command of the drone;

[0097] The speech model construction unit uses CTC as a loss function to train the traditional acoustic model, and according to the historical speech instructions issued by the user, obtains the phoneme sequence of each non-keyword with synonymous features, the phoneme sequence of each marked keyword, and the phoneme sequence of each non-keyword with non-synonymous features, takes the obtained phoneme sequences as input, and takes the standard phoneme sequence of each non-keyword with synonymous features, the standard phoneme sequence of each marked keyword, and the standard phoneme sequence of each non-keyword with non-synonymous features as output, to obtain the first speech model, the second speech model, and the third speech model respectively;

[0098] The task instruction output unit splits the voice instruction issued by the user, inputs the non-keywords with synonymous features, keywords and non-keywords with non-synonymous features obtained by the splitting into the first voice model, the second voice model and the third voice model respectively, and arranges the output words and output phrases of each voice model according to the composition logic of the standard control instruction to obtain a plurality of task instructions;

[0099] The UAV motion state control module is used to simulate the motion scene of the UAV through the surrounding images collected by the camera device, analyze the feasibility of each task instruction, and the UAV control terminal controls the motion state of the UAV according to the analyzed target control instruction;

[0100] The UAV motion state control module includes a nearest associated obstacle determination unit, a flight acceleration analysis unit, a voice command feasibility prediction unit, and a UAV motion state control unit;

[0101] The nearest associated obstacle determination unit performs a three-dimensional simulation of the motion scene of the UAV through the surrounding images collected by the camera device, constructs a three-dimensional space coordinate system in the three-dimensional motion scene, calculates the distance value between the UAV and each associated obstacle, and determines the nearest associated obstacle based on the calculation result;

[0102] The flight acceleration analysis unit analyzes the flight acceleration of the drone according to the last voice command issued by the user;

[0103] The voice command feasibility prediction unit calculates the time required for the drone to move from the coordinate origin to the location of the nearest associated obstacle according to the constructed mathematical formula, compares the calculation result with the emergency braking time of the drone at the corresponding flight speed, and predicts the feasibility of each task instruction through the constructed mathematical models according to the comparison result, and determines the target control instruction of the drone based on the prediction result;

[0104] The UAV motion state control unit controls the motion state of the UAV through the target control command received by the UAV control terminal;

[0105] The early warning response situation analysis module is used to obtain the real-time motion status of the drone through the drone status receiving device, analyze the early warning response of the drone, and based on the analysis results, choose whether to change the target control instructions received by the drone control terminal, and determine the change results;

[0106] The early warning response situation analysis module includes a UAV motion state acquisition unit, an early warning response judgment index prediction unit, and a change result determination unit;

[0107] The UAV motion state acquisition unit acquires the real-time position coordinates and real-time flight speed of the UAV after the UAV is controlled in the motion state through the UAV state receiving device;

[0108] The early warning response judgment index prediction unit calculates the real-time movement speed of the nearest associated obstacle according to the real-time distance value between the UAV and the nearest associated obstacle, predicts the real-time early warning response judgment index of the UAV based on the calculation result, and selects whether to change the target control instruction received by the UAV control terminal based on the prediction result;

[0109] The change result determination unit changes the target control instruction received by the UAV control terminal;

[0110] The three-dimensional motion scene display module modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state is adjusted and the surrounding images collected by the camera installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and the modified three-dimensional motion scene is displayed through the display software loaded in the user's smart terminal.

[0111] Example 1: Assume that the emergency braking time of the drone is 4 minutes when the flight speed is V0+a(tT′), T=5 minutes, then T≥the emergency braking time of the drone when the flight speed is V0+a(tT′), assume that the average movement speed of the nearest associated obstacle in the time period [T′, t] is V′=10 m / min, V0+a(tT′)=8 m / min, ut1 =-1m / min 2 ,u t2 =-3 m / min 2 ,u t3 =-5 m / min 2 ,u t4 =1m / min 2 ,u t5 =3 m / min 2 ,u t6 =5 m / min 2 , d = 4 m;

[0112] When there is no speed descriptor in the task instruction, the feasibility of the task instruction is:

[0113]

[0114] Then the feasibility of the task instruction without speed descriptors is 0.25;

[0115] When there is a speed descriptor in the task instruction, the feasibility of the task instruction is:

[0116]

[0117] Then the feasibility of the task instruction with the speed description word “decrease” is 0.31;

[0118]

[0119] Then the feasibility of the task instruction with the speed description word "rapid reduction" is 0.46;

[0120]

[0121] Then the feasibility of the task instruction with the speed description word "rapidly reduce" is 0.64;

[0122]

[0123] Then the feasibility of the task instruction with the speed descriptor “add” is 0.19;

[0124]

[0125] Then the feasibility of the task instruction with the speed description word "fast addition" is 0.08;

[0126]

[0127] Then the feasibility of the task instruction with the speed description word "fast addition" is 0.

[0128] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0129] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for intelligent voice control of UAV flight based on big data, characterized by: The method comprises: S10: receiving historical voice commands issued by the user through a voice receiving device to obtain a voice command error correction set, training a traditional acoustic model using the keyword and non-keyword phoneme sequences analyzed through the voice command error correction set to obtain a first voice model, a second voice model, and a third voice model, and processing the real-time voice commands received by the voice receiving device using the first voice model, the second voice model, and the third voice model to obtain a plurality of task commands; The S10 includes: S101: Acquire historical voice commands issued by the user, compare the acquired historical voice commands with standard control commands of the drone, store inconsistent historical voice commands in a set, obtain a voice command error correction set, and mark the keywords in the voice command error correction set according to the keywords in the standard control commands of the drone; S102: non-keywords with synonymous features are obtained from the voice command error correction set, CTC is used as a loss function to train the traditional acoustic model, and the phoneme sequences of non-keywords with synonymous features are obtained according to the historical voice commands issued by the user, and the phoneme sequences of non-keywords with synonymous features are used as input, and the standard phoneme sequences corresponding to the non-keywords with synonymous features are used as output, and the trained traditional acoustic model is trained again to obtain a first voice model; According to the historical voice commands issued by the user, the phoneme sequence of each marked keyword is obtained, the phoneme sequence of each marked keyword is used as input, the standard phoneme sequence corresponding to the marked keyword is used as output, and the trained traditional acoustic model is retrained to obtain a second voice model; Acquire non-keywords with non-synonymous features from the voice command error correction set, acquire phoneme sequences of non-keywords with non-synonymous features according to historical voice commands issued by the user, use the phoneme sequences of non-keywords with non-synonymous features as input, use the standard phoneme sequences corresponding to the non-keywords with non-synonymous features as output, perform secondary training on the trained traditional acoustic model, and obtain a third voice model; S103: splitting the voice command issued by the user at time t, inputting the non-keywords with synonymous features, keywords, and non-keywords with non-synonymous features obtained by splitting into the first voice model, the second voice model, and the third voice model, respectively, and combining and arranging the output words and output phrases of the first voice model, the second voice model, and the third voice model according to the composition logic of the standard control command to obtain a plurality of task instructions; S20: Analyze the feasibility of each task instruction by simulating the motion scene of the drone using the surrounding images collected by the camera device, determine the target control instruction of the drone based on the analysis result, and the drone control terminal controls the motion state of the drone according to the target control instruction; The S20 includes: S201: installing a plurality of camera devices on the drone, collecting images around the drone through the camera devices, performing a three-dimensional simulation of the motion scene of the drone based on the collected images, constructing a three-dimensional space coordinate system in the three-dimensional motion scene with the position of the drone at time t as the coordinate origin, obtaining the position coordinates of each associated obstacle in the three-dimensional motion scene, calculating the distance value between the drone and each associated obstacle according to a three-dimensional distance calculation formula, determining the calculated minimum distance value d, and the associated obstacle corresponding to the minimum distance value d is the nearest associated obstacle; S202: The issuing time T′ of the previous voice command issued by the user at time t is obtained, and the flight acceleration of the UAV in the time period [T′, t] is analyzed according to the previous voice command of the voice command issued by the user at time t. If there is no speed description word in the previous voice command of the voice command issued by the user at time t, and the speed description words include addition, subtraction, rapid addition, rapid subtraction, rapid addition and rapid subtraction, then a=0; if there is a speed description word in the previous voice command of the voice command issued by the user at time t, then a=u i , i = 1, 2, 3, 4, 5, 6, indicating the number corresponding to each speed description word, u i Indicates the flight acceleration of the drone when there is a speed descriptor numbered i in the voice command. When i=1, it means the speed descriptor is subtraction; when i=2, it means the speed descriptor is rapid subtraction; when i=3, it means the speed descriptor is rapid subtraction; when i=4, it means the speed descriptor is addition; when i=5, it means the speed descriptor is rapid addition; when i=6, it means the speed descriptor is rapid addition; S203: Calculate the time required for the UAV to move from the coordinate origin to the position of the nearest associated obstacle according to T=d / [V0+a(tT′)], where V0 represents the flight speed of the UAV at time T′, and a represents the flight acceleration of the UAV in the time period [T′, t] determined according to the previous voice command issued by the user at time t; When T < the emergency braking time of the UAV when the flight speed is V0+a(tT′), the feasibility of the task instruction without speed descriptor E0=0, and a mathematical model is constructed to predict the feasibility of each task instruction with speed descriptor. The specific mathematical model is: Where V′ represents the average speed of the nearest associated obstacle in the time period [T′, t], and u ti Indicates the flight acceleration of the drone when the user's voice command at time t contains a speed description word numbered i; When T≥the emergency braking time of the UAV at the flight speed of V0+a(tT′), according to The feasibility of task instructions without speed descriptors was calculated; according to Calculate the feasibility of task instructions with speed descriptors; E i and E0 are put into the set M, and the maximum value in the set M is searched. If the maximum value found = E0, then the task instruction without speed description word is the target control instruction. If the maximum value found = E k , then there is a task instruction with a speed description word numbered k as the target control instruction, k = 1, 2, 3, 4, 5, 6; S204: The UAV control terminal controls the motion state of the UAV according to the target control instruction; S30: When the UAV control terminal controls the motion state of the UAV according to the target control instruction, the real-time motion state of the UAV is obtained through the UAV state receiving device, and the early warning response of the UAV is analyzed. Based on the analysis result, it is selected whether to change the target control instruction received by the UAV control terminal, and the change result is determined; S40: The drone control terminal adjusts the motion state of the drone according to the change result, and modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state adjustment and the surrounding images collected by the camera device installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and displays the modified three-dimensional motion scene through the display software loaded in the user's smart terminal.

2. The method for intelligent voice control of unmanned aerial vehicle flight based on big data according to claim 1 is characterized in that: The S30 includes: S301: The time x at which the drone control terminal controls the motion state of the drone according to the target control command is determined, and the position coordinates (X) of the drone at the time x+y*g are received by the drone state receiving device. x+y*g ,Y x+y*g ) and flight speed V x+y*g To obtain, g represents the acquisition time interval of the UAV motion state, y = 1, 2, ..., n, represents the number corresponding to each acquisition time of the UAV motion state, and n represents the total number; S302: The camera installed on the drone is used to obtain the position coordinates (X′ x+y*g ,Y′ x+y*g ) is obtained, and the distance value D of the drone to the nearest associated obstacle at the time x+y*g is calculated according to the three-dimensional distance calculation formula. x+y*g Calculate according to c x+(y+1)*g =(D x+(y+1)*g -D x+y*g ) / g calculates the movement speed of the nearest associated obstacle at the time x+(y+1)*g, and according to γ x+(y+1)*g =[(c x+(y+1)*g -V x+(y+1)*g )*g+D x+y*g ]-V x+(y+1)*g *f x+(y+1)*g Predict the early warning response judgment index of the UAV at time x+(y+1)*g; Among them, V x+(y+1)*g represents the flight speed of the drone at the time x+(y+1)*g, f x+(y+1)*g Indicates that the drone is flying at a speed of V x+(y+1)*g Emergency braking time: When γ x+(y+1)*g ≤0, the target control command received by the UAV control terminal is changed. x+(y+1)*g When > 0, the target control command received by the UAV control terminal will not be changed; S303: Determine the change result of the target control instruction received by the UAV control terminal.

3. The method for intelligent voice control of unmanned aerial vehicle flight based on big data according to claim 2 is characterized in that: The S303 includes: According to p = [(c x+*y+1)*g ) 2 -(V x+(y+1)*g) 2 ] / (2*|γ x+(y+1)*g |) Calculate the target flight acceleration of the UAV, and adjust the keywords of the target control command according to the target flight acceleration of the UAV. The specific adjustment method is: Let p = u i The corresponding number is determined, and the determined number is q, q = 1, 2, 3, 4, 5, 6, then the speed description word corresponding to the number q is the adjustment word of the target control instruction keyword; Based on the adjustment results, the modified target control instruction is obtained, where p represents the target flight acceleration of the UAV.

4. A drone flight intelligent voice control system based on big data applied to the drone flight intelligent voice control method based on big data according to any one of claims 1 to 3, characterized in that: The system includes a voice model building module, a UAV motion state control module, a warning response situation analysis module and a three-dimensional motion scene display module; The speech model building module is used to train the traditional acoustic model using the phoneme sequences of keywords and non-keywords analyzed through the speech instruction error correction set to obtain a plurality of speech models, and use the obtained plurality of speech models to process the real-time speech instructions received by the speech receiving device to obtain a plurality of task instructions; The UAV motion state control module is used to simulate the motion scene of the UAV through the surrounding images collected by the camera device, analyze the feasibility of each task instruction, and the UAV control terminal controls the motion state of the UAV according to the analyzed target control instruction; The warning response situation analysis module is used to obtain the real-time motion state of the drone through the drone state receiving device, analyze the warning response situation of the drone, and based on the analysis result, select whether to change the target control instruction received by the drone control terminal, and determine the change result; The three-dimensional motion scene display module is used to modify the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state is adjusted, and the surrounding images collected by the camera device installed on the drone, and display the modified three-dimensional motion scene through the display software loaded in the user's smart terminal.

5. The intelligent voice control system for UAV flight based on big data according to claim 4 is characterized by: The speech model construction module includes a speech instruction error correction set marking unit, a speech model construction unit and a task instruction output unit; The voice command error correction set marking unit compares the user's historical voice commands with the standard control commands of the drone to obtain a voice command error correction set, and marks the keywords in the voice command error correction set according to the keywords in each standard control command of the drone; The speech model construction unit uses CTC as a loss function to train the traditional acoustic model, and according to the historical speech instructions issued by the user, obtains the phoneme sequence of each non-keyword with synonymous features, the phoneme sequence of each marked keyword, and the phoneme sequence of each non-keyword with non-synonymous features, takes the obtained phoneme sequences as input, and takes the standard phoneme sequence of each non-keyword with synonymous features, the standard phoneme sequence of each marked keyword, and the standard phoneme sequence of each non-keyword with non-synonymous features as output, to obtain the first speech model, the second speech model, and the third speech model respectively; The task instruction output unit splits the voice instructions issued by the user, and inputs the non-keywords with synonymous features, keywords, and non-keywords with non-synonymous features obtained by the splitting into the first voice model, the second voice model, and the third voice model respectively, and combines and arranges the output words and output terms of each voice model according to the composition logic of the standard control instructions to obtain a plurality of task instructions.

6. The intelligent voice control system for UAV flight based on big data according to claim 5 is characterized by: The UAV motion state control module includes a nearest associated obstacle determination unit, a flight acceleration analysis unit, a voice command feasibility prediction unit and a UAV motion state control unit; The nearest associated obstacle determination unit performs a three-dimensional simulation of the motion scene of the drone using surrounding images collected by the camera device, constructs a three-dimensional space coordinate system in the three-dimensional motion scene, calculates the distance value between the drone and each associated obstacle, and determines the nearest associated obstacle based on the calculation result; The flight acceleration analysis unit analyzes the flight acceleration of the drone according to the last voice command of the voice command issued by the user; The voice command feasibility prediction unit calculates the time required for the UAV to move from the coordinate origin to the position of the nearest associated obstacle according to the constructed mathematical formula, compares the calculation result with the emergency braking time of the UAV at the corresponding flight speed, and predicts the feasibility of each task instruction through the constructed mathematical models according to the comparison result, and determines the target control instruction of the UAV based on the prediction result; The UAV motion state control unit controls the motion state of the UAV through the target control instruction received by the UAV control terminal.

7. The intelligent voice control system for UAV flight based on big data according to claim 6 is characterized by: The warning response situation analysis module includes a UAV motion state acquisition unit, a warning response judgment index prediction unit and a change result determination unit; The UAV motion state acquisition unit acquires the real-time position coordinates and real-time flight speed of the UAV after the motion state is controlled through the UAV state receiving device; The early warning response judgment index prediction unit calculates the real-time movement speed of the nearest associated obstacle according to the real-time distance value between the UAV and the nearest associated obstacle, predicts the real-time early warning response judgment index of the UAV based on the calculation result, and selects whether to change the target control instruction received by the UAV control terminal based on the prediction result; The change result determination unit changes the target control instruction received by the drone control terminal.

8. The intelligent voice control system for UAV flight based on big data according to claim 7 is characterized by: The three-dimensional motion scene display module modifies the three-dimensional motion scene of the drone in real time through the real-time motion state of the drone after the motion state is adjusted and the surrounding images collected by the camera device installed on the drone. The drone control terminal transmits the modified three-dimensional motion scene to the user's smart terminal, and the modified three-dimensional motion scene is displayed through the display software loaded in the user's smart terminal.

Images