Action recognition response system based on deep learning

Abstract

The invention discloses an action recognition response system based on deep learning, and relates to the field of unmanned aerial vehicles, the action recognition response system comprises an equipment control center, and the equipment control center is in communication connection with a data acquisition module, a data processing module, a data analysis module, an interference analysis module and an action response module; the data acquisition module is used for acquiring environment monitoring data and flight data; the data processing module is used for analyzing the obtained environment monitoring data to obtain a risk assessment level; elastic analysis nodes are preset in the data analysis module, and the operation state of each elastic analysis node is set according to the risk assessment level corresponding to the environment monitoring data; the interference analysis module is used for acquiring an interference coefficient corresponding to the environment monitoring data; the action response module is used for analyzing and processing the flight data according to the obtained interference coefficient to generate corresponding response information; according to the invention, the accuracy in the action response process of the unmanned aerial vehicle equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles, and in particular to an action recognition and response system based on deep learning. Background Art

[0002] With the continuous development of science and technology, the technology of unmanned aerial vehicles is also constantly improving, and is widely used in various fields. Due to the diversity and convertibility of monitoring sensors in drone equipment, it is widely used in agriculture, express logistics, resource exploration, environmental monitoring and facility maintenance, etc., thus bringing convenience to people's lives and work. With the continuous development of technology, the application field of drones will continue to expand;

[0003] However, with the continuous development of drone technology, the sensitivity of drone equipment in the process of motion recognition and response is greatly affected by external factors. Therefore, how to reduce the impact of external factors on the motion recognition of drone equipment during flight is a problem we need to solve. To this end, we now provide a motion recognition and response system based on deep learning. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an action recognition response system based on deep learning.

[0005] The object of the present invention can be achieved by the following technical solutions: an action recognition and response system based on deep learning includes a device control center, wherein the device control center is communicatively connected with a data acquisition module, a data processing module, a data analysis module, an interference analysis module and an action response module;

[0006] The data acquisition module is used for the UAV equipment to collect environmental monitoring data and flight data during its flight, and send the collected data information to the data processing module;

[0007] The data processing module is used to obtain environmental monitoring data, analyze and process the obtained environmental monitoring data, and obtain the risk assessment level corresponding to the environmental monitoring data;

[0008] The data analysis module is pre-set with elastic analysis nodes, and the operation status of each elastic analysis node is set according to the risk assessment level corresponding to the environmental monitoring data; the corresponding elastic analysis node pre-processes the flight data according to its operation status;

[0009] The interference analysis module is used to construct an environmental interference model based on the historical environmental monitoring data of the device control center, input the obtained environmental monitoring data into the environmental interference model, output the interference coefficient, and send it to the action response module;

[0010] The action response module is used to analyze and process the flight data according to the obtained interference coefficient, set a corresponding equipment analysis process according to the flight data, generate corresponding response information, and the drone equipment executes the corresponding response information.

[0011] Furthermore, the process of the UAV equipment in the data collection module collecting environmental monitoring data and flight data during its flight includes:

[0012] The data acquisition module is provided with an environment acquisition unit and a flight acquisition unit;

[0013] The environment acquisition unit is used to collect environmental monitoring data during the flight of the UAV equipment, wherein corresponding environment acquisition sensors are provided, including air sensors, wind sensors and weather sensors; the air sensor is used to obtain air data during the flight of the UAV equipment; the wind sensor is used to collect wind data during the flight of the UAV; the weather sensor is used to collect weather data during the flight of the UAV; the data information collected by the environment acquisition sensor is recorded as environmental monitoring data, and sent to the data processing module;

[0014] The flight acquisition unit is used for collecting corresponding flight data of the UAV equipment during the flight, and includes corresponding data acquisition terminals, which include a gyroscope, an accelerometer and a radar monitoring terminal respectively; the gyroscope is used to collect angular velocity data of the UAV equipment during the flight, and the accelerometer is used to collect speed data of the UAV equipment during the flight; the radar monitoring terminal is used to collect surrounding object data of the UAV equipment during the flight, and the data information collected by the data acquisition terminal is recorded as flight data and sent to the action response module.

[0015] Furthermore, the data processing module analyzes and processes the obtained environmental monitoring data to obtain the risk assessment level corresponding to the environmental monitoring data, including:

[0016] The data processing module obtains the corresponding air data, wind data and weather data in the environmental monitoring data, presets the standard evaluation indicators corresponding to the air data, wind data and weather data according to the different types of data information, compares and analyzes the obtained environmental monitoring data with the corresponding standard evaluation indicators, obtains the comparison difference between the corresponding environmental monitoring data and the standard evaluation indicators, presets the weight coefficient corresponding to the corresponding environmental monitoring data, integrates the comparison difference and the weight coefficient corresponding to the obtained environmental monitoring data, and obtains the risk assessment data corresponding to the obtained environmental monitoring data;

[0017] The risk assessment level intervals are preset, including low risk, medium risk and high risk respectively. The obtained risk assessment data are compared and analyzed with the set risk assessment level intervals to determine the risk assessment level interval to which the environmental monitoring data belongs, and the risk assessment level is obtained according to the risk assessment level interval to which it belongs.

[0018] Furthermore, the data analysis module is pre-set with elastic analysis nodes, and the process of setting the operation status of each elastic analysis node according to the risk assessment level corresponding to the environmental monitoring data includes:

[0019] The data analysis module is provided with a state analysis unit;

[0020] The status analysis unit is used to analyze and process the operating status of the elastic analysis node set in the data analysis module, and obtain the risk assessment level obtained by the data processing module; a temporary data storage pool is set in the data analysis module, and the temporary data storage pool is used to temporarily store the obtained flight data and is associated with multiple elastic analysis nodes. The elastic analysis node is used to pre-process the flight data in the temporary data storage pool, and its operating status is adjusted according to the risk assessment level. The operating number of the elastic analysis node is preset according to the risk assessment level, and the operating monitoring cycle is preset to obtain the highest risk assessment level in the operating monitoring cycle. The operating number of the corresponding elastic analysis node is obtained according to the highest risk assessment level, and the elastic analysis nodes of the corresponding operating number are placed in an operating state.

[0021] Furthermore, the process of the data analysis module preprocessing the flight data by the corresponding elastic analysis node according to its running state includes:

[0022] The data analysis module is provided with a data analysis unit, which is interconnected with the elastic analysis node in the running state to obtain the analysis result obtained by the elastic analysis node for preprocessing the flight data. The elastic analysis node obtains the angular velocity data, speed data and surrounding object data in the wind data, and stores the data optimization model obtained by training based on the machine learning algorithm. The obtained flight data are respectively input into the data optimization model, and the optimized flight data is output and sent to the action response module.

[0023] Furthermore, the interference analysis module constructs an environmental interference model according to the historical environmental monitoring data of the device control center, inputs the obtained environmental monitoring data into the environmental interference model, and outputs the interference coefficient, which includes:

[0024] The interference analysis module obtains historical environmental monitoring data and its corresponding interference coefficient; sets the obtained historical environmental monitoring data and its corresponding interference coefficient as a historical data set, divides the obtained historical data set into a training set and a verification set, analyzes and trains the obtained training set based on a convolutional neural network, constructs an environmental interference model of the corresponding type, inputs the verification set into the environmental interference model, obtains its verification result, analyzes and processes the verification result obtained from the verification set, determines whether the trained environmental interference model meets the preset standard, and outputs the environmental interference model that has completed the analysis and training until it meets the preset standard;

[0025] The obtained environmental monitoring data is input into the environmental interference model of the corresponding type that has completed the analysis and training. The environmental interference model analyzes and processes it, obtains the interference coefficient corresponding to the data type of the environmental monitoring data, and sends the obtained interference coefficient to the action response module.

[0026] Furthermore, the action response module is used to analyze and process the flight data according to the obtained interference coefficient, set a corresponding equipment analysis process according to the flight data, generate corresponding response information, and the process of executing the corresponding response information by the drone equipment includes:

[0027] The action response module is provided with a data correction unit and a response analysis unit;

[0028] The data correction unit is used to correct the obtained flight data according to the interference coefficients corresponding to different types of environmental monitoring data at the current moment, generate corresponding types of linear models according to the interference coefficients corresponding to each type of environmental monitoring data, and the linear models respectively describe the influence of corresponding types of environmental monitoring data on the flight data of the UAV equipment during flight, generate corresponding polynomial models from each linear model, and superimpose and correct the polynomial model with the corresponding flight data through the correction algorithm to obtain the original flight data corresponding to the UAV equipment, and send it to the response analysis unit;

[0029] The response analysis unit is used to obtain original flight data, wherein a data response model is constructed based on a deep learning algorithm, the obtained original flight data is input into the data response model, corresponding response information is output, and the corresponding response information is executed by the drone device.

[0030] Compared with the prior art, the beneficial effects of the present invention are: by acquiring environmental monitoring data of the UAV equipment during flight, the flight data is corrected according to the interference in its flight process based on the obtained environmental monitoring data, thereby obtaining the original flight data, and generating corresponding response information based on the original flight data, thereby reducing the interference of environmental data on the action recognition response process to a certain extent, thereby improving the accuracy of the response process. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the deep learning-based action recognition response system of an embodiment of the present application. DETAILED DESCRIPTION

[0032] like Figure 1 As shown, the action recognition and response system based on deep learning includes a device control center, and the device control center is communicatively connected with a data acquisition module, a data processing module, a data analysis module, an interference analysis module and an action response module;

[0033] The data acquisition module is used to collect environmental monitoring data, flight mission data and flight data during the flight of the UAV, and sends the collected data information to the data processing module. The specific implementation process includes:

[0034] The data acquisition module is provided with an environment acquisition unit and a flight acquisition unit;

[0035] The environment acquisition unit is used to collect environmental monitoring data during the flight of the UAV equipment, wherein corresponding environment acquisition sensors are provided, including air sensors, wind sensors and weather sensors respectively. The air sensor is used to obtain air data during the flight of the UAV equipment, and the air data includes its temperature, humidity and atmospheric pressure data; the wind sensor is used to collect wind data during the flight of the UAV, and the wind data includes wind speed data and wind direction data; the weather sensor is used to collect weather data during the flight of the UAV, and the weather data includes corresponding snowfall data or rainfall data; the environment acquisition unit records the collected air data, wind data and weather data as environmental monitoring data, and sends the collected environmental monitoring data to the data analysis module;

[0036] The flight acquisition unit is used for collecting corresponding flight data of the UAV equipment during flight, including corresponding data acquisition sensors, and the data acquisition sensors include cameras, gyroscopes, accelerometers and radar monitoring sensors. The camera is used to collect surrounding image data, the gyroscope is used to collect angular velocity data of the UAV equipment during flight, and the accelerometer is used to collect acceleration data and speed data of the UAV equipment during flight; the radar monitoring sensor is used to collect surrounding object data during the flight of the UAV equipment, and the surrounding object data includes other data such as corresponding target object data and obstacle object data; the flight acquisition unit records the data information collected in the unit as flight data and sends it to the action response module.

[0037] The data processing module analyzes and processes the obtained environmental monitoring data to obtain the risk assessment level corresponding to the environmental monitoring data, including:

[0038] The data processing module obtains the corresponding air data, wind data and weather data in the environmental monitoring data, presets the standard evaluation indicators corresponding to the air data, wind data and weather data according to the different types of data information, compares and analyzes the obtained environmental monitoring data with the corresponding standard evaluation indicators, obtains the comparison difference between the corresponding environmental monitoring data and the standard evaluation indicators, presets the weight coefficient corresponding to the corresponding environmental monitoring data, integrates the comparison difference and the weight coefficient corresponding to the obtained environmental monitoring data, and obtains the risk assessment data corresponding to the obtained environmental monitoring data;

[0039] It should be further explained that, in the specific implementation process, the data processing module integrates the contrast difference and weight coefficient corresponding to the obtained environmental monitoring data in the following manner:

[0040] ;

[0041] Among them, KC, FC and TC are the contrast differences corresponding to air data, wind data and weather data in environmental monitoring data respectively; , and are the weight coefficients corresponding to the air data, wind data and weather data in the environmental monitoring data; FP is the risk assessment data corresponding to the environmental monitoring data.

[0042] The risk assessment level intervals are preset, including low risk, medium risk and high risk respectively. The obtained risk assessment data are compared and analyzed with the set risk assessment level intervals to determine the risk assessment level interval to which the environmental monitoring data belongs, and the risk assessment level is obtained according to the risk assessment level interval to which it belongs.

[0043] The data analysis module is pre-set with elastic analysis nodes, and the process of setting the operation status of each elastic analysis node according to the risk assessment level corresponding to the environmental monitoring data includes:

[0044] The data analysis module is provided with a state analysis unit and a data analysis unit;

[0045] The state analysis unit is used to analyze and process the operating state of the elastic analysis node set in the data analysis module, and obtain the risk assessment level obtained by the data processing module; a temporary data storage pool is set in the data analysis module, and the temporary data storage pool is used to temporarily store the obtained flight data and is associated with multiple elastic analysis nodes. The elastic analysis node is used to pre-process the flight data in the temporary data storage pool, and its operating state is adjusted according to the risk assessment level. The operating number of the elastic analysis node is preset according to the risk assessment level, and the operating monitoring cycle is preset to obtain the highest risk assessment level in the operating monitoring cycle. The operating number of the corresponding elastic analysis node is obtained according to the highest risk assessment level, and the elastic analysis nodes of the corresponding operating number are placed in the operating state;

[0046] The data analysis unit is interconnected with the elastic analysis node in the running state to obtain the analysis results obtained by the elastic analysis node for preprocessing the flight data. The elastic analysis node obtains the angular velocity data, speed data and surrounding object data in the flight data, which stores the data optimization model obtained by training based on the machine learning algorithm. The obtained flight data are respectively input into the data optimization model, and the optimized flight data is output and sent to the action response module.

[0047] The interference analysis module constructs an environmental interference model according to the historical environmental monitoring data of the device control center, inputs the obtained environmental monitoring data into the environmental interference model, and outputs the interference coefficient, including:

[0048] The interference analysis module obtains historical environmental monitoring data and its corresponding interference coefficient; sets the obtained historical environmental monitoring data and its corresponding interference coefficient as a historical data set, divides the obtained historical data set into a training set and a verification set, analyzes and trains the obtained training set based on a convolutional neural network, constructs a corresponding environmental interference model, inputs the verification set into the environmental interference model, obtains its verification result, analyzes and processes the verification result obtained from the verification set, determines whether the trained environmental interference model meets the preset standard, and outputs the environmental interference model that has completed the analysis and training until it meets the preset standard;

[0049] The obtained environmental monitoring data is input into the environmental interference model that has completed analysis and training, and the environmental interference model analyzes and processes it to obtain the interference coefficient corresponding to the environmental monitoring data, and the obtained interference coefficient is sent to the action response module.

[0050] It should be further explained that, in the specific implementation process, the interference coefficient corresponding to the historical environmental monitoring data set in the interference analysis module is obtained according to the operating instructions of the corresponding model of UAV equipment in the laboratory and after experiments.

[0051] The action response module is used to analyze and process the flight data according to the obtained interference coefficient, set a corresponding equipment analysis process according to the flight data, generate corresponding response information, and the process of executing the corresponding response information by the drone equipment includes:

[0052] The action response module is provided with a data correction unit and a response analysis unit;

[0053] The data correction unit is used to correct the obtained flight data according to the interference coefficients corresponding to different types of environmental monitoring data at the current moment, generate corresponding types of linear models according to the interference coefficients corresponding to each type of environmental monitoring data, and the linear models respectively describe the influence of corresponding types of environmental monitoring data on the flight data of the UAV equipment during flight, generate corresponding polynomial models from each linear model, and superimpose and correct the polynomial model with the corresponding flight data through the correction algorithm to obtain the original flight data corresponding to the UAV equipment, and send it to the response analysis unit;

[0054] The response analysis unit is used to obtain original flight data, wherein a data response model is constructed based on a deep learning algorithm, the obtained original flight data is input into the data response model, corresponding response information is output, and the corresponding response information is executed by the drone device.

[0055] It should be further explained that, in the specific implementation process, by obtaining environmental monitoring data of the UAV equipment during its flight, the flight data is corrected according to the interference during its flight process based on the obtained environmental monitoring data, thereby obtaining the original flight data, and generating corresponding response information based on the original flight data, thereby reducing the interference of environmental data on the action recognition response process to a certain extent, thereby improving the accuracy of the response process.

[0056] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A motion recognition response system based on deep learning, including a device control center, characterized in that: The device control center is communicatively connected to a data acquisition module, a data processing module, a data analysis module, an interference analysis module and an action response module; The data acquisition module is used for the UAV equipment to collect environmental monitoring data and flight data during its flight, and send the collected data information to the data processing module; The data processing module is used to obtain environmental monitoring data, analyze and process the obtained environmental monitoring data, and obtain the risk assessment level corresponding to the environmental monitoring data; The data analysis module is pre-set with elastic analysis nodes, and the operation status of each elastic analysis node is set according to the risk assessment level corresponding to the environmental monitoring data; The corresponding elastic analysis node pre-processes the flight data according to its operating status; The interference analysis module is used to construct an environmental interference model based on the historical environmental monitoring data of the device control center, input the obtained environmental monitoring data into the environmental interference model, output the interference coefficient, and send it to the action response module; The action response module is used to analyze and process the flight data according to the obtained interference coefficient, set a corresponding equipment analysis process according to the flight data, generate corresponding response information, and the drone equipment executes the corresponding response information.

2. The deep learning-based action recognition response system according to claim 1, characterized in that: The process of the UAV equipment collecting environmental monitoring data and flight data during its flight in the data collection module includes: The data acquisition module is provided with an environment acquisition unit and a flight acquisition unit; The environment acquisition unit is used to collect environmental monitoring data during the flight of the UAV equipment, wherein corresponding environment acquisition sensors are provided, including air sensors, wind sensors and weather sensors; the air sensor is used to obtain air data during the flight of the UAV equipment; the wind sensor is used to collect wind data during the flight of the UAV; the weather sensor is used to collect weather data during the flight of the UAV; the data information collected by the environment acquisition sensor is recorded as environmental monitoring data, and sent to the data processing module; The flight acquisition unit is used for collecting corresponding flight data of the UAV equipment during the flight, and includes corresponding data acquisition terminals, which include a gyroscope, an accelerometer and a radar monitoring terminal respectively; the gyroscope is used to collect angular velocity data of the UAV equipment during the flight, and the accelerometer is used to collect speed data of the UAV equipment during the flight; the radar monitoring terminal is used to collect surrounding object data of the UAV equipment during the flight, and the data information collected by the data acquisition terminal is recorded as flight data and sent to the action response module.

3. The deep learning-based action recognition response system according to claim 2, characterized in that: The data processing module analyzes and processes the obtained environmental monitoring data to obtain the risk assessment level corresponding to the environmental monitoring data, including: The data processing module obtains the corresponding air data, wind data and weather data in the environmental monitoring data, presets the standard evaluation indicators corresponding to the air data, wind data and weather data according to the different types of data information, compares and analyzes the obtained environmental monitoring data with the corresponding standard evaluation indicators, obtains the comparison difference between the corresponding environmental monitoring data and the standard evaluation indicators, presets the weight coefficient corresponding to the corresponding environmental monitoring data, integrates the comparison difference and the weight coefficient corresponding to the obtained environmental monitoring data, and obtains the risk assessment data corresponding to the obtained environmental monitoring data; The risk assessment level intervals are preset, including low risk, medium risk and high risk respectively. The obtained risk assessment data are compared and analyzed with the set risk assessment level intervals to determine the risk assessment level interval to which the environmental monitoring data belongs, and the risk assessment level is obtained according to the risk assessment level interval to which it belongs.

4. The deep learning-based action recognition response system according to claim 3, characterized in that: The data analysis module is pre-set with elastic analysis nodes, and the process of setting the operating status of each elastic analysis node according to the risk assessment level corresponding to the environmental monitoring data includes: The data analysis module is provided with a state analysis unit; The status analysis unit is used to analyze and process the operating status of the elastic analysis node set in the data analysis module, and obtain the risk assessment level obtained by the data processing module; a temporary data storage pool is set in the data analysis module, and the temporary data storage pool is used to temporarily store the obtained flight data and is associated with multiple elastic analysis nodes. The elastic analysis node is used to pre-process the flight data in the temporary data storage pool, and its operating status is adjusted according to the risk assessment level. The operating number of the elastic analysis node is preset according to the risk assessment level, and the operating monitoring cycle is preset to obtain the highest risk assessment level in the operating monitoring cycle. The operating number of the corresponding elastic analysis node is obtained according to the highest risk assessment level, and the elastic analysis nodes of the corresponding operating number are placed in an operating state.

5. The deep learning-based action recognition response system according to claim 4, characterized in that: The process of preprocessing the flight data by the corresponding elastic analysis node according to the running state of the data analysis module includes: The data analysis module is provided with a data analysis unit, which is interconnected with the elastic analysis node in the running state to obtain the analysis result obtained by the elastic analysis node for preprocessing the flight data. The elastic analysis node obtains the angular velocity data, speed data and surrounding object data in the wind data, and stores the data optimization model obtained by training based on the machine learning algorithm. The obtained flight data are respectively input into the data optimization model, and the optimized flight data is output and sent to the action response module.

6. The deep learning-based action recognition response system according to claim 5, characterized in that: The interference analysis module constructs an environmental interference model according to the historical environmental monitoring data of the device control center, inputs the obtained environmental monitoring data into the environmental interference model, and outputs the interference coefficient, including: The interference analysis module obtains historical environmental monitoring data and its corresponding interference coefficient; sets the obtained historical environmental monitoring data and its corresponding interference coefficient as a historical data set, divides the obtained historical data set into a training set and a verification set, analyzes and trains the obtained training set based on a convolutional neural network, constructs an environmental interference model of the corresponding type, inputs the verification set into the environmental interference model, obtains its verification result, analyzes and processes the verification result obtained from the verification set, determines whether the trained environmental interference model meets the preset standard, and outputs the environmental interference model that has completed the analysis and training until it meets the preset standard; The obtained environmental monitoring data is input into the environmental interference model of the corresponding type that has completed the analysis and training. The environmental interference model analyzes and processes it, obtains the interference coefficient corresponding to the data type of the environmental monitoring data, and sends the obtained interference coefficient to the action response module.

7. The deep learning-based action recognition response system according to claim 6, characterized in that: The action response module is used to analyze and process the flight data according to the obtained interference coefficient, set a corresponding equipment analysis process according to the flight data, generate corresponding response information, and the process of executing the corresponding response information by the drone equipment includes: The action response module is provided with a data correction unit and a response analysis unit; The data correction unit is used to correct the obtained flight data according to the interference coefficients corresponding to different types of environmental monitoring data at the current moment, generate corresponding types of linear models according to the interference coefficients corresponding to each type of environmental monitoring data, and the linear models respectively describe the influence of corresponding types of environmental monitoring data on the flight data of the UAV equipment during flight, generate corresponding polynomial models from each linear model, and superimpose and correct the polynomial model with the corresponding flight data through the correction algorithm to obtain the original flight data corresponding to the UAV equipment, and send it to the response analysis unit; The response analysis unit is used to obtain original flight data, wherein a data response model is constructed based on a deep learning algorithm, the obtained original flight data is input into the data response model, corresponding response information is output, and the corresponding response information is executed by the drone device.

Images