Motion state recognition method, device and equipment based on multilayer neural network

Abstract

The invention belongs to the technical field of neural networks, and particularly discloses a motion state recognition method, device and equipment based on a multilayer neural network. According to the application, a to-be-identified data sample is generated according to internal situation data and external environment data; performing dynamic standardization processing on the to-be-identified data sample; matrix operation is carried out on the current standard data sample based on the first-layer neural network; and based on the second-layer neural network, determining a target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and identifying the motion state of the unmanned platform according to the target hidden state. According to the mode, the first-layer neural network and the second-layer neural network are adopted for matrix operation and state updating, and the current unreasonable input is discarded at a certain probability in the second-layer neural network stage, so that the efficiency and accuracy of identifying the motion state can be effectively improved, and a property guard is provided for an unmanned platform.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of neural networks, and more particularly relates to a motion state recognition method, device and equipment based on a multi-layer neural network. BACKGROUND

[0002] For an unmanned platform, a task is usually completed in an offline manner, and is retrieved by manual operation. However, the environment in which the unmanned platform performs the task is relatively harsh. For example, compared with a land or lake surface, a sea unmanned platform is subjected to more intense wind and waves, and is easily corroded by seawater. The unmanned platform cannot perceive the harsh environment, and cannot respond to or report the environment. Therefore, how to accurately and efficiently recognize the motion state of the unmanned platform is particularly important.

[0003] At present, a commonly used manner for recognizing the motion state of an unmanned platform relies on a single-layer neural network, for example, a single-layer LSTM (Long Short-Term Memory) neural network. However, the single-layer neural network uses all input data, including unreasonable input data, when performing recognition, which leads to multiple false alarms, and the larger the amount of data, the longer the time consumption. Therefore, the above manner has low efficiency and accuracy in recognizing the motion state. SUMMARY

[0004] In view of the defects of the prior art, the purpose of the present application is to provide a motion state recognition method, device and equipment based on a multi-layer neural network, which aims to solve the problem of low efficiency and accuracy in recognizing the motion state due to the single-layer neural network not discarding the current unreasonable input data.

[0005] To achieve the above purpose, in a first aspect, the present application provides a motion state recognition method based on a multi-layer neural network, comprising: obtaining internal situation data and external environment data of an unmanned platform, and generating a to-be-recognized data sample according to the internal situation data and the external environment data; performing dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample; performing matrix operation on the current standard data sample based on a first-layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step; determining a target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step based on a second-layer neural network, and recognizing a motion state of the unmanned platform according to the target hidden state.

[0006] In an embodiment, the step of obtaining internal situation data and external environment data of the unmanned platform, and generating a to-be-recognized data sample according to the internal situation data and the external environment data, comprises: obtaining internal situation data and external environment data of the unmanned platform, and fusing the internal situation data and the external environment data to obtain a single fusion data point; setting a time label for the single fusion data point; in response to a data inflow instruction, filling the single fusion data point into a target sliding window according to the time label; wherein the length of the target sliding window is consistent with the data sampling frequency; when the target sliding window is completely filled with the single fusion data point, controlling the target sliding window to slide forward, and generating a to-be-recognized data sample according to the sliding target sliding window.

[0007] In an embodiment, the step of dynamically standardizing the to-be-recognized data sample to obtain a current standard data sample comprises: counting the data amount of the to-be-recognized data sample, and calculating a sample mean according to the data amount and the to-be-recognized data sample; calculating a sample standard deviation according to the sample mean and the to-be-recognized data sample; performing difference calculation on the sample mean and the value corresponding to the to-be-recognized data sample; performing ratio calculation on the current difference calculation result and the sample standard deviation, and dynamically standardizing the to-be-recognized data sample according to the ratio calculation result to obtain a current standard data sample.

[0008] In an embodiment, the step of performing matrix operation on the current standard data sample based on the first layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step comprises: splicing the current standard data sample and a first dimension hidden state of a previous time step to obtain a second dimension vector; performing matrix operation on the second dimension vector based on multiple gates in the first layer neural network, and fusing each first matrix operation result; determining a primary feature vector sequence of each time step according to the first matrix operation fusion result; performing data update on cells in the first layer neural network according to the first matrix operation fusion result to obtain a target cell state and a target hidden state; determining a complete hidden state of a target time step according to the target cell state and the target hidden state.

[0009] In an embodiment, the step of determining the target hidden state based on the sequence of primary feature vectors of each time step and the complete hidden state of the target time step based on the second-layer neural network comprises: The second-layer neural network is used to discard unreasonable data in the sequence of primary feature vectors, and the sequence of discarded feature vectors is spliced with the hidden state of the current time step to obtain a third-dimensional vector; The third-dimensional vector matrix is operated based on multiple gates in the second-layer neural network, and each second matrix operation result is fused; the second-layer neural network is additionally provided with a dropout layer, and part of neurons are randomly closed during training; The cells in the second-layer neural network are updated based on the second matrix operation fusion result, and the target hidden state is determined based on the updated cell state and the updated hidden state.

[0010] In an embodiment, the step of identifying the motion state of the unmanned platform based on the target hidden state further comprises: The target hidden state is combined by a first fully connected layer, and the combined target hidden state is mapped; The score of the motion state category to which the mapped target hidden state belongs is calculated by a second fully connected layer; The score of the motion state category is converted by a target output layer to obtain motion state category probability distribution data; The maximum probability is extracted from the motion state category probability distribution data, and the type of abnormal motion state of the unmanned platform is determined based on the maximum probability.

[0011] In a second aspect, the application provides a motion state recognition device based on a multi-layer neural network, comprising: An acquisition module is configured to acquire internal situation data and external environment data of an unmanned platform, and generate a to-be-recognized data sample based on the internal situation data and the external environment data; A processing module is configured to perform dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample; An operation module is configured to perform matrix operation on the current standard data sample based on a first-layer neural network to obtain a sequence of primary feature vectors of each time step and a complete hidden state of a target time step; An identification module is configured to identify the motion state of the unmanned platform based on a second-layer neural network and based on the sequence of primary feature vectors of each time step and the complete hidden state of the target time step.

[0012] In a third aspect, the present application provides an electronic device, comprising: at least one memory configured to store a program; and at least one processor configured to execute the program stored in the memory, wherein the processor is configured to execute the method described in the first aspect or any possible implementation manner of the first aspect when the program stored in the memory is executed.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is run on a processor, the processor executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0014] In a fifth aspect, the present application provides a computer program product, which, when run on a processor, causes the processor to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0015] It can be understood that the beneficial effects of the above-mentioned second aspect to fifth aspect can be referred to the related description in the first aspect, which will not be repeated here.

[0016] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects: (1) After the internal situation data and the external environment data of the unmanned platform are used to generate the to-be-identified data sample, the present application performs matrix operation based on the first layer neural network and determines the target hidden state based on the second layer neural network, that is, a multi-layer neural network is used, and in the second layer neural network stage, the current unreasonable input is discarded with a certain probability, which is intersected with a single-layer network, so that the situation of multiple false alarms can be avoided, and the data amount of the primary feature vector sequence can be reduced, thereby the efficiency and accuracy of identifying the motion state can be effectively improved.

[0017] (2) The present application additionally sets a dropout layer (dropout layer) in the second layer neural network, selects more than two layers for training when training the model, and adopts an experience value 0.2 for the dropout rate, so that 20% of the neurons are randomly "turned off", and the network is forced not to depend on a certain neuron, so that more robust features are learned, and overfitting is effectively prevented. In addition, the present application also deletes data that does not affect the situation of the unmanned platform, and further reduces the data amount on the X, Y and Z axes, which can effectively improve the efficiency of identifying the motion state.

[0018] In summary, the internal situation data and the external environment data of the unmanned platform are acquired, and the to-be-recognized data sample is generated according to the internal situation data and the external environment data; the current standard data sample is obtained by performing dynamic standardization processing on the to-be-recognized data sample; the matrix operation is performed on the current standard data sample based on the first-layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step; the target hidden state is determined according to the primary feature vector sequence of each time step and the complete hidden state of the target time step based on the second-layer neural network, and the motion state of the unmanned platform is recognized according to the target hidden state. In the foregoing manner, after the current standard data sample is determined, the matrix operation and state updating are performed by the first-layer neural network and the second-layer neural network, and the current unreasonable input is discarded with a certain probability in the second-layer neural network stage, so that the efficiency and accuracy of recognizing the motion state can be effectively improved, and property protection is provided for the unmanned platform. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is one of flowcharts of the motion state recognition method based on the multi-layer neural network provided by the embodiments of the present application.

[0020] Figure 2 is another of flowcharts of the motion state recognition method based on the multi-layer neural network provided by the embodiments of the present application.

[0021] Figure 3 is a module structure schematic diagram of the motion state recognition device based on the multi-layer neural network provided by the embodiments of the present application.

[0022] Figure 4 is a structure schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0024] The term “and / or” in this paper is a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The symbol “ / ” in this paper represents the relationship of or in the associated objects, for example, A / B represents A or B.

[0025] The terms "first" and "second" and the like in the description and claims of this patent document are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. For example, the first response message and the second response message are used for distinguishing between similar messages and not necessarily for describing a particular sequential or chronological order.

[0026] In this application embodiment, the words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in this application embodiment should not be construed as being preferred or superior over other embodiments or design solutions. In fact, the words such as "exemplary" or "for example" are used to present concepts in a concrete manner.

[0027] Based on this, the application embodiment provides a motion state recognition method based on a multi-layer neural network, referring to Figure 1 , Figure 1 is one of the flowcharts of the motion state recognition method based on the multi-layer neural network provided by the application embodiment. In this embodiment, the motion state recognition method based on the multi-layer neural network includes steps S10 to S40: Step S10, obtaining internal situation data and external environment data of an unmanned platform, and generating a to-be-recognized data sample according to the internal situation data and the external environment data.

[0028] It should be noted that the unmanned platform refers to a device capable of automatically performing tasks in a harsh environment, especially a strong wind and wave sea environment, under unattended conditions. The internal situation data includes motion posture dimension data and internal state dimension data, wherein the motion posture dimension data includes but is not limited to posture, motion speed, and rotation speed, and the internal state dimension data includes but is not limited to internal space temperature, water leakage detection data, etc. The external environment data can be collected by an external image device to determine whether an object is approaching the unmanned platform. The dimension of the to-be-recognized data sample can be 9 dimensions.

[0029] It should be understood that in the selection of input dimensions in this embodiment, through a large number of tests and training, time information must be retained, the external environment data can be output according to the image device arranged on the unmanned platform, only 1 dimension is used, the input of the internal situation data, through tests, those data that do not affect the situation of the unmanned platform are deleted, and the data amount is further reduced on the X, Y, and Z axes, which can effectively improve the efficiency of recognizing the motion state.

[0030] Step S20, performing dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample.

[0031] It can be understood that, considering the zero drift of the sensor, the data sample input to the motion state recognition model needs to be dynamically standardized, at this time the current standard data sample is subject to standard normal distribution, which can effectively adapt to the slow drift of the sensor data.

[0032] Further, step S20 comprises: counting the data quantity of the to-be-identified data sample, and calculating a sample mean value according to the data quantity and the to-be-identified data sample; calculating a sample standard deviation according to the sample mean value and the to-be-identified data sample; performing difference calculation on the sample mean value and the value corresponding to the to-be-identified data sample; performing ratio calculation on the current difference calculation result and the sample standard deviation, and performing dynamic standardization processing on the to-be-identified data sample according to the ratio calculation result to obtain a current standard data sample.

[0033] It should be understood that, in the embodiment, the dynamic standardization processing mode needs to calculate a sample mean value according to the data quantity and the to-be-identified data sample, and calculate a sample standard deviation according to the sample mean value and the to-be-identified data sample, at this time the to-be-identified data sample can be dynamically standardized according to the sample mean value and the sample standard deviation. Taking the to-be-identified data sample [10, 12, 14] as an example, the sample mean value is 12 and the sample standard deviation is 1.63, the current difference calculation result is-2, 0 and 2 respectively, and the ratio calculation result is-1.23, 0 and 1.23 respectively, at this time the current standard data sample can be [-1.23, 0, 1.23], which is subject to standard normal distribution with a mean value of 0 and a standard deviation of 1, and can be input to the motion state recognition model.

[0034] It should be noted that, each time a new group of data is read, the sample mean value and the sample standard deviation need to be dynamically updated, and the dynamic standardization processing needs to be performed again to calculate the unique standard data sample of each different window in real time, wherein the dynamic updating can be embodied by the following formula:

[0035] wherein, x represents the updated sample mean value, t represents the time when the data is read, x represents the known sample mean value before updating, σ represents the updated sample variance, used to determine the updated sample standard deviation, σ represents the known sample variance before updating, x represents the newly read data.

[0036] Step S30, performing matrix operation on the current standard data sample based on the first layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of the target time step. ​

[0037] It should be understood that for the motion state recognition model in the present embodiment, a double-layer neural network is arranged in the intermediate layer, for example, a first layer neural network and a second layer neural network, the first layer neural network can be a first layer LSTM neural network, and the second layer neural network can be a second layer LSTM neural network, both of which have the same number of neurons, and the difference is that the second layer neural network is additionally provided with a dropout layer, and the number of weights contained in each cell depends on the output number of the previous layer. Because a cell contains four gates, the single cell weight number of the multi-layer neural network is four times that of the ordinary layer.

[0038] It can be understood that after obtaining the current standard data sample meeting the requirement, the current standard data sample can be input to the first layer neural network. Because each cell of the first layer neural network has multiple gates, multiple matrix operations are actually performed to generate 32 outputs, and the hidden layer size is the same, that is, the primary feature vector sequence of each time step and the complete hidden state of the target time step, the target time step can be the last time step, and the complete hidden state carries the context information of the entire primary feature vector sequence. At this time, the calculation of the first layer neural network has been completed. The input current standard data sample is 1x10 rows, and matrix operation is performed with 32 cells. At this time, each cell needs to contain 10 weights, otherwise it cannot be calculated.

[0039] Further, step S30 comprises: splicing the current standard data sample and the first dimension hidden state of the previous time step to obtain a second dimension vector; performing matrix operation on the second dimension vector based on the multiple gates in the first layer neural network respectively, and fusing according to each first matrix operation result; determining the primary feature vector sequence of each time step according to the first matrix operation fusion result; updating the cells in the first layer neural network according to the first matrix operation fusion result to obtain a target cell state and a target hidden state; and determining the complete hidden state of the target time step according to the target cell state and the target hidden state.

[0040] It can be understood that the plurality of gates in the first layer neural network are respectively a forget gate, an input gate, a candidate gate and an output gate, wherein the forget gate is used to determine which old information is discarded from the cell state, the input gate is used to determine how much new information is worth storing, the candidate gate is used to create a new, candidate cell state value as a potential new memory content, and the output gate is used to determine which part of the current cell state will be output. The primary feature vector of each time step is calculated according to the first matrix operation fusion result, and the primary feature vectors are arranged in order to obtain a primary feature vector sequence. In addition, after processing the last time step, the cell in the first layer neural network is updated to obtain the final target cell state and target hidden state, and at this time the target time step complete hidden state is composed of the target cell state and the target hidden state, which contains the "memory summary" of the first layer neural network to the entire input sequence.

[0041] In step S40, based on the second layer neural network, the target hidden state is determined according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and the motion state of the unmanned platform is identified according to the target hidden state.

[0042] It can be understood that for the second layer neural network, the same way as the first layer neural network can be used to determine the target hidden state, only the input and output are different, that is, the input of the second layer neural network is no longer the original current standard data sample, but the primary feature vector sequence and the complete hidden state of the target time step output by the first layer neural network, that is, the second layer neural network completely inherits the full understanding of the first layer neural network to the current standard data sample.

[0043] It should be understood that for the motion state recognition model in the embodiment, in addition to including the first layer neural network and the second layer neural network, it also includes a plurality of fully connected layers, a target output layer, a loss function, an activation function and a hidden layer, etc. After determining the target hidden state based on the second layer neural network, the motion state of the unmanned platform can be identified according to the target hidden state to determine the type of abnormal motion state of the unmanned platform.

[0044] Further, the step of determining the target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step based on the second-layer neural network comprises: discarding unreasonable data in the primary feature vector sequence based on the second-layer neural network, splicing the discarded feature vector sequence with the hidden state of the current time step to obtain a third-dimensional vector; performing matrix operation on the third-dimensional vector matrix based on multiple gates in the second-layer neural network respectively, and fusing according to each second matrix operation result; wherein the second-layer neural network is additionally provided with a dropout layer, and part of neurons are randomly turned off during training; updating the cells in the second-layer neural network according to the second matrix operation fusion result, and determining the target hidden state according to the updated cell state and the updated hidden state.

[0045] It should be understood that, due to unreasonable data in the primary feature vector sequence, if not discarded in time, it will cause the need to use all data when determining the target hidden state, resulting in low efficiency of identifying the motion state, and in addition, unreasonable data will cause multiple false alarms. Therefore, the embodiment discards the current unreasonable input at a certain probability in the second-layer neural network stage, then splices it with the hidden state of the current time step, and after updating the cells in the second-layer neural network according to the second matrix operation fusion result, determines the target hidden state according to the updated cell state and the updated hidden state.

[0046] It should be noted that, compared with the first-layer neural network, the second-layer neural network is additionally provided with a dropout layer, and when training the model, more than two layers are selected for training, the dropout rate takes an empirical value of 0.2, and 20% of the neurons will be randomly turned off, forcing the network not to rely on a certain neuron, so as to learn more robust features, effectively prevent overfitting, and the loss function can select the cross-entropy function, which is more suitable for probability class classification problems. In addition, the size of the hidden layer should not exceed 64 to reduce the overhead of the model itself.

[0047] Further, the step of identifying the motion state of the unmanned platform according to the target hidden state further comprises: performing nonlinear combination on the target hidden state through a first fully connected layer, and mapping the combined target hidden state; calculating the score of the motion state category to which the mapped target hidden state belongs through a second fully connected layer; converting the score of the motion state category through a target output layer to obtain motion state category probability distribution data; extracting the maximum probability from the motion state category probability distribution data, and determining the type of abnormal motion state of the unmanned platform according to the maximum probability.

[0048] It should be understood that after the target hidden state is obtained, the target hidden state can be nonlinearly combined by a first fully connected layer using a ReLU activation function, and the combined target hidden state is mapped, in the motion state recognition model, the first fully connected layer is connected with a second layer neural network, the first fully connected layer can be provided with a plurality of neurons, a second fully connected layer is connected with the first fully connected layer, and the number of neurons thereof is the same as the number of classifications, for the second fully connected layer, no activation function is used, after calculating the score of the motion state category to which the mapped target hidden state belongs through the second fully connected layer, the score of the motion state category is converted into a motion state category probability distribution data through a target output layer, the target output layer can be a layer of a softmax function, at this time, the maximum probability is selected, and the type of the abnormal motion state of the unmanned platform is determined, for example, being attacked by other close objects, equipment failure, being intercepted and salvaged by a third party, etc. When the type is an obvious abnormal type, a response is immediately made to report a remote manned platform and destroy local resources, thereby realizing property guardianship of the unmanned platform.

[0049] It should be noted that the embodiment can be configured with a security module on the unmanned platform, the security module is deployed with a perception program, the perception program runs the motion state recognition method based on the multi-layer neural network of the embodiment, performs real-time perception and self-learning, and continuously improves and updates the motion state recognition model.

[0050] The embodiment obtains internal situation data and external environment data of the unmanned platform, and generates a to-be-recognized data sample according to the internal situation data and the external environment data; performs dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample; performs matrix operation on the current standard data sample based on a first layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step; determines a target hidden state based on a second layer neural network according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and recognizes a motion state of the unmanned platform according to the target hidden state. Through the above manner, after the current standard data sample is determined, matrix operation and state updating are performed by the first layer neural network and the second layer neural network, and the current unreasonable input is discarded with a certain probability in the second layer neural network stage, so that the efficiency and accuracy of recognizing the motion state can be effectively improved, and property guardianship of the unmanned platform is provided.

[0051] In a specific embodiment, the present application provides a step of generating a to-be-recognized data sample. Please refer to Figure 2 , Figure 2 FIG. 2 is a flowchart of a motion state recognition method based on a multi-layer neural network according to an embodiment of the present application. Step S10 includes steps S101-S104: In step S101, internal situation data and external environment data of the unmanned platform are acquired, and the internal situation data and the external environment data are fused to obtain a single fused data point.

[0052] It should be noted that the internal situation data includes motion posture dimension data and internal state dimension data, wherein the motion posture dimension data includes but is not limited to posture, motion speed, and rotation speed, etc., and can be read by an IMU (Inertial Measurement Unit), and the internal state dimension data includes but is not limited to internal space temperature, water leakage detection data, etc., and at this time, the internal situation data and the external environment data are fused into a single fused data point, and the data dimension can be 10 dimensions.

[0053] In step S102, a time label is set for the single fused data point.

[0054] In step S103, in response to a data inflow instruction, the single fused data point is filled into a target sliding window according to the time label, and a length of the target sliding window is consistent with a data sampling frequency.

[0055] It should be understood that after setting an accurate time label for the single fused data point, a target sliding window is created, and the core idea of the target sliding window is to cut continuous data points into multiple segments of fixed length and continuous time sequence, so that the model can analyze the dynamic mode of the data in time. Before filling the single fused data point into the target sliding window, a window length and a window step need to be set, wherein the window length represents the number of data points contained in a single window, and the window step represents the number of data points moved forward each time the target sliding window slides.

[0056] In step S104, when the target sliding window is completely filled with the single fused data point, the target sliding window is controlled to slide forward, and a to-be-recognized data sample is generated according to the target sliding window after sliding.

[0057] It can be understood that when the target sliding window is completely filled with the single fused data point, the target sliding window is controlled to slide forward, and then each time a group of data is read, the target sliding window is controlled to slide forward, the earliest data point corresponding to the time label is removed from the target sliding window, and the latest data point is included, and only the multiple data points in the target sliding window at the last moment are taken as output, that is, the to-be-recognized data sample.

[0058] The embodiment obtains internal situation data and external environment data of an unmanned platform, fuses the internal situation data and the external environment data, and obtains a single fusion data point; a time label is set for the single fusion data point; in response to a data inflow instruction, the single fusion data point is filled into a target sliding window according to the time label; the length of the target sliding window is consistent with a data sampling frequency; when the target sliding window is completely filled with the single fusion data point, the target sliding window is controlled to slide forward, and a to-be-recognized data sample is generated according to the target sliding window after sliding. In the foregoing manner, after an accurate time label is set for the fused single fusion data point, the single fusion data point is filled into the target sliding window according to the time label, then continuous data points are cut into multiple segments of fixed length and continuous time sequences according to the target sliding window, so that the model can analyze the dynamic mode of the data in time, and a target sliding window is generated according to the data points in the target sliding window after sliding, thereby effectively improving the accuracy of generating a to-be-recognized data sample.

[0059] The motion state recognition device based on the multi-layer neural network provided in the application is described below, and the motion state recognition device based on the multi-layer neural network described below can be mutually corresponding with the motion state recognition method based on the multi-layer neural network described above. Please refer to Figure 3 , Figure 3 is a module structure schematic diagram of the motion state recognition device based on the multi-layer neural network provided in the embodiment of the application, comprising: The acquisition module T10 is configured to acquire internal situation data and external environment data of an unmanned platform, and generate a to-be-recognized data sample according to the internal situation data and the external environment data.

[0060] The processing module T20 is configured to perform dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample.

[0061] The operation module T30 is configured to perform matrix operation on the current standard data sample based on a first layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step.

[0062] The recognition module T40 is configured to recognize a motion state of the unmanned platform based on a second layer neural network according to the primary feature vector sequence of each time step and the complete hidden state of the target time step.

[0063] The embodiment obtains internal situation data and external environment data of the unmanned platform, and generates a to-be-recognized data sample according to the internal situation data and the external environment data; performs dynamic standardization processing on the to-be-recognized data sample to obtain a current standard data sample; performs matrix operation on the current standard data sample based on a first-layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step; determines a target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step based on a second-layer neural network, and identifies a motion state of the unmanned platform according to the target hidden state. In the foregoing manner, after the current standard data sample is determined, matrix operation and state updating are performed by the first-layer neural network and the second-layer neural network, and unreasonable input at the present moment is discarded with a certain probability in the second-layer neural network stage, so that the efficiency and accuracy of identifying the motion state can be effectively improved, and property protection is provided for the unmanned platform.

[0064] It can be understood that the detailed function implementation of each module can be referred to the foregoing description in the method embodiment, and will not be described herein.

[0065] It should be understood that the above device is used to execute the method in the above embodiment, and the corresponding program module in the device has similar implementation principles and technical effects to the description in the above method, and the working process of the device can be referred to the corresponding process in the above method, which will not be described herein.

[0066] Based on the method in the above embodiment, an electronic device is provided in the embodiment of the present application, please refer to Figure 4 , Figure 4 is a structural schematic diagram of the electronic device provided in the embodiment of the present application.

[0067] It should be noted that the electronic device can include a processor (Processor) 10, a communication interface (Communications Interface) 20, a memory (Memory) 30 and a communication bus 40, wherein the processor 10, the communication interface 20 and the memory 30 complete mutual communication through the communication bus 40. The processor 10 can call the logic instructions in the memory 30 to execute the method in the above embodiment.

[0068] Further, the logic instructions in the memory 30 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application.

[0069] Based on the method in the above embodiments, the embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiments.

[0070] Based on the method in the above embodiments, the embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiments.

[0071] It can be understood that the processor in the embodiments of the present application can be a central processing unit, and can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, or any conventional processor.

[0072] The method steps in the embodiments of the present application can be realized by hardware or by the processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor.

[0073] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for convenient differentiation, and do not limit the scope of the embodiments of the present application. Those skilled in the art will readily understand that the above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of motion state recognition based on a multi-layer neural network, characterized by, The method comprises the following steps: obtaining internal situation data and external environment data of the unmanned platform, and generating a to-be-identified data sample according to the internal situation data and the external environment data; performing dynamic standardization processing on the to-be-identified data sample to obtain a current standard data sample; performing matrix operation on the current standard data sample based on a first-layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step; determining a target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step based on a second-layer neural network, and identifying a motion state of the unmanned platform according to the target hidden state.

2. The method of claim 1, wherein, The step of obtaining internal situation data and external environment data of the unmanned platform, and generating a to-be-identified data sample according to the internal situation data and the external environment data comprises the following steps: obtaining internal situation data and external environment data of the unmanned platform, and fusing the internal situation data and the external environment data to obtain a single fused data point; setting a time label for the single fused data point; in response to a data flow instruction, filling the single fused data point into a target sliding window according to the time label; wherein the length of the target sliding window is consistent with the data sampling frequency; when the target sliding window is completely filled with the single fused data point, controlling the target sliding window to slide forward, and generating a to-be-identified data sample according to the target sliding window after sliding.

3. The method of claim 1, wherein, The step of performing dynamic standardization processing on the to-be-identified data sample to obtain a current standard data sample comprises the following steps: statistically analyzing the data amount of the to-be-identified data sample, and calculating a sample mean according to the data amount and the to-be-identified data sample; calculating a sample standard deviation according to the sample mean and the to-be-identified data sample; performing difference calculation on the sample mean and the value corresponding to the to-be-identified data sample; performing ratio calculation on the current difference calculation result and the sample standard deviation, and performing dynamic standardization processing on the to-be-identified data sample according to the ratio calculation result to obtain a current standard data sample.

4. The method of claim 1, wherein, The step of performing matrix operation on the current standard data sample based on a first-layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step comprises the following steps: splicing the current standard data sample and a first-dimensional hidden state of a previous time step to obtain a second-dimensional vector; performing matrix operation on the second-dimensional vector based on multiple gates in the first-layer neural network, and fusing each first matrix operation result; determining a primary feature vector sequence of each time step according to the first matrix operation fusion result; performing data update on cells in the first-layer neural network according to the first matrix operation fusion result to obtain a target cell state and a target hidden state; determining a complete hidden state of a target time step according to the target cell state and the target hidden state.

5. The method of claim 1, wherein, The step of determining the target hidden state based on the second-layer neural network according to the primary feature vector sequence of each time step and the complete hidden state of the target time step comprises: Discarding unreasonable data in the primary feature vector sequence based on the second-layer neural network, and splicing the discarded feature vector sequence and the hidden state of the current time step to obtain a third-dimensional vector; Respectively performing matrix operations on the third-dimensional vector matrix based on multiple gates in the second-layer neural network, and fusing each second matrix operation result; wherein the second-layer neural network is additionally provided with a dropout layer, and part of neurons are randomly closed during training; Updating the cells in the second-layer neural network based on the second matrix operation fusion result, and determining the target hidden state according to the updated cell state and the updated hidden state.

6. The method of claim 1, wherein, The step of identifying the motion state of the unmanned platform according to the target hidden state further comprises: Performing nonlinear combination on the target hidden state through a first fully connected layer, and mapping the combined target hidden state; Calculating the score of the motion state category to which the mapped target hidden state belongs through a second fully connected layer; Converting the score of the motion state category through a target output layer to obtain motion state category probability distribution data; Extracting the maximum probability from the motion state category probability distribution data, and determining the type of the abnormal motion state of the unmanned platform according to the maximum probability.

7. A motion state recognition apparatus based on a multi-layer neural network, characterized by comprising: Comprise: An acquisition module configured to acquire internal situation data and external environment data of an unmanned platform, and generate a to-be-identified data sample based on the internal situation data and the external environment data; A processing module configured to perform dynamic standardization processing on the to-be-identified data sample to obtain a current standard data sample; An operation module configured to perform matrix operations on the current standard data sample based on a first-layer neural network to obtain a primary feature vector sequence of each time step and a complete hidden state of a target time step; An identification module configured to identify a motion state of the unmanned platform based on a second-layer neural network according to the primary feature vector sequence of each time step and the complete hidden state of the target time step.

8. An electronic device, comprising: Comprise: At least one memory configured to store a computer program; At least one processor configured to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program runs on the processor, the processor is caused to execute the method according to any one of claims 1-6.

10. A computer program product, characterised in that, When the computer program product runs on the processor, the processor is caused to execute the method according to any one of claims 1-6.

Images