Motion state recognition method, device and equipment based on multi-layer neural network

Abstract

The application belongs to the technical field of neural networks, and specifically discloses a motion state recognition method, device and equipment based on a multi-layer neural network. According to the application, internal state data and external environment data are used to generate a to-be-recognized data sample; the to-be-recognized data sample is dynamically standardized; a first-layer neural network is used to perform matrix operation on the current standard data sample; based on a second-layer neural network, a target hidden state is determined according to a primary feature vector sequence of each time step and a complete hidden state of a target time step, and the motion state of an unmanned platform is recognized according to the target hidden state. In the above manner, the first-layer neural network and the second-layer neural network are used to perform matrix operation and state updating, and the current unreasonable input is discarded with a certain probability in the second-layer neural network stage, so that the efficiency and accuracy of recognizing the motion state can be effectively improved, and the unmanned platform can be provided with property protection.

Description

Technical Field

[0001] This application belongs to the field of neural network technology, and more specifically, relates to a method, apparatus and device for motion state recognition based on multilayer neural networks. Background Technology

[0002] For unmanned platforms, tasks are typically completed offline and retrieved manually. However, the environments in which unmanned platforms operate are often harsh. For example, offshore unmanned platforms face much stronger waves and are more susceptible to corrosion from seawater compared to land-based lakes. Since unmanned platforms cannot perceive these harsh environments, nor can they react to or report them, accurately and efficiently identifying the movement status of unmanned platforms is crucial.

[0003] Currently, common methods for identifying the motion state of unmanned platforms rely on single-layer neural networks, such as single-layer LSTM (Long Short-Term Memory) neural networks. However, single-layer neural networks use all input data during identification, including unreasonable input data, leading to multiple false alarms. Furthermore, the larger the data volume, the longer the processing time. Therefore, the efficiency and accuracy of the above methods for identifying motion state are relatively low. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide a motion state recognition method, apparatus, and device based on a multi-layer neural network, aiming to solve the problem that the efficiency and accuracy of motion state recognition are low in existing technologies because single-layer neural networks do not discard unreasonable input data.

[0005] To achieve the above objectives, in a first aspect, this application provides a motion state recognition method based on a multi-layer neural network, comprising:

[0006] Acquire internal situational data and external environment data of the unmanned platform, and generate data samples to be identified based on the internal situational data and the external environment data;

[0007] The data sample to be identified is dynamically standardized to obtain the current standard data sample;

[0008] Matrix operations are performed on the current standard data samples based on the first layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step;

[0009] Based on the second layer neural network, the target hidden state is determined according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and the motion state of the unmanned platform is identified according to the target hidden state.

[0010] In one embodiment, the step of acquiring internal situational data and external environment data of the unmanned platform, and generating a data sample to be identified based on the internal situational data and the external environment data, includes:

[0011] Acquire internal situational data and external environment data of the unmanned platform, and fuse the internal situational data and the external environment data to obtain a single fused data point;

[0012] Set a time stamp for the single fused data point;

[0013] In response to a data inflow instruction, the single fused data point is filled into a target sliding window according to the time tag; wherein the length of the target sliding window is consistent with the data sampling frequency;

[0014] When the target sliding window is completely filled by the single fused data point, the target sliding window is controlled to slide forward, and a data sample to be identified is generated based on the target sliding window after sliding.

[0015] In one embodiment, the step of dynamically standardizing the data sample to be identified to obtain the current standard data sample includes:

[0016] The data volume of the data sample to be identified is counted, and the sample mean is calculated based on the data volume and the data sample to be identified;

[0017] Calculate the sample standard deviation based on the sample mean and the data sample to be identified;

[0018] Calculate the difference between the sample mean and the corresponding value of the data sample to be identified;

[0019] The ratio of the current difference calculation result to the sample standard deviation is calculated, and the data sample to be identified is dynamically standardized based on the ratio calculation result to obtain the current standard data sample.

[0020] In one embodiment, the step of performing matrix operations on the current standard data samples based on a first-layer neural network to obtain the primary feature vector sequence at each time step and the complete hidden state at the target time step includes:

[0021] The current standard data sample is concatenated with the first-dimensional hidden state of the previous time step to obtain the second-dimensional vector.

[0022] The second dimension vector is subjected to matrix operations by multiple gates in the first layer of the neural network, and the results of each first matrix operation are fused together.

[0023] The primary feature vector sequence for each time step is determined based on the fusion result of the first matrix operation;

[0024] Based on the fusion result of the first matrix operation, the data of the cells in the first layer of the neural network is updated to obtain the target cell state and the target hidden state.

[0025] The complete hidden state of the target time step is determined based on the target cell state and the target hidden state.

[0026] In one embodiment, the step of determining the target hidden state based on the primary feature vector sequence of each time step and the complete hidden state of the target time step using a second-layer neural network includes:

[0027] The second-layer neural network discards unreasonable data in the primary feature vector sequence, and concatenates the discarded feature vector sequence with the hidden state at the current time step to obtain the third-dimensional vector.

[0028] The third-dimensional vector matrix is ​​operated on by multiple gates in the second-layer neural network, and the results of the second matrix operations are fused together; wherein, the second-layer neural network is additionally equipped with a dropout layer, which randomly shuts down some neurons during training;

[0029] The data of the cells in the second layer of the neural network is updated based on the fusion result of the second matrix operation, and the target hidden state is determined based on the updated cell state and the updated hidden state.

[0030] In one embodiment, the step of identifying the motion state of the unmanned platform based on the target concealment state further includes:

[0031] The target hidden state is nonlinearly combined through the first fully connected layer, and the combined target hidden state is mapped.

[0032] The score of the motion state category to which the mapped target hidden state belongs is calculated through the second fully connected layer;

[0033] The target output layer transforms the scores of the corresponding motion state categories to obtain motion state category probability distribution data.

[0034] Extract the maximum probability from the motion state category probability distribution data, and determine the type of abnormal motion state of the unmanned platform based on the maximum probability.

[0035] Secondly, this application provides a motion state recognition device based on a multi-layer neural network, comprising:

[0036] The acquisition module is used to acquire internal situational data and external environment data of the unmanned platform, and generate data samples to be identified based on the internal situational data and the external environment data.

[0037] The processing module is used to perform dynamic standardization processing on the data sample to be identified to obtain the current standard data sample;

[0038] The computation module is used to perform matrix operations on the current standard data samples based on the first layer of the neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step;

[0039] The recognition module is used to identify the motion state of the unmanned platform based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, using a second-layer neural network.

[0040] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0041] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0042] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0043] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0044] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0045] (1) After generating the data samples to be identified from the internal situational data and external environment data of the unmanned platform, this application performs matrix operations based on the first layer neural network and determines the target hidden state based on the second layer neural network. That is, it uses a multi-layer neural network and discards the current unreasonable input with a certain probability in the second layer neural network stage. It intersects with the single-layer network, which can avoid multiple false alarms. At the same time, it will also reduce the amount of data in the primary feature vector sequence, thereby effectively improving the efficiency and accuracy of motion state identification.

[0046] (2) This application includes an additional dropout layer in the second layer of the neural network. When training the model, two or more layers are selected for training, and the dropout rate is set to an empirical value of 0.2. 20% of the neurons are randomly "shut down," forcing the network to not rely on any specific neuron, thereby learning more robust features and effectively preventing overfitting. In addition, this application also deletes data that does not affect the status of the unmanned platform and further reduces the amount of data on the X, Y, and Z axes, which can effectively improve the efficiency of motion state recognition.

[0047] In summary, this application acquires internal situational data and external environment data of an unmanned platform, and generates data samples to be identified based on the internal situational data and the external environment data; it performs dynamic standardization processing on the data samples to be identified to obtain current standard data samples; it performs matrix operations on the current standard data samples based on a first-layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step; based on a second-layer neural network, it determines the target hidden state based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, and identifies the motion state of the unmanned platform based on the target hidden state. Through the above method, after determining the current standard data samples, matrix operations and state updates are performed by the first and second layers of neural networks. In the second-layer neural network stage, unreasonable inputs are discarded with a certain probability, thereby effectively improving the efficiency and accuracy of motion state identification, and thus providing property protection for the unmanned platform. Attached Figure Description

[0048] Figure 1 This is one of the flowcharts illustrating the motion state recognition method based on a multi-layer neural network provided in this application embodiment.

[0049] Figure 2 This is the second flowchart of the motion state recognition method based on a multi-layer neural network provided in the embodiments of this application.

[0050] Figure 3 This is a schematic diagram of the module structure of the motion state recognition device based on a multi-layer neural network provided in the embodiments of this application.

[0051] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0054] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0055] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] Based on this, embodiments of this application provide a motion state recognition method based on a multi-layer neural network, referring to... Figure 1 , Figure 1 This is one of the flowcharts illustrating the motion state recognition method based on a multi-layer neural network provided in this application. In this embodiment, the motion state recognition method based on a multi-layer neural network includes steps S10 to S40:

[0057] Step S10: Obtain internal situational data and external environment data of the unmanned platform, and generate a data sample to be identified based on the internal situational data and the external environment data.

[0058] It should be noted that an unmanned platform refers to equipment capable of automatically performing tasks in harsh environments, especially in rough seas. Internal situational data includes motion attitude dimension data and internal state dimension data. Motion attitude dimension data includes, but is not limited to, attitude, speed, and rotation speed. Internal state dimension data includes, but is not limited to, internal space temperature and leak detection data. External environmental data can be acquired by external imaging equipment to determine if any object is approaching the unmanned platform. The dimension of the data sample to be identified can be up to nine dimensions.

[0059] It should be understood that in this embodiment, the selection of input dimensions is based on extensive testing and training. Time information must be retained, and external environmental data can be output from the image device set on the unmanned platform. Only one dimension is used. For the input of internal situational data, data that does not affect the situational status of the unmanned platform is deleted after testing, and the amount of data is further reduced on the X, Y and Z axes respectively, which can effectively improve the efficiency of motion state recognition.

[0060] Step S20: Perform dynamic standardization processing on the data sample to be identified to obtain the current standard data sample.

[0061] Understandably, considering the zero-drift phenomenon of sensors, the data samples input to the motion state recognition model must undergo dynamic standardization. At this time, the current standard data samples follow a standard normal distribution, which can effectively adapt to the slow drift of sensor data.

[0062] Further, step S20 includes: counting the amount of data in the data sample to be identified, and calculating the sample mean based on the amount of data and the data sample to be identified; calculating the sample standard deviation based on the sample mean and the data sample to be identified; calculating the difference between the sample mean and the corresponding value of the data sample to be identified; calculating the ratio between the current difference calculation result and the sample standard deviation, and performing dynamic standardization processing on the data sample to be identified based on the ratio calculation result to obtain the current standard data sample.

[0063] It should be understood that this embodiment adopts a dynamic standardization processing method, which requires calculating the sample mean based on the amount of data and the data samples to be identified, and calculating the sample standard deviation based on the sample mean and the data samples to be identified. At this time, the data samples to be identified can be dynamically standardized based on the sample mean and the sample standard deviation. Taking the data samples to be identified as [10, 12, 14] as an example, the sample mean is 12 and the sample standard deviation is 1.63. The current difference calculation results are -2, 0 and 2 respectively, and the ratio calculation results are -1.23, 0 and 1.23. At this time, the current standard data sample can be [-1.23, 0, 1.23], which conforms to the standard normal distribution with a mean of 0 and a standard deviation of 1, and can be input into the motion state recognition model.

[0064] It should be noted that for each new set of data read, the sample mean and sample standard deviation need to be dynamically updated, and dynamic standardization is re-performed. A unique standard data sample is calculated in real time for each different window. This dynamic updating can be represented by the following formula:

[0065]

[0066] in, This represents the updated sample mean. Indicates the time when data is read. This represents the known sample mean before the update. This represents the updated sample variance, used to determine the updated sample standard deviation. This represents the known sample variance before the update. This indicates newly read data.

[0067] Step S30: Perform matrix operations on the current standard data samples based on the first layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step.

[0068] It should be understood that, for the motion state recognition model in this embodiment, a two-layer neural network is set in the middle layer, for example, a first-layer neural network and a second-layer neural network. The first-layer neural network can be a first-layer LSTM neural network, and the second-layer neural network can be a second-layer LSTM neural network. Both have the same number of neurons. The difference is that the second-layer neural network has an additional dropout layer. The number of weights in each cell depends on the number of outputs of the previous layer. Because a cell in a multi-layer neural network contains four gates, its single-cell weights are four times that of a normal layer.

[0069] Understandably, after obtaining the current standard data sample that meets the requirements, it can be input into the first layer of the neural network. Since each cell in the first layer has multiple gates, it actually performs multiple matrix operations, producing 32 outputs, the same size as the hidden layer. These outputs represent the primary feature vector sequences at each time step and the complete hidden state at the target time step, which can be the last time step. This complete hidden state carries the contextual information of the entire primary feature vector sequence. At this point, the computation of the first layer of the neural network is complete. When the input current standard data sample is 1x10 rows, matrix operations are performed with the 32 cells. Each cell needs to contain 10 weights; otherwise, computation is impossible.

[0070] Further, step S30 includes: concatenating the current standard data sample with the first-dimensional hidden state of the previous time step to obtain a second-dimensional vector; performing matrix operations on the second-dimensional vector based on multiple gates in the first-layer neural network, and fusing the results of each first matrix operation; determining the primary feature vector sequence of each time step based on the first matrix operation fusion result; updating the data of the cells in the first-layer neural network based on the first matrix operation fusion result to obtain the target cell state and the target hidden state; and determining the complete hidden state of the target time step based on the target cell state and the target hidden state.

[0071] Understandably, the multiple gates in the first layer of the neural network are the forget gate, input gate, candidate gate, and output gate. The forget gate determines which old information to discard from the cell state, the input gate determines how much new information is worth storing, the candidate gate creates a new, candidate cell state value as potential new memory content, and the output gate determines which parts of the current cell state will be output. The primary feature vector for each time step is calculated based on the fusion result of the first matrix operation, and these primary feature vectors are arranged in order to obtain a sequence of primary feature vectors. Furthermore, after processing the last time step, the cells in the first layer of the neural network are updated to obtain the final target cell state and target hidden state. At this point, the target cell state and target hidden state constitute the complete hidden state of the target time step, containing the "memory summary" of the entire input sequence from the first layer of the neural network.

[0072] Step S40: Based on the second layer neural network, determine the target hidden state according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and identify the motion state of the unmanned platform according to the target hidden state.

[0073] Understandably, the second layer of the neural network can also use the same method as the first layer to determine the target hidden state, only the input and output are different. That is, the input of the second layer of the neural network is no longer the original current standard data sample, but the primary feature vector sequence output by the first layer of the neural network and the complete hidden state of the target time step. In other words, the second layer of the neural network completely inherits the first layer of the neural network's understanding of the current standard data sample.

[0074] It should be understood that the motion state recognition model in this embodiment includes not only the first and second neural networks, but also multiple fully connected layers, a target output layer, a loss function, an activation function, and hidden layers. Based on the second neural network, the target hidden state is determined, and the motion state of the unmanned platform can be identified according to the target hidden state to determine the type of abnormal motion state of the unmanned platform.

[0075] Further, the step of determining the target hidden state based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, using the second-layer neural network, includes: discarding unreasonable data in the primary feature vector sequence based on the second-layer neural network, and concatenating the discarded feature vector sequence with the hidden state of the current time step to obtain a third-dimensional vector; performing matrix operations on the third-dimensional vector using multiple gates in the second-layer neural network, and fusing the results of each second matrix operation; wherein the second-layer neural network additionally includes a discarding layer, which randomly shuts down some neurons during training; updating the data of the cells in the second-layer neural network based on the fusion results of the second matrix operations, and determining the target hidden state based on the updated cell states and the updated hidden states.

[0076] It should be understood that if unreasonable data in the primary feature vector sequence is not discarded in time, it will require all the data to be used when determining the target's hidden state, resulting in low efficiency in recognizing motion states. In addition, unreasonable data will cause multiple false alarms. Therefore, in this embodiment, unreasonable inputs are discarded with a certain probability in the second layer of the neural network stage, and then concatenated with the hidden state of the current time step. After updating the data of the cells in the second layer of the neural network according to the fusion result of the second matrix operation, the target's hidden state is determined based on the updated cell state and the updated hidden state.

[0077] It should be noted that, compared to the first layer of the neural network, the second layer of the neural network has an additional dropout layer. When training the model, two or more layers are selected for training, and the dropout rate is set to an empirical value of 0.2. This randomly "shuts down" 20% of the neurons, forcing the network to not rely on any particular neuron, thereby learning more robust features and effectively preventing overfitting. The loss function can be the cross-entropy function, which is more suitable for probabilistic classification problems. In addition, the size of the hidden layer should not exceed 64 to reduce the cost of the model itself.

[0078] Furthermore, the step of identifying the motion state of the unmanned platform based on the target hiding state further includes: performing a nonlinear combination of the target hiding state through a first fully connected layer and mapping the combined target hiding state; calculating the score of the motion state category to which the mapped target hiding state belongs through a second fully connected layer; converting the score of the motion state category to which the target hiding state belongs through a target output layer to obtain motion state category probability distribution data; extracting the maximum probability from the motion state category probability distribution data, and determining the type of abnormal motion state of the unmanned platform based on the maximum probability.

[0079] It should be understood that after obtaining the target's hidden state, the target's hidden state can be non-linearly combined using the ReLU activation function through the first fully connected layer, and then mapped. In the motion state recognition model, the first fully connected layer is connected to the second neural network layer. This first fully connected layer can have multiple neurons. The second fully connected layer is connected to the first fully connected layer, and its number of neurons is the same as the number of categories. For the second fully connected layer, no activation function is used. After calculating the score of the motion state category to which the mapped target's hidden state belongs through the second fully connected layer, the target output layer converts the score of the motion state category into motion state category probability distribution data. This target output layer can be the layer containing the softmax function. At this time, the maximum probability is selected, and the type of abnormal motion state of the unmanned platform is determined, such as being attacked by other approaching objects, equipment failure, or being intercepted and salvaged by a third party. When the type is obviously abnormal, an immediate response is made to report to the remote manned platform and destroy local resources, thereby achieving the protection of the unmanned platform's assets.

[0080] It should be noted that in this embodiment, a security module can be configured on the unmanned platform. This security module is equipped with a perception program. The perception program runs the motion state recognition method based on multi-layer neural networks in this embodiment to perform real-time perception and self-learning, and continuously improve and update the motion state recognition model.

[0081] This embodiment acquires internal situational awareness data and external environment data of the unmanned platform, and generates data samples to be identified based on the internal situational awareness data and the external environment data. The data samples to be identified are dynamically standardized to obtain current standard data samples. Matrix operations are performed on the current standard data samples based on a first-layer neural network to obtain the primary feature vector sequence for each time step and the complete hidden state of the target time step. Based on a second-layer neural network, the target hidden state is determined based on the primary feature vector sequence for each time step and the complete hidden state of the target time step, and the motion state of the unmanned platform is identified based on the target hidden state. Through the above method, after determining the current standard data samples, matrix operations and state updates are performed using a first-layer and a second-layer neural network. In the second-layer neural network stage, unreasonable inputs are discarded with a certain probability, thereby effectively improving the efficiency and accuracy of motion state identification, and thus providing property protection for the unmanned platform.

[0082] In one specific implementation, this application provides steps for generating a data sample to be identified. Please refer to... Figure 2 , Figure 2 This is the second flowchart illustrating the motion state recognition method based on a multi-layer neural network provided in this application. Step S10 includes steps S101 to S104:

[0083] Step S101: Obtain internal situational data and external environment data of the unmanned platform, and fuse the internal situational data and the external environment data to obtain a single fused data point.

[0084] It should be noted that the internal situational data includes motion posture dimension data and internal state dimension data. Among them, motion posture dimension data includes, but is not limited to, attitude, motion speed and rotation speed, which can be read by IMU (Inertial Measurement Unit). Internal state dimension data includes, but is not limited to, internal space temperature, water leakage detection data, etc. At this time, the internal situational data and external environment data are fused into a single fused data point, and the data dimension can be 10 dimensions.

[0085] Step S102: Set a time stamp for the single fused data point.

[0086] Step S103: In response to the data inflow instruction, the single fused data point is filled into the target sliding window according to the time tag; wherein the length of the target sliding window is consistent with the data sampling frequency.

[0087] It should be understood that after setting a precise time label for a single fused data point, a target sliding window is created. The core idea of ​​this target sliding window is to cut continuous data points into multiple time series of fixed length so that the model can analyze the dynamic patterns of the data over time. Before filling the target sliding window with a single fused data point, the window length and window step size need to be set. The window length represents the number of data points contained in a single window, and the window step size represents the number of data points that the target sliding window moves forward each time it slides.

[0088] Step S104: When the target sliding window is completely filled by the single fused data point, control the target sliding window to slide forward, and generate the data sample to be identified based on the target sliding window after sliding.

[0089] It is understandable that when the target sliding window is completely filled by the single fused data point, the target sliding window is controlled to slide forward. Then, for each set of data read, the target sliding window is controlled to slide forward, the earliest data point corresponding to the time tag is removed from the target sliding window, the latest data point is included, and only the multiple data points in the target sliding window at the last moment are taken as the output, which is the data sample to be identified.

[0090] This embodiment acquires internal situational awareness data and external environment data from an unmanned platform, and fuses the internal situational awareness data and the external environment data to obtain a single fused data point; a time label is set for the single fused data point; in response to a data inflow command, the single fused data point is filled into a target sliding window according to the time label; wherein, the length of the target sliding window is consistent with the data sampling frequency; when the target sliding window is completely filled by the single fused data point, the target sliding window is controlled to slide forward, and a data sample to be identified is generated based on the slid target sliding window. Through the above method, after setting a precise time label for the fused single data point, the single fused data point is filled into the target sliding window according to the time label, and then the continuous data points are cut into multiple fixed-length and continuous time series according to the target sliding window, so that the model can analyze the dynamic pattern of the data over time and generate the target sliding window based on the data points within the slid target sliding window, thereby effectively improving the accuracy of generating the data sample to be identified.

[0091] The motion state recognition device based on a multi-layer neural network provided in this application is described below. The motion state recognition device based on a multi-layer neural network described below can be referred to in conjunction with the motion state recognition method based on a multi-layer neural network described above. Please refer to... Figure 3 , Figure 3 This is a schematic diagram of the module structure of the motion state recognition device based on a multi-layer neural network provided in this application embodiment, including:

[0092] The acquisition module T10 is used to acquire the internal situational data and external environment data of the unmanned platform, and generate a data sample to be identified based on the internal situational data and the external environment data.

[0093] The processing module T20 is used to perform dynamic standardization processing on the data sample to be identified to obtain the current standard data sample.

[0094] The operation module T30 is used to perform matrix operations on the current standard data samples based on the first layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step.

[0095] The recognition module T40 is used to identify the motion state of the unmanned platform based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, using a second-layer neural network.

[0096] This embodiment acquires internal situational awareness data and external environment data of the unmanned platform, and generates data samples to be identified based on the internal situational awareness data and the external environment data. The data samples to be identified are dynamically standardized to obtain current standard data samples. Matrix operations are performed on the current standard data samples based on a first-layer neural network to obtain the primary feature vector sequence for each time step and the complete hidden state of the target time step. Based on a second-layer neural network, the target hidden state is determined based on the primary feature vector sequence for each time step and the complete hidden state of the target time step, and the motion state of the unmanned platform is identified based on the target hidden state. Through the above method, after determining the current standard data samples, matrix operations and state updates are performed using a first-layer and a second-layer neural network. In the second-layer neural network stage, unreasonable inputs are discarded with a certain probability, thereby effectively improving the efficiency and accuracy of motion state identification, and thus providing property protection for the unmanned platform.

[0097] It is understood that the detailed functional implementation of each of the above modules can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0098] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0099] Based on the methods in the above embodiments, this application provides an electronic device, please refer to... Figure 4 , Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0100] It should be noted that the system may include: a processor 10, a communications interface 20, a memory 30, and a communication bus 40. The processor 10, communications interface 20, and memory 30 communicate with each other via the communication bus 40. The processor 10 can invoke logical instructions stored in the memory 30 to execute the methods described in the above embodiments.

[0101] Furthermore, the logical instructions in the aforementioned memory 30 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0102] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0103] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0104] It is understood that the processor in the embodiments of this application can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0105] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory, flash memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, hard disks, portable hard disks, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor.

[0106] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. Those skilled in the art will readily understand that the above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A motion state recognition method based on a multi-layer neural network, characterized in that, include: Acquire internal situational data and external environment data of the unmanned platform, and generate data samples to be identified based on the internal situational data and the external environment data; The data sample to be identified is dynamically standardized to obtain the current standard data sample; Matrix operations are performed on the current standard data samples based on the first layer neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step; Based on the second-layer neural network, the target hidden state is determined according to the primary feature vector sequence of each time step and the complete hidden state of the target time step, and the motion state of the unmanned platform is identified according to the target hidden state. The step of determining the target hidden state based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, using a second-layer neural network, includes: The second-layer neural network discards unreasonable data in the primary feature vector sequence, and concatenates the discarded feature vector sequence with the hidden state at the current time step to obtain the third-dimensional vector. The third-dimensional vector matrix is ​​operated on by multiple gates in the second-layer neural network, and the results of the second matrix operations are fused together; wherein, the second-layer neural network is additionally equipped with a dropout layer, which randomly shuts down some neurons during training; The data of the cells in the second layer of the neural network is updated based on the fusion result of the second matrix operation, and the target hidden state is determined based on the updated cell state and the updated hidden state. The step of identifying the motion state of the unmanned platform based on the target concealment state includes: The target hidden state is nonlinearly combined through the first fully connected layer, and the combined target hidden state is mapped. The score of the motion state category to which the mapped target hidden state belongs is calculated through the second fully connected layer; The target output layer transforms the scores of the corresponding motion state categories to obtain motion state category probability distribution data. Extract the maximum probability from the motion state category probability distribution data, and determine the type of abnormal motion state of the unmanned platform based on the maximum probability.

2. The method as described in claim 1, characterized in that, The step of acquiring internal situational data and external environment data of the unmanned platform, and generating a data sample to be identified based on the internal situational data and the external environment data, includes: Acquire internal situational data and external environment data of the unmanned platform, and fuse the internal situational data and the external environment data to obtain a single fused data point; Set a time stamp for the single fused data point; In response to a data inflow instruction, the single fused data point is filled into a target sliding window according to the time tag; wherein the length of the target sliding window is consistent with the data sampling frequency; When the target sliding window is completely filled by the single fused data point, the target sliding window is controlled to slide forward, and a data sample to be identified is generated based on the target sliding window after sliding.

3. The method as described in claim 1, characterized in that, The step of dynamically standardizing the data sample to be identified to obtain the current standard data sample includes: The data volume of the data sample to be identified is counted, and the sample mean is calculated based on the data volume and the data sample to be identified; Calculate the sample standard deviation based on the sample mean and the data sample to be identified; Calculate the difference between the sample mean and the corresponding value of the data sample to be identified; The ratio of the current difference calculation result to the sample standard deviation is calculated, and the data sample to be identified is dynamically standardized based on the ratio calculation result to obtain the current standard data sample.

4. The method as described in claim 1, characterized in that, The step of performing matrix operations on the current standard data samples based on the first-layer neural network to obtain the primary feature vector sequence at each time step and the complete hidden state at the target time step includes: The current standard data sample is concatenated with the first-dimensional hidden state of the previous time step to obtain the second-dimensional vector. The second dimension vector is subjected to matrix operations by multiple gates in the first layer of the neural network, and the results of each first matrix operation are fused together. The primary feature vector sequence for each time step is determined based on the fusion result of the first matrix operation; Based on the fusion result of the first matrix operation, the data of the cells in the first layer of the neural network is updated to obtain the target cell state and the target hidden state. The complete hidden state of the target time step is determined based on the target cell state and the target hidden state.

5. A motion state recognition device based on a multi-layer neural network, characterized in that, include: The acquisition module is used to acquire internal situational data and external environment data of the unmanned platform, and generate data samples to be identified based on the internal situational data and the external environment data. The processing module is used to perform dynamic standardization processing on the data sample to be identified to obtain the current standard data sample; The computation module is used to perform matrix operations on the current standard data samples based on the first layer of the neural network to obtain the primary feature vector sequence of each time step and the complete hidden state of the target time step; The identification module is used to identify the motion state of the unmanned platform based on the primary feature vector sequence of each time step and the complete hidden state of the target time step, using a second-layer neural network. The recognition module is further configured to discard unreasonable data in the primary feature vector sequence based on the second-layer neural network, and concatenate the discarded feature vector sequence with the hidden state at the current time step to obtain a third-dimensional vector; perform operations on the third-dimensional vector matrix based on multiple gates in the second-layer neural network, and fuse the results of each second matrix operation; wherein, the second-layer neural network is additionally equipped with a discarding layer, which randomly shuts down some neurons during training; update the data of the cells in the second-layer neural network according to the second matrix operation fusion results, and determine the target hidden state based on the updated cell state and the updated hidden state; The recognition module is further configured to perform nonlinear combination of the target hidden state through the first fully connected layer, and map the combined target hidden state; and calculate the score of the motion state category to which the mapped target hidden state belongs through the second fully connected layer. The target output layer transforms the score of the corresponding motion state category to obtain motion state category probability distribution data; the maximum probability is extracted from the motion state category probability distribution data, and the type of abnormal motion state of the unmanned platform is determined based on the maximum probability.

6. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the method as described in any one of claims 1-4.

8. A computer program product, characterized in that, When the computer program product is run on a processor, the processor causes the processor to perform the method as described in any one of claims 1-4.

Images