A ST-GRU memristor neural network circuit and training method for gait prediction

Abstract

The present invention discloses an ST-GRU memristor neural network circuit and training method for realizing gait prediction, wherein the circuit includes obtaining the output result of the GRU unit circuit for processing the time series as a spatial characteristic pulse voltage signal after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, and inputting the spatial characteristic pulse voltage signal into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit. The present invention solves the problem that the existing GRU hardware circuit cannot train the target in multi-layer dynamic timing prediction, realizes the sequence prediction of multiple dimensions and uses it for the step length prediction problem of the timing, has high accuracy and good computing efficiency, and does not need to consume a large amount of software computing resources.

Description

Technical Field

[0001] The present invention relates to the field of neural network circuits, and in particular to an ST-GRU memristor neural network circuit and a training method for realizing gait prediction. Background Art

[0002] In recent years, the high attention paid to artificial intelligence is largely due to the progress of deep neural networks. There are many structures of deep neural networks, and recurrent neural networks (RNN) are one of the important structures. There are many improved versions of recurrent neural networks, and the more commonly used ones are long short-term memory (LSTM) and gated recurrent unit (GRU). These versions can effectively improve the gradient vanishing and gradient exploding problems in RNN; however, due to the hysteresis of unidirectional recurrent neural networks, the effect is not very ideal when dealing with prediction problems.

[0003] Memristor is a nanoscale device with the advantages of high volatility, high density, low power consumption, and compatibility with CMOS devices. It is highly suitable for hardware neural networks. Therefore, hardware neural networks based on memristors have attracted extensive attention from researchers. Figure 1 As shown in the figure, a single GRUCell circuit module based on the threshold memristor of AIST in the prior art is shown. For the output function softmax of the fully connected layer connected to the GRU network, due to the exponential operation, if an op amp is used to build it, a larger power supply voltage must be provided, which does not meet the original intention of energy saving and high efficiency of this neural network circuit design. The softmax function mainly performs the normalization of M outputs and compares the size operations, so a competitive neural network circuit (winner take all circuit) with a higher degree of integration can be used. Figure 1 The AIST-based threshold memristor is used in this neural network circuit. This unit is integrated in a vertical line, which improves the integration level. At the same time, due to the characteristics of the conductivity adjustment circuit, there will be no mutual interference between the conductivity information.

[0004] Although the existing GRU has a hardware implementation solution, it lacks an effective hardware neural network algorithm and is difficult to complete actual projects. Existing gait prediction mostly uses software-level deep learning algorithms, which often overemphasize temporal features and ignore some spatial features. In addition, the software operation consumes huge computing resources.

[0005] In view of this, the present invention is proposed. Summary of the invention

[0006] The purpose of the present invention is to provide a ST-GRU memristor neural network circuit and training method for realizing gait prediction, so as to solve the above-mentioned technical problems existing in the prior art. The present invention improves the deficiencies existing in the existing GRU hardware circuit in timing prediction, solves the problem that the existing GRU hardware circuit cannot train the target in multi-layer dynamic timing prediction, realizes sequence prediction in multiple dimensions and uses it for the step length prediction problem of timing, has high accuracy and good computing efficiency, and does not need to consume a large amount of software computing resources.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A ST-GRU memristor neural network circuit for realizing gait prediction, comprising: a GRU unit circuit for processing time series, a GRU unit circuit for processing space series, and a fully connected layer circuit;

[0009] Each moment in the time series corresponds to a spatial sequence;

[0010] The GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at each moment in the time series, and the GRU unit circuit for processing the space series processes the characteristic pulse voltage signal of the space series corresponding to each moment;

[0011] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has completed processing all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit.

[0012] Preferably, the GRU unit circuit for processing time series is mainly composed of a memristor array; the GRU unit circuit for processing space series is mainly composed of a memristor array.

[0013] The training method of the ST-GRU memristor neural network circuit for realizing gait prediction comprises the following steps:

[0014] Step 1: Cut a 3-second video from the training pedestrian walking video, and extract 30 128×128 images frame by frame from this 3-second video. Then blur these images, use Fourier transform to extract the edges in the images for plotting, and take 50 anchor points of the lower body of each image as feature vectors. A total of 1500 anchor points are taken from the 30 images, which are evenly divided into 30 groups and processed according to 30 time steps, with one group of anchor points processed in each time step.

[0015] Step 2, converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and then inputting the time characteristic pulse voltage signal at each moment in the time series into the GRU unit circuit for processing the time series of the ST-GRU memristor neural network circuit, and inputting the characteristic pulse voltage signal of the spatial sequence corresponding to each moment into the GRU unit circuit for processing the spatial sequence of the ST-GRU memristor neural network circuit;

[0016] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has processed all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the weight of the GRU unit circuit for processing the time series and the weight of the GRU unit circuit for processing the spatial sequence are updated according to the output result of the GRU unit circuit for processing the spatial sequence, and the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit;

[0017] Step 3: Repeat steps 1 and 2 above and run for 60 epochs to complete the training of the ST-GRU memristor neural network circuit.

[0018] The method for using the above-mentioned ST-GRU memristor neural network circuit for realizing gait prediction includes: intercepting a 3-second video from a pedestrian walking video to be predicted, and extracting 30 128×128 pictures frame by frame from the 3-second video, and then blurring the pictures, extracting the edges in the pictures by Fourier transform for plotting points, taking 50 anchor points of the lower body of each picture as feature vectors, and taking a total of 1500 anchor points from the 30 pictures, which are evenly divided into 30 groups, and processed according to 30 time steps, with 1 group of anchor points processed for each time step; converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and inputting them into the trained ST-GRU memristor neural network circuit for realizing gait prediction for processing, thereby obtaining a predicted gait.

[0019] A ST-GRU memristor neural network circuit system for realizing gait prediction adopts the above-mentioned ST-GRU memristor neural network circuit for realizing gait prediction and is trained according to the above-mentioned training method for the ST-GRU memristor neural network circuit for realizing gait prediction.

[0020] Compared with the prior art, the present invention changes the existing GRU hardware circuit into an ST-GRU circuit, sets up a GRU unit circuit for processing time series and a GRU unit circuit for processing space series, and obtains the output result of the GRU unit circuit for processing time series as a spatial characteristic pulse voltage signal, and inputs the spatial characteristic pulse voltage signal into the GRU unit circuit for processing the space series to process the characteristic pulse voltage signal of the space series corresponding to the moment, which can complete the time and space multi-dimensional sequence prediction of the hardware, greatly enhances the ability of the GRU circuit to handle dynamic prediction problems, solves the problem that the existing GRU hardware circuit cannot train the target in multi-layer dynamic timing prediction, realizes the sequence prediction of multiple dimensions and uses it for the step length prediction problem of timing, has high accuracy and good computing efficiency, and does not require a large amount of software computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 Schematic diagram of a single GRUCell circuit module of a threshold-type memristor based on AIST in the prior art.

[0023] Figure 2Schematic diagram of the principle of the ST-GRU memristor neural network circuit for realizing gait prediction in an embodiment of the present invention Figure 1 .

[0024] Figure 3 Schematic diagram of the principle of the ST-GRU memristor neural network circuit for realizing gait prediction in an embodiment of the present invention Figure 2 .

[0025] Figure 4 Schematic diagram of the structure of the GRU unit circuit in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0027] First, the terms that may be used in this article are explained as follows:

[0028] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0029] The ST-GRU memristor neural network circuit and training method for realizing gait prediction provided by the present invention are described in detail below. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If the specific conditions are not specified in the embodiments of the present invention, they are carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer. The reagents or instruments used in the embodiments of the present invention, if the manufacturer is not specified, are all conventional products that can be purchased commercially.

[0030] like Figure 2 , Figure 3 and Figure 4 As shown, the present invention provides an ST-GRU memristor neural network circuit for realizing gait prediction, and its structure includes: a GRU unit circuit for processing time series, a GRU unit circuit for processing space series, and a fully connected layer circuit;

[0031] Each moment in the time series corresponds to a spatial sequence; the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal of each moment in the time series, and the GRU unit circuit for processing the spatial sequence processes the characteristic pulse voltage signal of the spatial sequence corresponding to each moment.

[0032] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has completed processing all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit.

[0033] Specifically, the ST-GRU memristor neural network circuit for implementing gait prediction may include the following implementation schemes:

[0034] (1) The GRU unit circuit for processing time series is mainly composed of a memristor array; the GRU unit circuit for processing space series is mainly composed of a memristor array; for example: the GRU unit circuit for processing time series can be composed of Figure 4 The circuit shown ( Figure 4 The three boxes in the figure represent a memristor array), and the GRU unit circuit for processing spatial sequences can be used as follows Figure 4 The circuit shown ( Figure 4 The three boxes in the figure represent a memristor array). Since the memristor has a non-volatile memory function, its size and power consumption are smaller than those of a traditional memory, and it does not need to be refreshed regularly like a DRAM, the present invention preferably uses a memristor as a unit for constructing a neural network.

[0035] (2) The GRU unit circuit for processing time series uses the hidden layer characteristic pulse voltage signal at the previous moment as input to process the time characteristic pulse voltage signal at the current moment in the time series.

[0036] Furthermore, the present invention provides a training method for the ST-GRU memristor neural network circuit for realizing gait prediction, which may include the following steps:

[0037] Step 1: Cut a 3-second video from the training pedestrian walking video, and extract 30 128×128 images frame by frame from this 3-second video, then blur these images, use Fourier transform to extract the edges of the images for plotting, take 50 anchor points of the lower body of each image as feature vectors, and take a total of 1500 anchor points from the 30 images, which are evenly divided into 30 groups and processed in 30 time steps, with one group of anchor points processed in each time step.

[0038] Step 2, converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and then inputting the time characteristic pulse voltage signal at each moment in the time series into the GRU unit circuit for processing the time series of the ST-GRU memristor neural network circuit, and inputting the characteristic pulse voltage signal of the spatial sequence corresponding to each moment into the GRU unit circuit for processing the spatial sequence of the ST-GRU memristor neural network circuit;

[0039] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has processed all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the weight of the GRU unit circuit for processing the time series and the weight of the GRU unit circuit for processing the spatial sequence are updated according to the output result of the GRU unit circuit for processing the spatial sequence, and the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit.

[0040] Step 3: Repeat steps 1 and 2 above and run for 60 epochs to complete the training of the ST-GRU memristor neural network circuit.

[0041] Furthermore, the present invention also provides a method for using the above-mentioned ST-GRU memristor neural network circuit for realizing gait prediction, comprising: intercepting a 3-second video from a pedestrian walking video to be predicted, and extracting 30 128×128 pictures frame by frame from the 3-second video, and then blurring the pictures, extracting the edges in the pictures by Fourier transform for plotting points, taking 50 anchor points of the lower body of each picture as feature vectors, taking a total of 1500 anchor points from the 30 pictures, and dividing them into 30 groups, processing them according to 30 time steps, and processing 1 group of anchor points for each time step; converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and inputting them into the trained ST-GRU memristor neural network circuit for realizing gait prediction for processing, thereby obtaining a predicted gait.

[0042] In the prior art, the prediction problem based on time series has always been the focus of the field of artificial intelligence, but most algorithms tend to consume huge computing resources. Starting from the hardware solution of saving computing power, transfer learning, and solving spatiotemporal problems, the present invention proposes a neural network circuit constructed by a memristor based on the ST-GRU algorithm to actually solve the gait prediction problem. The present invention improves the existing GRU hardware circuit and turns it into an ST-GRU circuit. This improvement enables the original GRU circuit to perform prediction analysis from multiple dimensions, greatly enhancing the ability of the GRU circuit to handle dynamic prediction problems.

[0043] The present invention mainly improves the original circuit structure by adding a bidirectional head-tail loop GRUCell, which can complete the time-space multi-dimensional sequence prediction of the hardware. In order to demonstrate the practicality of the circuit, the present invention is applied to the step length prediction of the time series and shows good results.

[0044] In summary, it can be seen that the embodiments of the present invention improve the deficiencies of the existing GRU hardware circuit in timing prediction, solve the problem that the existing GRU hardware circuit cannot train the target in multi-layer dynamic timing prediction, realize sequence prediction in multiple dimensions and use it for the step size prediction problem of timing, have high accuracy and good computing efficiency, and do not require a large amount of software computing resources.

[0045] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the ST-GRU memristor neural network circuit and training method for realizing gait prediction provided by the embodiments of the present invention are described in detail below with specific embodiments.

[0046] Example 1

[0047] like Figure 2 , Figure 3 and Figure 4As shown, a ST-GRU memristor neural network circuit for realizing gait prediction, the structure of which includes: a GRU unit circuit for processing time series, a GRU unit circuit for processing space series and a fully connected layer circuit; each moment in the time series corresponds to a space sequence; the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal of each moment in the time series, and the GRU unit circuit for processing the space sequence processes the characteristic pulse voltage signal of the space sequence corresponding to each moment.

[0048] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has completed processing all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit.

[0049] Among them, the GRU unit circuit for processing time series is mainly composed of a memristor array; the GRU unit circuit for processing space series is mainly composed of a memristor array; for example: the GRU unit circuit for processing time series can be composed of Figure 4 The circuit shown in FIG. 1 can be used as follows: Figure 4 The circuit shown.

[0050] Specifically, the ST-GRU memristor neural network circuit for realizing gait prediction provided by Embodiment 1 of the present invention is used to perform gait prediction, and its training method includes the following steps:

[0051] Step A: A 3-second video is captured from the training pedestrian walking video, and 30 128×128 images are extracted frame by frame from this 3-second video. Then, these images are blurred, and the edges of the images are extracted using Fourier transform for plotting. 50 anchor points of the lower body of each image are taken as feature vectors. A total of 1,500 anchor points are taken from the 30 images, which are evenly divided into 30 groups and processed according to 30 time steps, with one group of anchor points processed in each time step.

[0052] Step B, converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and then inputting the time characteristic pulse voltage signal at each moment in the time series into the GRU unit circuit for processing the time series of the ST-GRU memristor neural network circuit, and inputting the characteristic pulse voltage signal of the spatial sequence corresponding to each moment into the GRU unit circuit for processing the spatial sequence of the ST-GRU memristor neural network circuit;

[0053] For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has processed all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the weight of the GRU unit circuit for processing the time series and the weight of the GRU unit circuit for processing the spatial sequence are updated according to the output result of the GRU unit circuit for processing the spatial sequence, and the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit.

[0054] Step C: Repeat the above steps A and B and run 60 epochs to complete the training of the ST-GRU memristor neural network circuit.

[0055] Further, when the ST-GRU memristor neural network circuit for realizing gait prediction provided by the trained embodiment 1 of the present invention is used for gait prediction, a 3-second video is intercepted from the pedestrian walking video to be predicted, and 30 128×128 pictures are extracted frame by frame from the 3-second video, and then the pictures are blurred, and the edges in the pictures are extracted by Fourier transform for plotting points. 50 anchor points of the lower body are taken from each picture as feature vectors, and 1500 anchor points are taken from the 30 pictures, which are evenly divided into 30 groups and processed according to 30 time steps, with 1 group of anchor points processed for each time step; each group of anchor points is converted into a matrix form, and Gaussian encoded in sequence according to the time sequence, and input into the trained ST-GRU memristor neural network circuit for realizing gait prediction for processing, so as to obtain the predicted gait; after testing, the accuracy of the present invention can reach 95.4%, which realizes sequence prediction in multiple dimensions, has high accuracy and good computing efficiency, and does not require a large amount of software computing resources.

[0056] In summary, it can be seen that the embodiments of the present invention improve the deficiencies of the existing GRU hardware circuit in timing prediction, solve the problem that the existing GRU hardware circuit cannot train the target in multi-layer dynamic timing prediction, realize sequence prediction in multiple dimensions and use it for the step size prediction problem of timing, have high accuracy and good computing efficiency, and do not require a large amount of software computing resources.

[0057] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A ST-GRU memristor neural network circuit for realizing gait prediction, characterized in that: include: GRU unit circuit for processing time series, GRU unit circuit for processing spatial series, and fully connected layer circuit; Each moment in the time series corresponds to a spatial sequence; The GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at each moment in the time series, and the GRU unit circuit for processing the space series processes the characteristic pulse voltage signal of the space series corresponding to each moment; For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence to process the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has completed processing all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit; The GRU unit circuit for processing time series adopts a memristor array to form a neural network; the GRU unit circuit for processing space series adopts a memristor array to form a neural network.

2. The training method of the ST-GRU memristor neural network circuit for realizing gait prediction as described in claim 1 above is characterized in that: The following steps are involved: Step 1: Cut a 3-second video from the training pedestrian walking video, and extract 30 128×128 images frame by frame from this 3-second video. Then blur these images, use Fourier transform to extract the edges in the images for plotting, and take 50 anchor points of the lower body of each image as feature vectors. A total of 1500 anchor points are taken from the 30 images, which are evenly divided into 30 groups and processed according to 30 time steps, with one group of anchor points processed in each time step. Step 2, converting each group of anchor points into a matrix form, and Gaussian encoding them in chronological order, and then inputting the time characteristic pulse voltage signal at each moment in the time series into the GRU unit circuit for processing the time series of the ST-GRU memristor neural network circuit, and inputting the characteristic pulse voltage signal of the spatial sequence corresponding to each moment into the GRU unit circuit for processing the spatial sequence of the ST-GRU memristor neural network circuit; For the time characteristic pulse voltage signal at each moment in the time series, after the GRU unit circuit for processing the time series processes the time characteristic pulse voltage signal at a moment in the time series, the output result of the GRU unit circuit for processing the time series is obtained as the spatial characteristic pulse voltage signal, and the spatial characteristic pulse voltage signal is input into the GRU unit circuit for processing the spatial sequence for processing the characteristic pulse voltage signal of the spatial sequence corresponding to the moment; when the GRU unit circuit for processing the spatial sequence has processed all the characteristic pulse voltage signals of the spatial sequence corresponding to the moment, the weight of the GRU unit circuit for processing the time series and the weight of the GRU unit circuit for processing the spatial sequence are updated according to the output result of the GRU unit circuit for processing the spatial sequence, and the output result of the GRU unit circuit for processing the spatial sequence is input into the fully connected layer circuit; Step 3: Repeat steps 1 and 2 above and run 60 epochs to complete the training of the ST-GRU memristor neural network circuit.

3. The method for using the ST-GRU memristor neural network circuit for realizing gait prediction as described in claim 1 above is characterized in that: include: A 3-second video is intercepted from the pedestrian walking video to be predicted, and 30 128×128 pictures are extracted frame by frame from the 3-second video, and then the pictures are blurred, and the edges in the pictures are extracted by Fourier transform for plotting points. 50 anchor points of the lower body are taken from each picture as feature vectors, and a total of 1500 anchor points are taken from the 30 pictures, which are evenly divided into 30 groups and processed according to 30 time steps, with one group of anchor points processed for each time step; each group of anchor points is converted into a matrix form, and Gaussian encoded in sequence according to time order, and input into the trained ST-GRU memristor neural network circuit for realizing gait prediction for processing, so as to obtain the predicted gait.

4. A ST-GRU memristor neural network circuit system for realizing gait prediction, characterized in that: The ST-GRU memristor neural network circuit for realizing gait prediction described in claim 1 is adopted, and training is performed according to the training method for the ST-GRU memristor neural network circuit for realizing gait prediction described in claim 2.

Images