A human motion rapid recognition method based on parallel time delay optical reservoir computing

Abstract

The application discloses a human motion rapid recognition method based on parallel time delay optical reserve pool calculation, and specifically comprises the following steps: after data compression preprocessing, the feature data of 5 to 7 key frames is extracted from the preprocessed data by combining sampling and key frame screening technology, and the splicing mode of the state sequence is adjusted from different regions of a static image to different frames of a related video; subsequently, multiple reserve pools are introduced to realize the parallel input of multiple frames of data, and a parallel semiconductor laser array is introduced inside a single reserve pool; in the output layer, all virtual node states of the reserve pool laser are combined for training and testing, and the output weight is calculated by a ridge regression algorithm. While maintaining the high bandwidth advantage of optical calculation, the application significantly improves the system calculation efficiency, and provides a new solution for real-time image recognition, dynamic visual tasks and the like.

Description

Technical Field

[0001] This invention belongs to the field of reservoir computing applications, and particularly relates to a method for rapid human motion recognition based on parallel time-delay optical reservoir computing. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence, image recognition has been increasingly widely used in fields such as autonomous driving, intelligent monitoring, and human-computer interaction. Limited by early algorithms and computing power, image recognition technology mainly relies on manually designed feature extraction and traditional machine learning methods, which can only process simple, structured images. Meanwhile, the computing power demands of artificial intelligence have significantly exceeded the hardware performance improvement capabilities described by Moore's Law. Constrained by the "memory wall" and data transfer overhead caused by the separation of storage and computation in the von Neumann architecture, computing power consumption and energy efficiency bottlenecks are becoming increasingly prominent. Therefore, there is an urgent need to develop new computing architectures with high computing power and low power consumption.

[0003] Reservoir computing (RC) is a neuromorphic computing framework that simulates the information processing mechanism of neural circuits in the human brain. Its connection weights between the input layer and the reservoir layer remain fixed, requiring only simple algorithms such as linear regression to train the output layer readout weights, thus significantly simplifying the training process and possessing good physical implementation potential. Optical platforms inherently possess physical characteristics such as parallelism, high speed, large bandwidth, and low power consumption; therefore, time-delay optical reservoir computing systems based on semiconductor lasers have become one of the important physical implementation paths for reservoir computing. However, traditional time-delay optical reservoir computing, due to its time-division multiplexing method for constructing the reservoir state space, suffers from low data processing rates when handling large-scale, high-dimensional data tasks such as image and video analysis and human motion video recognition, limited by the serial data input method, making it difficult to meet real-time processing requirements. Summary of the Invention

[0004] To overcome the above limitations, this invention provides a method for rapid human motion recognition based on parallel time-delay optical reservoir computation.

[0005] The present invention provides a fast human motion recognition method based on parallel time-delay optical reservoir computation, specifically as follows:

[0006] First, histogram of oriented gradients (HARGs) features are extracted from all image frames of the action video dataset, and then the features are further dimensionality-reduced using principal component analysis.

[0007] After data preprocessing, by combining sampling and keyframe filtering techniques, feature data of 5 to 7 keyframes are extracted from the preprocessed data, and the stitching method of the state sequence is adjusted from different regions of static images to different frames of related videos.

[0008] Subsequently, multiple reservoirs are introduced to achieve parallel input of multi-frame data. At the same time, a parallel semiconductor laser array is introduced inside a single reservoir to construct a high-dimensional state space under high-speed computing conditions. Different frame data are synchronously loaded into the phase of the light source through multiple modulators and injected into K parallel response lasers, with each response laser having the same number of virtual nodes.

[0009] In the output layer, the states of all virtual nodes of K lasers are merged for training and testing, and the output weights are calculated using the ridge regression algorithm.

[0010] The dynamic characteristics of each semiconductor laser are described by the Lang-Kobayashi equations:

[0011] (1)

[0012] (2)

[0013] in, and Let represent the slowly varying electric field intensity and carrier density of the i-th responding laser, respectively; It is the gain coefficient. This represents the feedback intensity of each responding laser. Indicates the light injection intensity. It is the linewidth factor. It is the carrier density during the transparent period. These are the photon lifetime and the carrier lifetime, respectively. It responds to the injection current of the laser. This represents the current factor of the i-th responding laser; in the input layer, it represents the preprocessed data. After sampling and holding for one operating period T, the resulting signal is multiplied by a time mask. This ensures the variability of the input signal across the virtual nodes of the reservoir layer; finally, the mask signal... The phase modulated to the light source is input into the response laser.

[0014] The mapping relationship from the input layer to the reservoir layer is determined by the relationship between the sample-and-hold time and the delay time. Here, a desynchronization method is used to adjust the sample-and-hold time T and the delay time of each laser. The relationship is:

[0015] (3)

[0016] The obtained node state correspondence is represented as follows:

[0017] (4)

[0018] In the formula, This represents the state of the i-th node within the n-th delay time. This indicates the nonlinear response of the laser. This represents the input mask signal, and k represents the multiple of the node interval between the delay time and the sampling time.

[0019] The output layer is trained using virtual nodes, whose state values ​​are taken from the output intensities of all K responding lasers; the virtual node state values ​​are obtained as follows:

[0020] (5)

[0021] The node state is taken from the state between two virtual nodes.

[0022] When all Then, the virtual node states of all reserve pool layers are concatenated, and the output weights of the reserve pools are calculated using ridge regression. The final output of the storage pool is obtained. The specific calculation formula is as follows:

[0023] (6)

[0024] (7)

[0025] in, The ridge regression coefficients are introduced to prevent overfitting. Represents the identity matrix.

[0026] After training, an output dataset with 6 dimensions is generated. Train a corresponding number of outputs based on the number of action categories; the output formula is defined as:

[0027] (8)

[0028] In the formula, j represents the output category. When the current input data is in category c, y c Take the higher value, and take the other output y. j The lower value is selected, and the "winner-takes-all" strategy is used to assign the input to the category c with the highest output value. The input is then compared with the one-hot code in the input data. If the numbers are the same, the prediction is correct.

[0029] To quantify the performance of action recognition, the KTH human action dataset was used as a benchmark for testing, with the correct action recognition rate as the metric for recognition accuracy.

[0030] (9)

[0031] in, This indicates the number of correctly identified actions. Indicates the number of test sets.

[0032] The beneficial technical effects of this invention compared to the prior art are as follows:

[0033] Compared with the human motion recognition method based on traditional time-delay optical reservoirs, this invention has the following obvious advantages: (1) Video data compression: By preprocessing the video data, redundant information is reduced, and the input data dimension is significantly simplified by combining principal component analysis. (2) Improved accuracy of human motion recognition: By using a parallel reservoir computing architecture, the nonlinear representation capability of the state space for different frames of video data is enhanced, thereby improving the accuracy of human motion recognition. (3) Significantly improved training efficiency: By adopting a multi-input mechanism, the compressed data is synchronously injected into multiple parallel time-delay optical reservoirs, and the system computing efficiency is significantly improved through the parallel processing mechanism, increasing the processing speed of human motion recognition by three orders of magnitude. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of a human motion recognition method based on a traditional time-delay optical reservoir.

[0035] Figure 2 Example of KTH database preprocessing.

[0036] Figure 3 This is a schematic diagram of the fast human motion recognition method based on parallel time-delay optical reservoir calculation according to the present invention.

[0037] Figure 4 The impact of varying the number of virtual nodes N on the predictive performance of the reservoir system.

[0038] Figure 5 The effect of varying the number of parallel lasers K on the recognition performance of the reservoir system. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0040] A method for human motion recognition based on traditional time-delay optical reservoirs. Figure 1 Implement its flowchart. Before the input layer, preprocess the human motion videos from the KTH database. Figure 2The flowchart describes a process where a binary contour is generated using an attention mechanism, and the directional gradient histogram is calculated to extract feature vectors. Features from all frames of approximately 1.5 seconds of video are used as input data and injected serially into a time-delayed light reservoir to excite the nonlinear response of the reservoir laser. Human motion recognition is then achieved using its high-dimensional state space. However, the large data volume of a single video and the serial input method result in a low data processing rate.

[0041] To further improve the action recognition rate and processing speed, this invention proposes a fast human action recognition method based on parallel time-delay optical reservoir computation. Figure 3 Create a flowchart for it. Specifically:

[0042] First, histogram of oriented gradients (HARGs) features are extracted from all image frames of the action video dataset, and then the features are further dimensionality-reduced using principal component analysis.

[0043] After data preprocessing, by combining sampling and keyframe filtering techniques, feature data of 5 to 7 keyframes are extracted from the preprocessed data while maintaining the integrity of the motion information. The stitching method of the state sequence is changed from different regions of static images to different frames of related videos to reduce information redundancy.

[0044] To avoid the rate bottleneck caused by traditional serial input, multiple reservoirs are introduced to enable parallel input of multi-frame data in the reservoir construction. Simultaneously, a parallel semiconductor laser array is introduced within each reservoir to construct a high-dimensional state space under high-speed computing conditions. Different frame data are synchronously loaded into the phase of the light source through multiple modulators and injected into K parallel response lasers, each with the same number of virtual nodes.

[0045] In the output layer, the states of all virtual nodes of K lasers are merged for training and testing, and the output weights are calculated using the ridge regression algorithm.

[0046] The dynamic characteristics of each semiconductor laser are described by the Lang-Kobayashi equations:

[0047] (1)

[0048] (2)

[0049] in, and Let represent the slowly varying electric field intensity and carrier density of the i-th responding laser, respectively; It is the gain coefficient. This represents the feedback intensity of each responding laser. Indicates the light injection intensity. It is the linewidth factor. It is the carrier density during the transparent period. These are the photon lifetime and the carrier lifetime, respectively. It responds to the injection current of the laser. This represents the current factor of the i-th responding laser; in the input layer, it represents the preprocessed data. After sampling and holding for one operating period T, the resulting signal is multiplied by a time mask. This ensures the variability of the input signal across the virtual nodes of the reservoir layer; finally, the mask signal... The phase modulated to the light source is input into the response laser.

[0050] The mapping relationship from the input layer to the reservoir layer is determined by the relationship between the sample-and-hold time and the delay time. Here, a desynchronization method is used to adjust the sample-and-hold time T and the delay time of each laser. The relationship is:

[0051] (3)

[0052] At this point, a good balance is achieved between the system's required nonlinear dynamic characteristics and memory capability. Desynchronization makes the nonlinear characteristics of the laser delay loop more complex. The obtained node state correspondence is represented as:

[0053] (4)

[0054] In the formula, This represents the state of the i-th node within the n-th delay time. This indicates the nonlinear response of the laser. This represents the input mask signal, and k represents the multiple of the node interval between the delay time and the sampling time.

[0055] Because a desynchronization method is used, the state relationships between virtual nodes do not correspond one-to-one with the virtual node states of the previous delay period. The selected node states start with N nodes from the first delay, and then nodes are selected along each delay loop thereafter.

[0056] The output layer is trained using virtual nodes, whose state values ​​are taken from the output intensities of all K responding lasers; the virtual node state values ​​are obtained as follows:

[0057] (5)

[0058] The node state is taken from the state between two virtual nodes.

[0059] When all Then, the virtual node states of all reserve pool layers are concatenated, and the output weights of the reserve pools are calculated using ridge regression. The final output of the storage pool is obtained. The specific calculation formula is as follows:

[0060] (6)

[0061] (7)

[0062] in, The ridge regression coefficients are introduced to prevent overfitting. Represents the identity matrix.

[0063] After training, an output dataset with 6 dimensions is generated. Train a corresponding number of outputs based on the number of action categories (in this task, there are 6 actions); the output formula is defined as:

[0064] (8)

[0065] In the formula, j represents the output category. When the current input data is in category c, y c Take the higher value (e.g., 1), while other outputs y j If the value is low (e.g., 0), the input is assigned to the category c with the highest output value using the "winner-takes-all" strategy, and compared with the one-hot code in the input data. If the numbers are the same, the prediction is correct.

[0066] To quantify the performance of action recognition, the KTH human action dataset was used as a benchmark for testing, and the correct action recognition rate (Accuracy) was used as a metric for recognition accuracy.

[0067] (9)

[0068] in, This indicates the number of correctly identified actions. Indicates the number of test sets.

[0069] This invention improves action recognition accuracy by introducing a parallel reservoir computing architecture and expanding the spatial state dimension of the reservoir. Simultaneously, it combines video data compression and a multi-input mechanism to further enhance data processing speed. While maintaining the high bandwidth advantage of optical computing, it significantly improves system computational efficiency, providing a new solution for applications such as real-time image recognition and dynamic vision tasks.

[0070] Example:

[0071] This invention uses the KTH human motion dataset for testing. In the preprocessing stage, histogram of oriented gradients (HARGs) features are first extracted from all image frames of the motion video dataset, and then principal component analysis is used to reduce the dimensionality of these features. Combining sampling and keyframe selection techniques, feature data from keyframes is extracted from the preprocessed data; 5-7 frame segments can achieve performance comparable to a complete video (this example uses 5 frame segments). Subsequently, different frame data are synchronously loaded into the phase of the light source through multiple modulators and injected into K parallel response lasers, each with the same number of virtual nodes. In the output layer, the states of all virtual nodes of the K lasers are merged for training and testing, and the output weights are calculated using a ridge regression algorithm.

[0072] Figure 4 The figure shows a comparison of the computational performance of the proposed scheme and the traditional time-delay optical reservoir scheme in a human motion recognition task under different numbers of virtual nodes N. While maintaining the same training cost for both, i.e., the traditional scheme uses K times more virtual nodes in the semiconductor laser than the parallel laser (in this example, N is selected as N), the computational performance is compared. The average recognition accuracy of the proposed solution is significantly higher than that of traditional solutions. When the number of virtual nodes N increases from 200 to 1700, the recognition accuracy improves from 0.79 to 0.93. The results demonstrate that employing a multi-input mechanism combined with a parallel semiconductor laser array can effectively improve the overall performance of the reservoir computation, and the operating speed is approximately three orders of magnitude faster than the traditional time-delay optical reservoir architecture.

[0073] Furthermore, this experiment explored the impact of the number of parallel lasers, K, on ​​the system's recognition performance, while keeping the training cost constant. For example... Figure 5 As shown, with other conditions remaining constant, as K increases, the accuracy of human motion recognition first rises and then falls. At that time, it achieved good recognition results, with an accuracy rate of 0.95, and the running speed was also further improved.

[0074] However, as K increases further, the recognition accuracy decreases. This may be because the input data has 111 dimensions, and when... At that time, the number of virtual nodes in a single reservoir layer was only 160. This meant that when the input signal underwent nonlinear projection mapping on the high-dimensional state space of the reservoir, the input in the task could not be effectively separated, thus affecting the final recognition accuracy. Nevertheless, when... At the same time, the recognition accuracy is on par with the traditional time-delay optical reservoir architecture, while the operating speed is improved by about three orders of magnitude.

[0075] In summary, this invention performs dimensionality reduction on high-dimensional video data through feature extraction and principal component analysis. Then, utilizing a multi-input mechanism, the dimensionality-reduced video data is injected into multiple parallel time-delay optical storage pools, achieving parallel nonlinear mapping and computation of the data. Compared to traditional time-delay optical storage pool architectures, this invention improves the data computation rate by three orders of magnitude.

Claims

1. A method for fast human motion recognition based on parallel time-delay optical reservoir computation, characterized in that, Specifically: First, histogram of oriented gradients (HARQs) features are extracted from all image frames of the action video dataset, and then the features are further dimensionality reduced by principal component analysis. After data preprocessing, by combining sampling and keyframe filtering techniques, feature data of 5 to 7 keyframes are extracted from the preprocessed data, and the splicing method of the state sequence is adjusted from different regions of static images to different frames of related videos. Subsequently, multiple reservoirs were introduced to achieve parallel input of multi-frame data, and a parallel semiconductor laser array was introduced inside a single reservoir to construct a high-dimensional state space under high-speed computing conditions. Different frame data are synchronously loaded into the phase of the light source through multiple modulators and injected into K parallel response lasers, each response laser having the same number of virtual nodes. In the output layer, the states of all virtual nodes of K lasers are merged for training and testing, and the output weights are calculated using the ridge regression algorithm.

2. The method for fast human motion recognition based on parallel time-delay optical reservoir computation according to claim 1, characterized in that, The dynamic characteristics of the semiconductor laser are described by the Lang-Kobayashi equation: （1） （2） in, and Let represent the slowly varying electric field intensity and carrier density of the i-th responding laser, respectively; It is the gain coefficient. This represents the feedback intensity of each responding laser. Indicates the light injection intensity. It is the linewidth factor. It is the carrier density during the transparent period. These are the photon lifetime and the carrier lifetime, respectively. It responds to the injection current of the laser. This represents the current factor of the i-th responding laser; in the input layer, it represents the preprocessed data. After sampling and holding for one operating period T, the resulting signal is multiplied by a time mask. This ensures the variability of the input signal across the virtual nodes of the reservoir layer; finally, the mask signal... The phase modulated to the light source is input into the response laser; The mapping relationship from the input layer to the reservoir layer is determined by the relationship between the sample-and-hold time and the delay time. Here, a desynchronization method is used to adjust the sample-and-hold time T and the delay time of each laser. The relationship is: （3） The obtained node state correspondence is represented as follows: （4） In the formula, This represents the state of the i-th node within the n-th delay time. This indicates the nonlinear response of the laser. This represents the input mask signal, where k represents the multiple of the node interval between the delay time and the sampling time. The output layer is trained using virtual nodes, whose state values ​​are taken from the output intensities of all K responding lasers; the virtual node state values ​​are obtained as follows: （5） The node state is taken from the state between two virtual nodes; When all Then, the virtual node states of all reserve pool layers are concatenated, and the output weights of the reserve pools are calculated using ridge regression. The final output of the storage pool is obtained. The specific calculation formula is as follows: （6） （7） in, The ridge regression coefficients are introduced to prevent overfitting. Represents the identity matrix; After training, an output dataset with 6 dimensions is generated. Train a corresponding number of outputs based on the number of action categories; the output formula is defined as: （8） In the formula, j represents the output category. When the current input data is in category c, y c Take the higher value, and take the other output y. j The lower value is selected, and the "winner-takes-all" strategy is used to assign the input to the category c with the highest output value. The value is then compared with the one-hot code in the input data. If the numbers are the same, the prediction is correct.

3. The method for fast human motion recognition based on parallel time-delay optical reservoir computation according to claim 1, characterized in that, To quantify the performance of action recognition, the KTH human action dataset was used as a benchmark for testing, with the correct action recognition rate as the metric for recognition accuracy. （9） in, This indicates the number of correctly identified actions. Indicates the number of test sets.

Images