Human body gender recognition method and system based on space-time diagram convolutional neural network
Through the spatiotemporal graph convolution neural network and the improved STGCN model, human body skeleton spatiotemporal graph data is used to identify human gender, solving the problems of large scale of data sets and the influence of interference factors in the existing technology, and achieving efficient and accurate recognition of human gender.
Patent Information
- Application Number
- CN202310623594.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
The existing human gender recognition method based on image data has low efficiency, low accuracy and reduced accuracy due to the huge data set size and irrelevant interference factors.
The human gender recognition method based on the spatiotemporal graph convolution neural network is adopted. By obtaining video data, preprocessing is performed to obtain the spatiotemporal graph data of the human body skeleton, and classification is used using the improved STGCN network model. The improved fully connected layer is a binary classification function. Shield background and light interference factors to improve training efficiency and accuracy.
The model training time is reduced, the model accuracy and recognition accuracy are improved, and the system's robustness and efficiency are enhanced through transfer learning and data normalization processing.
Smart Images

Figure CN119027974A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of human gender recognition, and in particular relates to a human gender recognition method and system based on a spatiotemporal graph convolutional neural network. Background Art
[0002] At present, the research on human gender recognition in the field of deep learning mostly adopts the method of collecting images or videos and based on convolutional neural network (CNN). However, due to the large storage capacity of images and videos, the storage volume of data sets required for model training is also huge. It will inevitably take a lot of time to train the model using CNN method with such a large amount of data. Since a single image and video has a large storage capacity, and due to the limitations of hardware equipment such as storage devices and GPU devices and the time-consuming training model, the scale of gender recognition data sets constructed using images and videos is generally small, and the use of images and videos is more easily affected by environmental complexity and background diversity. The accuracy and robustness of the model trained with such data sets are not strong.
[0003] For example, in the 58th volume, issue 16 of Laser and Optoelectronics Progress, the authors Yu Fusheng, Yu Jiang, etc., titled "Gender Classification of Iris Images Based on Residual Network", used human iris image data to implement gender classification using residual network. The iris image dataset used was collected by the Shenzhen Institute of Advanced Technology of the Chinese Academy of Sciences, which contains 3,200 iris images with a resolution of 640×480. The total size of the image files is about 4GB, which is only a small-scale dataset. In the 43rd volume, issue 5 of Computer Engineering and Science, the authors Chen Wenbing, Li Yulin, etc., titled "A Model of Age and Gender Recognition Based on CNN-SE-ELM", introduced a method of using face images to implement gender recognition based on CNN theory. The datasets used in the experiment are IMDB-WIKI face dataset and Adience face dataset. The IMDB-WIKI face dataset contains two parts with a total of 523,051 face images. The Adience face dataset is a collection of photos uploaded by volunteers through their mobile phones. There are 26,580 images in total. Since they are taken by mobile phones, there must be interference from factors such as lighting, noise, posture, and occlusion. In the 24th issue of Modern Computer, the authors Yin Ziwei and Chen Shurong wrote an article called "Face Gender Recognition Based on Dual-Way Convolutional Neural Network". By constructing a dual-way convolutional neural network, the convolutional neural network was improved and the interference problem of some irrelevant factors was solved. The datasets used for the training model are the Adience dataset, the Wiki dataset (a dataset containing nearly 460,000 images of size 250×300) and the FERET dataset (a dataset containing 1,400 face images of size 80×80).
[0004] Based on the literature research in the field of gender recognition, it can be seen that the methods used in current research on gender recognition are mostly based on the theory of convolutional neural networks (CNN) and use image data sets to conduct experiments and train models. The scale of such data sets is very large, and the amount of data is very large. At the same time, the image data contains too many interference factors (lighting, occlusion, etc.). Using such a large data set to train a model also places very high requirements on hardware such as GPUs, and the model training time is also very long. In addition, due to the existence of irrelevant interference factors, the accuracy of the trained model will also be affected to a certain extent, and then the accuracy of the results obtained by using this trained model will also be reduced accordingly. Summary of the invention
[0005] The purpose of the present invention is to provide a human gender recognition method and system based on a spatiotemporal graph convolutional neural network, so as to solve the problem that the existing method of obtaining human gender recognition results based on image data input into a network model is low in training efficiency due to the large capacity of the data set used in training the network model, and the existence of irrelevant interference factors makes the model obtained by training low in accuracy, and then the accuracy of the results obtained by using the trained model is correspondingly reduced.
[0006] To solve the above technical problems, the present invention provides a human gender recognition method based on spatiotemporal graph convolutional neural network, comprising the following steps:
[0007] 1) Obtaining video data for human gender recognition;
[0008] 2) preprocessing the video data to obtain human skeleton spatiotemporal graph data corresponding to the video data;
[0009] 3) The human skeleton spatiotemporal graph data is first normalized and then input into the trained improved STGCN network model to obtain the classification result of the human skeleton spatiotemporal graph data; the improvement of the improved STGCN network model is that the fully connected layer used is a fully connected layer with a binary classification function;
[0010] 4) Determine the gender of the human body based on the classification results.
[0011] The beneficial effects are as follows: the method of the present invention finds that due to the physiological differences between men and women, there are greater differences in the skeleton proportions and movement changes of different parts of the male and female bodies. Therefore, the process of judging the gender of a human body based on the human skeleton can be realized by identifying the skeleton proportions or movement changes of different parts of the human body. The present invention processes the acquired video data to obtain the corresponding human skeleton spatiotemporal graph, and obtains the classification result of the human skeleton spatiotemporal graph by improving the STGCN network model. The present invention improves the STGCN network model by improving the fully connected layer into a fully connected layer with a binary classification function, and then after the human skeleton spatiotemporal graph is input into the improved STGCN network model, the output classification result is one of the two classification results (that is, one of the two classification results corresponds to one human gender), so the human gender can be judged based on the obtained classification result. Because the present invention adopts the improved STGCN network model to realize human gender recognition, and the improved STGCN network model uses a human skeleton spatiotemporal graph data set when training, and the human skeleton spatiotemporal graph data set has a small storage capacity compared to the existing image data (such as iris image data or face image data) under the same number of data sets, so based on the human skeleton spatiotemporal graph data set, the time consumed in the model training process is also reduced, and the human skeleton and joint trajectory are robust to illumination changes and scene changes, that is, the human skeleton spatiotemporal graph data can shield the influence of irrelevant factors such as background and light, so not only when training the improved STGCN network model, the training efficiency and the accuracy of the model can be improved, but also when using the trained improved STGCN network model for human gender recognition, the accuracy of recognition can be guaranteed. The present invention also takes into account that since different nodes of the human skeleton spatiotemporal graph need to share weights, the coherence of the input data must be maintained, so the present invention must first perform a data normalization process before the human skeleton spatiotemporal graph data is input into the model, and the data is normalized, thereby ensuring the accuracy of the final output result.
[0012] Furthermore, in step 2), the preprocessing is: performing posture estimation on the video data by using a posture estimation algorithm.
[0013] The present invention uses a simple posture estimation algorithm to realize the process of converting video data into human skeleton spatiotemporal graph data, that is, the process of obtaining human skeleton spatiotemporal graph data in a simple way to improve the recognition efficiency of the entire human gender recognition process.
[0014] Furthermore, when training the improved STGCN network model, the data set used is: video data is selected from the existing RGB video data set, and preprocessing is performed to convert the selected video data into corresponding human skeleton spatiotemporal graph data, and the obtained human skeleton spatiotemporal graph data is used as the data set used.
[0015] Since there is no large-scale public data set for gender recognition, the present invention needs to produce the data set required in the present invention when training the improved STGCN network model. The data set required in the present invention is based on the data selected from the existing RGB video data set, and then the existing RGB video data set is converted into the human skeleton spatiotemporal graph data set required by the present invention after corresponding processing, and then the improved STGCN network model is trained using the data set. Because the present invention performs corresponding processing on the existing data set to obtain the data set used by the present invention, the method of obtaining the data set used for training in the present invention is more convenient than the method of re-collecting data to form a data set, thereby reducing the acquisition time of the data set used for training and improving the efficiency of the entire process from obtaining the training set to completing the model training.
[0016] Furthermore, when training the improved STGCN network model, the weight parameters of the remaining network layers in the improved STGCN network model except the fully connected layer are first determined through transfer learning, and then the improved STGCN network model is trained.
[0017] The biggest advantage of transfer learning is that it can use small sample data to train the model while ensuring that the model accuracy remains unchanged or slightly improved. This improves the learning efficiency of the model, avoids learning the model from scratch, and greatly reduces redundant work. In addition, the present invention takes into account the differences in learning tasks, and loads the weight parameters of other network layers except the fully connected layer when loading the initial weight parameters. During transfer learning, only the weight parameters of the fully connected layer are trained.
[0018] Furthermore, during training, the improved STGCN network model optimizes the parameters of the spatiotemporal convolutional layer specified in the improved STGCN network model through transfer learning.
[0019] The present invention only trains the weight parameters of the fully connected layer during transfer learning, but in the process of parameter optimization, it also trains and optimizes the weight parameters of the parameters of the specified spatiotemporal convolutional layer, that is, only the weight parameters of the non-specified part of the existing model network are loaded for transfer learning, so as to improve the accuracy of the network model by optimizing the parameters of the specified spatiotemporal convolutional layer, and further improve the accuracy of the human gender recognition results obtained using this model.
[0020] In order to solve the above technical problems, the present invention also provides a human gender recognition system based on a spatiotemporal graph convolutional neural network, including a detection module for obtaining video data for human gender recognition, and a processing module, wherein the detection module is also used to transmit the acquired video data to the processing module, and the processing module is used to pre-process the video data to obtain human skeleton spatiotemporal graph data corresponding to the video data; and the human skeleton spatiotemporal graph data is first normalized and then input into a trained improved STGCN network model to obtain a classification result of the human skeleton spatiotemporal graph data, and then the human gender is judged according to the obtained classification result; the improvement of the improved STGCN network model is that the fully connected layer used is a fully connected layer with a binary classification function.
[0021] The beneficial effects are as follows: the present invention finds that due to the physiological differences between men and women, there are greater differences in the skeleton proportions and movement changes of different parts of the male and female bodies. Therefore, the process of judging the gender of a human body based on the human skeleton can be realized by identifying the skeleton proportions or movement changes of different parts of the human body. The present invention processes the acquired video data to obtain the corresponding human skeleton spatiotemporal graph, and obtains the classification result of the human skeleton spatiotemporal graph by improving the STGCN network model. The present invention improves the STGCN network model by improving the fully connected layer to a fully connected layer with a binary classification function, and then after the human skeleton spatiotemporal graph is input into the improved STGCN network model, the output classification result is one of the two classification results (that is, one of the two classification results corresponds to one human gender), so the human gender can be judged based on the obtained classification result. Because the present invention adopts the improved STGCN network model to realize human gender recognition, and the improved STGCN network model uses a human skeleton spatiotemporal graph data set when training, and the human skeleton spatiotemporal graph data set has a small storage capacity compared to the existing image data (such as iris image data or face image data) under the same number of data sets, so based on the human skeleton spatiotemporal graph data set, the time consumed in the model training process is also reduced, and the human skeleton and joint trajectory are robust to illumination changes and scene changes, that is, the human skeleton spatiotemporal graph data can shield the influence of irrelevant factors such as background and light, so not only when training the improved STGCN network model, the training efficiency and the accuracy of the model can be improved, but also when using the trained improved STGCN network model for human gender recognition, the accuracy of recognition can be guaranteed. The present invention also takes into account that since different nodes of the human skeleton spatiotemporal graph need to share weights, the coherence of the input data must be maintained, so the present invention must first perform a data normalization process before the human skeleton spatiotemporal graph data is input into the model, and the data is normalized, thereby ensuring the accuracy of the final output result.
[0022] Furthermore, the preprocessing is: performing posture estimation on the video data by using a posture estimation algorithm.
[0023] The present invention uses a simple posture estimation algorithm to realize the process of converting video data into human skeleton spatiotemporal graph data, that is, the process of obtaining human skeleton spatiotemporal graph data in a simple way to improve the recognition efficiency of the entire human gender recognition process.
[0024] Furthermore, when training the improved STGCN network model, the data set used is: video data is selected from the existing RGB video data set, and preprocessing is performed to convert the selected video data into corresponding human skeleton spatiotemporal graph data, and the obtained human skeleton spatiotemporal graph data is used as the data set used.
[0025] Since there is no large-scale public data set for gender recognition, the present invention needs to produce the data set required in the present invention when training the improved STGCN network model. The data set required in the present invention is based on the data selected from the existing RGB video data set, and then the existing RGB video data set is converted into the human skeleton spatiotemporal graph data set required by the present invention after corresponding processing, and then the improved STGCN network model is trained using the data set. Because the present invention performs corresponding processing on the existing data set to obtain the data set used by the present invention, the method of obtaining the data set used for training in the present invention is more convenient than the method of re-collecting data to form a data set, thereby reducing the acquisition time of the data set used for training and improving the efficiency of the entire process from obtaining the training set to completing the model training.
[0026] Furthermore, when training the improved STGCN network model, the weight parameters of the remaining network layers in the improved STGCN network model except the fully connected layer are first determined through transfer learning, and then the improved STGCN network model is trained.
[0027] The biggest advantage of transfer learning is that it can use small sample data to train the model while ensuring that the model accuracy remains unchanged or slightly improved. This improves the learning efficiency of the model, avoids learning the model from scratch, and greatly reduces redundant work. In addition, the present invention takes into account the differences in learning tasks, and loads the weight parameters of other network layers except the fully connected layer when loading the initial weight parameters. During transfer learning, only the weight parameters of the fully connected layer are trained.
[0028] Furthermore, during training, the improved STGCN network model optimizes the parameters of the spatiotemporal convolutional layer specified in the improved STGCN network model through transfer learning.
[0029] The present invention only trains the weight parameters of the fully connected layer during transfer learning, but in the process of parameter optimization, it also trains and optimizes the weight parameters of the parameters of the specified spatiotemporal convolutional layer, that is, only the weight parameters of the non-specified part of the existing model network are loaded for transfer learning, so as to improve the accuracy of the network model by optimizing the parameters of the specified spatiotemporal convolutional layer, and further improve the accuracy of the human gender recognition results obtained using this model. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of the space-time diagram of the human skeleton;
[0031] Figure 2 It is a flow chart of the training process of the improved STGCN network model used in the present invention;
[0032] Figure 3 is a schematic diagram of a human skeleton diagram obtained after preprocessing of the present invention;
[0033] Figure 4 It is a structural diagram of the improved STGCN network model of the present invention. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.
[0035] Example of human gender recognition system based on spatiotemporal graph convolutional neural network:
[0036] The system of this embodiment takes into account the fact that due to the physiological differences between men and women, there are large differences in the skeleton proportions and movement changes of different parts of the male and female bodies. By extracting these different movement features of men and women from the human skeleton spatiotemporal graph for gender recognition, the required data set storage capacity is small, and only information with a large correlation with gender, such as skeleton nodes and time, is retained. Therefore, under the condition of the same number of data sets, its storage capacity is greatly reduced, and the time spent in the model training process is also reduced. Therefore, the system of this embodiment constructs a human skeleton spatiotemporal graph data set, such as Figure 1 It is an intuitive display of the human skeleton space-time graph. By improving the space-time graph convolutional neural network model (i.e., improving the STGCN network model), the function of human gender recognition is realized. Space-time graph convolutional neural network (STGCN) is a type of graph neural network (GCN). The STGCN model is mainly used for deep learning tasks of space-time graph prediction. Human gender recognition methods based on the STGCN model are widely used in Figure 1 Gender recognition is realized on the spatiotemporal graph of the human skeleton, which shields the influence of irrelevant factors such as background and light. In addition, the data storage required for training the model is small, the time consumed is short, and the model is more robust.
[0037] The structure of the improved STGCN network model used by the system in this embodiment is shown as follows: Figure 4 As shown in the figure, the entire ST-GCN network consists of 9 different spatiotemporal convolutional layers. Different from the CNN model, the spatiotemporal convolutional layer is a new type of convolutional layer. Each spatiotemporal convolutional layer is divided into 1 GCN graph convolutional layer unit and 3 TCN temporal convolutional layer units. The spatiotemporal graph convolutional network is also used for action recognition because of its characteristics of human skeleton spatiotemporal feature extraction. Based on the ST-GCN network structure for action recognition, the fully connected layer of its network is modified, so that the fully connected layer of the original multi-classification ST-GCN network is changed to the fully connected layer of the binary classification ST-GCN network, thereby realizing the function of gender recognition. Specifically, Figure 4 As shown, FC represents the fully connected layer, which is to connect the first digits of all the output multi-dimensional vectors to convert them into one-dimensional vectors, and then change the fully connected layer of the multi-classification ST-GCN network into the fully connected layer of the binary ST-GCN network. That is, the present invention improves the full connection of the ST-GCN network model into a fully connected layer with binary classification function, and then inputs the human skeleton spatiotemporal graph into the improved STGCN network model. The output classification result is one of the two classification results (that is, one of the two classification results corresponds to one human gender). Therefore, the human gender can be judged based on the obtained classification result. And as Figure 4 As shown in the figure, the human skeleton spatiotemporal graph data must undergo a data normalization process (Batch Normalization, BN) before being input into the model. This is because different nodes of the human skeleton spatiotemporal graph need to share weights, so the consistency of the input data must be maintained. The middle 9 layers are ST-GCN convolution units, which are mainly used to extract gender features. The 4th and 7th convolution units contain average pooling layers to more accurately extract the features of the human skeleton spatiotemporal graph, and then the purpose of gender classification is achieved through the fully connected layer (FC) and SoftMax function.
[0038] Based on the improved STGCN network model of this embodiment, Figure 2 As shown, when training the improved STGCN network model, the following steps are included:
[0039] 1) Environment Construction: The hardware environment and software environment used in this embodiment are shown in Table 1 and Table 2.
[0040] Table 1 Hardware environment parameter configuration
[0041]
[0042] Table 2 Detailed description of experimental software environment
[0043]
[0044] 2) Video data acquisition. The training of the model is inseparable from a large amount of data. Since there is no large-scale public gender recognition data set, according to the experimental needs, the original RGB video data set with obvious differences in male and female characteristics in the NTU_RGB+D data set is selected and manually divided into male and female categories. In this embodiment, considering that there is no gender recognition data set in video format in the existing data set, it is necessary to make a data set before the model training process can be carried out. In this embodiment, the process of making a data set is not to collect corresponding video data from the existing environment to make a data set, but to use the existing video data set, that is, to select samples with obvious male and female characteristics from the existing data set for action recognition, and then make the data set used in this embodiment based on the sample. Because this embodiment is to process the existing data set accordingly to obtain the data set used in this embodiment, compared with the method of re-collecting data to form a data set, the method of obtaining the data set used for training in this embodiment is more convenient, thereby reducing the acquisition time of the data set used for training, and improving the efficiency of the entire process from obtaining the training set to the end of model training. The resolution of all video data is 1920*1080 and the frame rate is 30FPS. In this embodiment, the gender recognition data set is made using 10,000 video samples as a training set (5,000 male and female samples each), 2,000 video samples as a validation set (1,000 male and female samples each), and 800 as a test set (400 male and female samples each).
[0045] 3) Video data preprocessing. The preprocessing process uses the OpenPose algorithm to preprocess the NTU_RGB+D original RGB dataset, extract the human skeleton spatiotemporal graph, and generate a JSON file containing the human skeleton spatiotemporal graph. According to most current deep learning methods, for model training in the visual direction, most researchers directly use RGB images or videos to train CNN models without much preprocessing of the dataset. The present invention uses the OpenPose posture estimation algorithm to process each sample in the RGB video dataset into a human skeleton spatiotemporal graph containing 18 human skeleton key points, and then form a human skeleton spatiotemporal graph dataset to retrain the ST-GCN model.
[0046] Since there is no large-scale public data set for gender recognition, the present invention needs to produce the data set required in the present invention when training the improved STGCN network model. The data set required in the present invention is based on the data selected from the existing RGB video data set, and then the existing RGB video data set is converted into the human skeleton spatiotemporal graph data set required by the present invention after corresponding processing, and then the improved STGCN network model is trained using the data set. Because the present invention performs corresponding processing on the existing data set to obtain the data set used by the present invention, the method of obtaining the data set used for training in the present invention is more convenient than the method of re-collecting data to form a data set, thereby reducing the acquisition time of the data set used for training and improving the efficiency of the entire process from obtaining the training set to completing the model training.
[0047] like Figure 3 This is a schematic diagram of the skeleton nodes of 18 different parts of the human body obtained by preprocessing the data set using the OpenPose posture estimation algorithm, such as Figure 3 As shown, each skeleton node contains its index coordinates and confidence information.
[0048] Table 3 shows the specifications of the gender recognition human skeleton spatiotemporal graph dataset constructed by the experiment.
[0049] Table 3 Dataset specifications
[0050]
[0051] From Table 3, it can be concluded that the use of the human skeleton spatiotemporal graph dataset has a very obvious advantage: small data storage capacity. According to the analysis of the datasets used in the prior art in the background technology, the storage capacity of the RGB video dataset of the same size is more than ten times that of the human skeleton spatiotemporal graph dataset. Therefore, the use of graph convolutional networks to train the ST-GCN model on the human skeleton spatiotemporal graph dataset greatly reduces the data storage capacity, which is conducive to expanding the size of the dataset and thus improving the accuracy of the model. Secondly, when using a smaller capacity dataset for training, the key points of gender recognition are captured, irrelevant feature data is shielded, and the data is more targeted, so the time spent on model training is greatly reduced.
[0052] 4) Build a network model. Use the human skeleton spatiotemporal graph dataset obtained in step 3) to train the improved STGCN model.
[0053] 5) Parameter optimization: Different from most gender recognition methods, this embodiment uses transfer learning technology when training the model and adjusting the network hyperparameters. Since the ST-GCN (i.e., STGCN) network model is also used in the field of action recognition, in addition to conventional hyperparameters such as batch size, learning rate, optimizer, etc., the present invention uses the ST-GCN human action recognition task trained on the Kinetics large public data set in the article "Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action" by Sijie Yan, Yuanjun Xiong, etc. from AAAI 2018 (Proceedings of the International Top Conference on Artificial Intelligence). The model is used for transfer learning. Taking into account the difference in deep learning recognition tasks (i.e., the above-mentioned "Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action" is action recognition, while in this embodiment it is gender recognition), the model convolutional layer weights are loaded when the model is loaded, the convolutional layer model is kept unchanged during training, and only the fully connected layer is trained using the transfer learning method, and then a model for gender recognition is obtained through training. Deep learning is to obtain the parameters of the model by inputting a large amount of data and giving the output results and repeating this process many times, wherein the model is saved in a file in .pb format and exists in the form of a multidimensional vector. The more complex the model, the more complex the parameters, and the parameters obtained by each training of a model are different. Therefore, this embodiment adopts the model stored in the existing .pd format file, uses the transfer learning method to train only the fully connected layer, and then obtains the model for gender recognition through training. Therefore, the improvement of this embodiment is the improvement of the training method, that is, the improvement of a model method, not the specific modification of certain weight parameters.
[0054] The biggest advantage of transfer learning is that it can use small sample data to train the model while ensuring that the model accuracy remains unchanged or slightly improved. This improves the learning efficiency of the model, avoids the model from learning from scratch, and greatly reduces redundant work.
[0055] In the actual parameter optimization process, in addition to training only the parameters of the fully connected layer, the present invention also trains the weight parameters of the last few ST-GCN unit layers of the network model (such as the last two convolutional layers and the fully connected layer), that is, only the weight parameters of the first few layers of the existing model network are loaded for transfer learning, so as to compare the model accuracy.
[0056] 6) Optimal model and model evaluation. Finally, through a large number of parameter optimization experiments, the optimal parameter combination shown in Table 4 was obtained.
[0057] Table 4 Optimal model training parameter combinations
[0058]
[0059] The optimal model is predicted on the test set. Table 5 shows the loss value and accuracy of the model obtained by predicting the model accuracy on the test set.
[0060] Table 5 Optimal model evaluation results
[0061]
[0062] It can be seen that the accuracy of the optimal gender recognition model obtained by training reaches nearly 90%. The model trained using such a small-capacity data set achieves such accuracy, so the system of this embodiment has strong practical value.
[0063] Because the present invention adopts the improved STGCN network model to realize human gender recognition, and the improved STGCN network model uses a human skeleton spatiotemporal graph data set when training, and the human skeleton spatiotemporal graph data set has a small storage capacity compared to the existing image data (such as iris image data or face image data) under the same number of data sets, so based on the human skeleton spatiotemporal graph data set, the time consumed in the model training process is also reduced, and the human skeleton and joint trajectory are robust to illumination changes and scene changes, that is, the human skeleton spatiotemporal graph data can shield the influence of irrelevant factors such as background and light, so not only when training the improved STGCN network model, the training efficiency and the accuracy of the model can be improved, but also when using the trained improved STGCN network model for human gender recognition, the accuracy of recognition can be guaranteed. The present invention also takes into account that since different nodes of the human skeleton spatiotemporal graph need to share weights, the coherence of the input data must be maintained, so the present invention must first perform a data normalization process before the human skeleton spatiotemporal graph data is input into the model, and the data is normalized, thereby ensuring the accuracy of the final output result.
[0064] Embodiment of a human gender recognition method based on spatiotemporal graph convolutional neural network:
[0065] The method of this embodiment obtains the video data for human gender recognition, and pre-processes the video data to obtain the human skeleton spatiotemporal graph data corresponding to the video data, and then normalizes the human skeleton spatiotemporal graph data, and then inputs it into the trained improved STGCN network model to obtain the classification result of the human skeleton spatiotemporal graph data, and then judges the human gender according to the obtained classification result. The specific structure of the improved STGCN network model and the steps of the human gender recognition method based on the spatiotemporal graph convolutional neural network have been introduced in detail in the embodiment of the human gender recognition system based on the spatiotemporal graph convolutional neural network, and will not be repeated here.
[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. The patent protection scope of the present invention shall be based on the claims. All equivalent structural changes made using the contents of the description and drawings of the present invention should also be included in the protection scope of the present invention.
Claims
1. A human gender recognition method based on spatiotemporal graph convolutional neural network, characterized in that: The steps include: 1) Obtaining video data for human gender recognition; 2) preprocessing the video data to obtain human skeleton spatiotemporal graph data corresponding to the video data; 3) The human skeleton spatiotemporal graph data is first normalized and then input into the trained improved STGCN network model to obtain the classification result of the human skeleton spatiotemporal graph data; the improvement of the improved STGCN network model is that the fully connected layer used is a fully connected layer with a binary classification function; 4) Determine the gender of the human body based on the classification results.
2. The method for human gender recognition based on spatiotemporal graph convolutional neural network according to claim 1, characterized in that: In step 2), the preprocessing is: performing posture estimation on the video data by using a posture estimation algorithm.
3. The method for human gender recognition based on spatiotemporal graph convolutional neural network according to claim 1, characterized in that: When training the improved STGCN network model, the data set used is: video data is selected from the existing RGB video data set, preprocessing is performed to convert the selected video data into corresponding human skeleton spatiotemporal graph data, and the obtained human skeleton spatiotemporal graph data is used as the data set used.
4. The method for human gender recognition based on spatiotemporal graph convolutional neural network according to claim 1, characterized in that: When training the improved STGCN network model, the weight parameters of the remaining network layers in the improved STGCN network model except the fully connected layer are first determined through transfer learning, and then the improved STGCN network model is trained.
5. The method for human gender recognition based on spatiotemporal graph convolutional neural network according to claim 4 is characterized in that: During the training of the improved STGCN network model, the parameters of the spatiotemporal convolutional layer specified in the improved STGCN network model are optimized through transfer learning.
6. A human gender recognition system based on a spatiotemporal graph convolutional neural network, comprising a detection module for acquiring video data to be recognized as human gender, and a processing module, wherein the detection module is further configured to transmit the acquired video data to the processing module, characterized in that: The processing module is used to pre-process the video data to obtain the human skeleton spatiotemporal graph data corresponding to the video data; and the human skeleton spatiotemporal graph data is first normalized and then input into the trained improved STGCN network model to obtain the classification result of the human skeleton spatiotemporal graph data, and then the human gender is judged according to the obtained classification result; The improvement of the improved STGCN network model is that the fully connected layer used is a fully connected layer with a binary classification function.
7. The human gender recognition system based on spatiotemporal graph convolutional neural network according to claim 6 is characterized in that: The preprocessing is: performing posture estimation on the video data by using a posture estimation algorithm.
8. The human gender recognition system based on spatiotemporal graph convolutional neural network according to claim 6, characterized in that: When training the improved STGCN network model, the data set used is: video data is selected from the existing RGB video data set, preprocessing is performed to convert the selected video data into corresponding human skeleton spatiotemporal graph data, and the obtained human skeleton spatiotemporal graph data is used as the data set used.
9. The human gender recognition system based on spatiotemporal graph convolutional neural network according to claim 6, characterized in that: When training the improved STGCN network model, the weight parameters of the remaining network layers in the improved STGCN network model except the fully connected layer are first determined through transfer learning, and then the improved STGCN network model is trained.
10. The human gender recognition system based on spatiotemporal graph convolutional neural network according to claim 9, characterized in that: During the training of the improved STGCN network model, the parameters of the spatiotemporal convolutional layer specified in the improved STGCN network model are optimized through transfer learning.
