Gesture recognition detection method and system based on deep learning

Abstract

The application discloses a gesture recognition detection method and system based on deep learning, relates to the technical field of image recognition, and lays a good foundation for subsequent model training by collecting a panoramic picture of a patrol scene and performing pretreatment to generate a data set, then obtains a usable segmentation model by training an image segmentation model on the pretreated data set, aligns segmented gesture pictures and divides a training set and a verification set, and then establishes a gesture recognition integrated model, replaces a first full connection layer of a CNN module with a GRU module to form a CGRU module, and performs gesture recognition with the help of a GPR module. The whole process is trained and verified by using the training set and the verification set, so that gestures in the panoramic picture of the patrol scene can be effectively and accurately obtained, the efficiency and accuracy of gesture recognition detection are improved, and the demand of patrol and other related scenes for gesture recognition is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and more particularly to a gesture recognition detection method and system based on deep learning. BACKGROUND

[0002] At present, with the continuous development of human-computer interaction technology, gesture recognition as a natural and intuitive interaction method has a wide application prospect in many fields, such as intelligent security monitoring, virtual reality / augmented reality interaction, smart home control, etc. Traditional gesture recognition methods are often based on hand-crafted features such as contour features and texture features, and combined with machine learning classifiers for recognition, but these methods have poor recognition accuracy and robustness in the face of complex backgrounds, diversified gesture poses, and different lighting conditions.

[0003] Deep learning technology has achieved outstanding results in image recognition due to its powerful ability to automatically learn features. Applying deep learning to gesture recognition detection can more effectively extract gesture features and improve recognition accuracy and stability. Therefore, how to provide a recognition method based on deep learning to improve the accuracy of gesture recognition is a problem that needs to be solved by those skilled in the art. SUMMARY

[0004] Therefore, the present application provides a gesture recognition detection method and system based on deep learning to solve the problems in the background art.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] A gesture recognition detection method based on deep learning, comprising:

[0007] Collecting a panoramic picture of a patrol scene and preprocessing the collected panoramic picture of the patrol scene, including mask labeling and data enhancement of the panoramic picture of the patrol scene, to generate a data set;

[0008] Training an image segmentation model on the preprocessed data set to obtain a trained segmentation model;

[0009] Sending the panoramic picture of the patrol scene for gesture feature recognition into the trained segmentation model and aligning the segmented gesture picture;

[0010] Dividing the aligned gesture image into a training set and a validation set;

[0011] Establishing a gesture recognition integrated model, in which the first fully connected layer of the CNN module is replaced by the GRU module to form the CGRU module, and the gesture features output by the CGRU module are regarded as input data for gesture recognition in the GPR module.

[0012] training the gesture recognition ensemble model using the training set to obtain a trained gesture recognition ensemble model;

[0013] inputting the verification set into the trained gesture recognition ensemble model to obtain the gesture in the inspection scene panoramic picture.

[0014] Optionally, the specific method for preprocessing the collected inspection scene panoramic picture is:

[0015] using a labeling tool to label the gesture edge in the inspection scene panoramic picture, and then exporting the labeling content as a json file;

[0016] performing image enhancement on the labeled image, and the enhancement methods include random sharpness enhancement, random color enhancement, random contrast enhancement, random brightness enhancement, and picture fusion, so that one labeled picture is changed into five pictures after enhancement.

[0017] Optionally, the CNN module comprises a hidden layer and a full connection layer; the first hidden layer comprises a convolution layer, a normalization layer and a nonlinear activation layer, a feature detector is used in each convolution layer, and data batch normalization is performed after convolution; the second hidden layer is an average pooling layer; and a hyperbolic tangent function is used to activate the full connection layer.

[0018] Optionally, the function used by the nonlinear activation layer is:

[0019] LReLU(x)={x(x≥0);αx(x<0)};

[0020] wherein x represents an input value, and a is a parameter, and a=0.01.

[0021] Optionally, the calculation process of the GRU module is:

[0022] s t =δ(I s x t +H s m t-1 );

[0023] q t =δ(I q x t +H q m t-1 );

[0024] m t =tanm(Ix t +H(q t ⊙m t-1 ));

[0025] mt =(1-s t )⊙m t-1 +s t ⊙ m t ;

[0026] y = δ(m) t I y );

[0027] Among them, I s H s I q H q ,I,H,I y The weights of the GRU model are represented by I, where I represents the weights from the input layer to the hidden layer, and H represents the weights of the hidden layer itself. y Indicates the weights from the hidden layer to the output layer; x t It is the input vector at time t, m t-1 It stores information from the previous time t-1; m t It represents the output at time t; δ represents the sigmoid function; m t It is the activation state of the hidden layer at time t; I s s represents the weight from the hidden layer to the update gate. t Indicates the update gate, H s It is the weight from the hidden layer to the update gate in the previous time t-1; I q Indicates the weight from the reset gate; q t Indicates resetting the door, H q y represents the weight from the hidden layer to the reset gate in the previous time step; y is the output of the GRU module; ⊙ represents the multiplication of the corresponding element in the matrix.

[0028] Optionally, the specific method for training the image segmentation model on the preprocessed dataset is as follows:

[0029] Change the batch normalization layer in the model to a group normalization layer, and adjust the initialization content to skip the batch normalization initialization.

[0030] The network is initialized with pre-trained weights, using weights trained on the COOC or PASCAL_VOC datasets, and the final output layer is randomly initialized.

[0031] Each time, preprocessed training samples and corresponding labels are fed into the network model, and the model is trained repeatedly through forward propagation and back propagation until the maximum number of iterations is reached, so as to minimize the loss function value.

[0032] Optionally, the specific method for aligning the segmented gesture images is as follows:

[0033] The preprocessed picture is sent into the trained segmentation model to obtain a segmentation result of the image, and the area outside the segmentation mask result in the image is set to 0, that is, black;

[0034] The segmentation result is used to calculate the segmentation edge by OpenCV, and then the minimum circumscribed rectangle of the segmentation edge is calculated by OpenCV again;

[0035] According to the four points of the minimum circumscribed rectangle, the short sides and the straight lines corresponding to the short sides are found out;

[0036] The center of the rectangle is moved by 1 / 4 of the long side distance from the two short sides respectively, and the edge area between the two short sides is calculated respectively, and the smaller one is the position of the finger and the larger one is the position of the palm;

[0037] According to the positions of the finger and the palm, the four coordinate points of the rectangle are rearranged so that the finger and the palm obtained by using affine transformation are horizontal and upward;

[0038] The aligned original gesture picture is cropped according to the rectangle size obtained by OpenCV.

[0039] A gesture recognition detection system based on deep learning, comprising:

[0040] A data acquisition and preprocessing module acquires a panoramic picture of an inspection scene, and preprocesses the acquired panoramic picture of the inspection scene, including mask annotation and data enhancement of the panoramic picture of the inspection scene, to generate a data set;

[0041] A data segmentation module trains an image segmentation model on the preprocessed data set to obtain a trained segmentation model;

[0042] A data alignment module sends the panoramic picture of the inspection scene for gesture feature recognition into the trained segmentation model, and aligns the segmented gesture picture;

[0043] A data division module divides the aligned gesture image into a training set and a validation set;

[0044] A model establishment module establishes a gesture recognition integrated model, in which a GRU module replaces the first fully connected layer of a CNN module to form a CGRU module, and the gesture feature output by the CGRU module is regarded as input data for gesture recognition in a GPR module;

[0045] A model training module trains the gesture recognition integrated model using the training set to obtain a trained gesture recognition integrated model;

[0046] A model verification module inputs the verification set into the trained gesture recognition integrated model to obtain the gesture in the inspection scene panoramic picture.

[0047] Compared with the prior art, the gesture recognition detection method and system based on deep learning provided by the present application can effectively and accurately obtain the gesture in the inspection scene panoramic picture through the whole process of training and verification by using the training set and the verification set, improve the efficiency and accuracy of gesture recognition detection, and adapt to the demand for gesture recognition in the inspection and other related scenes. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0049] Figure 1 The method flowchart provided by the present application is shown in the figure.

[0050] Figure 2 The system structure diagram provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] The embodiment of the present application discloses a gesture recognition detection method based on deep learning, as shown in the figure, which comprises the following steps: Figure 1

[0053] Collecting an inspection scene panoramic picture and preprocessing the collected inspection scene panoramic picture, including inspection scene panoramic picture mask annotation and data enhancement, to generate a data set;

[0054] ​Train the image segmentation model on the preprocessed dataset to obtain a trained segmentation model;

[0055] Send the inspection scene panoramic picture for gesture feature recognition into the trained segmentation model, and align the segmented gesture picture;

[0056] Divide the aligned gesture image into a training set and a validation set;

[0057] Establish a gesture recognition integrated model, in which the GRU module replaces the first fully connected layer of the CNN module to form a CGRU module, and the gesture features output by the CGRU module are regarded as input data for gesture recognition in the GPR module;

[0058] Train the gesture recognition integrated model using the training set to obtain a trained gesture recognition integrated model;

[0059] Input the validation set into the trained gesture recognition integrated model to obtain the gesture in the inspection scene panoramic picture.

[0060] In a specific embodiment, the specific method for preprocessing the collected inspection scene panoramic picture is:

[0061] Use the annotation tool to annotate the gesture edge in the inspection scene panoramic picture, and then export the annotation content as a json file;

[0062] Perform image enhancement on the annotated image, and the enhancement methods include random sharpness enhancement, random color enhancement, random contrast enhancement, random brightness enhancement, and picture fusion. After enhancement, one annotated picture becomes five pictures.

[0063] Specifically, the specific method for picture data annotation is: first, import the picture data to be annotated into the annotation tool labelme or via, then create a polygon annotation, annotate the edge of the gesture with a polygon, and annotate the gesture edge as much as possible.

[0064] The specific method for data enhancement is: the enhanced content mainly includes sharpness enhancement, color enhancement, contrast enhancement, and brightness enhancement. The four enhancement methods are performed using the ImageEnhance library of PIL. The picture fusion enhancement method needs to first obtain random pictures for fusion, which can be downloaded from the Internet. Then, scale the background picture to the size of the gesture picture to be fused, and then open the two pictures in RGBA mode. Randomly select the transparency of the background to superimpose the two pictures.

[0065] In one specific embodiment, the CNN (Convolutional Neural Network) module in the present application is mainly composed of three hidden layers and two fully connected layers; the first hidden layer includes a convolution layer, a normalization layer and a nonlinear activation layer. Among them, in the 32 convolution layers, a feature detector with a size of 1x2 and a step of 1 is used in each convolution layer, and data batch normalization is performed after convolution to improve the convergence speed and stability in model analysis; the second hidden layer is an average pooling layer to simplify the data dimension and prevent overfitting phenomenon in analysis, and the third hidden layer has 128 convolution kernels, and a 1x3 feature detector with a step of 1 is used to extract more data features, and a dropout strategy layer is connected to make the model more general. Two fully connected layers are activated by hyperbolic tangent function, and the output sequences are 1 128-length vector and 1 2-length vector respectively. The function used in the nonlinear activation layer is:

[0066] LReLU(x)={x(x≥0);αx(x<0)};

[0067] Where x represents the input value, and a is a parameter, a=0.01.

[0068] Further, the gated recurrent unit (GRU) module: the GRU model is also used to process sequence data, which has the advantage of making up for the gradient disappearance or explosion phenomenon that the CNN model cannot solve. The GRU module can master the important characteristics of the data through the reset gate (q t ) and update gate (s t ). The calculation process of the GRU module is:

[0069] s t =δ(I s x t +H s m t-1 );

[0070] q t =δ(I q x t +H q m t-1 );

[0071] m t =tanm(Ix t +H(q t ⊙m t-1 ));

[0072] m t =(1-s t )⊙m t-1 +s t ⊙ m t ;

[0073] y = δ(m t I y );

[0074] where I s , H s , I q , H q , I, H, I y denote the weights of the GRU model, I denotes the weights from the input layer to the hidden layer; H is the weights of the hidden layer itself; I y denotes the weights from the hidden layer to the output layer; x t is the input vector at the t-th time, m t-1 holds the information at the previous time t-1; m t is the output at the t-th time; δ represents the S-shaped function; m t is the activation state of the hidden layer at time t; I s denotes the weights from the hidden layer to the update gate, s t denotes the update gate, H s is the weights from the hidden layer to the update gate at the previous time t-1; I q denotes the weights from the reset gate; q t denotes the reset gate, H q denotes the weights from the hidden layer to the reset gate at the previous time; y is the output of the GRU module; ⊙ denotes the multiplication of corresponding elements in the matrix.

[0075] Further, Gaussian regression (GPR) model: the GPR module when processing a set of N sample data, i.e. where x i ∈R d is the input variable matrix, y n is the corresponding target value. The value between the input matrix and the output matrix is as follows:

[0076] y n = f(x n )+ λ n ;

[0077] where f is a function defined in the data set D, λ n is an independent Gaussian noise subject to distribution . The set F = {f(x1), f(x2), …, f(x N )} has a joint Gaussian distribution:

[0078] p(F︱x1,x2,…,x N )~(0,K);

[0079] where K is the kernel matrix, k(·) is the K ij= k(x i ,x j ) is the kernel function. The square exponential is used as the kernel function, which is defined as:

[0080]

[0081] where, is the signal equation of the kernel function; Γ is a hyperparameter symmetric matrix; is the variance of the noise, and the marginal distribution of y is shown as follows:

[0082]

[0083] p(y︱F,X) = ∫p(y︱f,X)p(f︱X);

[0084]

[0085] The set of hyperparameters is determined, and the log-likelihood function of the GPR module is established, as shown in the following equation:

[0086]

[0087] Then the optimal hyperparameters are determined under the maximum likelihood Bayesian framework shown in the above equation:

[0088]

[0089] Finally, the determined GPR model is used to identify new input data x' and calculate the joint distribution p(x'|X,x'), that is: where K' represents the kernel matrix of (N+1) × (N+1), and U represents the unit matrix.

[0090] The joint distribution following the Gaussian distribution is shown in the following equation:

[0091]

[0092] y' = [y1, y2, …, y N ,y'] T ;

[0093] k' = [k(x1, x'), …, k(x N ,x') T ;

[0094] k” = k(x', x');

[0095] Then, the posterior distribution is determined according to the Bayesian regression, and the prediction distribution of the new sample x' is calculated:

[0096]

[0097] According to the above formula, the prediction interval at a 95% confidence level determined based on probability distribution theory is as shown in the following equation:

[0098] [L 1-α (x'),H 1-α (x')]=[μ'-s (1-α) / 2 δ',μ'+s (1-α) / 2 δ']。

[0099] In the integrated model of CGRU-GPR, the GRU module behind the convolutional layer (CNN) replaces the first fully connected layer of the CNN module. The point prediction result (gesture feature) of the CGRU module is regarded as input data (μ') for interval prediction in the GPR module. That is, the gesture recognition result in the panoramic picture of the inspection scene can be obtained.

[0100] In a specific embodiment, the specific method for training the image segmentation model on the preprocessed data set is as follows:

[0101] The batch normalization layer in the model is modified into a group normalization layer, and the initialization content is adjusted, and the initialization of batch normalization is skipped;

[0102] The network is initialized with pre-trained weights, the weights trained on the COOC or PASCAL_VOC data set are used to initialize the network, and the last output layer is randomly initialized;

[0103] Each time the preprocessed training sample and the corresponding label are fed into the network model, the network model is repeatedly trained through the two steps of forward propagation and back propagation until the maximum number of iterations is reached, so that the loss function value is minimized.

[0104] In a specific embodiment, the specific method for aligning the segmented gesture picture is as follows:

[0105] The preprocessed picture is fed into the trained segmentation model to obtain the segmentation result of the image, and the area outside the segmentation mask result in the image is set to 0, i.e. black;

[0106] The segmentation result is used to calculate the segmentation edge using OpenCV, and then the minimum bounding rectangle of the segmentation edge is calculated using OpenCV again;

[0107] According to the four points of the minimum bounding rectangle, the short side and the straight line corresponding to the short side are found;

[0108] Respectively from the two short sides to the center of the rectangle, move 1 / 4 of the long side distance, the moving direction is parallel to the long side of the rectangle, and the edge area between the two short sides is calculated respectively, the smaller area is the position of the finger, and the larger area is the position of the palm;

[0109] According to the positions of the fingers and the palm, the four coordinate points of the rectangle are rearranged so that the fingers and the palm obtained after the affine transformation are definitely horizontally upward;

[0110] The aligned original gesture picture is cropped according to the size of the rectangle obtained by OpenCV.

[0111] The specific method of alignment is: first, the gesture picture is sent into the trained segmentation model to obtain the segmentation result of the gesture. From the gesture annotation information, it can be known that the finger annotation area is relatively small and the palm annotation area is relatively large. By using this point, the positions of the palm and the fingers can be determined, and the alignment of the gesture can be performed according to the positions of the palm and the fingers. First, findContours in OpenCV is used to find the edge information in the segmentation result, and the largest one is taken. Then, minAreaRect in OpenCV is used to calculate the minimum circumscribed rectangle of the edge. After finding the minimum circumscribed rectangle, the positions of the long side and the short side of the rectangle are found according to the four coordinate points of the rectangle. The long side corresponds to the side of the hand, and the short side corresponds to the palm or the finger. Then, from the two short sides, move along the long side to the center of the rectangle by 1 / 4 of the long side distance to obtain two small rectangles. The contourArea in OpenCV is used to calculate the area of the edge in the two small rectangles. The small area represents the finger, and the large area represents the palm. In this way, the four coordinate points of the rectangle can be obtained. The four points of the rectangle are arranged according to the coordinate sequence of the four points of the aligned gesture, and then the affine transformation is used to perform the alignment operation of the gesture to complete the alignment. After alignment, the gesture is cropped according to the size of the rectangle.

[0112] A gesture recognition detection system based on deep learning, as shown in Figure 2 , comprising:

[0113] A data acquisition and preprocessing module acquires a panoramic picture of an inspection scene and pre-processes the acquired panoramic picture of the inspection scene, including panoramic picture mask annotation and data enhancement, to generate a data set;

[0114] A data segmentation module trains an image segmentation model on the pre-processed data set to obtain a trained segmentation model;

[0115] A data alignment module sends the panoramic picture of the inspection scene for gesture feature recognition into the trained segmentation model and aligns the segmented gesture picture;

[0116] A data division module divides the aligned gesture picture into a training set and a validation set;

[0117] The model establishing module establishes the gesture recognition integrated model, in the gesture recognition integrated model, the first full connection layer of the CNN module is replaced by the GRU module to constitute the CGRU module, and the gesture feature output by the CGRU module is regarded as input data to perform gesture recognition in the GPR module;

[0118] The model training module trains the gesture recognition integrated model by using the training set, and obtains the trained gesture recognition integrated model;

[0119] The model verification module inputs the verification set into the trained gesture recognition integrated model, and obtains the gesture in the inspection scene panoramic picture.

[0120] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other.

[0121] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A gesture recognition detection method based on deep learning, characterized in that, The method comprises the following steps: Collecting a panoramic picture of an inspection scene and preprocessing the collected panoramic picture of the inspection scene, including mask labeling and data enhancement of the panoramic picture of the inspection scene, to generate a data set; Training an image segmentation model on the preprocessed data set to obtain a trained segmentation model; Sending the panoramic picture of the inspection scene for gesture feature recognition into the trained segmentation model and aligning the segmented gesture picture; Dividing the aligned gesture picture into a training set and a validation set; Establishing a gesture recognition integrated model, in which a GRU module is used to replace the first fully connected layer of a CNN module to form a CGRU module, and the gesture features output by the CGRU module are regarded as input data for gesture recognition in a GPR module; Training the gesture recognition integrated model using the training set to obtain a trained gesture recognition integrated model; Inputting the validation set into the trained gesture recognition integrated model to obtain gestures in the panoramic picture of the inspection scene. 2.The gesture recognition detection method based on deep learning according to claim 1, characterized in that, The specific method for preprocessing the collected panoramic picture of the inspection scene is as follows: Using a labeling tool to label the gesture edges in the panoramic picture of the inspection scene, and then exporting the labeling content as a json file; Performing image enhancement on the labeled image, and the enhancement methods include random sharpness enhancement, random color enhancement, random contrast enhancement, random brightness enhancement and picture fusion, so that one labeled picture is changed into five pictures after enhancement. 3.The gesture recognition detection method based on deep learning according to claim 1, characterized in that, The CNN module comprises a hidden layer and a fully connected layer; the first hidden layer comprises a convolution layer, a normalization layer and a nonlinear activation layer, a feature detector is used in each convolution layer, and data batch normalization is performed after convolution; the second hidden layer is an average pooling layer; and a hyperbolic tangent function is used to activate the fully connected layer. 4.The gesture recognition detection method based on deep learning according to claim 3, characterized in that, The function used in the nonlinear activation layer is as follows: LReLU(x) = {x(x≥0); alpha x(x<0)}; wherein x represents an input value, alpha is a parameter, and alpha = 0.

01. 5.The gesture recognition detection method based on deep learning according to claim 1, characterized in that, The calculation process of the GRU module is as follows: s t = δ(I s x t + H s m t-1 ); q t = δ(I q x t + H q m t-1 ); m t = tanm(Ix t + H(q t ⊙ m t-1 )); m t = (1 - s t ) ⊙ m t-1 + s t ⊙ m t ; y = δ(m t I y ); where I s , H s , I q , H q , I, H, I y denotes the weight of the GRU model, I denotes the weight from the input layer to the hidden layer; H is the weight of the hidden layer itself; I y denotes the weight from the hidden layer to the output layer; x t is the input vector at the t-th time, m t-1 holds the information at the previous time t-1; m t is the output at the t-th time; δ represents the S-shaped function; m t is the activation state of the hidden layer at time t; I s denotes the weight from the hidden layer to the update gate, s t denotes the update gate, H s is the weight from the hidden layer to the update gate at the previous time t-1; I q denotes the weight from the reset gate; q t denotes the reset gate, H q denotes the weight from the hidden layer to the reset gate at the previous time; y is the output of the GRU module; ⊙ denotes the multiplication of corresponding elements in the matrix. 6.The gesture recognition detection method based on deep learning according to claim 1, characterized in that, The specific method for training the image segmentation model on the preprocessed data set is as follows: The batch normalization layer in the model is modified into a group normalization layer, and the initialization content is adjusted, and the initialization of the batch normalization is skipped; Pretrained weight is used to initialize the network, the weight trained on the COOC or PASCAL_VOC data set is used to initialize the network, and the last output layer is randomly initialized; Each time, the preprocessed training sample and the corresponding label are sent into the network model, and the network model is repeatedly trained through the two steps of forward propagation and backward propagation until the maximum iteration number is reached, so that the loss function value is minimized. 7.The gesture recognition detection method based on deep learning according to claim 1, characterized in that, The specific method for aligning the segmented gesture picture is as follows: The preprocessed picture is sent into the trained segmentation model to obtain the segmentation result of the picture, and the area outside the segmentation mask result in the picture is set to 0, i.e. black; OpenCV is used to calculate the segmentation edges of the segmentation result, and then OpenCV is used again to calculate the minimum bounding rectangle of the segmentation edges; According to the four points of the minimum bounding rectangle, the short side and the straight line corresponding to the short side are found out; Respectively from two short sides to the rectangular center moving 1 / 4 long side distance, moving direction is parallel to the long side of the rectangle, respectively calculate the edge area between the two short sides, the smaller area is the position of the finger, the larger area is the position of the palm; According to the position of the finger and the palm, rearrange the four coordinate points of the rectangle, so that the finger and the palm obtained after the affine transformation are horizontal upward; Cut the aligned original gesture picture according to the size of the rectangle obtained by OpenCV. 8.A gesture recognition detection system based on deep learning, characterized in that, The gesture recognition detection method based on deep learning according to any one of claims 1-7, comprising: a data acquisition and preprocessing module, which acquires a panoramic picture of an inspection scene and pre-processes the acquired panoramic picture of the inspection scene, including mask labeling and data enhancement of the panoramic picture of the inspection scene, to generate a data set; a data segmentation module, which trains an image segmentation model on the pre-processed data set to obtain a trained segmentation model; a data alignment module, which inputs the panoramic picture of the inspection scene for gesture feature recognition into the trained segmentation model and aligns the segmented gesture picture; a data division module, which divides the aligned gesture picture into a training set and a validation set; a model establishment module, which establishes a gesture recognition integrated model, in which a GRU module replaces the first fully connected layer of a CNN module to form a CGRU module, and the gesture feature output by the CGRU module is regarded as input data for gesture recognition in a GPR module; a model training module, which trains the gesture recognition integrated model using the training set to obtain a trained gesture recognition integrated model; a model verification module, which inputs the validation set into the trained gesture recognition integrated model to obtain the gesture in the panoramic picture of the inspection scene.

Images