A gesture recognition model determination method, gesture recognition method and device

Abstract

The application relates to a gesture recognition model determination method, a gesture recognition method and a device. The method comprises the following steps: obtaining an initial gesture recognition model, the initial gesture recognition model comprising a mixed convolution module, the mixed convolution module comprising a first branch unit and a second branch unit; determining the first branch unit according to a maximum pooling layer and a first convolution layer connected in sequence; determining the second branch unit according to a second convolution layer, a separation and combination layer and a third convolution layer connected in sequence; processing the initial gesture recognition model by using the determined first branch unit and second branch unit to obtain a processed initial gesture recognition model; obtaining a gesture image training set; and training the processed initial gesture recognition model by using the gesture image training set to obtain a target gesture recognition model. Thus, the application can ensure that the size of the image is reduced and the features of the image are not lost, thereby ensuring the accuracy of the gesture recognition model.

Description

Technical Field

[0001] The present invention relates to the field of robotics, and in particular to a method for determining a gesture recognition model, a gesture recognition method, and a device. Background Technology

[0002] Currently, to improve the human-computer interaction experience when using robots, users often interact with robots through gestures. After the robot collects the user's gesture image, it can recognize the gesture image to obtain the gesture recognition result, and then the robot can perform corresponding control based on the gesture recognition result.

[0003] In existing technologies, robots often use the YOLO-v7 network model to recognize user gestures from images. The YOLO-v7 network model includes a hybrid convolutional module with two branches. The first branch consists of a max-pooling layer and a 1x1 convolutional layer, while the second branch consists of a 1x1 convolutional layer and a 3x2 convolutional layer. However, using the second branch during convolution can lead to the loss of some image features, ultimately resulting in inaccurate gesture recognition. Summary of the Invention

[0004] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the present invention provides a gesture recognition model determination method, gesture recognition method and device.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining a gesture recognition model, comprising:

[0006] An initial gesture recognition model is obtained, the initial gesture recognition model including a hybrid convolution module, the hybrid convolution module including a first branch unit and a second branch unit;

[0007] The first branch unit is determined based on the sequentially connected max pooling layer and the first convolutional layer, wherein the first convolutional layer is a 1x1 convolutional layer.

[0008] The second branch unit is determined based on the second convolutional layer, the split-merge layer and the third convolutional layer connected in sequence. The second convolutional layer and the third convolutional layer are both 1x1 convolutional layers. The split-merge layer is used to split an image according to a preset number and merge the resulting preset number of sub-images according to channels.

[0009] The initial gesture recognition model is processed using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model;

[0010] Obtain a training set of gesture images;

[0011] The processed initial gesture recognition model is trained using the gesture image training set to obtain the target gesture recognition model.

[0012] In an optional implementation, processing the initial gesture recognition model using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model includes:

[0013] The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model.

[0014] The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit.

[0015] The convolutional attention unit is used to determine local features of the image;

[0016] The self-attention unit is used to determine the global features of the image;

[0017] The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

[0018] In an optional implementation, training the processed initial gesture recognition model using the gesture image training set to obtain the target gesture recognition model includes:

[0019] During the training of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the processed initial gesture recognition model is determined;

[0020] When the loss function value reaches a preset threshold, the initial gesture recognition model is determined to be trained successfully, and the target gesture recognition model is obtained.

[0021] Secondly, an embodiment of the present invention provides a gesture recognition method, comprising:

[0022] Obtain the target gesture image;

[0023] The target gesture image is input into a pre-trained target gesture recognition model so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is the gesture recognition model as described above, and the target gesture recognition model is obtained by training through a gesture image training set.

[0024] In an optional implementation, the method further includes:

[0025] When the gesture recognition result is empty, the gesture image training set is updated according to the target gesture image;

[0026] The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

[0027] Thirdly, embodiments of the present invention provide a gesture recognition model determination device, comprising:

[0028] An acquisition module is used to acquire an initial gesture recognition model, the initial gesture recognition model including a hybrid convolution module, the hybrid convolution module including a first branch unit and a second branch unit;

[0029] A determination module is used to determine the first branch unit based on a sequentially connected max pooling layer and a first convolutional layer, wherein the first convolutional layer is a 1x1 convolutional layer.

[0030] The determining module is further configured to determine the second branch unit based on the second convolutional layer, the split-merge layer and the third convolutional layer connected in sequence. The second convolutional layer and the third convolutional layer are both 1x1 convolutional layers. The split-merge layer is configured to split an image according to a preset number and merge the resulting preset number of sub-images according to channels.

[0031] The processing module is used to process the initial gesture recognition model using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model.

[0032] The acquisition module is also used to acquire a gesture image training set;

[0033] The training module is used to train the processed initial gesture recognition model using the gesture image training set to obtain the target gesture recognition model.

[0034] In an optional implementation, the processing module is further configured to;

[0035] The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model.

[0036] The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit.

[0037] The convolutional attention unit is used to determine local features of the image;

[0038] The self-attention unit is used to determine the global features of the image;

[0039] The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

[0040] In an optional implementation, the training module is further configured to:

[0041] During the training of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the processed initial gesture recognition model is determined;

[0042] When the loss function value reaches a preset threshold, the initial gesture recognition model is determined to be trained successfully, and the target gesture recognition model is obtained.

[0043] Fourthly, embodiments of the present invention provide a gesture recognition device, comprising:

[0044] The acquisition module is used to acquire the target gesture image;

[0045] The recognition module is used to input the target gesture image into a pre-trained target gesture recognition model, so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is obtained by training the gesture recognition model as described above using a gesture image training set.

[0046] In an optional implementation, the apparatus further includes: an update module;

[0047] The update module is used to update the gesture image training set according to the target gesture image when the gesture recognition result is empty;

[0048] The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

[0049] This invention provides a method for determining a gesture recognition model, comprising: obtaining an initial gesture recognition model, the initial gesture recognition model including a hybrid convolutional module, the hybrid convolutional module including a first branch unit and a second branch unit; determining a first branch unit based on a sequentially connected max pooling layer and a first convolutional layer, the first convolutional layer being a 1x1 convolutional layer; determining a second branch unit based on a sequentially connected second convolutional layer, a split-merge layer, and a third convolutional layer, the second and third convolutional layers being both 1x1 convolutional layers, the split-merge layer being used to split an image according to a preset number, and merging the resulting preset number of sub-images according to channels; processing the initial gesture recognition model using the determined first and second branch units to obtain a processed initial gesture recognition model; obtaining a gesture image training set; and training the processed initial gesture recognition model using the gesture image training set to obtain a target gesture recognition model. Through the above method, this invention improves the hybrid convolutional model in the initial gesture recognition model. Because 1x1 convolutional layers are used before and after the split-merge layer, the image size is reduced without loss of image features, thus ensuring the accuracy of the gesture recognition model. Attached Figure Description

[0050] Figure 1 A flowchart illustrating a gesture recognition model determination method provided in an embodiment of the present invention;

[0051] Figure 2 A flowchart illustrating a gesture recognition method provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of a gesture recognition model determination device provided in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the structure of a gesture recognition device provided in an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0057] refer to Figure 1 , Figure 1 This is a flowchart illustrating a gesture recognition model determination method provided by an embodiment of the present invention. The gesture recognition model determination method provided by an embodiment of the present invention includes the following steps:

[0058] S101: Obtain the initial gesture recognition model.

[0059] In this embodiment, to ensure the speed and accuracy of gesture recognition, the initial gesture recognition model is a YOLO-v7 network model. This initial model includes a hybrid convolutional module, which is part of the backbone network of the YOLO-v7 network model. The hybrid convolutional module can be denoted as the MPConv module. The hybrid convolutional model expands the receptive field of the current feature layer and then fuses it with the feature information after normal convolution processing to improve the network's generalization ability. Specifically, the hybrid convolutional module includes a first branch unit and a second branch unit. The first branch unit can be the same as a traditional first branch unit. However, to ensure the integrity of image features during the convolution process of the second branch unit, the second branch unit needs to be improved.

[0060] S102: Determine the first branch unit based on the sequentially connected max pooling layer and the first convolutional layer.

[0061] In this embodiment, the traditional first branch unit consists of a max pooling layer and a first convolutional layer cascaded together to extract image feature information. Since no image features are lost during the convolution process of the first branch unit, there is no need to improve the first branch unit when improving the hybrid convolutional model. Specifically, the first convolutional layer is a 1x1 convolutional layer.

[0062] S103: Determine the second branch unit based on the sequentially connected second convolutional layer, split-merge layer, and third convolutional layer.

[0063] In this embodiment, the traditional second branch unit typically consists of a cascaded second and fourth convolutional layers to extract more detailed features from the image. By fusing the various feature information extracted by the traditional first and second branch units, better image feature information can be obtained. However, currently, the second convolutional layer is a 1x1 convolutional layer and the fourth convolutional layer is a 3x2 convolutional layer. The way the fourth convolutional layer is configured results in the loss of some image features, which affects the establishment of the gesture recognition model and consequently the accuracy of gesture recognition. Therefore, the second branch unit needs to be improved.

[0064] Specifically, the third convolutional layer is a 1x1 convolutional layer. The second convolutional layer, the split-and-merge layer, and the third convolutional layer are cascaded. The split-and-merge layer is used to separate an image according to a preset number of sub-images and merge the resulting preset number of sub-images according to channels. The preset number can be set according to actual needs; in this embodiment, the specific value of the preset number is not limited, for example, the preset number can be four. In this embodiment, after the image is convolved by the 1x1 convolutional layer, the convolved image is processed by the split-and-merge layer to obtain four sub-images corresponding to the image. These four sub-images are merged according to channels to obtain a merged image. The merged image is then convolved by the 1x1 convolutional layer to obtain an image with half the size. For example, for an image containing MxM pixels, after passing through the 1x1 convolutional layer, it enters the split-and-merge layer, which separates the MxM pixel image into four M / 2xM / 2 sub-images. These four sub-images are then merged according to channels to obtain the merged image. By improving the second branch unit in the above way, the image size can be halved while avoiding the loss of image features during the convolution process.

[0065] S104: The initial gesture recognition model is processed using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model.

[0066] In this embodiment, after determining the first branch unit and the second branch unit, the hybrid convolution module is processed according to the first branch unit and the second branch unit to obtain the processed hybrid convolution module. Since the initial gesture recognition model includes the hybrid convolution module, the processed initial gesture recognition model can be obtained after obtaining the processed hybrid convolution module.

[0067] Specifically, to further improve the accuracy of gesture recognition in this embodiment, the initial gesture recognition model is processed using the determined first and second branch units to obtain a processed initial gesture recognition model, including:

[0068] The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model.

[0069] The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit.

[0070] Convolutional attention units are used to determine local features of an image;

[0071] Self-attention units are used to determine global features of an image;

[0072] The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

[0073] In this embodiment, since convolutional attention units focus more on the relationship between input and output, while self-attention units focus more on the relationship between inputs, an attention module including convolutional attention units and self-attention units is added to the backbone network to further improve the accuracy of gesture recognition. This improves the attention of the gesture recognition model to small targets during gesture recognition. Specifically, convolutional attention units focus on local features of the image, while self-attention units focus on global features. After merging the local and global features of the image through a feature processing unit, the target features of the image can be obtained, enabling the gesture recognition model to more comprehensively focus on various features of the image. For example, after an H×W×C feature is convolved by three 1x1 convolutional layers, it is divided into N pieces, resulting in 3xN H×W×C / N sub-features. For the convolutional attention unit, its kernel is a k-fold convolutional path. After the sub-image passes through a 3N×K2N fully connected layer, the generated features are shifted, aggregated, and convolved to obtain the H×W×C feature. For the self-attention unit, the 3N sub-features correspond to three feature maps of size H×W×C / N. These are then processed through shifting, aggregation, and convolution to obtain H×W×C features. Merging these two features yields the target features of the image. The target features obtained through the attention model simultaneously consider both global and local features of the image, improving the accuracy of gesture recognition.

[0074] S105: Obtain the gesture image training set.

[0075] In this embodiment, the gesture image training set can be an existing gesture image training set. Of course, various gesture images can also be collected by an image acquisition device, and the collected gesture images can be combined to obtain the gesture image training set.

[0076] S106: Train the processed initial gesture recognition model using the gesture image training set to obtain the target gesture recognition model.

[0077] In this embodiment, after obtaining the processed initial gesture recognition model, the processed initial gesture recognition model is trained using a gesture image training set to obtain a target gesture recognition model that accurately recognizes gestures.

[0078] Specifically, the initial gesture recognition model is trained using a gesture image training set to obtain the target gesture recognition model, including:

[0079] During the training process of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the initial gesture recognition model is determined.

[0080] When the loss function value reaches a preset threshold, the initial gesture recognition model is considered to have been trained and the target gesture recognition model is obtained.

[0081] In this embodiment, the loss function includes three parts: a coordinate error loss function, a confidence loss function, and a classification error loss function. Adding these three functions together yields the overall loss function for the processed initial gesture recognition model. During the training of this model, the overall loss function value is calculated via forward propagation, and the weights of the model are optimized via backpropagation until the overall loss function value reaches a preset threshold, indicating that the training of the processed gesture recognition model is complete. Specifically, the preset threshold can be selected according to actual needs; this embodiment does not limit the specific value of the preset threshold.

[0082] More specifically, the confidence loss function and classification error loss function are both determined using the BCEWithLogitsLoss function, while the coordinate error loss function is determined using CloU. The coordinate error loss function can be determined using the following formula:

[0083]

[0084]

[0085]

[0086] In the above formula, LCoU represents the coordinate error loss function. represents the balance parameter, and v represents the similarity measured by aspect ratio.

[0087] This embodiment provides a gesture recognition model determination method. By improving the hybrid convolutional model in the initial gesture recognition model, since 1x1 convolutional layers are used before and after the separation and merging layers, the image size is reduced without the loss of image features, thereby ensuring the accuracy of the gesture recognition model.

[0088] refer to Figure 2 , Figure 2 This is a flowchart illustrating a gesture recognition method provided in an embodiment of the present invention. The gesture recognition method provided in this embodiment of the present invention includes the following steps:

[0089] S201: Obtain the target gesture image.

[0090] In this embodiment, the executing entity is a robot, which may be equipped with a camera device. During the interaction between the user and the robot, the robot collects the user's target gesture image through the camera device, and the camera device sends the collected target gesture image to the target gesture image for processing in order to perform gesture recognition on the target gesture image.

[0091] S202: Input the target gesture image into the trained target gesture recognition model so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image.

[0092] In this embodiment, the trained target gesture recognition model is the gesture recognition model described above, which is obtained by training on a gesture image training set. To accurately and quickly recognize the target gesture image, before executing step S202, this embodiment provides a gesture recognition method that further includes the following steps:

[0093] Determine if the target gesture image is complete;

[0094] If the target gesture image is confirmed to be complete, proceed to step S202;

[0095] If the target gesture image is determined to be incomplete, a prompting device will be used to provide a prompt.

[0096] Specifically, the integrity of the target gesture image can be determined using existing technologies, which will not be elaborated upon in this embodiment. The prompting device includes a display device, a signal light device, and a voice device. If the target gesture image is determined to be incomplete, a prompt message is displayed through the display device, an audio-visual prompt is provided through the signal light device, or a voice prompt is broadcast through the voice device. This embodiment does not limit the specific form of the prompting device; it can be selected according to actual needs. This embodiment, by judging the integrity of the target gesture image after acquisition, can improve the speed and accuracy of gesture recognition.

[0097] In this embodiment, to further improve the accuracy of the target gesture recognition model in recognizing gestures, the gesture recognition method provided in this embodiment further includes the following steps:

[0098] When the gesture recognition result is empty, the gesture image training set is updated according to the target gesture image;

[0099] The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

[0100] Specifically, when the gesture recognition result is empty, it indicates that the target gesture recognition model has failed to accurately recognize the gesture of the target gesture image. Therefore, in order to accurately recognize the gesture of the target gesture image in the future, the gesture image training set is updated using the target gesture image, so as to complete the update of the target gesture recognition model using the updated gesture image training set.

[0101] The present invention provides a gesture recognition method that, when a robot performs recognition, uses an improved initial gesture recognition model to recognize gesture images, thereby improving the accuracy of gesture recognition by the robot and further enhancing the interactive experience between the user and the robot.

[0102] refer to Figure 3 , Figure 3This is a schematic diagram of a gesture recognition model determination device provided in an embodiment of the present invention. The gesture recognition model provided in this embodiment includes an acquisition module 101, a determination module 102, a processing module 103, and a training module 104. The acquisition module 101 is used to acquire an initial gesture recognition model, which includes a hybrid convolutional module, comprising a first branch unit and a second branch unit. The determination module 102 is used to determine the first branch unit based on a sequentially connected max pooling layer and a first convolutional layer, wherein the first convolutional layer is a 1x1 convolutional layer. The determination module 102 is further used to determine the second branch unit based on a sequentially connected second convolutional layer, a split-merge layer, and a third convolutional layer. The branching unit, the second convolutional layer and the third convolutional layer are both 1x1 convolutional layers, the separation and merging layer is used to separate an image according to a preset number, and merge the obtained preset number of sub-images according to channels; the processing module 103 is used to process the initial gesture recognition model using the determined first branching unit and second branching unit to obtain the processed initial gesture recognition model; the acquisition module 101 is also used to acquire a gesture image training set; the training module 104 is used to train the processed initial gesture recognition model using the gesture image training set to obtain a target gesture recognition model.

[0103] In this embodiment, the processing module 103 is further configured to;

[0104] The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model.

[0105] The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit.

[0106] The convolutional attention unit is used to determine local features of the image;

[0107] The self-attention unit is used to determine the global features of the image;

[0108] The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

[0109] In this embodiment, the training module 104 is further configured to:

[0110] During the training of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the processed initial gesture recognition model is determined;

[0111] When the loss function value reaches a preset threshold, the initial gesture recognition model is determined to be trained successfully, and the target gesture recognition model is obtained.

[0112] This embodiment provides a gesture recognition model determination device that improves the hybrid convolutional model in the initial gesture recognition model. Since 1x1 convolutional layers are used before and after the separation and merging layers, the image size is reduced without the loss of image features, thereby ensuring the accuracy of the gesture recognition model.

[0113] refer to Figure 4 , Figure 4 This is a schematic diagram of a gesture recognition device provided in an embodiment of the present invention. The gesture recognition device provided in this embodiment includes an acquisition module 201 and a recognition module 202. The acquisition module 201 is used to acquire a target gesture image; the recognition module 202 is used to input the target gesture image into a pre-trained target gesture recognition model, so that the target gesture recognition model outputs a gesture recognition result corresponding to the target gesture image; wherein the target gesture recognition model is obtained by training a gesture image training set.

[0114] The gesture recognition device provided in this embodiment also includes an update module, which is used for:

[0115] When the gesture recognition result is empty, the gesture image training set is updated according to the target gesture image;

[0116] The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

[0117] This embodiment provides a gesture recognition device that uses an improved initial gesture recognition model to recognize gesture images when the robot is performing recognition, thereby improving the accuracy of gesture recognition by the robot and further enhancing the interactive experience between the user and the robot.

[0118] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 5 The illustrated electronic device 400 includes at least one processor 401, a memory 402, at least one network interface 404, and other user interfaces 403. The various components in the electronic device 400 are coupled together via a bus system 405. It is understood that the bus system 405 is used to implement communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 5 The general designated all buses as Bus System 405.

[0119] The user interface 403 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0120] It is understood that the memory 402 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 402 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0121] In some implementations, memory 402 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 4021 and application program 4022.

[0122] The operating system 4021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 4022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 4022.

[0123] In this embodiment of the invention, by calling the program or instructions stored in memory 402, specifically the program or instructions stored in application program 4022, processor 401 executes the method steps provided in each method embodiment, for example including: obtaining an initial gesture recognition model, the initial gesture recognition model including a hybrid convolution module, the hybrid convolution module including a first branch unit and a second branch unit; determining the first branch unit based on a sequentially connected max pooling layer and a first convolution layer, the first convolution layer being a 1x1 convolution layer; determining the second branch unit based on a sequentially connected second convolution layer, a split-merge layer, and a third convolution layer, the second convolution layer and the third convolution layer being both 1x1 convolution layers, the split-merge layer being used to split an image according to a preset number, and merging the obtained preset number of sub-images according to channels; processing the initial gesture recognition model using the determined first branch unit and second branch unit to obtain a processed initial gesture recognition model; obtaining a gesture image training set; training the processed initial gesture recognition model using the gesture image training set to obtain a target gesture recognition model. Alternatively,

[0124] Acquire a target gesture image; input the target gesture image into a pre-trained target gesture recognition model so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is obtained by training a gesture image training set.

[0125] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 402. Processor 401 reads the information in memory 402 and, in conjunction with its hardware, completes the steps of the above method.

[0126] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0127] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0128] The electronic device provided in this embodiment may be as follows: Figure 5 The electronic device shown can perform the following: Figure 1 and Figure 2 All steps of the method, thus achieving Figure 1 and Figure 2 For details on the technical effects of the method shown, please refer to [link / reference]. Figure 1 and Figure 2 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0129] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0130] The above method can be implemented when one or more programs in the storage medium can be executed by one or more processors.

[0131] The processor is used to execute the gesture recognition model determination and gesture recognition program stored in the memory to implement the following steps: obtaining an initial gesture recognition model, the initial gesture recognition model including a hybrid convolutional module, the hybrid convolutional module including a first branch unit and a second branch unit; determining the first branch unit based on a sequentially connected max pooling layer and a first convolutional layer, the first convolutional layer being a 1x1 convolutional layer; determining the second branch unit based on a sequentially connected second convolutional layer, a split-merge layer, and a third convolutional layer, the second convolutional layer and the third convolutional layer being both 1x1 convolutional layers, the split-merge layer being used to split an image according to a preset number and merge the resulting preset number of sub-images according to channels; processing the initial gesture recognition model using the determined first branch unit and second branch unit to obtain a processed initial gesture recognition model; obtaining a gesture image training set; training the processed initial gesture recognition model using the gesture image training set to obtain a target gesture recognition model. Alternatively,

[0132] Acquire a target gesture image; input the target gesture image into a pre-trained target gesture recognition model so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is obtained by training a gesture image training set.

[0133] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0134] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0135] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining a gesture recognition model, characterized in that, include: An initial gesture recognition model is obtained, the initial gesture recognition model including a hybrid convolution module, the hybrid convolution module including a first branch unit and a second branch unit; The first branch unit is determined based on the sequentially connected max pooling layer and the first convolutional layer, wherein the first convolutional layer is a 1x1 convolutional layer. The second branch unit is determined based on the second convolutional layer, the split-merge layer and the third convolutional layer connected in sequence. The second convolutional layer and the third convolutional layer are both 1x1 convolutional layers. The split-merge layer is used to split an image according to a preset number and merge the resulting preset number of sub-images according to channels. The initial gesture recognition model is processed using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model; Obtain a training set of gesture images; The processed initial gesture recognition model is trained using the gesture image training set to obtain the target gesture recognition model.

2. The method according to claim 1, characterized in that, The step of processing the initial gesture recognition model using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model includes: The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model. The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit. The convolutional attention unit is used to determine local features of the image; The self-attention unit is used to determine the global features of the image; The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

3. The method according to claim 1 or 2, characterized in that, The step of training the processed initial gesture recognition model using the gesture image training set to obtain the target gesture recognition model includes: During the training of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the processed initial gesture recognition model is determined; When the loss function value reaches a preset threshold, the initial gesture recognition model is determined to be trained successfully, and the target gesture recognition model is obtained.

4. A gesture recognition method, characterized in that, include: Obtain the target gesture image; The target gesture image is input into a pre-trained target gesture recognition model so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is a gesture recognition model determined by the method of any one of claims 1 to 3, and the target gesture recognition model is obtained by training through a gesture image training set.

5. The method according to claim 4, characterized in that, The method further includes: When the gesture recognition result is empty, the gesture image training set is updated according to the target gesture image; The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

6. A gesture recognition model determination device, characterized in that, include: An acquisition module is used to acquire an initial gesture recognition model, the initial gesture recognition model including a hybrid convolution module, the hybrid convolution module including a first branch unit and a second branch unit; A determination module is used to determine the first branch unit based on a sequentially connected max pooling layer and a first convolutional layer, wherein the first convolutional layer is a 1x1 convolutional layer. The determining module is further configured to determine the second branch unit based on the second convolutional layer, the split-merge layer and the third convolutional layer connected in sequence. The second convolutional layer and the third convolutional layer are both 1x1 convolutional layers. The split-merge layer is configured to split an image according to a preset number and merge the resulting preset number of sub-images according to channels. The processing module is used to process the initial gesture recognition model using the determined first branch unit and second branch unit to obtain the processed initial gesture recognition model. The acquisition module is also used to acquire a gesture image training set; The training module is used to train the processed initial gesture recognition model using the gesture image training set to obtain the target gesture recognition model.

7. The apparatus according to claim 6, characterized in that, The processing module is also used for: The initial gesture recognition model is processed using the determined first branch unit, second branch unit, and attention module to obtain the processed initial gesture recognition model. The attention module includes a convolutional attention unit, a self-attention unit, and a feature processing unit. The convolutional attention unit is used to determine local features of the image; The self-attention unit is used to determine the global features of the image; The feature processing unit is used to merge the local features and global features of the image to determine the target features of the image.

8. The apparatus according to claim 6 or 7, characterized in that, The training module is also used for: During the training of the processed initial gesture recognition model using the gesture image training set, the loss function value corresponding to the processed initial gesture recognition model is determined; When the loss function value reaches a preset threshold, the initial gesture recognition model is determined to be trained successfully, and the target gesture recognition model is obtained.

9. A gesture recognition device, characterized in that, include: The acquisition module is used to acquire the target gesture image; The recognition module is used to input the target gesture image into a pre-trained target gesture recognition model, so that the target gesture recognition model outputs the gesture recognition result corresponding to the target gesture image; wherein, the target gesture recognition model is the gesture recognition model according to any one of claims 6 to 8, and the target gesture recognition model is obtained by training through a gesture image training set.

10. The apparatus according to claim 9, characterized in that, The device further includes: an update module; The update module is used to update the gesture image training set according to the target gesture image when the gesture recognition result is empty; The target gesture recognition model is trained based on the updated gesture image training set to complete the update of the target gesture recognition model.

Images