A gesture detection method, apparatus and electronic device

Abstract

This disclosure provides a gesture detection method, apparatus, and electronic device, relating to the field of image recognition technology. It addresses the problem in existing technologies where gesture detection models deployed on offline electronic devices sacrifice accuracy to ensure low inference latency, a small number of parameters, and low computational cost, resulting in low gesture detection accuracy. The method includes: acquiring a gesture image to be recognized; inputting the gesture image to be recognized into a pre-configured detection model to determine the actual regression result and the actual classification result; and determining the detection result of the gesture image to be recognized based on the actual regression result and the actual classification result.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and in particular to a gesture detection method, apparatus and electronic device. Background Technology

[0002] In gesture recognition scenarios, the accuracy of the gesture detection model determines the accuracy of gesture detection. For example, gesture detection models deployed on offline electronic devices sacrifice accuracy in order to ensure low inference latency, a small number of parameters, and low computational cost, resulting in lower gesture detection accuracy. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a gesture detection method, apparatus, and electronic device.

[0004] In a first aspect, this disclosure provides a gesture detection method, comprising: acquiring a gesture image to be recognized; inputting the gesture image to be recognized into a pre-configured detection model to determine the actual regression result and the actual classification result; wherein the pre-configured detection model is obtained by knowledge distillation training based on a teacher model with dynamic convolutional layers and attention mechanism; and determining the detection result of the gesture image to be recognized based on the actual regression result and the actual classification result.

[0005] As an optional implementation of this disclosure, the training process of the pre-configured detection model is as follows: acquiring training sample images and labeling results of training sample images; wherein, the training sample images include foreground and background, the foreground includes gestures, and the labeling results include the detection results of gestures; inputting the training sample images into a teacher model with dynamic convolutional layers and attention mechanisms to determine a first target feature map and target detection results; and performing knowledge distillation training on the student model based on the first target feature map, target detection results, training sample images, and labeling results to obtain the pre-configured detection model.

[0006] As an optional implementation of this disclosure, inputting training sample images into a teacher model having dynamic convolutional layers and an attention mechanism to determine a first target feature map and a target detection result includes: inputting training sample images into the teacher model to obtain at least one first feature map; inputting the first feature map into a dynamic convolutional layer for convolution operation to determine at least one second feature map; processing the second feature map using an attention mechanism to determine the first target feature map; and inputting the first feature map into the detection head of the teacher model for detection to determine the target detection result.

[0007] As an optional implementation of this disclosure, a pre-configured detection model is trained by knowledge distillation based on a first target feature map, target detection results, training sample images, and labeling results. This includes: inputting training sample images into the student model to determine a second target feature map; inputting the second target feature map into the detection head of the student model for detection, and determining predicted regression results and predicted classification results; determining a first loss value between the second target feature map and the first feature map, a second loss value between the predicted regression results and the target detection results, and a third loss value between the predicted classification results and the labeling results based on a target loss function; and adjusting the network parameters of the student model according to the first loss value, the second loss value, and the third loss value until the student model converges, thus obtaining the pre-configured detection model.

[0008] As an optional implementation of this disclosure, the target loss function includes one or more of the following: a smooth minimum absolute deviation loss function, a regression loss function, and a weighted cross-entropy loss function.

[0009] Secondly, this disclosure provides a gesture detection device, comprising: an acquisition unit for acquiring a gesture image to be recognized; a processing unit for inputting the gesture image to be recognized acquired by the acquisition unit into a pre-configured detection model to determine the actual regression result and the actual classification result; wherein the pre-configured detection model is obtained by knowledge distillation training based on a teacher model with dynamic convolutional layers and attention mechanisms; the processing unit is further configured to determine the detection result of the gesture image to be recognized based on the actual regression result and the actual classification result.

[0010] As an optional implementation of this disclosure, the training process of the pre-configured detection model is as follows: an acquisition unit is further configured to acquire training sample images and labeling results of training sample images; wherein, the training sample images include foreground and background, the foreground includes gestures, and the labeling results include detection results of gestures; a processing unit is further configured to input the training sample images acquired by the acquisition unit into a teacher model with dynamic convolutional layers and attention mechanisms to determine a first target feature map and target detection results; the processing unit is further configured to perform knowledge distillation training on the student model based on the first target feature map, target detection results, training sample images acquired by the acquisition unit, and labeling results acquired by the acquisition unit to obtain the pre-configured detection model.

[0011] As an optional implementation of this disclosure, the processing unit is further configured to input the training sample image acquired by the acquisition unit into the teacher model to obtain at least one first feature map; the processing unit is further configured to input the first feature map into a dynamic convolutional layer for convolution operation to determine at least one second feature map; the processing unit is further configured to process the second feature map using an attention mechanism to determine a first target feature map; the processing unit is further configured to input the first feature map into the detection head of the teacher model for detection to determine the target detection result.

[0012] As an optional implementation of this disclosure, the processing unit is further configured to input the training sample image acquired by the acquisition unit into the student model to determine the second target feature map; the processing unit is further configured to input the second target feature map into the detection head of the student model for detection to determine the predicted regression result and the predicted classification result; the processing unit is further configured to determine, based on the target loss function, a first loss value between the second target feature map and the first feature map, a second loss value between the predicted regression result and the target detection result, and a third loss value between the predicted classification result and the labeling result; the processing unit is further configured to adjust the network parameters of the student model according to the first loss value, the second loss value, and the third loss value until the student model converges to obtain a pre-configured detection model.

[0013] As an optional implementation of this disclosure, the target loss function includes one or more of the following: a smooth minimum absolute deviation loss function, a regression loss function, and a weighted cross-entropy loss function.

[0014] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the electronic device to implement the gesture detection method provided in the first aspect when executing the computer program.

[0015] Fourthly, this disclosure provides a computer program product that, when run on a computer, enables the computer to implement the gesture detection method provided in the first aspect.

[0016] Fifthly, this disclosure provides a computer program product that, when run on a computer, causes the computer to execute a gesture detection method as described in the first aspect.

[0017] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the gesture detection device, or it may be packaged separately from the processor of the gesture detection device; this disclosure does not impose any limitations on this.

[0018] The descriptions of the second, third, fourth, and fifth aspects in this disclosure can be referenced to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0019] In this disclosure, the name of the aforementioned gesture detection device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this disclosure, it falls within the scope of the claims of this disclosure and its equivalents.

[0020] These or other aspects of this disclosure will become more readily apparent in the following description.

[0021] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0022] By introducing a knowledge distillation framework, a teacher model with dynamic convolutional layers and attention mechanisms is used to train the student model through knowledge distillation. This yields a pre-configured detection model once the student model converges. This approach ensures the accuracy of the pre-configured detection model without increasing the number of network parameters or computational load. Because the pre-configured detection model has higher accuracy, it improves the accuracy of gesture detection when used to recognize gesture images, thus enhancing the user experience.

[0023] Furthermore, when the pre-configured detection model is deployed on an offline electronic device, since the number of network parameters and computational cost are the same between the pre-configured detection model and the gesture detection model in the prior art, but the model accuracy of the pre-configured detection model provided in the gesture detection method provided in this disclosure embodiment is higher, the accuracy of gesture detection can be improved when using the pre-configured detection model to recognize the gesture image to be recognized. This solves the problem that in the prior art, the gesture detection model deployed on offline electronic devices sacrifices the accuracy of the gesture detection model in order to ensure low inference latency, small number of parameters and low computational cost, resulting in low gesture detection accuracy. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is one of the flowcharts illustrating the gesture detection method provided in this embodiment of the disclosure;

[0027] Figure 2 This is a second schematic flowchart of the gesture detection method provided in this embodiment of the present disclosure;

[0028] Figure 3 The third schematic flowchart of the gesture detection method provided in this embodiment of the present disclosure;

[0029] Figure 4 This is a schematic diagram of the knowledge distillation structure in the gesture detection method provided in the embodiments of this disclosure;

[0030] Figure 5 The fourth schematic flowchart of the gesture detection method provided in this embodiment of the present disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of the gesture detection device provided in the embodiments of this disclosure;

[0032] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure;

[0033] Figure 8 This is a schematic diagram of the structure of a computer program product for a gesture detection method provided in an embodiment of this disclosure. Detailed Implementation

[0034] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0035] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0036] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0037] Knowledge distillation, as described in this embodiment, is a commonly used method for model compression. Unlike pruning and quantization in model compression, knowledge distillation involves building a lightweight small model and using the supervision information from a larger, higher-performing model to train this small model, aiming to achieve better performance and accuracy.

[0038] Figure 1 This is a flowchart illustrating a gesture detection method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following S11-S13.

[0039] S11. Obtain the image of the gesture to be recognized.

[0040] In some examples, users can use gestures to control electronic devices to perform corresponding functions. For instance, a user inputs a gesture through air gestures, at which point the electronic device's image acquisition device captures an image of the gesture to be recognized. The electronic device then executes the gesture detection method provided in this embodiment to determine the detection result of the gesture image to be recognized, such as: the gesture image to be recognized corresponds to a "fist". In this way, the electronic device can perform the corresponding function corresponding to the gesture "fist".

[0041] S12. Input the gesture image to be recognized into the pre-configured detection model to determine the actual regression result and the actual classification result. The pre-configured detection model is trained by knowledge distillation based on a teacher model with dynamic convolutional layers and attention mechanisms.

[0042] In some examples, the gesture detection method provided in this disclosure introduces a knowledge distillation framework. It utilizes a teacher model with dynamic convolutional layers and an attention mechanism to perform knowledge distillation on a student model. This allows for the acquisition of a pre-configured detection model once the student model converges. This ensures the accuracy of the pre-configured detection model without increasing the number of network parameters or computational cost. Subsequently, when the pre-configured detection model is deployed on an offline electronic device, it guarantees low inference latency, a small number of parameters, and low computational cost. Furthermore, because the pre-configured detection model has higher accuracy, the gesture detection accuracy is also higher, thus ensuring a better user experience.

[0043] S13. Based on the actual regression results and the actual classification results, determine the detection results of the gesture image to be recognized.

[0044] In some examples, the actual regression result represents the location information of the gesture features in the image to be recognized, while the actual classification result represents the classification result of the gesture features in the image to be recognized, such as classification results including "fist" or "OK". In this way, the electronic device can summarize the actual regression result and the actual classification result to obtain the detection result of the gesture image to be recognized.

[0045] As described above, the gesture detection method provided in this disclosure introduces dynamic convolutional layers and an attention mechanism within the framework of knowledge distillation. This allows the dynamic convolutional layers to reduce the product-moment correlation of the Pearson correlation coefficient in the middle feature map of the teacher model. Simultaneously, the attention mechanism further reduces the redundancy of background information in the middle feature map of the teacher model, effectively increasing the proportion of foreground gesture features and ensuring the robustness and generalization of the pre-configured detection model.

[0046] As an optional implementation of this disclosure, combined with Figure 1 ,like Figure 2 As shown, in the gesture detection method provided in this embodiment, the training process of the pre-configured detection model includes: S14-S16.

[0047] S14. Obtain the training sample images and their labeling results. The training sample images include foreground and background, the foreground includes gestures, and the labeling results include the gesture detection results.

[0048] S15. Input the training sample images into the teacher model with dynamic convolutional layers and attention mechanism to determine the first target feature map and target detection results.

[0049] Specifically, a dynamic convolutional layer includes at least two standard convolutional kernels.

[0050] S16. Based on the first target feature map, target detection results, training sample images and labeling results, the student model is trained by knowledge distillation to obtain a pre-configured detection model.

[0051] As can be seen from the above, the gesture detection method provided in this embodiment introduces a knowledge distillation framework and uses a teacher model with dynamic convolutional layers and attention mechanisms to perform knowledge distillation on the student model. Thus, when the student model converges, a pre-configured detection model is obtained, ensuring the model accuracy of the pre-configured detection model.

[0052] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 3 As shown, the above S15 can be implemented by the following S150-S153.

[0053] S150. Input the training sample images into the teacher model to obtain at least one first feature map.

[0054] S151. Input the first feature map into the dynamic convolutional layer for convolution operation to determine at least one second feature map.

[0055] S152. The second feature map is processed using an attention mechanism to determine the first target feature map.

[0056] S153. Input the first feature map into the detection head of the teacher model for detection and determine the target detection result.

[0057] In some examples, the knowledge distillation structure in the gesture detection method provided in this disclosure embodiments is as follows: Figure 4As shown, the training image contains one or more training sample images. When training the student model, the training sample images can be input into the teacher model (Teacher BackboneNet) for convolution operations, resulting in one or more first feature maps. Then, each first feature map is input into a dynamic convolutional layer (DyamicKernels) for convolution operations, resulting in one or more second feature maps. Next, an attention mechanism is used to find the optimal second feature map as the first target feature map (e.g., when using the attention mechanism to find the optimal second feature map, it sorts the second feature maps according to their corresponding weight factors, selecting the second feature map with the largest weight factor as the optimal second feature map, and then using this second feature map as the first target feature map). Simultaneously, each first feature map is input into the teacher model's detection head (TeacherDetHead) for convolution operations, resulting in a heatmap, the width and height information of the detection boxes, and soft labels.

[0058] Specifically, in the following example, the soft label can be referred to as the target detection result.

[0059] As can be seen from the above, the gesture detection method provided in this embodiment of the present disclosure, by introducing a knowledge distillation framework and using a teacher model with dynamic convolutional layers and attention mechanisms to perform knowledge distillation on the student model, can improve the model accuracy of the student model and enhance its generalization ability to complex backgrounds, thus ensuring the user experience.

[0060] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 5 As shown, the above S16 can be specifically implemented through the following S160-S163.

[0061] S160. Input the training sample images into the student model to determine the second target feature map.

[0062] S161. Input the second target feature map into the detection head of the student model for detection, and determine the predicted regression result and the predicted classification result.

[0063] S162. Based on the target loss function, determine the first loss value between the second target feature map and the first feature map, predict the second loss value between the regression result and the target detection result, and predict the third loss value between the classification result and the labeling result.

[0064] S163. Adjust the network parameters of the student model according to the first loss value, the second loss value and the third loss value until the student model converges to obtain the pre-configured detection model.

[0065] In some examples, referring to the example given in S153 above, when training the student model, training sample images are input into the student model (Student BackboneNet) for convolution operations to obtain a second target feature map corresponding to the training sample image. Then, through forward propagation, a first loss value is calculated between the second target feature map and the first feature map using the target loss function. When this first loss value is greater than or equal to a first threshold, the network parameters of the student model and the teacher model are adjusted separately through backpropagation. Simultaneously, the second target feature map is input into the student model's detection head (Student DetHead) for convolution operations to obtain the predicted regression result and the predicted classification result. Then, through forward propagation, a second loss value is calculated between the predicted regression result and the target detection result using the target loss function. When the second loss value is greater than or equal to a second threshold, the network parameters of the student model are adjusted through backpropagation. Similarly, through forward propagation, a third loss value is calculated between the predicted classification result and the ground truth result using the target loss function. When the third loss value is greater than or equal to the third threshold, the network parameters of the student model are adjusted through backpropagation. Thus, after training, if the first loss value is less than the first threshold, the second loss value is less than the second threshold, and the third loss value is less than the third threshold, it indicates that the student network has converged, and the pre-configured detection model is obtained.

[0066] As can be seen from the above, the gesture detection method provided in this embodiment introduces a knowledge distillation framework and uses a teacher model with dynamic convolutional layers and attention mechanisms to perform knowledge distillation on the student model. This optimizes the supervised learning process of the pre-configured detection model, improves the model accuracy of the student model, enhances its generalization ability to complex backgrounds, and ensures the user experience.

[0067] As an optional implementation of this disclosure, the target loss function in the embodiments of this disclosure includes one or more of the following: a smooth minimum absolute deviation (LAD) loss function, a regression loss function, and a weighted cross-entropy loss function.

[0068] In some examples, the target loss function includes the smooth LAD loss function, the regression loss function, and the weighted loss function. In this case, the electronic device can determine a first loss value between the second target feature map and the first feature map based on the smooth LAD loss function. The electronic device can determine a second loss value between the predicted regression result and the target detection result based on the regression loss function. The electronic device can determine a third loss value between the predicted classification result and the labeled result based on the weighted loss function.

[0069] It should be noted that the above examples illustrate the use of a target loss function including smooth LAD loss, regression loss, and weighted loss. In other examples, the target loss function may include any one or two of the smooth LAD loss, regression loss, and weighted loss functions; this disclosure does not limit this. For example, when the target loss function includes smooth LAD loss, the electronic device can determine the first loss value between the second target feature map and the first feature map based on the smooth LAD loss function. The electronic device can determine the second loss value between the predicted regression result and the target detection result based on the smooth LAD loss function. The electronic device can determine the third loss value between the predicted classification result and the labeling result based on the smooth LAD loss function. Alternatively, when the target loss function includes smooth LAD loss and regression loss, the electronic device can determine the first loss value between the second target feature map and the first feature map based on the smooth LAD loss function. The electronic device can determine the second loss value between the predicted regression result and the target detection result based on the regression loss function. Electronic devices can determine a third loss value for the predicted classification and labeling results based on the smooth LAD loss function, which will not be elaborated here.

[0070] The foregoing primarily describes the solutions provided by the embodiments of the present invention from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0071] In this embodiment of the invention, the gesture detection device can be divided into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0072] like Figure 6 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a gesture detection device 10. The electronic device includes an acquisition unit 101 and a processing unit 102.

[0073] The acquisition unit 101 is used to acquire a gesture image to be recognized; the processing unit 102 is used to input the gesture image to be recognized acquired by the acquisition unit 101 into a pre-configured detection model to determine the actual regression result and the actual classification result; wherein, the pre-configured detection model is trained by knowledge distillation based on a teacher model with dynamic convolutional layers and attention mechanism; the processing unit 102 is also used to determine the detection result of the gesture image to be recognized based on the actual regression result and the actual classification result.

[0074] As an optional implementation of this disclosure, the training process of the pre-configured detection model is as follows: the acquisition unit 101 is further configured to acquire training sample images and labeling results of training sample images; wherein, the training sample images include foreground and background, the foreground includes gestures, and the labeling results include the detection results of gestures; the processing unit 102 is further configured to input the training sample images acquired by the acquisition unit 101 into a teacher model with dynamic convolutional layers and attention mechanisms to determine a first target feature map and a target detection result; the processing unit 102 is further configured to perform knowledge distillation training on the student model based on the first target feature map, the target detection result, the training sample images acquired by the acquisition unit 101, and the labeling results acquired by the acquisition unit 101 to obtain the pre-configured detection model.

[0075] As an optional implementation of this disclosure, the processing unit 102 is further configured to input the training sample image acquired by the acquisition unit 101 into the teacher model to obtain at least one first feature map; the processing unit 102 is further configured to input the first feature map into a dynamic convolutional layer for convolution operation to determine at least one second feature map; the processing unit 102 is further configured to process the second feature map using an attention mechanism to determine a first target feature map; the processing unit 102 is further configured to input the first feature map into the detection head of the teacher model for detection to determine the target detection result.

[0076] As an optional implementation of this disclosure, the processing unit 102 is further configured to input the training sample image acquired by the acquisition unit 101 into the student model to determine the second target feature map; the processing unit 102 is further configured to input the second target feature map into the detection head of the student model for detection to determine the predicted regression result and the predicted classification result; the processing unit 102 is further configured to determine, based on the target loss function, a first loss value between the second target feature map and the first feature map, a second loss value between the predicted regression result and the target detection result, and a third loss value between the predicted classification result and the labeling result; the processing unit 102 is further configured to adjust the network parameters of the student model according to the first loss value, the second loss value and the third loss value until the student model converges to obtain a pre-configured detection model.

[0077] As an optional implementation of this disclosure, the target loss function includes one or more of the following: a smooth minimum absolute deviation loss function, a regression loss function, and a weighted cross-entropy loss function.

[0078] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and their functions will not be repeated here.

[0079] Of course, the gesture detection device 10 provided in this embodiment of the invention includes, but is not limited to, the modules described above. For example, the gesture detection device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the gesture detection device 10, and may also be used to store data generated by the writing electronic device during operation, such as data in a write request.

[0080] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 7 As shown, the electronic device may include at least one processor 51, a memory 52, a communication interface 53, and a communication bus 54.

[0081] The following is combined with Figure 7 A detailed introduction to each component of the electronic device:

[0082] The processor 51 is the control center of the electronic device. It can be a single processor or a collective term for multiple processing elements. For example, the processor 51 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field-programmable gate arrays (FPGAs).

[0083] In a specific implementation, as one example, the processor 51 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram. Furthermore, as one embodiment, the electronic device may include multiple processors, such as... Figure 7 The processors 51 and 56 shown are illustrated. Each of these processors can be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0084] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0085] In a specific implementation, memory 52 is used to store data from this invention and the software program for executing this invention. Processor 51 can perform various functions of the air conditioner by running or executing the software program stored in memory 52 and by calling the data stored in memory 52.

[0086] Communication interface 53 uses any transceiver-like device for communicating with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, and the cloud. Communication interface 53 may include acquisition unit 101 to implement acquisition functions.

[0087] The communication bus 54 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0088] As an example, combined Figure 6 The functions implemented by the acquisition unit 101 in the gesture detection device 10 are the same as those of the acquisition unit 101 in the gesture detection device 10. Figure 7 The communication interface 53 in the gesture detection device 10 has the same function as the processing unit 102 in the gesture detection device 10. Figure 7 The processor 51 in the gesture detection device 10 has the same function as the storage unit 103 in the gesture detection device 10. Figure 7 The memory 52 in it has the same function.

[0089] Another embodiment of the present invention also provides a computer-readable storage medium that, when a computer program product is run on a computer, causes the computer to implement the method shown in the above-described method embodiment.

[0090] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0091] Figure 8A conceptual partial view of a computer program product provided in an embodiment of the present invention is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0092] In one embodiment, a computer program product is provided using a signal bearer medium 410. The signal bearer medium 410 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 1 The described function or part of the function. Therefore, for example, refer to... Figure 1 In the embodiment shown, one or more features of S11-S13 can be fulfilled by one or more instructions associated with the signal carrying medium 410. Furthermore, Figure 8 The program instructions in the document also describe example instructions.

[0093] In some examples, the signal carrying medium 410 may include a computer-readable medium 411, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0094] In some implementations, the signal carrying medium 410 may include a computer recordable medium 412, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0095] In some implementations, the signal carrying medium 410 may include a communication medium 413, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0096] The signal-bearing medium 410 can be transmitted by a wireless communication medium 413 (e.g., a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocols). One or more program instructions can be, for example, computer-executable instructions or logical implementation instructions.

[0097] In some examples, such as targeting Figure 6 The described gesture detection device can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 411, a computer-recordable medium 412, and / or a communication medium 413.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0099] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0103] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A gesture detection method, characterized in that, include: Acquire the image of the gesture to be recognized; The gesture image to be recognized is input into a pre-configured detection model to determine the actual regression result and the actual classification result; wherein, the pre-configured detection model is trained by knowledge distillation based on a teacher model with dynamic convolutional layers and attention mechanism; Based on the actual regression results and the actual classification results, the detection result of the gesture image to be recognized is determined; The training process of the pre-configured detection model includes: sequentially executing dynamic convolutional layers and attention mechanisms.

2. The gesture detection method according to claim 1, characterized in that, The training process of the pre-configured detection model is as follows: Obtain training sample images and their labeling results; wherein, the training sample images include foreground and background, the foreground includes gestures, and the labeling results include gesture detection results; The training sample images are input into a teacher model with dynamic convolutional layers and an attention mechanism to determine the first target feature map and the target detection result; The student model is trained by knowledge distillation based on the first target feature map, the target detection result, the training sample image, and the labeling result to obtain the pre-configured detection model.

3. The gesture detection method according to claim 2, characterized in that, The step of inputting the training sample images into a teacher model with dynamic convolutional layers and an attention mechanism to determine the first target feature map and the target detection result includes: The training sample images are input into the teacher model to obtain at least one first feature map; The first feature map is input into a dynamic convolutional layer for convolution operation to determine at least one second feature map; An attention mechanism is used to process the second feature map to determine the first target feature map; The first feature map is input into the detection head of the teacher model for detection, and the target detection result is determined.

4. The gesture detection method according to claim 2, characterized in that, The step of training the student model through knowledge distillation based on the first target feature map, the target detection result, the training sample image, and the labeling result to obtain the pre-configured detection model includes: The training sample images are input into the student model to determine the second target feature map; The second target feature map is input into the detection head of the student model for detection, and the predicted regression result and predicted classification result are determined. Based on the target loss function, a first loss value is determined between the second target feature map and the first feature map, a second loss value is determined between the predicted regression result and the target detection result, and a third loss value is determined between the predicted classification result and the labeling result; Based on the first loss value, the second loss value, and the third loss value, the network parameters of the student model are adjusted until the student model converges, thus obtaining the pre-configured detection model.

5. The gesture detection method according to claim 4, characterized in that, The target loss function includes one or more of the following: a smooth minimum absolute deviation loss function, a regression loss function, and a weighted cross-entropy loss function.

6. A gesture detection device, characterized in that, include: The acquisition unit is used to acquire the image of the gesture to be recognized; The processing unit is used to input the gesture image to be recognized acquired by the acquisition unit into a pre-configured detection model to determine the actual regression result and the actual classification result; wherein, the pre-configured detection model is obtained by knowledge distillation training based on a teacher model with dynamic convolutional layers and attention mechanism; The processing unit is further configured to determine the detection result of the gesture image to be recognized based on the actual regression result and the actual classification result; The training process of the pre-configured detection model includes: sequentially executing dynamic convolutional layers and attention mechanisms.

7. The gesture detection device according to claim 6, characterized in that, The training process of the pre-configured detection model is as follows: The acquisition unit is further configured to acquire training sample images and labeling results of the training sample images; wherein, the training sample images include foreground and background, the foreground includes gestures, and the labeling results include detection results of gestures; The processing unit is further configured to input the training sample image acquired by the acquisition unit into a teacher model with dynamic convolutional layers and attention mechanisms to determine the first target feature map and the target detection result; The processing unit is further configured to perform knowledge distillation training on the student model based on the first target feature map, the target detection result, the training sample image obtained by the acquisition unit, and the labeling result obtained by the acquisition unit, to obtain the pre-configured detection model.

8. The gesture detection device according to claim 7, characterized in that, The processing unit is further configured to input the training sample image acquired by the acquisition unit into the teacher model to obtain at least one first feature map; The processing unit is further configured to input the first feature map into a dynamic convolutional layer for convolution operation to determine at least one second feature map; The processing unit is further configured to process the second feature map using an attention mechanism to determine the first target feature map; The processing unit is further configured to input the first feature map into the detection head of the teacher model for detection, and determine the target detection result.

9. The gesture detection device according to claim 7, characterized in that, The processing unit is further configured to input the training sample image acquired by the acquisition unit into the student model to determine the second target feature map; The processing unit is further configured to input the second target feature map into the detection head of the student model for detection, and determine the predicted regression result and the predicted classification result; The processing unit is further configured to determine, based on the target loss function, a first loss value between the second target feature map and the first feature map, a second loss value between the predicted regression result and the target detection result, and a third loss value between the predicted classification result and the labeling result; The processing unit is further configured to adjust the network parameters of the student model according to the first loss value, the second loss value and the third loss value until the student model converges to obtain the pre-configured detection model.

10. The gesture detection device according to claim 9, characterized in that, The target loss function includes one or more of the following: a smooth minimum absolute deviation loss function, a regression loss function, and a weighted cross-entropy loss function.

11. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to cause the electronic device to implement the gesture detection method according to any one of claims 1-5 when executing the computer program.

12. A computer program product, characterized in that, When the computer program product is run on a computer, the computer implements the gesture detection method as described in any one of claims 1-5.

Images