Gesture detection methods, devices, electronic devices, and storage media

Abstract

This application relates to a gesture detection method, apparatus, electronic device, and storage medium. The gesture detection method includes: acquiring a single-channel image to be detected; inputting the image to be detected into the input layer of a trained gesture detection model, converting the image from a single channel to multiple feature channels to obtain a multi-channel feature map, and performing hierarchical feature extraction processing on the multi-channel feature map via the backbone network of the gesture detection model to obtain first-level features and second-level features; inputting the first-level features and second-level features into the detection layer of the gesture detection model for parallel detection, outputting detection boxes, and generating a target gesture detection result based on the detection boxes. This application solves the problem of missed detections and false detections that easily occur when performing gesture detection on single-channel images.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to gesture detection methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Gesture interaction is one of the most natural and intuitive human-computer interaction methods in augmented reality (AR) applications. To achieve a smooth user experience on wearable devices such as AR glasses, an algorithm that can accurately detect the position and type of the user's hands in real time is needed.

[0003] Currently, general-purpose object detection algorithms, represented by the YOLO series, perform well on various public datasets. However, their model structures are complex, with a huge number of parameters and computational costs (GFLOPs). If they are directly deployed on edge devices such as AR glasses, which are characterized by low power consumption and low computing power, they will face problems such as excessive inference latency, excessive power consumption, and severe device overheating, failing to meet the basic requirements of real-time interaction.

[0004] Furthermore, to minimize power consumption and data transmission bandwidth, embedded devices such as AR glasses typically employ single-channel (monochrome) image sensors, directly outputting or processing single-channel grayscale images. However, the vast majority of existing mainstream detection models are designed for processing three-channel (RGB) color images, with their network input layers and pre-trained weights based on three-channel data. Directly applying such models to single-channel grayscale images not only fails to leverage their original design advantages but may also lead to low information processing efficiency or accuracy loss due to channel mismatch.

[0005] Currently, no effective solution has been proposed to address the issues of missed detections and false detections that easily occur when performing gesture detection on single-channel images in related technologies. Summary of the Invention

[0006] This application provides a gesture detection method, apparatus, electronic device, and storage medium to at least solve the problem of missed detection and false detection that easily occur when performing gesture detection on single-channel images in related technologies.

[0007] In a first aspect, embodiments of this application provide a gesture detection method, the method comprising:

[0008] Acquire the image to be detected in a single channel;

[0009] The image to be detected is input into the input layer of the trained gesture detection model, and the image to be detected is converted from the single channel to multiple feature channels to obtain a multi-channel feature map. The multi-channel feature map is then subjected to hierarchical feature extraction processing through the backbone network in the gesture detection model to obtain the first-level features and the second-level features.

[0010] The first-level features and the second-level features are respectively input into the detection layer of the gesture detection model for parallel detection, outputting each set of detection boxes, and generating the target gesture detection result based on each set of detection boxes.

[0011] In some embodiments, the hierarchical feature extraction process performed on the multi-channel feature map via the backbone network of the gesture detection model to obtain first-level features and second-level features includes:

[0012] The multi-channel feature map is input into the initial convolutional layer of the backbone network for processing, and a shallow extracted feature map is output.

[0013] The shallow extracted feature map is input into multiple downsampling convolutional layers and parallel feature fusion modules in the backbone network; the shallow extracted feature map is processed sequentially and alternately by each downsampling convolutional layer and the parallel feature fusion module to obtain a deep extracted feature map, and the highest layer feature in the deep extracted feature map is retained as the first level feature, and the second highest layer feature in the deep extracted feature map is retained as the second level feature.

[0014] In some embodiments, the process of sequentially and alternately processing the shallow extracted feature map via each of the downsampling convolutional layers and the parallel feature fusion module to obtain the deep extracted feature map includes:

[0015] The shallow extracted feature map is input into the downsampling convolutional layer at the current level for downsampling processing to obtain the extracted feature map at the current level;

[0016] The extracted feature map at the current level is input to the parallel feature fusion module at the current level. The extracted feature map at the current level is processed through the first feature branch and the second feature branch in the parallel feature fusion module, respectively, and the first feature and the second feature are output. The first feature and the second feature are then concatenated to obtain the fused feature map at the current level. The fineness of the second feature is higher than that of the first feature.

[0017] The fused feature map of the current level is input into the downsampling convolutional layer of the next level to obtain the extracted feature map of the next level. The extracted feature map of the next level is then input into the parallel feature fusion module of the next level until the fused feature map of the highest level is obtained. The deep extracted feature map includes fused feature maps of different levels. The stacking degree of the parallel feature fusion module of the next level is greater than that of the parallel feature fusion module of the current level.

[0018] In some embodiments, the process of processing the extracted feature map at the current level via the first feature branch and the second feature branch in the parallel feature fusion module, and outputting the first feature and the second feature, includes:

[0019] The extracted feature maps at the current level are respectively input into the first feature branch and the second feature branch in parallel;

[0020] The first feature is obtained by sequentially performing a two-layer chained process on the extracted feature map at the current level through the first convolutional layer and the second convolutional layer of the first feature branch; the convolutional kernels between the first convolutional layer and the second convolutional layer are different.

[0021] The second feature is obtained by performing a single-layer transformation on the extracted feature map at the current level through the third convolutional layer of the second feature branch.

[0022] In some embodiments, concatenating the first feature and the second feature to obtain a fused feature map at the current level includes:

[0023] The first feature and the second feature are aggregated to obtain an aggregated feature map;

[0024] Weight learning is performed based on the aggregated feature map to obtain a first weight and a second weight. Based on the first weight and the second weight, the first feature and the second feature are concatenated to obtain the fused feature map.

[0025] In some embodiments, the gesture detection model further includes a detection head; the step of inputting the first-level features and the second-level features into the detection layer of the gesture detection model for parallel detection and outputting each set of detection boxes includes:

[0026] The first-level features and the second-level features are respectively input into the first detection branch and the second detection branch in the detection head, and the first-level feature map and the second-level feature map are output.

[0027] The first-level feature map and the second-level feature map are input into the detection layer in the detection head, and the detection boxes are output.

[0028] In some embodiments, generating target gesture detection results based on the respective sets of detection boxes includes:

[0029] Based on the comparison results between the confidence scores of each group of detection boxes and the preset confidence threshold, candidate detection boxes are retained from each group of detection boxes;

[0030] The candidate detection boxes are traversed, and the target detection box is determined based on the degree of overlap between the current candidate detection box and each candidate detection box. The target gesture detection result is generated based on the target detection box.

[0031] Secondly, embodiments of this application provide a gesture detection device, including:

[0032] The acquisition module is used to acquire the image to be detected in a single channel.

[0033] The hierarchical extraction module is used to input the image to be detected into the input layer of the trained gesture detection model, convert the image to be detected from the single channel to multiple feature channels to obtain a multi-channel feature map, and perform hierarchical feature extraction processing on the multi-channel feature map through the backbone network in the gesture detection model to obtain the first-level features and the second-level features.

[0034] The detection module is used to fuse the first-level features and the second-level features using the gesture detection model to obtain a fused feature map, and to perform detection based on the fused feature map to generate a target gesture detection result.

[0035] Thirdly, embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the gesture detection method as described in the first aspect above.

[0036] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the gesture detection method as described in the first aspect above.

[0037] Compared to related technologies, the gesture detection method, apparatus, electronic device, and storage medium provided in this application acquire a single-channel image to be detected; input the image to be detected into the input layer of a trained gesture detection model, converting the image from a single channel to multiple feature channels to obtain a multi-channel feature map; perform hierarchical feature extraction processing on the multi-channel feature map through the backbone network in the gesture detection model to obtain first-level features and second-level features; input the first-level features and second-level features into the detection layer of the gesture detection model for parallel detection, output each set of detection boxes, and generate the target gesture detection result based on each set of detection boxes.

[0038] By inputting the acquired single-channel image to be detected into the trained gesture detection model, the input layer first transforms it from a single channel to multiple feature channels to obtain a multi-channel feature map. This multi-channel approach enriches the image information and reduces the risk of feature loss due to the limited information in a single channel. Next, the backbone network performs hierarchical feature extraction on the multi-channel feature map, obtaining features at different levels. These different levels of features can capture information at different scales and semantic levels, enhancing the ability to characterize complex gesture features. Then, the first and second level features are input into the detection layer for parallel detection, outputting various detection boxes. Based on these boxes, the target gesture detection result is generated. This parallel detection and comprehensive utilization of different level features fully leverages the multi-dimensional information of the image, effectively improving the accuracy and comprehensiveness of gesture detection. This effectively addresses the problem of missed and false detections that easily occur when performing gesture detection on single-channel images.

[0039] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 This is a hardware structure block diagram of a terminal for a gesture detection method according to an embodiment of this application;

[0042] Figure 2 This is a flowchart of a gesture detection method according to an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of a gesture detection model according to an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the internal structure of a CBL module according to an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the internal structure of a parallel feature fusion module according to an embodiment of this application;

[0046] Figure 6 This is a structural block diagram of a gesture detection device according to an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0048] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of a terminal for a gesture detection method according to an embodiment of this application. Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0051] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the gesture detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0053] This embodiment provides a gesture detection method. Figure 2 This is a flowchart of a gesture detection method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0054] Step S210: Obtain the image to be detected in a single channel.

[0055] In this embodiment, it can be applied to various scenarios requiring real-time gesture interaction, and is particularly suitable for edge devices that use single-channel cameras to save power. For example, this application scenario is AR / VR interaction, where the monochrome camera, as the core interaction technology of AR / VR glasses, enables bare-hand menu selection, virtual object dragging, and other operations. Alternatively, this application scenario is a smart cockpit, which is applied in the cockpit of a car, using infrared or ordinary monochrome cameras to capture gestures and control the in-vehicle infotainment system, improving driving safety. Another application scenario is smart homes, where, by deploying low-power monochrome cameras or smart terminals, users can control smart devices such as lights and curtains with gestures. Yet another application scenario is human-machine collaborative robots, where, in the field of industrial or service robots, operators can issue commands to robots with gestures, achieving more natural and efficient human-machine collaborative work.

[0056] Accordingly, the aforementioned image to be detected refers to a single-channel grayscale image of the gesture to be detected, acquired by an edge device; for example, a grayscale image frame directly acquired from a monochrome camera of an AR device, or a grayscale image obtained by real-time conversion of a color image frame. As described in the background section, to minimize power consumption and data transmission bandwidth, edge devices such as AR glasses typically employ single-channel image sensors to directly output or process single-channel grayscale images. These edge devices include, but are not limited to, various AR / VR devices, infrared or ordinary monochrome cameras, etc.

[0057] Step S220: Input the image to be detected into the input layer of the trained gesture detection model, convert the image to be detected from a single channel to multiple feature channels to obtain a multi-channel feature map, and perform hierarchical feature extraction processing on the multi-channel feature map through the backbone network in the gesture detection model to obtain the first-level features and the second-level features.

[0058] It should be noted that the training dataset for the aforementioned gesture detection model mainly consists of a series of single-channel grayscale training images. These single-channel grayscale training images can be sourced from gesture images captured by a single-channel device in historical or standard scenarios, or they can be obtained by converting collected color gesture images to single-channel grayscale format. Furthermore, in this embodiment, global parameters are preset for the gesture detection model, such as a model depth coefficient (0.33), a width coefficient (0.25), and two sets of anchor boxes optimized for gesture size.

[0059] Specifically, after the standardized single-channel image is input into the input layer of the gesture detection network, this layer is configured to expand the single-channel pixel data into a multi-channel feature map (e.g., 16 channels) through the first-layer convolution operation, completing the initial encoding of spatial features. For example, the input image is first convolved with a set of learnable convolutional kernels (e.g., 16 1×1 convolutional kernels). Each convolutional kernel extracts features of specific patterns (e.g., edges, texture directions) on the image through a sliding window mechanism, and maps these features to independent channels, thereby converting the original single-channel image into a multi-channel feature map.

[0060] Subsequently, the aforementioned multi-channel feature maps are fed into the backbone network for hierarchical processing: the network downsamples and deepens the feature representation through multiple convolutional stages, with shallow stages preserving high-resolution details (such as edge textures) and deep stages extracting abstract semantics (such as the overall structure of gestures). The final output consists of two key features: first-level features (highest-level high-level semantic features, low resolution but rich semantics) and second-level features (second-highest-level mid-level semantic features, balancing details and semantics), forming complementary feature representations. This ultimately achieves a mapping and hierarchical abstraction from the original pixel space to the deep feature space.

[0061] Step S230: Input the first-level features and the second-level features into the detection layer of the gesture detection model for parallel detection, output each set of detection boxes, and generate the target gesture detection result based on each set of detection boxes.

[0062] In the target gesture detection process, this step achieves efficient recognition through a two-level feature parallel detection mechanism. Specifically, the extracted first-level features (deep high-semantic features) and second-level features (second-high-level detail-preserving features) are simultaneously input into two independently optimized detection heads. The high-level feature detection head focuses on recognizing the overall shape of large-scale gestures based on its wide receptive field, while the second-high-level feature detection head uses its preserved spatial details to accurately locate medium-scale gestures. Each detection head performs bounding box regression (predicting the precise coordinates x, y, width, height of the gesture) and classification confidence calculation (outputting the probability distribution of each gesture category) to generate two sets of candidate detection boxes. Finally, the two sets of detection boxes are fused and deduplicated using the non-maximum suppression (NMS) algorithm, overlapping predictions are filtered out, and the optimal detection result is selected based on the confidence threshold. The output is a target detection result containing the gesture category and precise bounding box coordinates, thereby improving the robustness of recognition of gestures of different scales while ensuring real-time performance.

[0063] Through steps S210 to S230, the acquired single-channel image to be detected is input into the trained gesture detection model. First, the input layer transforms the single-channel image into a multi-channel feature map, enriching the image information and reducing the risk of feature loss due to limited single-channel information. Then, the backbone network performs hierarchical feature extraction on the multi-channel feature map, obtaining features at different levels. These different levels of features can capture information at different scales and semantic levels, enhancing the ability to characterize complex gesture features. Next, the first and second level features are input into the detection layer for parallel detection, outputting detection boxes. Based on these boxes, the target gesture detection result is generated. Parallel detection and comprehensive utilization of different level features fully leverage multi-dimensional image information, effectively improving the accuracy and comprehensiveness of gesture detection. This effectively addresses the problem of missed and false detections that easily occur when performing gesture detection on single-channel images.

[0064] In some embodiments, the hierarchical feature extraction process of the multi-channel feature map via the backbone network in the gesture detection model to obtain the first-level features and the second-level features may further include the following steps:

[0065] The multi-channel feature map is input into the initial convolutional layer in the backbone network for processing, and a shallow extracted feature map is output. The shallow extracted feature map is then input into multiple downsampling convolutional layers and a parallel feature fusion module in the backbone network. The shallow extracted feature map is processed alternately by each downsampling convolutional layer and the parallel feature fusion module to obtain a deep extracted feature map. The highest-level feature in the deep extracted feature map is retained as the first-level feature, and the second-highest-level feature in the deep extracted feature map is retained as the second-level feature.

[0066] When the multi-channel feature map is input into the initial convolutional layer of the backbone network for processing, the initial convolutional layer performs local feature extraction on the input multi-channel feature map through convolution operations. It uses learnable convolutional kernels to scan each local region of the feature map, capturing low-level visual features such as basic texture and edges of the image. These shallowly extracted feature maps serve as the basis for subsequent deep feature extraction, effectively preserving the rich local detail information after the single-channel image is converted to multi-channel, providing more complete low-level feature support for subsequent detection, thereby reducing the risk of missed detection caused by limited single-channel image information, such as blurred gesture edges and missing textures.

[0067] After the shallow feature maps are input into multiple downsampling convolutional layers and a parallel feature fusion module in the backbone network, each downsampling convolutional layer gradually reduces the spatial dimension of the feature maps through convolution or pooling operations with a stride greater than 1, while increasing the number of channels to extract higher-level semantic features (such as abstract features like the shape and structure of gestures). The parallel feature fusion module then fuses features from different downsampling stages through cross-layer connections or feature concatenation, preserving the detailed information of shallow features while integrating the semantic information of deep features. This alternating processing mechanism enables the deep feature maps to simultaneously possess the dual expressive power of local details and global semantics, effectively solving the problem of gesture misjudgment caused by the single-channel image's limited feature representation.

[0068] The highest-level features extracted from the deep feature map are retained as the first-level features, and the next-highest-level features as the second-level features. This is because the highest-level features, after multiple downsampling steps, have the smallest spatial size and the largest receptive field, enabling them to capture the global semantic information of gestures (such as the overall gesture category and approximate location), making them suitable for detecting gestures of larger scale or complex structures. The next-highest-level features, on the other hand, retain more spatial detail information (such as the degree of finger bending and joint position), making them suitable for detecting gestures of smaller scale or with obvious local features. By detecting these two types of features in parallel and combining them to output a detection box, we can utilize both global features to avoid missing large-scale gestures and local features to avoid false detections of small-scale non-gesture regions. The final target gesture detection results are significantly improved in both accuracy and comprehensiveness.

[0069] For example, the specific process of feature extraction described above is as follows: First, the input image to be detected is processed by Conv(s=2), reducing the resolution and expanding the receptive field, allowing for the extraction of basic edge features. After another Conv(s=2), the resolution is further reduced, and the receptive field expands again, enabling the network to learn some simple textures and component combinations. During the generation of P3, P4, and P5, each downsampling by Conv(s=2) is a compression of spatial information and a refinement of semantic information. Simultaneously, at each scale, feature refinement is performed through stacked C3Libra modules (a self-designed balanced convolutional module). These modules, without changing the resolution, repeatedly perform deep processing and fusion of features at the current scale through their dual-path parallel structure, resulting in richer and more robust semantic representation. Finally, the P5 feature map generated after the 7th convolutional layer has the lowest resolution, but each pixel contains the highest level of abstract understanding of the global information of the original image, i.e., the "highest-level semantic feature map." It is best suited for determining the presence of large targets such as "hands" in an image.

[0070] In some embodiments, the shallow extracted feature map is processed sequentially and alternately by each downsampling convolutional layer and parallel feature fusion module to obtain a deep extracted feature map, including:

[0071] The shallow extracted feature map is input into the downsampling convolutional layer of the current layer for downsampling processing to obtain the extracted feature map of the current layer;

[0072] The extracted feature map at the current level is input into the parallel feature fusion module at the current level. The extracted feature map at the current level is processed through the first feature branch and the second feature branch in the parallel feature fusion module, respectively, and the first feature and the second feature are output. The first feature and the second feature are then concatenated to obtain the fused feature map at the current level. The fineness of the second feature is higher than that of the first feature.

[0073] The fused feature map of the current level is input into the downsampling convolutional layer of the next level to obtain the extracted feature map of the next level. The extracted feature map of the next level is then input into the parallel feature fusion module of the next level until the fused feature map of the highest level is obtained. The deep extracted feature map includes the fused feature maps of different levels. The stacking degree of the parallel feature fusion module of the next level is greater than that of the parallel feature fusion module of the current level.

[0074] Specifically, after the above steps, by using a convolutional layer with a stride of 2 to reduce the resolution of the multi-channel feature map from P0 to P2 level, completing the initial shallow feature extraction and obtaining the extracted feature map at the current level, the extracted feature map at the current level is alternately passed through downsampling convolutional layers and multiple stacked dual-path parallel feature fusion modules; this process is repeated until the highest-level semantic feature map at P5 level is generated. The following is a detailed explanation of the above deep feature extraction process.

[0075] The shallow feature map is first input to the downsampling convolutional layer of the current level (such as Stage 1). This layer performs spatial dimensionality reduction operation through a convolutional kernel with a stride of 2 and outputs the extracted feature map of the current level. At this time, the spatial information is compressed but the semantic information is initially improved.

[0076] Subsequently, the extracted feature map at the current level is input to the parallel feature fusion module at the same level. The module refines features through dual-branch heterogeneous processing: in the first feature branch, local spatial features with a larger receptive field are extracted, and the first feature with medium fine granularity is output; in the second feature branch, the features are only linearly transformed to avoid information loss caused by spatial convolution, and the second feature with higher fine granularity is output, which retains the original pixel-level positional accuracy and can preserve microscopic details such as fingertip position and nail edge; then the dual outputs are spliced ​​and fused through the channel dimension, and finally the fused feature map at the current level is output.

[0077] The fused feature map is then fed into the next level (such as Stage 2) downsampling convolutional layer, initiating a new processing loop: the resolution decreases further, while the receptive field expands further; the features are then refined again in the parallel fusion module of this level. At this stage, the fine-grained features output by the second feature branch begin to shift from geometric details to semantic details (such as the relative positional relationship of finger joints); this process is repeated iteratively, with each level's output retaining the fused feature map of the current level, forming a feature pyramid structure: low-level features have high resolution and strong fine-grainedness but low semantic abstraction, while high-level features have low resolution and weak fine-grainedness but contain rich global semantics (such as gesture category intent).

[0078] Ultimately, the deep feature map set contains multi-level fused feature maps. The highest-level feature is selected as the first-level feature (with the lowest resolution but each pixel encodes complete gesture semantics, suitable for large-scale detection), and the next-highest-level feature is selected as the second-level feature (preserving the spatial relationship of gesture components, suitable for medium-scale target localization).

[0079] For more details, please refer to Figure 3The figure primarily illustrates the processing flow of the backbone network in the gesture detection model. A series of single-channel images of size 320×320×1 are input to the output layer of the gesture detection model, resulting in a multi-channel feature map. This multi-channel feature map is first convolved by the initial convolutional layer (CBL) in the backbone network, outputting a shallow feature map. This shallow feature map is then processed alternately by multiple downsampling layers (CBL) and parallel fusion feature modules (C3Libra). The stacking degree of the parallel fusion feature modules varies across different levels; for example, the stacking degree of the first-level parallel fusion module is 1, the second-level is 2, and the third-level is 3. The second-level features (i.e., the P4 feature map) are directly output from the third level to the subsequent detection head. Simultaneously, the feature map output from this third level undergoes an additional processing layer with a stacking degree of 1 to output the first-level features (i.e., the P5 feature map). The first-level and second-level features are then input into the detection head in the gesture detection model, and are further processed by the detection head and prediction head to finally output the target gesture detection result.

[0080] Please continue reading. Figure 4 The figure also shows the internal structure of the CBL module. In this CBL (Conv, BatchNormalization, Leaky ReLU) module, the input data is processed sequentially through the Conv convolutional layer, the BN batch normalization layer, and the Leaky activation function.

[0081] Through the above embodiments, the receptive field is gradually expanded by downsampling, enabling the network to upgrade from local texture perception to global gesture understanding. The fine-grained dual-path design of each fusion module continuously protects key details during spatial compression, and the pyramid-shaped feature structure provides scale-adaptive input for subsequent detection layers. Thus, through a closed loop of spatial compression, feature refinement, and fine-grained conservation, the optimal balance between accuracy and efficiency can be achieved on resource-constrained devices.

[0082] In some embodiments, the above-mentioned processing of the extracted feature map at the current level via the first feature branch and the second feature branch in the parallel feature fusion module to output the first feature and the second feature may further include the following steps:

[0083] The extracted feature map at the current level is input into the first feature branch and the second feature branch in parallel. The extracted feature map at the current level is processed in a double-layer chain through the first convolutional layer and the second convolutional layer of the first feature branch to obtain the first feature. The convolutional kernels between the first convolutional layer and the second convolutional layer are different. The extracted feature map at the current level is processed in a single layer transformation through the third convolutional layer of the second feature branch to obtain the second feature.

[0084] In this step, the feature maps extracted at the current level are simultaneously distributed to two structurally heterogeneous processing paths to perform differentiated feature extraction.

[0085] Specifically, in the first feature branch, the input features first enter the first convolutional layer (usually a 1×1 convolution). This layer performs cross-channel information recombination through pointwise convolutional kernels (mathematically, a linear transformation of the channel dimension), compressing or expanding the channel dimension without changing the spatial resolution, and generating transitional features. Subsequently, the transitional features are input to the second convolutional layer (usually a 3×3 convolution), and weighted summation is performed in the local neighborhood (3×3 window) through spatial convolutional kernels to capture spatial correlations (such as edge direction and texture continuity), and finally output the first feature containing structured semantics (such as palm outline and finger gap boundaries).

[0086] Meanwhile, in the second feature branch, the input features undergo a single-step linear mapping via another independent third convolutional layer (e.g., a 1×1 convolution). This operation is essentially a lightweight affine transformation, ultimately outputting a second feature with significantly higher granularity. This second feature retains more of the original feature information compared to the first feature.

[0087] in, Figure 5 This is a schematic diagram of the internal structure of a parallel feature fusion module according to an embodiment of this application, as shown below. Figure 5 As shown, in this parallel feature fusion module (C3Libra), the input feature map x at the current level is simultaneously fed into two parallel processing branches within the C3Libra module. The first feature branch (deep feature path) involves x undergoing a 1×1 convolution for channel transformation, followed by a 3×3 convolution to extract local spatial features with a larger receptive field, resulting in the output first feature y1. The second feature branch (direct feature path) involves x undergoing another independent 1×1 convolution for lightweight linear transformation, resulting in the output second feature y2. This path retains more original, fine-grained feature information. Then, the first feature y1 and the second feature y2 are concatenated along the channel dimension (Concat) to form a wider, more information-rich feature map y_cat. Finally, a 1×1 convolution is used to process y_cat, performing dimensionality reduction and cross-channel information integration to obtain the final output of the C3Libra module at this level.

[0088] In the above embodiments, the first feature branch adopts a chain structure of "1×1 convolution + 3×3 convolution". The 1×1 convolution first compresses the number of channels to reduce the input dimension of the subsequent 3×3 convolution (reducing computational cost), while the 3×3 convolution captures local spatial features through a larger receptive field to ensure semantic expression. The second feature branch uses only 1×1 convolution for lightweight linear transformation, preserving the original fine-grained features with extremely low computational cost. When the two paths are processed in parallel, the first feature branch focuses on complex feature extraction, while the second feature branch focuses on efficient information preservation. Overall, through differentiated combination of computational modules and channel number adjustment, the number of operations is significantly reduced while ensuring feature richness, thereby effectively improving the computational efficiency of the model in single-channel image gesture detection tasks.

[0089] In some embodiments, the above-mentioned concatenation of the first feature and the second feature to obtain the fused feature map at the current level may further include the following steps:

[0090] The first feature and the second feature are aggregated to obtain an aggregated feature map; weights are learned based on the aggregated feature map to obtain a first weight and a second weight, and the first feature and the second feature are concatenated based on the first weight and the second weight to obtain a fused feature map.

[0091] Specifically, the first and second features output from the two branches are first aggregated, for example, by element-wise addition (y_agg = y1 + y2), to generate an aggregated feature map to capture the global context of the dual-path features. Then, attention-gated generation is performed on this aggregated feature map: features are compressed along the spatial dimension using global average pooling, compressing [Batch, Channel, H, W] into a feature vector of dimension [Batch, Channel]. This vector can be seen as the global response of the current feature map on each channel. The compressed feature vector is then incorporated into a small feedforward neural network (usually composed of two fully connected layers, also known as an MLP) to learn the non-linear dependencies between channels. The fully connected layer FC1 reduces the number of channels from C to a smaller value (e.g., C / r, where r is the reduction rate) and activates it with ReLU; the other fully connected layer FC2 restores the number of channels to the C dimension. The output vector from FC2 is passed through a Sigmoid activation function to generate the final weight vector alpha, where each element has a value between 0 and 1. The dimension of alpha is [Batch, Channel]. To achieve weighting, we can set beta = 1 - alpha, which creates a complementary weighting relationship.

[0092] Thus, the generated weight vectors alpha and beta are applied to the outputs of the two branches respectively for feature recalibration. This is a broadcast, channel-by-channel multiplication operation, as shown in the following formula:

[0093] y1_att = alpha × y1;

[0094] y2_att = beta × y2;

[0095] Through this step, the network dynamically enhances the feature channels it deems more important (e.g., channels with larger alpha values ​​correspond to important edge features) while suppressing less important feature channels. The recalibrated feature maps are then concatenated, and finally fused using a 1×1 convolution. This ensures that the first feature emphasizes hand edges, and the second feature emphasizes hand joint positions, achieving dynamic binding with the gesture detection task. This concatenation and fusion process is illustrated in the following formula:

[0096] y_cat = torch.cat((y1_att, y2_att), 1);

[0097] output = cv3(y_cat);

[0098] Through the above embodiments, the model no longer statically fuses all features, but can dynamically adjust the emphasis on edge and joint semantic information based on the content of the input image (e.g., whether it is an open palm or a clenched fist), achieving intelligent feature selection. By adaptively weighting different types of features, the model can more accurately capture key discriminative information of gestures, thus exhibiting higher detection accuracy and stronger robustness in complex backgrounds, lighting changes, or pose variations. This improvement explicitly incorporates prior knowledge of gesture recognition tasks (the importance of edges and joints) into the network structure design, making the C3Libra-A module a highly efficient component "tailor-made" for gesture recognition tasks. In addition, the introduced attention gating unit (global pooling + two fully connected layers) has extremely low computational cost, almost no increase in the overall inference latency of the model, fully meeting the lightweight requirements for deployment on AR devices.

[0099] In some embodiments, the gesture detection model further includes a detection head; the above-mentioned inputting the first-level features and the second-level features into the detection layer of the gesture detection model for parallel detection and outputting each set of detection boxes may further include the following steps:

[0100] The first-level features and the second-level features are respectively input into the first detection branch and the second detection branch in the detection head, and the first-level feature map and the second-level feature map are output. The first-level feature map and the second-level feature map are input into the detection layer in the detection head, and each group of detection boxes is output.

[0101] In this step, a hierarchical-task binding strategy is used to achieve efficient multi-scale gesture detection: the first-level features are input to the first detection branch, which is optimized for large-scale gestures, focusing on global gesture shape recognition (such as the palm region and fist posture), and outputting a first-level feature map containing position regression and classification confidence; at the same time, the second-level features (mid-level detail features, 14×14 resolution) are input to the second detection branch, which is designed for medium-sized target detection and outputs a second-level feature map with precise localization. Finally, the two feature maps are simultaneously input into the detection layer (usually a 1×1 convolutional layer).

[0102] The detection layer receives multiple feature maps from the detection head, namely the P4 and P5 detection feature maps. These two feature maps have different resolutions and receptive fields, and are better suited to detecting gestures of different sizes. For each input feature map (P4 and P5), the detection layer applies one or more 1×1 convolutional layers as the "prediction head". The number of output channels of this 1×1 convolution is carefully calculated. The formula is: num_anchors × (num_classes + 5). In this formula, num_anchors represents the number of pre-defined anchor boxes at each feature map grid point. For example, there are 3 anchor boxes per scale. num_classes represents the number of classes to be detected (e.g., left hand, right hand, nc=2). 5 represents 4 bounding box coordinates (x, y, w, h) and 1 objectness score. For example, for a P4 feature map, if its size is 20×20, with 3 anchor boxes and 2 classes, then after a 1×1 convolution, the output feature map size will be 20 ×20 × [3 × (2 + 5)], which is 20 × 20 × 21.

[0103] The output 20 × 20 × 21 tensor contains all the original prediction information at the P4 scale. The detection layer needs to decode it. For each grid point on the P4 feature map (20 × 20 = 400 in total), and for each anchor box (3 in total) at each grid point, the model predicts 7 values:

[0104] ①x, ty, tw, th: The offset and scaling factor of the bounding box coordinates.

[0105] ②p_obj: Target confidence, representing the probability that the box contains a "hand".

[0106] ③c1, c2: Category confidence, representing the probability that if it is a hand, it is either "left hand" or "right hand".

[0107] The detection layer uses a decoding formula similar to YOLO to calculate the absolute coordinates (x_center, y_center, width, height) of the bounding box on the original image by combining these relative offsets and scaling factors with the position of the current grid point and the size of the corresponding anchor box.

[0108] In some embodiments, the above-described method of inputting the first-level features and the second-level features into the first detection branch and the second detection branch in the detection head, respectively, and outputting the first-level feature map and the second-level feature map, may further include the following steps:

[0109] The first detection branch upsamples the second-level features to obtain a first target feature whose resolution matches that of the first-level features. The first target feature is then concatenated with the first-level features to output a first-level feature map. The second detection branch upsamples the first-level features to obtain a second target feature whose resolution matches that of the second-level features. The second target feature is then concatenated with the second-level features to output a second-level feature map.

[0110] More specifically, the detection head is responsible for fusing multi-scale features and generating prediction results. The process of generating P4-level detection features is as follows: The P5 feature map from the highest layer of the backbone network is taken and channel-adjusted using a 1×1 convolution. It is then upsampled by a factor of 2 to match the resolution of the P4-level feature map. The upsampled feature map is then concatenated with the original P4-level feature map from the backbone network. Finally, the concatenated feature map is fed into a C3Libra module for thorough feature fusion, and its output is the P4-level detection feature map used for detecting medium-scale gestures.

[0111] Similarly, the process of generating P5-level detection features is as follows: The P4-level fusion features generated in the previous step are taken and downsampled through a convolutional layer with a stride of 2. The downsampled feature map is then concatenated with the original P5-level feature map in the backbone network. Finally, the concatenated feature map is fed into a C3Libra module for fusion, and its output is the P5-level detection feature map used for detecting larger-scale gestures.

[0112] Finally, the detection feature maps at levels P4 and P5 are fed into the final detection layer. This layer, combined with predefined anchor boxes, predicts the bounding box, confidence score, and class (left-handed / right-handed) of the target at each location, completing the entire detection process.

[0113] Through the above embodiments, the limitations of traditional one-way fusion are overcome by using a two-way information complementarity mechanism, thereby significantly optimizing multi-scale detection performance in AR glasses testing.

[0114] In some embodiments, the above-mentioned generation of target gesture detection results based on each set of detection boxes may further include the following steps:

[0115] Based on the comparison between the confidence scores of each group of detection boxes and the preset confidence threshold, candidate detection boxes are retained from each group of detection boxes; each candidate detection box is traversed, and the target detection box is determined according to the degree of overlap between the current candidate detection box and each candidate detection box, and the target gesture detection result is generated based on the target detection box.

[0116] Specifically, the multi-scale detection process described above is performed independently and in parallel on the P4 and P5 feature maps. The P4 feature map (i.e., the second-level feature map mentioned above) has a denser grid and smaller anchor boxes, so it is mainly responsible for detecting medium-sized gestures. The P5 feature map (i.e., the first-level feature map mentioned above) has a sparser grid and larger anchor boxes, so it is mainly responsible for detecting larger gestures. This results in two sets of detection box lists: one from P4 and one from P5. These two sets of detection box lists are then merged. At this point, the same gesture is likely to be detected simultaneously by multiple different anchor boxes in both P4 and P5, resulting in a large number of overlapping bounding boxes.

[0117] To obtain clean detection results, non-maximum suppression (NMS) is required. The NMS process is as follows: First, filter out detection boxes with excessively low confidence. Then, among the remaining boxes, select the box with the highest confidence and remove all other boxes whose area of ​​overlap (IoU) exceeds a certain threshold. Repeat this process until all boxes have been processed.

[0118] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0119] This embodiment also provides a gesture detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] Figure 6 This is a structural block diagram of a gesture detection device according to an embodiment of this application, such as... Figure 6 As shown, the device includes:

[0121] The acquisition module 10 is used to acquire the image to be detected in a single channel.

[0122] The hierarchical extraction module 20 is used to input the image to be detected into the input layer of the trained gesture detection model, convert the image to be detected from a single channel to multiple feature channels to obtain a multi-channel feature map, and perform hierarchical feature extraction processing on the multi-channel feature map through the backbone network in the gesture detection model to obtain the first-level features and the second-level features.

[0123] The detection module 30 is used to fuse the first-level features and the second-level features using a gesture detection model to obtain a fused feature map, and to perform detection based on the fused feature map to generate the target gesture detection result.

[0124] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0125] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0126] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0127] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0128] S1, acquire the image to be detected in a single channel.

[0129] S2, the image to be detected is input into the input layer of the trained gesture detection model, the image to be detected is converted from a single channel to multiple feature channels to obtain a multi-channel feature map, and the multi-channel feature map is subjected to hierarchical feature extraction processing through the backbone network in the gesture detection model to obtain the first-level feature and the second-level feature.

[0130] S3: Input the first-level features and the second-level features into the detection layer of the gesture detection model for parallel detection, output each set of detection boxes, and generate the target gesture detection result based on each set of detection boxes.

[0131] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0132] Furthermore, in conjunction with the gesture detection methods in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the gesture detection methods in the above embodiments.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0134] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0135] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method of gesture detection, characterized by, The method comprises: acquiring an image to be detected under a single channel; inputting the image to be detected into an input layer of a trained gesture detection model, converting the image to be detected from the single channel to a plurality of feature channels to obtain a multi-channel feature map, and performing hierarchical feature extraction processing on the multi-channel feature map via a backbone network in the gesture detection model to obtain first-level features and second-level features; inputting the first-level features and the second-level features into detection layers of the gesture detection model for parallel detection, outputting each group of detection boxes, and generating a target gesture detection result based on the detection boxes; the hierarchical feature extraction processing on the multi-channel feature map via the backbone network in the gesture detection model to obtain first-level features and second-level features comprises: inputting the multi-channel feature map into an initial convolutional layer in the backbone network for processing to output a shallow layer extraction feature map; inputting the shallow layer extraction feature map into a plurality of down-sampling convolutional layers and parallel feature fusion modules in the backbone network; sequentially and alternately processing the shallow layer extraction feature map via each down-sampling convolutional layer and parallel feature fusion module to obtain a deep layer extraction feature map, and retaining the highest layer feature in the deep layer extraction feature map as the first-level features, and retaining the next highest layer feature in the deep layer extraction feature map as the second-level features; the sequentially and alternately processing the shallow layer extraction feature map via each down-sampling convolutional layer and parallel feature fusion module to obtain a deep layer extraction feature map comprises: inputting the shallow layer extraction feature map into a down-sampling convolutional layer at a current level for down-sampling processing to obtain an extraction feature map at the current level; inputting the extraction feature map at the current level into a parallel feature fusion module at the current level, processing the extraction feature map at the current level via a first feature branch and a second feature branch in the parallel feature fusion module respectively to output first features and second features, and splicing the first features and the second features to obtain a fusion feature map at the current level; the second features have a higher granularity than the first features; inputting the fusion feature map at the current level into a down-sampling convolutional layer at a next level to obtain an extraction feature map at the next level, and inputting the extraction feature map at the next level into a parallel feature fusion module at the next level, until a fusion feature map at the highest level is obtained; wherein the deep layer extraction feature map comprises fusion feature maps at different levels; the parallel feature fusion module at the next level has a higher stacking degree than the parallel feature fusion module at the current level.

2. The gesture detection method of claim 1, wherein, the processing of the extraction feature map at the current level via the first feature branch and the second feature branch in the parallel feature fusion module to output first features and second features comprises: inputting the extraction feature map at the current level into the first feature branch and the second feature branch in parallel; The first feature is obtained by sequentially performing double-layer chain processing on the extracted feature map at the current level via a first convolutional layer and a second convolutional layer of the first feature branch, wherein convolution kernels of the first convolutional layer and the second convolutional layer are different; The second feature is obtained by performing single-layer transformation processing on the extracted feature map at the current level via a third convolutional layer of the second feature branch.

3. The gesture detection method of claim 1, wherein, The first feature and the second feature are spliced to obtain a fused feature map at the current level, including: The first feature and the second feature are aggregated to obtain an aggregated feature map; Based on the aggregated feature map, first and second weights are learned, and the first feature and the second feature are spliced based on the first and second weights to obtain the fused feature map.

4. The gesture detection method of claim 1, wherein, The gesture detection model further includes a detection head; the first level feature and the second level feature are input into the detection layer of the gesture detection model for parallel detection, and each group of detection boxes is output, including: The first level feature and the second level feature are input into a first detection branch and a second detection branch in the detection head to output a first level feature map and a second level feature map; The first level feature map and the second level feature map are input into the detection layer in the detection head, and the each group of detection boxes is output.

5. The gesture detection method of claim 1, wherein, Based on the each group of detection boxes, a target gesture detection result is generated, including: Based on a comparison result between a confidence of the each group of detection boxes and a preset confidence threshold, a candidate detection box is retained from the each group of detection boxes; Each candidate detection box is traversed, a target detection box is determined according to an overlapping degree between the current candidate detection box traversed and each candidate detection box, and the target gesture detection result is generated based on the target detection box.

6. A gesture detection apparatus, characterized by Including: An acquisition module is configured to acquire an image to be detected in a single channel; A hierarchical extraction module is configured to input the image to be detected into an input layer of a trained gesture detection model, convert the image to be detected from the single channel to a plurality of feature channels to obtain a multi-channel feature map, and perform hierarchical feature extraction processing on the multi-channel feature map via a backbone network in the gesture detection model to obtain a first level feature and a second level feature; A detection module is configured to fuse the first level feature and the second level feature by using the gesture detection model to obtain a fused feature map, and perform detection based on the fused feature map to generate a target gesture detection result; The hierarchical feature extraction processing on the multi-channel feature map via the backbone network in the gesture detection model to obtain the first level feature and the second level feature includes: The multi-channel feature map is input into an initial convolutional layer in the backbone network for processing to output a shallow layer extracted feature map; input the shallow layer extraction feature map into a plurality of down-sampling convolutional layers and parallel feature fusion modules in the backbone network; sequentially and alternately process the shallow layer extraction feature map via each of the down-sampling convolutional layers and the parallel feature fusion modules to obtain a deep layer extraction feature map, and retain the highest layer feature in the deep layer extraction feature map as the first hierarchical feature and retain the second highest layer feature in the deep layer extraction feature map as the second hierarchical feature; the sequentially and alternately processing the shallow layer extraction feature map via each of the down-sampling convolutional layers and the parallel feature fusion modules to obtain a deep layer extraction feature map comprises: inputting the shallow layer extraction feature map into a down-sampling convolutional layer at a current hierarchical level for down-sampling processing to obtain an extraction feature map at the current hierarchical level; inputting the extraction feature map at the current hierarchical level into a parallel feature fusion module at the current hierarchical level, processing the extraction feature map at the current hierarchical level via a first feature branch and a second feature branch in the parallel feature fusion module respectively, outputting a first feature and a second feature, and splicing the first feature and the second feature to obtain a fusion feature map at the current hierarchical level, wherein a granularity of the second feature is higher than a granularity of the first feature; inputting the fusion feature map at the current hierarchical level into a down-sampling convolutional layer at a next hierarchical level to obtain an extraction feature map at the next hierarchical level, and inputting the extraction feature map at the next hierarchical level into a parallel feature fusion module at the next hierarchical level, until a fusion feature map at a highest hierarchical level is obtained, wherein the deep layer extraction feature map comprises fusion feature maps at different hierarchical levels, and a stacking degree of the parallel feature fusion module at the next hierarchical level is greater than a stacking degree of the parallel feature fusion module at the current hierarchical level.

7. An electronic device, comprising: The memory and the processor are characterized in that the memory stores a computer program, and the processor is configured to execute the computer program to perform the gesture detection method in any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores a computer program, wherein the computer program is configured to execute the gesture detection method in any one of claims 1 to 5 when executed.

Images