Real-time gesture interaction method based on NPU edge computing platform and safety helmet

Abstract

A real-time gesture interaction method based on an NPU edge computing platform, steps comprising: S1 image acquisition facing the edge device use scene; S2 gesture recognition model construction; S3 depth model optimization facing the characteristics of a neural network processor NPU; S4 model conversion and end side deployment.In S3, for the inference characteristics of the NPU-based edge computing platform, the gesture recognition model obtained in S2 is adjusted in structure and compressed in parameters, the method being: firstly, by analyzing the inference delay and source occupation of each network layer of the gesture recognition model network structure on the target hardware, the network structure of the gesture recognition model is pruned and reconstructed, and the network layer with large calculation overhead is preferentially optimized; then, parameter compression and operator optimization are performed.The present application realizes a lightweight gesture recognition model with high precision and low delay through a technical path of layered performance query and iterative pruning and other hardware and software collaborative optimization techniques, and has the characteristics of lightweight, high inference efficiency and the like.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent safety protection technology, specifically to a real-time gesture interaction method based on an NPU edge computing platform and an intelligent safety helmet, which is particularly suitable for non-contact human-computer interaction and safety monitoring based on an edge computing platform in high-risk working environments such as power, chemical, and construction industries. Background Technology

[0002] In industrial work environments, workers are typically exposed to high noise and high-intensity work conditions, making traditional human-machine interaction methods significantly limited. Firstly, regarding voice interaction, the accuracy of voice command recognition is significantly reduced due to continuous mechanical noise or sudden bursts of high-intensity noise in industrial settings. Furthermore, voice interaction is difficult to implement effectively in scenarios requiring quiet, such as power line inspections, or in work environments where masks are required, such as chemical protective equipment. Secondly, regarding touch interaction, workers typically wear protective gloves or use hand tools, making it difficult to accurately operate the physical buttons or touch interfaces on smart safety helmets, easily leading to interaction interruptions or misoperations.

[0003] Therefore, there is an urgent need for an interaction method that requires no voice input or physical contact to enable stable and efficient operation by workers in complex industrial environments. Vision-based gesture interaction, due to its non-contact and intuitive nature, has become a viable alternative. However, existing gesture recognition methods largely rely on high-performance computing platforms, making it difficult to achieve low-latency, real-time operation on resource-constrained edge devices such as smart safety helmets. This is especially true on edge computing platforms based on NPUs (Neural Processing Units, AI acceleration chips optimized for neural network operations), where there is a lack of model optimization and deployment solutions tailored to hardware characteristics, thus limiting their application in real-world industrial scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a real-time gesture interaction method based on an NPU edge computing platform. This method is applicable to resource-constrained edge devices, such as smart safety helmets.

[0005] This method integrates the training, structural optimization, and edge deployment processes of the gesture recognition model, enabling the gesture recognition model to adapt to the NPU hardware architecture while meeting the recognition accuracy requirements, thus achieving low-latency inference and stable operation.

[0006] The steps of this method are as follows:

[0007] Step S1: Image acquisition for edge device scenarios. This method first acquires the gesture images or video stream data of the operators through the image acquisition device installed on the edge device, and constructs a gesture dataset suitable for industrial environments.

[0008] Step S2: Gesture recognition model construction. A target detection neural network is used to model and train the gesture target to obtain an initial model with basic recognition capabilities.

[0009] Step S3: Deep model optimization for NPU characteristics. This invention focuses on the inference characteristics of the NPU edge computing platform, adjusting the structure and compressing parameters of the gesture recognition model. Specifically, by analyzing the inference latency and resource consumption of each network layer on the target hardware, the model structure is pruned and reconstructed, prioritizing the optimization of computationally expensive network layers, thereby reducing the overall model inference latency. Simultaneously, through parameter compression and operator optimization, the model can more efficiently adapt to the NPU's computing mode, improving resource utilization efficiency during inference. Through the above optimization process, the model maintains high recognition accuracy while meeting the performance requirements of real-time operation of edge devices.

[0010] Step S4: Model Conversion and Edge Deployment. The optimized gesture recognition model is converted into an inference format suitable for NPU execution and loaded into the edge computing unit of the edge device. During device operation, the system acquires input image data in real time and performs preprocessing operations consistent with the training phase. Subsequently, the data is input into the deployed model for forward inference, and the gesture recognition result is output.

[0011] Step S5: Real-time interaction and command mapping. During the operation of the edge device, real-time feedback from perception to control is realized. The identification results of step S4 are further mapped into preset control commands to drive the functional modules of the edge device and realize contactless human-computer interaction.

[0012] Further optimization, in step S3, the model pruning optimization process is as follows:

[0013] First, the initial state of the model is measured:

[0014] ;

[0015] Where Net0 is the initial topology of the model; Res0 is the resource consumption (such as power consumption, latency or memory usage).

[0016] Secondly, during the iterative pruning process, a final constraint target budget Bud is set, while a smaller resource target constraint Constraint is set for each iteration of pruning.

[0017] ;

[0018] Then, local search and optimal layer selection are performed. For each layer k∈{1......K} in the network, a candidate solution is generated. Among the candidate solutions generated by the K layers, the one with the highest accuracy is selected as the system state for the next round.

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] Among them, f prune For pruning operations, determine how many filters (convolutional kernels) need to be retained in the k-th layer of the network so that the resource consumption of that layer conforms to the constraint; select For the filtering operation, an importance ranking strategy is adopted, retaining the filter with the largest L2 norm; f fine_tune For short-term fine-tuning, this layer is trained briefly to recover the accuracy loss caused by pruning; Net_Simp k These are the k simplified model schemes after pruning, selection, and short-term fine-tuning; Net i+i To select the most accurate solution from k simplified solutions as the initial network structure for the next iteration; ||W f ||2 is the L2 norm of each filter, C is the number of input channels, and H and W are the height and width of the convolution kernel, respectively;

[0024] Finally, the final stable model is obtained: when the optimized model satisfies the final budget Bud, the loop stops, and a final long-term fine-tuning is performed:

[0025] ;

[0026] This is the final stable model topology obtained after long-term fine-tuning.

[0027] The edge scenario of this invention is a real-time interactive environment using devices such as smart safety helmets. The modules and data flow in this scenario are as follows:

[0028] The module includes:

[0029] Image acquisition module: Composed of an industrial-grade camera at the front of the safety helmet, used to capture images of workers' hand gestures in real time under complex lighting and high-noise backgrounds;

[0030] Heterogeneous computing and AI inference module: Based on a chip such as Rockchip, it includes a dedicated NPU hardware acceleration unit to carry optimized dedicated format models (such as RKNN format) and perform high frame rate forward inference.

[0031] Command parsing and execution module: used to map the structured detection results (category labels and bounding boxes) output by the model into specific control commands, such as turning lights on and off, voice call control, or volume adjustment.

[0032] Data flow process:

[0033] Raw data acquisition: The raw image data stream captured by the camera is transmitted in real time to the temporary storage unit of the AI ​​inference circuit;

[0034] On-chip preprocessing: The edge terminal performs resolution scaling and normalization on the image to ensure that the data format is consistent with the input tensor of the lightweight model;

[0035] Hardware-accelerated inference: The preprocessed tensor data is directly fed into the NPU heterogeneous computing unit, and the on-chip memory mapping and operator optimization mechanism are used to complete the fast recognition and output the gesture category including confidence.

[0036] Closed-loop interactive feedback: The recognized commands trigger corresponding hardware actions or voice prompts through the internal bus of the safety helmet, completing low-latency contactless human-computer interaction, and the data can be processed locally without uploading to the cloud.

[0037] A safety helmet employing the aforementioned real-time gesture interaction method based on an NPU edge computing platform is disclosed. The safety helmet includes an image acquisition device, a graphics processor, and an edge AI inference circuit, and deploys a gesture recognition model. The AI ​​inference circuit is an edge computing platform centered on an NPU (such as a Rockchip chip).

[0038] Corresponding to step S5, the safety helmet, as an edge device, integrates a battery and its powered control unit, lighting unit, image acquisition unit, distress signal transmission unit, communication unit, and speaker unit; the control unit controls the operation of the other units;

[0039] The recognition results of the NPU edge computing platform are mapped to preset control commands. The control unit controls the lighting unit to turn the lights on and off, controls the image acquisition unit to take pictures, controls the distress signal sending unit to send distress signals, controls the communication unit to enable voice communication, and controls the speaker unit to adjust the volume.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. By establishing a hardware hierarchical lookup table, the operator response time and resource consumption of each layer of the model on the target NPU were measured, achieving deep hardware-specific co-optimization of software and hardware. This method is not a blind general algorithm compression, but rather prioritizes optimizing computationally expensive network layers for the inference characteristics of edge computing platforms such as Rockchip chips. It utilizes model quantization, operator fusion, and memory mapping mechanisms to ensure that the model can fully leverage the heterogeneous computing capabilities of the NPU, achieving high frame rate forward inference.

[0042] 2. For model optimization, a multi-criteria iterative model pruning and parameter compression strategy is proposed. This strategy searches for the optimal subnet in the candidate solution space by setting a resource constraint objective. Key filters are selected based on L2 norm importance ranking, and short-term fine-tuning and long-term refined training are combined to recover the accuracy loss caused by pruning. This gradual compression method significantly reduces the number of model parameters and computational cost while ensuring robust recognition accuracy even in resource-constrained environments at the edge.

[0043] 3. A complete contactless real-time interactive closed loop was designed for high-risk industrial environments such as power and chemical industries. The system not only constructs a scenario-based gesture dataset strongly related to the functions of safety helmets (such as turning lights on / off, taking photos, sending distress signals, etc.), but also introduces preprocessing technologies such as dynamic contrast enhancement and adaptive scaling to address the strong noise, low light, and dust interference in industrial environments. Recognition results can be mapped to hardware control commands locally without uploading to the cloud, solving the pain point of workers being unable to perform voice or physical touch interactions while wearing protective equipment. Attached Figure Description

[0044] Figure 1 This is the flowchart for the core phase;

[0045] Figure 2 This is a picture of the actual smart safety helmet;

[0046] Figure 3 It is a gesture category diagram;

[0047] Figure 4 This is a detailed flowchart of step S3;

[0048] Figures 5(a) to 5(c) show application scenarios for smart safety helmets, where:

[0049] Figure 5(a) shows a high-noise work scenario.

[0050] Figure 5(b) illustrates cross-language interaction barriers.

[0051] Figure 5(c) shows the operational degree of freedom constraints. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This embodiment proposes a real-time gesture interaction method for smart safety helmets based on an NPU edge computing platform. The aim is to resolve the contradiction between limited computing resources and real-time interaction requirements of wearable devices in high-risk industrial environments through hardware and software co-design. Figure 1 As shown, the main process of this method covers five core stages: image acquisition for safety helmet scenarios, gesture recognition model construction, deep model optimization for NPU characteristics, cross-platform deployment, and instruction mapping.

[0054] Step S1: Image acquisition for safety helmet scenarios. This method first acquires the gesture images or video stream data of the worker using an image acquisition device installed on the smart safety helmet, and then constructs a gesture dataset suitable for industrial environments. The specific steps are as follows:

[0055] Step S1.1: Device integration, such as... Figure 2 As shown, a miniature high-resolution camera, securely mounted on the front of a smart safety helmet, can acquire real-time images or video streams of the operator's interactive gestures from a first-person perspective.

[0056] Step S1.2: Scenario-based dataset construction. For typical working environments such as power and chemical industries, collect seven types of specific gestures that are strongly related to the functional modules of safety helmets, including turning on lights, turning off lights, taking photos, sending distress signals, making voice calls, and adjusting volume.

[0057] like Figure 3 As shown, (a) is an image of the gesture to turn on the light, (b) is an image of the gesture to turn off the light, (c) is an image of the gesture to take a picture, (d) is an image of the gesture to send a distress signal, (e) is an image of the gesture to make a voice call, (f) is an image of the gesture to increase the volume, and (g) is an image of the gesture to decrease the volume.

[0058] Step S1.3: Industrial environment preprocessing. To address common challenges in industrial environments such as low light, strong light interference, and dust obstruction, the input image is subjected to resolution adaptive scaling, normalization, and dynamic contrast enhancement to improve the model's robustness in complex environments.

[0059] Step S2: Gesture recognition model construction. An object detection neural network is used to model and train the gesture target to obtain an initial model with basic recognition capabilities. The specific steps are as follows:

[0060] Step S2.1: Network architecture selection. A basic model is built based on a lightweight object detection neural network (such as the YOLOv5n network). This architecture significantly reduces the initial computational load in the backbone feature extraction network by optimizing the depth and width design of the convolutional layers.

[0061] Step S2.2: Robustness enhancement. To address the complex lighting and dusty environments in industrial settings, the input image undergoes resolution adaptive scaling, normalization, and enhancement processing to simulate environmental noise.

[0062] Step S2.3: Training and Evaluation. End-to-end training is performed using the training set. Preliminary identification parameters are obtained by minimizing the joint loss function consisting of location regression, target confidence, and category prediction. During the validation phase, the forward inference floating-point operations (FLOPs) and total parameters (Params) of the model are monitored. The checkpoint with the best overall performance is selected as the baseline for further optimization.

[0063] Step S3: Deep model optimization for NPU characteristics, such as... Figure 4 As shown, deep compression is performed on the NPU edge computing platform integrated inside the smart safety helmet using a hardware-aware optimization strategy. The specific steps are as follows:

[0064] Step S3.1: Establish a hardware hierarchical lookup table, deploy the initially trained model in the target NPU execution environment, measure and record the operator response time and resource consumption data of each network layer structure in the forward inference process, and thus construct a model hierarchical performance lookup table.

[0065] Step S3.2: Multi-criteria iterative pruning and parameter compression strategy. First, based on the original model, K heterogeneous pruning configurations are derived by combining the hierarchical dimension of the neural network. Then, the performance of each configuration is verified through short-cycle fine-tuning, aiming to search for the optimal subnet with the minimum accuracy loss from the candidate space. If the current model state has not yet converged to the target threshold, this optimal solution will be used as the initial topology in the next round of pruning, thereby achieving progressive compression of parameters.

[0066] Step S3.3: Accuracy Restoration and Capability Release. The determined final pruning scheme is used as the starting point for training, and long-term fine-tuning and optimization are performed using the complete dataset. During this process, learning rate warm-up and refined hyperparameter configuration compensate for the representational loss caused by structural pruning, allowing the optimized model to fully release its inference performance potential on the NPU platform.

[0067] Step S4: Model Conversion and Edge Deployment. The deeply optimized model is migrated to the NPU computing unit on the safety helmet edge. The specific steps are as follows:

[0068] Step S4.1: Cross-format compilation. Utilize the target hardware's toolchain to convert the model into an efficient hardware-accelerated format (such as RKNN) and perform full quantization to match the NPU's fixed-point computation requirements.

[0069] Step S4.2: NPU hardware acceleration. The conversion process includes model quantization, operator fusion, and memory mapping optimization to fully utilize the heterogeneous computing units of the NPU. The converted model can be efficiently loaded by the runtime library of the smart safety helmet device, significantly reducing cold start latency.

[0070] Step S5: Real-time interaction and command mapping, enabling real-time feedback from perception to control during the operation of the smart safety helmet. The specific steps are as follows:

[0071] Step S5.1: Real-time streaming inference; the recognition results include gesture tags and confidence levels. Based on preset logic, the system maps the recognized gestures, such as "distress signals" and "taking photos," to the underlying hardware control commands or network alarm signals of the smart safety helmet in real time.

[0072] Step S5.2: Instruction mapping, through memory mapping and weight preloading technology, reduces the cold start time and runtime footprint of the model when the safety helmet end device is started.

[0073] Step S5.3: Implementation of typical application scenarios. Combining the above-mentioned real-time interaction and command mapping mechanism, as shown in Figure 5, the smart safety helmet of this embodiment has achieved stable human-computer interaction in the following complex industrial scenarios:

[0074] 1) As shown in Figure 5(a), in high-noise work scenarios, in environments where voice commands fail due to the noise of construction machinery, workers can directly control the helmet's mute or lighting modules through specific non-contact gestures without relying on voice wake-up.

[0075] 2) Cross-language interaction barrier scenario as shown in Figure 5(b): When multiple jobs or foreign personnel work together, the system can automatically map the instructions into corresponding control actions or general communication signals through standardized gesture commands, thus eliminating language barriers;

[0076] 3) As shown in Figure 5(c), when the operator is wearing heavy protective gloves or holding tools and it is inconvenient to touch the physical buttons, the operator can complete the instruction confirmation by simple single-hand gestures, which greatly improves the safety and efficiency of operation in high-risk environments.

[0077] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A real-time gesture interaction method based on an NPU edge computing platform, characterized in that... Includes the following steps: Step S1: Image acquisition for edge device usage scenarios: First, images or video streams of workers' gestures are acquired using an image acquisition device installed on an edge device; then, a gesture dataset suitable for industrial environments is constructed; the images are images of gestures representing specific interaction commands. Step S2: Gesture Recognition Model Construction: A target detection neural network is used to model and pre-train the gesture target to obtain an initial gesture recognition model with basic recognition capabilities; Step S3: Deep model optimization for the characteristics of the Neural Processor (NPU): To address the inference characteristics of NPU-based edge computing platforms, the gesture recognition model obtained from S2 undergoes structural adjustments and parameter compression, specifically as follows: First, by analyzing the inference latency and resource consumption of each network layer of the gesture recognition model on the target hardware, the network structure of the gesture recognition model is pruned and reconstructed, prioritizing the optimization of network layers with high computational overhead. Then, through parameter compression and operator optimization, the adaptation of the gesture recognition model to the NPU computing mode is optimized; Step S4: Model Conversion and Edge Deployment The S3-optimized gesture recognition model is converted into an inference format suitable for NPU execution and loaded into the edge computing unit of the edge device; During the operation of the edge device, firstly, image data is acquired in real time; then, the image data undergoes preprocessing consistent with the training phase; subsequently, the preprocessed data is input into the deployed gesture recognition model for forward inference. Finally, the gesture recognition results are output.

2. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1, characterized in that: It also includes step S5: Real-time interaction and command mapping: During the operation of the edge device, real-time feedback from perception to control is achieved, and the identification result of step S4 is further mapped into preset control commands to drive the corresponding functional modules of the edge device, thereby realizing contactless human-computer interaction.

3. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1 or 2, Its characteristic is that step S1 includes: Step S1.1: Acquire interactive gesture images or video stream data of the operator from a first-person perspective in real time using a camera that is securely mounted on the front of the edge device; Step S1.2: The construction of the scenario-based dataset involves collecting specific gesture images that are strongly related to the functions of edge devices, including turning on lights, turning off lights, taking pictures, sending distress signals, making voice calls, and adjusting volume, for typical working environments.

4. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1 or 2, Its characteristic is that step S2 includes: Step S2.1: Construct a basic model based on a lightweight object detection neural network; Step S2.2: Perform resolution adaptive scaling, normalization, and simulated environmental noise enhancement on the input image; Step S2.3: During the model training phase, end-to-end training is performed using the training set. Preliminary identification parameters are obtained by minimizing the joint loss function consisting of location regression, target confidence, and category prediction. During the validation phase, the focus is on monitoring the model's forward inference floating-point operations (FLOPs) and total parameters (Params), and the checkpoint with the best overall performance is selected as the baseline for optimization.

5. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1 or 2, characterized in that: Step S3, deep model optimization for NPU characteristics, is for edge computing platforms with integrated NPUs on edge devices. It uses hardware-aware optimization strategies for deep compression, and the specific steps are as follows: Step S3.1: Establish a hardware hierarchical lookup table: The gesture recognition model obtained from S2 was deployed in the target NPU execution environment. The operator response time and resource consumption data of each network layer structure of the model during the forward inference process were measured and recorded to construct a model hierarchical performance query table. Step S3.2: Multi-criteria iterative pruning and parameter compression strategy: First, based on the gesture recognition model obtained in S2, K heterogeneous pruning configurations are derived by combining the hierarchical dimension of the neural network; Subsequently, the performance of each configuration is verified through short-cycle fine-tuning, thereby searching for the optimal subnet with the least accuracy loss in the candidate space. If the current model state has not yet converged to the target threshold, the optimal pruning scheme will be used as the initial topology to enter the next round of pruning cycle, thereby achieving gradual compression of the number of parameters; Step S3.3: Precision Restoration and Capability Release: The final pruning scheme is used as the starting point for training, and long-term fine-tuning and optimization are performed using the complete dataset. In this process, by learning rate warm-up and refined hyperparameter configuration, the representation loss caused by structural pruning is compensated, so that the inference performance potential of the optimized gesture recognition model on the NPU platform can be fully released.

6. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1 or 2, Its characteristic is that step S4 includes: Step S4.1: Cross-format compilation: Using the target hardware's toolchain, the gesture recognition model is converted into a hardware-accelerated format and fully quantized to match the NPU's fixed-point computation requirements. Step S4.2: NPU Hardware Acceleration: The S4.1 conversion process includes model quantization, operator fusion, and memory mapping optimization to fully utilize the heterogeneous computing units of the NPU.

7. The real-time gesture interaction method based on an NPU edge computing platform according to claim 5, characterized in that: Step S3 specifically includes: 1) The initial state of the model is measured and represented as: ； Where Net0 represents the initial network structure of the model; Res0 represents the resource consumption; Indicates a measurement operation; 2) During the iterative pruning process, a final constraint target budget Bud is set, and a smaller resource target constraint Constraint is set for each iteration of pruning, expressed as: ； in, This represents the resource consumption during the i-th iteration; This indicates the reduction in resource consumption required for the i-th iteration; 3) Local search and optimal layer selection; For each layer k∈{1......K} in the network, a candidate scheme is generated; then, among the candidate schemes generated from the K layers, the one with the highest accuracy is selected as the initial network structure for the next iteration; this is represented as: ； ； ； ； Among them, f prune For pruning operations, determine how many filters (i.e., convolutional kernels) need to be retained in the k-th layer of the network so that the resource consumption of this layer conforms to the constraint. f select For the filtering operation, an importance ranking strategy is adopted to retain the filter with the largest L2 norm; f fine_tune For short-term fine-tuning, this layer is trained briefly to recover the accuracy loss caused by pruning. These are the k simplified model schemes after pruning, screening, and short-term fine-tuning. To select the scheme with the highest accuracy from k simplified schemes as the initial network structure for the next iteration; Here, L2 norm is given for each filter, C is the number of input channels, and H and W are the height and width of the convolution kernel, respectively. 4) Obtain the final stable model: When the optimized model satisfies the final budget Bud, the loop stops, and a final long-term fine-tuning is performed, as follows: ； This is the final stable model network structure obtained after long-term fine-tuning.

8. The real-time gesture interaction method based on an NPU edge computing platform according to claim 1, characterized in that... Edge scenarios are real-time interactive environments using smart helmets as the platform. The modules and data flow in this scenario are as follows: The module includes: Image acquisition module: Composed of an industrial-grade camera at the front of the helmet, used to capture images of workers' hand gestures in real time under complex lighting and high-noise backgrounds; Heterogeneous computing and AI inference module: Based on the Rockchip chip, it includes a dedicated NPU hardware acceleration unit to carry the optimized dedicated format model and perform high frame rate forward inference; Instruction parsing and execution module: used to map the structured detection results output by the model into specific control instructions; Data flow process: Raw data acquisition: The raw image data stream captured by the camera is transmitted in real time to the temporary storage unit of the AI ​​inference circuit; On-chip preprocessing: The edge terminal performs resolution scaling and normalization on the image to ensure that the data format is consistent with the input tensor of the lightweight model; Hardware-accelerated inference: The preprocessed tensor data is directly fed into the NPU heterogeneous computing unit, and the on-chip memory mapping and operator optimization mechanism are used to complete the fast recognition and output the gesture category including confidence. Closed-loop interactive feedback: The recognized commands trigger corresponding hardware actions or voice prompts through the helmet's internal bus, completing low-latency contactless human-computer interaction, and the data can be processed locally without uploading to the cloud.

9. A safety helmet employing the real-time gesture interaction method based on an NPU edge computing platform as described in claim 1 or 2, characterized in that: It features an image acquisition device, a graphics processor, and an edge AI inference circuit, and deploys a gesture recognition model.

10. The safety helmet according to claim 9, Its characteristic is that the safety helmet, as an edge device, integrates a battery and its powered control unit, lighting unit, image acquisition unit, distress signal transmission unit, communication unit, and speaker unit; the control unit controls the operation of the other units; The recognition results of the NPU edge computing platform are mapped to preset control commands. The control unit controls the lighting unit to turn the lights on and off, controls the image acquisition unit to take pictures, controls the distress signal sending unit to send distress signals, controls the communication unit to enable voice communication, and controls the speaker unit to adjust the volume.

Images