AR glasses-oriented lightweight insulation protector identification model optimization method, medium and program product

Abstract

The invention discloses an AR glasses-oriented lightweight insulation protector identification model optimization method, a medium and a program product, and relates to the technical field of model optimization, and the method comprises the steps: obtaining a training image data set containing overlooking, head-up, look-up and side-looking multi-view angles and marked shielding levels; taking the data set as input, based on a pre-training target detection model, constructing a visual angle sensitive weight matrix by calculating a multi-visual angle channel response of a convolutional layer / feature layer, performing selective cutting, and executing visual angle sensitive adaptive pruning to obtain a lightweight student model; then introducing a shielding guiding mechanism, executing knowledge distillation on the student model by using the uncompressed teacher model, and dynamically adjusting a distillation loss weight according to a sample shielding grade; and finally, outputting the optimized lightweight protector detection model. The method realizes the balance of light weight and high recognition performance of the model, adapts to the end side of AR glasses, improves the recognition precision of the protector in a multi-view and shielding scene, and guarantees the operation safety of an electric power field.

Description

Technical Field

[0001] This invention relates to the field of model optimization technology, and in particular to a method, medium, and program product for optimizing a lightweight insulating protective gear recognition model for AR glasses. Background Technology

[0002] In power field operations, AR glasses are often used to assist in the identification of insulating protective gear to ensure the safety of operators. However, current AR glasses-based insulating protective gear identification models face several challenges, including but not limited to: the wearer's perspective dynamically changes with head movements, encompassing various scenarios such as looking down (e.g., to check gloves and cables), looking up (e.g., to observe high-altitude equipment), and looking to the side, which can cause perspective shifts in the protective gear images captured by the camera; and the protective gear is easily obstructed by tools and wires, and factors such as lighting and perspective can weaken the edge and texture features of the protective gear, further increasing the difficulty of identification.

[0003] Existing lightweight target detection models, such as lightweight YOLO, have limited feature extraction capabilities and are prone to losing recognition ability in non-standard viewpoints and occluded scenes. Traditional lightweighting methods such as pruning and quantization only rely on channel importance or feature sparsity compression, without considering the impact of viewpoint changes in AR scenes on feature distribution. This leads to a significant performance degradation in real-world power field wearing scenarios, making it difficult to meet the requirements for real-time and accurate identification of protective gear.

[0004] Therefore, there is a need for a lightweight insulated protective gear recognition model optimization method, medium, and program product for AR glasses. Summary of the Invention

[0005] In response, the present invention provides a method, medium, and computer program product for optimizing a lightweight insulating protective gear recognition model for AR glasses, so as to at least partially solve the above-mentioned technical problems.

[0006] This invention provides a method for optimizing a lightweight insulating protective gear recognition model for AR glasses, comprising the following steps: A training image dataset containing multiple viewpoints and occlusion conditions is obtained, wherein the viewpoints include at least one of top-down, eye-level, top-down, and side-view, and the occlusion conditions are labeled with occlusion levels. Using the training image dataset as input, a viewpoint-sensitive adaptive pruning process is performed based on a pre-trained object detection model to obtain a lightweight student model. The adaptive pruning process includes: calculating the channel responses of convolutional layers or feature layers in the pre-trained object detection model under different viewpoints; constructing a viewpoint-sensitive weight matrix based on the channel responses, which characterizes the consistency of importance of each channel under multiple viewpoints; and sorting and selectively pruning the model channels according to the viewpoint-sensitive weight matrix. After the adaptive pruning process, a knowledge distillation optimization process is performed, using an uncompressed teacher model to train the lightweight student model. The knowledge distillation optimization process introduces an occlusion guidance mechanism, dynamically adjusting the weights of the distillation loss according to the occlusion level of the training samples, enabling the student model to learn the feature responses of the teacher model more effectively under occlusion conditions. The final lightweight protective gear detection model after pruning and distillation optimization is output.

[0007] Preferably, the construction of the viewpoint-sensitive weight matrix specifically includes: for each channel, calculating its average activation value under each viewpoint group; based on the average activation value under all viewpoint groups, calculating the cross-viewpoint consistency score of the channel, which is equal to the mean of the average activation values ​​under all viewpoint groups, divided by the sum of the standard deviation of the average activation values ​​under all viewpoint groups and a very small constant; weighting and fusing the consistency score with the overall importance index of the channel to obtain the final viewpoint-sensitive weight of each channel, which is equal to the first adjustable weight coefficient multiplied by the normalized overall importance index, plus the second adjustable weight coefficient multiplied by the normalized consistency score, wherein the sum of the first adjustable weight coefficient and the second adjustable weight coefficient is equal to 1.

[0008] Preferably, the adaptive pruning process adopts a staged pruning strategy, which includes at least two pruning stages. After each pruning stage, the model is briefly fine-tuned and its performance is verified on a dedicated multi-view validation set. If the performance degradation under any view exceeds a preset threshold, the pruning operation of that stage is rolled back and the pruning intensity of the corresponding layer is reduced.

[0009] Preferably, the distillation loss in the knowledge distillation optimization process includes at least feature map distillation loss and classification soft label distillation loss; the feature map distillation loss is the imitation loss of the student model to the key layer output feature map of the teacher model, which is obtained by calculating the L2 distance between the two feature maps; the classification soft label distillation loss is the imitation loss of the student model to the softened probability distribution of the classification head output of the teacher model, which is obtained by calculating the KL divergence between the two probability distributions.

[0010] Preferably, when calculating the feature map distillation loss, a dynamic weight coefficient is applied to the feature map distillation loss term according to the occlusion level of the target in the current training sample; the higher the occlusion level, the larger the dynamic weight coefficient.

[0011] Preferably, a lightweight feature gating unit is embedded in the backbone network or feature pyramid network of the student model; the feature gating unit generates channel attention weights through global pooling, a lightweight multilayer perceptron and a sigmoid activation function, and multiplies them channel by channel with the original feature map to enhance task-related features and suppress redundant features.

[0012] Preferably, when training the feature gating unit, a regularization term based on the relevance of channels to the task is introduced. The regularization term is calculated as follows: the third weight coefficient is multiplied by the sum of all channels, the sum is 1 minus the standardized evaluation score of the channel's relevance to the task, and then multiplied by the absolute value of the channel's gating coefficient. This regularization term penalizes the gating coefficients of channels with low relevance to the task by reducing their contribution to the total loss, thereby promoting sparsity.

[0013] Preferably, the overall importance index of the channel is determined based on the L1 norm of the channel or the scaling factor of the channel in the batch normalization layer.

[0014] In another aspect, this application also provides a lightweight insulating protective gear recognition model optimization device for AR glasses, comprising: A training set acquisition unit is used to acquire a training image dataset containing multiple viewpoints and multiple occlusion conditions, wherein the viewpoints include at least one of top-down, eye-level, top-down, and side-view, and the occlusion conditions are labeled with occlusion levels; a model pruning module is used to take the training image dataset as input and, based on a pre-trained object detection model, perform a viewpoint-sensitive adaptive pruning process to obtain a lightweight student model; wherein the adaptive pruning process includes: calculating the channel responses of convolutional layers or feature layers in the pre-trained object detection model under different viewpoints, constructing a viewpoint-sensitive weight matrix based on the channel responses, and using the viewpoint-sensitive weight matrix for... The model is characterized by the consistency of importance of each channel across multiple viewpoints; and the model channels are sorted and selectively pruned according to the viewpoint-sensitive weight matrix. A model distillation module is used to perform a knowledge distillation optimization process after the adaptive pruning process, training the lightweight student model using the uncompressed teacher model. The knowledge distillation optimization process introduces an occlusion guidance mechanism, dynamically adjusting the weights of the distillation loss based on the occlusion level of the training samples, enabling the student model to learn the feature responses of the teacher model more effectively under occlusion conditions. An output module is used to output the final lightweight protective gear detection model after pruning and distillation optimization.

[0015] In another aspect, this application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the lightweight insulating protective gear recognition model optimization method for AR glasses as described above.

[0016] In another aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor to implement the lightweight insulated protective gear recognition model optimization method for AR glasses as described above.

[0017] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the lightweight insulated protective gear recognition model optimization method for AR glasses as described above.

[0018] This application constructs a multi-view, multi-occlusion training dataset, employs viewpoint-sensitive adaptive pruning, occlusion-guided knowledge distillation, and lightweight feature gating unit embedding to obtain a lightweight insulating protective gear recognition model. This model significantly reduces the number of parameters and computational load to adapt to the computing power of AR glasses, while effectively addressing the issues of multi-view variations and protective gear occlusion in power field settings. It significantly improves the recognition accuracy and robustness of protective gear under different viewpoints and occlusion scenarios, achieving a balance between lightweight model and high recognition performance. This meets the needs of AR glasses in power field for real-time and accurate recognition of insulating protective gear, ensuring operational safety. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0020] Figure 1 This is a schematic diagram of a lightweight insulating protective gear recognition model optimization method for AR glasses provided by an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the implementation process of the adaptive pruning method provided in an embodiment of the present invention. Figure 3 A schematic diagram of the distillation loss construction process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a lightweight insulating protective gear recognition model optimization device for AR glasses provided in an embodiment of the present invention; Figure 5This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] It should be noted that all user information (including but not limited to user device information, user personal information, object information corresponding to device usage data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, device usage data, etc.) involved in all embodiments of this disclosure are information and data authorized by the user or fully authorized by all parties.

[0023] To address the aforementioned issues, this application proposes a method for optimizing a lightweight insulating protective gear recognition model for AR glasses. The technical solution of this application will be described in detail below with reference to various embodiments.

[0024] like Figure 1 As shown in the figure, an embodiment of the present invention discloses a flowchart of a lightweight insulating protective gear recognition model optimization method 100 for AR glasses, including the following method steps: S1, Obtain a training image dataset containing multiple viewpoints and multiple occlusion conditions, wherein the viewpoints include at least one of top view, eye view, top view and side view, and the occlusion conditions are labeled with occlusion levels; S2, using the training image dataset as input, and based on the pre-trained object detection model, a view-sensitive adaptive pruning process is performed to obtain a lightweight student model; wherein, the adaptive pruning process includes: calculating the channel responses of the convolutional layers or feature layers in the pre-trained object detection model under different viewpoints, constructing a view-sensitive weight matrix based on the channel responses, the view-sensitive weight matrix being used to characterize the consistency of importance of each channel under multiple viewpoints; and sorting and selectively pruning the model channels according to the view-sensitive weight matrix; S3, after the adaptive pruning process, a knowledge distillation optimization process is performed to train the lightweight student model using the uncompressed teacher model; wherein, the knowledge distillation optimization process introduces an occlusion guidance mechanism to dynamically adjust the weight of the distillation loss according to the occlusion level of the training samples, so that the student model learns the feature response of the teacher model more effectively under occlusion conditions. S4 outputs the final lightweight protective gear test model after pruning and distillation optimization.

[0025] In some embodiments, for step S1, the viewing angle is divided into four core groups according to the actual operating scenario of the AR glasses wearer at the power site. For example, the angle range of each group and the corresponding acquisition scenario may include: For the top-down view group, for example, the angle range is set to -30° to -90°, corresponding to scenarios such as the wearer looking down to check the operation status of the insulated gloves, the cable arrangement underfoot, or the placement of tools on the ground. When collecting data, it is necessary to ensure that the angle between the center axis of the lens and the horizontal plane is within this range, and focus on capturing images of the interaction area between the gloves, tools, and cables. For the eye-level group, for example, the angle range is set to -30° to +30°, corresponding to scenarios such as the wearer looking at electrical equipment in front, checking the operation and cooperation of colleagues at the same height, or reading equipment labels. When collecting data, the angle between the center axis of the lens and the horizontal plane is controlled within this range, focusing on covering the natural presentation state of the protective gear without obvious pitch angle. For the upward-looking group, for example, the angle range is set to +30°~+90°, corresponding to scenarios such as the wearer looking up to observe electrical equipment at a height or to inspect the wearing of high-altitude protective gear. When collecting data, the angle between the lens center axis and the horizontal plane is within this range, and the focus is on capturing images of the shoulders of the protective clothing, the upper edge of the mask, and the superimposed area of ​​the high-altitude protective gear and the background equipment. For the side-view group, for example, the angle range is set to approximately ±90°, that is, the angle between the lens and the wearer's line of sight is 85°~95°. This corresponds to scenarios such as the wearer turning to the side to check the status of the protective gear on the side of adjacent equipment, the presentation of protective gear when assisting others to pass tools from the side, or checking the protective condition of the terminal blocks on the side of the equipment. During the acquisition, the angle between the center axis of the lens and the wearer's line of sight is controlled within this range. The focus is on capturing the side contours of the protective gear, such as the texture of the side of the gloves, the edge of the sleeve of the protective clothing, and side obstructions, such as the edge of the equipment cabinet, and the superimposed image of the tool handle and the protective gear.

[0026] Optionally, the number of images collected in each group of perspectives shall not be less than 5,000, and it is necessary to ensure that the sample proportion of each type of insulating protective gear, such as insulating gloves, insulating protective clothing, and insulating face shield, is balanced in each group of perspectives. The number of samples of a single protective gear in a single perspective shall not be less than 1,200, so as to avoid model bias due to insufficient samples of a certain type of protective gear. Meanwhile, a hybrid sample composition method was adopted, combining real-scene data collection and synthetic data supplementation. Real-scene data collection was conducted in different power field environments, covering protective gear of different models and with varying degrees of wear and tear. Synthetic data supplementation used 3D modeling software to construct 3D models of the protective gear, simulating lighting and occlusion effects from different perspectives to generate virtual images. These virtual images were used to supplement samples from extreme perspectives or with special occlusions that were difficult to collect in real-scene scenarios. Synthetic samples accounted for no more than 30% of the total sample size in each group, ensuring a balance between sample authenticity and diversity.

[0027] In one embodiment, the occlusion conditions are divided into three levels according to the occlusion ratio of the key identification area of ​​the insulating protective gear, and the labeling is performed using the independent target of each type of protective gear in a single image as the labeling unit.

[0028] Specifically, Level 0, Unobstructed: Key identification areas of protective gear, such as the palm and knuckle areas of gloves, the torso and cuff areas of protective clothing, and the face-covering areas of face masks, are not covered by any obstructions, and the edges and texture features of the protective gear are fully visible. Level 1, Partial Obstruction: The area covered by the obstruction in the key identification area of ​​the protective gear is less than 50%, and the core features of the protective gear, such as the finger texture of the gloves, the markings on the protective clothing, and the edge of the mask window, are not obstructed. The type and status of the protective gear can still be determined by the unobstructed area. Level 2, Severe Obstruction: The critical identification area of ​​the protective gear is covered by an obstruction object by ≥50%, or the obstructed area contains the core features of the protective gear, such as the palm of the glove being completely covered by a tool, or the window of the mask being covered by fog or stains, and the presence of the protective gear can only be determined by some edges or non-critical areas.

[0029] At the same time, the specific types of obstructions are labeled, including power tools such as wrenches, screwdrivers, insulated operating rods, cable harnesses, equipment parts such as cabinet edges and terminals, and clothing edges such as protective clothing collars and cuff folds. These four categories are used for subsequent model analysis to determine the degree of influence of different obstructions on protective gear recognition.

[0030] In some embodiments, for step S2, such as Figure 2 The schematic diagram shown illustrates the implementation process of adaptive pruning, which, exemplarily, specifically includes: S201, Selection and initialization of pre-trained object detection model; for example, specifically including selecting YOLOv8n as the pre-trained object detection model, using it as the baseline model for subsequent adaptive pruning (i.e., the initial structure of the student model). This model has a lightweight infrastructure and high detection accuracy, adapts to the computing power requirements of AR glasses, and its network layer structure, including convolutional layers, feature layers, and batch normalization layers, supports channel-level pruning operations; Load the pre-trained weights of YOLOv8n on a general object detection dataset, such as the COCO dataset, and then use the multi-view, multi-occlusion insulating protective gear training image dataset constructed in step S1 to fine-tune the pre-trained model for 10-15 epochs to enable the model to initially learn the feature distribution of insulating protective gear, laying the foundation for subsequent channel response calculation. The Adam optimizer is used during the fine-tuning process, with the initial learning rate set to 1e-4 and the batch size set to 8.

[0031] S202, Convolutional layer / feature layer channel response calculation; Specifically, this includes grouping the training image dataset constructed in step S1 according to viewpoint, for example, top view group: -30°~-90°, eye view group: -30°~+30°, bottom view group: +30°~+90°, side view group: ±90°, ensuring that each group of data is independently input into the model for forward propagation; Each set of viewpoint data is input into the initialized pre-trained model. All convolutional and feature layers of the model, including the backbone network and the Feature Pyramid Network (FPN) layer, are traversed. For each channel in each layer, the feature response value of that channel on all samples of the corresponding viewpoint group is recorded, i.e., the absolute value of all pixel values ​​in the channel's output feature map. The average activation value of that channel under that viewpoint group is then calculated. The calculation formula is: Where c is the channel index and v is the view group index. Let v be the sample set of the vth viewpoint. E[·] represents the characteristic response output of the c-th channel to sample x, where E[·] represents the expectation (i.e., the average of the characteristic response values ​​of all samples in the sample set).

[0032] Finally, using the mapping relationship between layer index, channel index, view group index, and average activation value, a channel response statistics matrix is ​​constructed and stored in a local database for subsequent construction of the view-sensitive weight matrix.

[0033] S203, Construction of a perspective-sensitive weight matrix; Specifically, this includes, for example, (1) cross-perspective consistency score calculation. For each channel c in each layer of the model, based on the average activation value of each view group obtained in step two. Calculate its cross-perspective consistency score The specific steps are as follows: Calculate the average activation mean of the channels : The average activation value of channel c across all view groups, a total of 4 groups. The overall activation level of this channel is obtained by taking the arithmetic mean, using the following formula: Where |V| is the total number of viewpoint groups, and here |V|=4.

[0034] Calculate the channel mean activation standard deviation Calculate the average activation value of channel c across all view groups. The standard deviation reflects the activation stability of the channel under different perspectives. The formula for the mean is: Calculate the consistency score : Mean Divide by standard deviation The cross-perspective consistency score is obtained by summing the result with the minimum constant ϵ, where ϵ is taken as 1e-6 to avoid a denominator of 0. The formula is as follows: The higher the value, the more stable the activation level of the channel is under different perspectives, and the more consistent its importance for multi-view feature extraction, so it should be retained first.

[0035] (2) Calculation of overall importance index of the channel Calculate the overall importance index for each channel based on the L1 norm or batch normalization (BN) layer scaling factor. You only need to choose one of the two options. Specifically, The calculation method based on the L1 norm involves, for example, summing the absolute values ​​of the convolution kernel parameters or feature maps output by the feature layer corresponding to channel c. The larger this value, the richer the feature information carried by the channel and the higher its importance. The formula is as follows: in, Let i be the kernel parameters corresponding to channel c, i and j be the kernel spatial dimensions, and k be the number of input channels.

[0036] The scaling factor calculation method based on the BN layer, for example, extracts the scaling factor of the BN layer where channel c is located. The coefficients in the BN layer parameters used to scale the standardized features will... The absolute value is used as an overall importance indicator. ,Right now = This method uses the parameters of the Batch Normalization (BN) layer to directly reflect the influence of channels on the model output; the larger the scaling factor, the higher the importance of the channel.

[0037] (3) Weighted fusion For example, cross-view consistency scores for all channels Compared with overall importance indicators Normalization was performed separately, using the Min-Max normalization method to map the values ​​to the [0,1] interval, thus eliminating dimensional differences.

[0038] Weighted fusion calculation perspective-sensitive weights Introducing the first adjustable weighting coefficient With the second adjustable weighting coefficient ,satisfy ,and (Adjusted according to the model's lightweight objective and accuracy requirements, default value is 0.5), the normalized value is... and Weighted fusion yields the final view sensitivity weight for each channel, using the following formula: Constructing a weight matrix: using layer index - channel index - view-sensitive weights The structure is used to construct a perspective-sensitive weight matrix, which directly reflects the comprehensive value of each channel in terms of "overall feature importance" and "multi-perspective consistency", providing a basis for subsequent channel pruning.

[0039] S204, phased adaptive pruning execution; Specifically, at least two pruning stages are employed, preferably three, to progressively compress the model in a "coarse pruning - medium pruning - fine pruning" manner. This avoids the irrecoverable loss of model accuracy caused by a single large-scale pruning. For example, the objectives of each stage include: Coarse pruning stage: The target pruning ratio is 30%-40%. For intermediate convolutional layers with high feature redundancy in the model, such as early convolutional layers of the backbone network, viewpoint-sensitive weights are pruned first. Lower number of channels quickly reduces the number of model parameters and computational cost (FLOPs). Mid-pruning stage: The target pruning ratio is 20%-30%. Pruning is carried out on the middle and late layers of the backbone network and the FPN layer. In this stage, the channel retention rate of key layers, such as the output layer of FPN, needs to be appropriately increased to avoid affecting the high-level fusion of protective gear features. Fine-tuning stage: The target pruning ratio is 10%-15%. Fine-tuning is performed on the remaining channels of the model. The focus is on verifying the channel weights of the detection head association layers, such as the feature layers connected to the classification head and regression head, to ensure that the model's detection logic is not damaged after pruning. The ultimate goal is to reduce the overall FLOPs by 60%-75%.

[0040] Preferably, each pruning stage follows a process of weight sorting - channel pruning - short-term fine-tuning, with the specific steps as follows: Channel weight sorting: For all channels of the layer to be clipped in the current stage, sort them by view-sensitive weight. Sort the channels from largest to smallest to determine the priority of channel clipping. Selective channel pruning: Based on the target pruning ratio for the current stage, retain the channels with the highest weight (1 - pruning ratio) and directly delete the channels with the lowest weight; at the same time, adjust the connection structure of the subsequent network of this layer (e.g., the number of input channels of the subsequent convolutional layer needs to match the number of output channels of the current layer) to ensure the integrity of the network structure. Short-term fine-tuning to restore accuracy: Using the training image dataset from step S1 as input, the cropped model is fine-tuned for 3-5 epochs. During the fine-tuning process, a low learning rate (1 / 10 of the initial learning rate, i.e., 1e-5) is used, the optimizer remains Adam, and the batch size is set to 8. By fine-tuning, the feature expression gaps caused by channel cropping are repaired, and the basic detection accuracy of the model is restored.

[0041] After completing all the pruning stages, the final output is a lightweight model that serves as a student model for subsequent knowledge distillation.

[0042] In some embodiments, for step S3, an uncompressed YOLOv8 series model is selected as the teacher model, with preference given to YOLOv8x or a YOLOv8 version with a larger number of parameters. This invention does not impose any restrictions. This model has stronger feature extraction capabilities and occlusion robustness, and can provide high-quality distillation guidance signals for the student model.

[0043] Preferably, the teacher model is pre-trained and fine-tuned. For example, the pre-trained weights of the teacher model on a general object detection dataset, such as the COCO dataset, are first loaded. Then, the multi-view, multi-occlusion insulating protective gear training image dataset constructed in step S1 is used to fine-tune the teacher model for 20-30 epochs to ensure that the teacher model fully learns the feature distribution of insulating protective gear, especially the features in occluded scenes. The fine-tuning process uses the AdamW optimizer, with an initial learning rate set to 5e-5, a batch size set to 16, and the default detection loss of YOLOv8, such as classification loss and regression loss, used as the loss function.

[0044] In one embodiment, the distillation loss function consists of two parts: feature map distillation loss and classification soft label distillation loss. These are weighted and fused to form the total distillation loss. For example, as shown below... Figure 3 The diagram shown illustrates the distillation loss construction process, with the specific construction method as follows: S301 calculates the feature map distillation loss, including key layer selection, L2 distance calculation, application of dynamic occlusion weights, and calculation of the total feature map distillation loss. Specifically... For key layer selection, representative key layers from both the teacher and student models can be chosen as distillation targets. Priority should be given to mid-to-late layers of the backbone network, such as the output layer of the C3 module in YOLOv8, and output layers of the feature pyramid network, such as the P3, P4, and P5 layers in YOLOv8. These layers are responsible for extracting high-level semantic features and multi-scale features of protective gear, which is crucial for protective gear recognition in occluded scenarios. It is also necessary to ensure that the key layers of the teacher and student models match in feature map size and number of channels. If the number of channels does not match, a 1×1 convolutional layer can be added after the key layers in the student model to adjust the number of channels. L2 distance calculation, for each selected key layer, teacher layer Student level Where l is the layer index, the L2 distance between the output feature maps of the two layers is calculated and used as the feature map distillation loss for a single layer. The formula is: Where H and W are the height and width of the feature map, respectively, and C is the number of channels in the feature map. This represents the square of the L2 norm.

[0045] Occlusion dynamic weights are applied, introducing dynamic weight coefficients for occlusion levels. The weights of the feature map distillation loss are adjusted based on the occlusion level of the protective gear in the current training samples. The higher the occlusion level, the larger the weight coefficient. The formula is as follows: Where r represents the occlusion level of the current sample, r=0 indicates no occlusion, r=1 indicates partial occlusion, and r=2 indicates severe occlusion. k is the weight adjustment coefficient (ranging from 0.5 to 1.0, with a default value of 0.8). For example, when the occlusion level is 2, =1 + 0.8 × 2 = 2.6; When the occlusion level is 0, =1.

[0046] Total feature map distillation loss: The total feature map distillation loss is obtained by weighted summation of the feature map distillation losses of all selected key layers. The formula is: Where L is the set of key layers. This is the global weight coefficient for the feature map distillation loss, preferably ranging from 0.3 to 0.7, with a default value of 0.5, used to balance the contributions of the feature map distillation loss and the classification soft label distillation loss.

[0047] S302, calculate the distillation loss of the classification soft label; Specifically, the first step is to generate soft labels. Specifically, this involves obtaining the raw logits output by the teacher model's classification head, denoted as... The temperature softening function is used to convert it into a soft label probability distribution. The formula is: Where T is a temperature parameter, preferably ranging from 2 to 4, with a default value of 3. The higher the temperature, the smoother the probability distribution of the soft label, and the richer the inter-category correlation information contained, which helps the student model learn the teacher model's category judgment logic for fuzzy samples.

[0048] The raw logits output by the student model classification head are denoted as . Similarly, it is converted into a probability distribution through the temperature softening function. The formula is: Calculate the KL divergence, specifically, calculate the soft label probability distribution. With student model probability distribution The KL divergence between them serves as the distillation loss for classification soft tags. The formula is: Where i is the category index, which in this case is the protective gear category, such as gloves, protective clothing, and face shield. KL divergence is used to measure the difference between two probability distributions. The smaller the difference, the closer the student model is to the classification judgment of the teacher model.

[0049] Classification soft-label distillation loss weighting: Set global weight coefficients for classification soft-label distillation loss. The value range is 0.3-0.7, with a default value of 0.5, and it satisfies the following conditions: This ensures a weighted balance of total distillation losses.

[0050] S303, calculate the total distillation loss; Specifically, total distillation loss The weighted sum of the feature map distillation loss and the classification soft label distillation loss is given by the following formula: During distillation training, the total loss of the student model is the sum of the distillation loss and the student model's own detection loss. The weight of the student model's own detection loss is set to 0.2-0.4 to avoid the student model from over-relying on the teacher's signal. in, The student model itself has detection loss, classification loss, and regression loss. It loses weight for itself.

[0051] In one embodiment, during the distillation training process, real-time data augmentation is performed on the training image dataset from step S1, such as random flipping, scaling, and brightness adjustment. Each training sample also carries the occlusion level information labeled in step S1, which is used to dynamically adjust the weights of the feature map distillation loss. ; For example, the distillation training cycle is set to 20-40 epochs, the batch size is set to 8-16 (adjustable according to hardware computing power), the optimizer is AdamW, and the learning rate adopts a Cosine decay strategy, with the initial learning rate set to 5e-5 and decaying to 1e-6 at the end. During training, the teacher model parameters remain fixed, and only the student model parameters are updated. In each training iteration, for each sample in the current batch, first read its occlusion level r, and then calculate the corresponding dynamic weight coefficient. Then substitute it into the characteristic map distillation loss formula to calculate Ultimately combined Total distillation loss The student model parameters are then updated via backpropagation. This mechanism allows the student model to focus more on learning the feature responses of the teacher model when dealing with samples with high occlusion levels, thus enhancing its ability to recognize occluded protective gear.

[0052] In one embodiment, preferably, a lightweight feature gating unit is embedded in the backbone network or feature pyramid network of the student model; the feature gating unit generates channel attention weights through global pooling, a lightweight multilayer perceptron, and a sigmoid activation function, and multiplies them channel by channel with the original feature map to enhance task-related features and suppress redundant features.

[0053] When training the feature gating unit, a regularization term based on the relevance of channels to the task is introduced. The regularization term is calculated by multiplying the third weight coefficient by the sum of all channels, where the sum is 1 minus the standardized evaluation score of the channel's relevance to the task, and then multiplying by the absolute value of the channel's gating coefficient. This regularization term penalizes the gating coefficients of channels with low relevance to the task by reducing their contribution to the total loss, thereby promoting sparsity.

[0054] Specifically, the Feature Gating Unit (FGU) adopts the principle of low parameter quantity and high computational efficiency. The overall structure consists of three parts connected in series: a global pooling layer, a lightweight multilayer perceptron (MLP), and a sigmoid activation layer.

[0055] Specifically, for global pooling layers, for example, global average pooling (GAP) is used to compress the spatial dimensions of the input feature map. Let the input feature map have dimensions H×W×C, where H is the height, W is the width, and C is the number of channels. Global average pooling calculates the mean of each channel across all spatial locations (H×W pixels), resulting in an output channel feature vector compressed to 1×1×C. This operation aggregates global spatial information for each channel, avoiding local noise interference and significantly reducing subsequent computational costs.

[0056] For lightweight MLP layers, for example, a lightweight design that uses "1×1 convolutions instead of fully connected layers" contains only two convolutional modules: The first module uses a 1×1 convolution kernel to compress the dimension of the channel feature vector from C to C / r, where r is the compression ratio, ranging from 4 to 8, with a default value of 6, to reduce the number of parameters and computational cost; the convolution is followed by a ReLU activation function to introduce non-linear feature transformation. The second module: A 1×1 convolutional kernel is used again to restore the channel dimension from C / r to C, outputting a feature vector with the same number of input channels. This module has no activation function, ensuring that the output features cover the numerical range of the original channels. The total number of parameters in the lightweight MLP is only C×(C / r)+(C / r)×C=2C² / r, compared to the traditional fully connected MLP (C×C), the number of parameters is reduced by (r-2) / r (e.g., when r=6, the number of parameters is reduced by about 67%, adapting to the computing power limitations of AR glasses on-device.

[0057] Sigmoid activation layer: The 1×1×C feature vector output by the lightweight MLP is input into the Sigmoid function to generate channel attention weights gc with values ​​ranging from [0,1], where c is the channel index. The closer the weight gc is to 1, the stronger the correlation between the corresponding channel features and the protective gear recognition task; the closer it is to 0, the higher the redundancy of the channel features.

[0058] In one embodiment, based on the student model, i.e., the pruned YOLOv8n network structure, FGU is embedded into the later layers of the backbone network and the feature pyramid network layer, specifically in the following locations: Backbone Network: In the C3 modules of YOLOv8n, especially the C3 modules close to the FPN, such as the output of the 6th and 7th C3 modules, FGU is embedded. The features at this position are the mid-to-high-level semantic features of the protective gear. FGU can enhance the expression of key semantics, such as the expression of glove texture and mask outline.

[0059] FPN layer: FGU is embedded in the P3, P4, and P5 layers of FPN, which are multi-scale feature output layers. The features of these layers are directly used for classification and regression of subsequent detection heads. FGU can suppress background redundant features at different scales, such as equipment cabinets and cable harnesses, and improve the recognizability of protective gear features.

[0060] For example, a plug-and-play embedding method is used, where the FGU is concatenated at the output of the target layer. The specific process is as follows: The target layer, such as the C3 module or P3 layer, outputs the original feature map Fin, with dimensions H×W×C; Fin is input into the FGU to generate channel attention weights g, with dimensions 1×1×C; g is multiplied by Fin channel by channel through a broadcast mechanism to obtain the enhanced feature map Fout=Fin⊗g, where ⊗ represents channel-by-channel multiplication; Fout is used as the input to subsequent network layers, such as convolutional layers and detectors. During the embedding process, there is no need to modify the original network structure and parameters of the student model; only lightweight parameters of the FGU need to be added, ensuring that the overall lightweight characteristics of the model are not affected.

[0061] In one embodiment, the Pearson correlation coefficient (Rc) is used as a quantitative indicator of the correlation between channels and tasks. This coefficient measures the degree of linear correlation between the feature response of a single channel and the protective gear identification task target (tag), and its value ranges from -1 to 1. The closer Rc is to 1, the stronger the positive correlation between the channel features and the task target, and the greater its contribution to protective gear identification; the closer it is to 0 or negative, the weaker or irrelevant the correlation between the channel features and the task target, indicating that they are redundant features.

[0062] For example, select representative samples from the training set in step S1, 100 images for each viewpoint and each occlusion level, for a total of 1200 images, input them into the student model embedded with FGU, record the feature response values ​​Fc,i of each channel before FGU input, where c is the channel index, i is the sample index, and the protective gear label of the corresponding sample, using one-hot encoding, such as gloves as [1,0,0], protective clothing as [0,1,0], and face mask as [0,0,1]. Map the protective gear labels to binary task target vectors Yi (Yi=1 if the sample contains the target protective gear, and Yi=0 if it does not, simplifying the correlation calculation). The Pearson coefficient is calculated specifically by taking each channel c and calculating its feature response Fc,i and the task objective Yi, using the following formula: Where N is the number of samples (N=1200 here). The average characteristic response of channel c. For the mission objective The mean; Subsequently, Min-Max normalization is performed on all channels Rc, mapping them to the [0,1] interval to obtain the standardized evaluation score Rc,norm.

[0063] In one embodiment, for the construction of the correlation regularization term, for example, a regularization term based on channel-task correlation is introduced. The formula is Where γ is the third weight coefficient (ranging from 0.1 to 0.3, with a default value of 0.2), used to control the contribution of the regularization term to the total loss and avoid over-regularization leading to underfitting of the model; c is the channel index, and C is the total number of channels; Channel redundancy coefficient: The smaller the value (the lower the correlation between the channel and the task), the larger the coefficient, and the higher the gating coefficient. The harsher the punishment; The FGU output represents the absolute value of the channel attention weights. The regularization term penalizes channels with low correlation. This causes it to approach 0, thus achieving sparsity of redundant channels.

[0064] Understandably, for highly correlated channels →1: →0, regularization term pairs The penalty approaches 0, allowing Maintaining larger values ​​enhances the expression of highly correlated features; For low correlation channels →0: →1, Regularization terms The punishment was significantly increased, forcing Decrease the value to approach 0, suppress the interference of redundant features, and reduce the computational load of the model.

[0065] For training with regularized total loss, the total loss function is constructed by incorporating the FGU regularization term. The total training loss incorporated into the student model is calculated using the following formula: in, The distillation loss in step S3 (feature map distillation loss and classification soft label distillation loss); The detection loss of the student model itself, The default value is 0.3; This is the correlation regularization term for the feature gating unit. This loss term is used to simultaneously optimize the gating parameters of the FGU and the detection performance of the student model.

[0066] In some embodiments, for step S4, the pruned and distilled model files, model configuration files, including input / output dimensions, class label mappings, default values ​​for inference parameters, and model version information files, including training time, test accuracy, and lightweight metrics, are packaged into a unified model file package to facilitate AR device access and version management.

[0067] For example, the model configuration file adopts JSON format, annotating the input image preprocessing method, such as normalization parameters, image size scaling method, output parsing rules, such as detection box coordinate decoding method, confidence threshold default value of 0.5, and category labels, such as 0-gloves, 1-protective clothing, 2-face mask, to ensure that the edge application can correctly parse the model output.

[0068] Figure 4 A lightweight insulated protective gear recognition model optimization device 400 for AR glasses is shown. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0069] like Figure 4 As shown in the embodiment of this application, the lightweight insulated protective gear recognition model optimization device 400 for AR glasses includes: The training set acquisition module 401 is used to acquire a training image dataset containing multiple viewpoints and multiple occlusion conditions, wherein the viewpoints include at least top view, eye view, top view and side view, and the occlusion conditions are labeled with occlusion levels. The model pruning module 402 is used to take the training image dataset as input and perform a view-sensitive adaptive pruning process based on the pre-trained object detection model to obtain a lightweight student model. The adaptive pruning process includes: calculating the channel responses of the convolutional layers or feature layers in the pre-trained object detection model under different viewpoints; constructing a view-sensitive weight matrix based on the channel responses, whereby the view-sensitive weight matrix characterizes the consistency of importance of each channel under multiple viewpoints; and sorting and selectively pruning the model channels according to the view-sensitive weight matrix. The model distillation module 403 is used to perform a knowledge distillation optimization process after the adaptive pruning process, and to train the lightweight student model using the uncompressed teacher model; wherein the knowledge distillation optimization process introduces an occlusion guidance mechanism, dynamically adjusting the weight of the distillation loss according to the occlusion level of the training samples, so that the student model learns the feature response of the teacher model more effectively under occlusion conditions. Output module 404 is used to output the final lightweight protective gear test model after pruning and distillation optimization.

[0070] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0071] Figure 5 The diagram illustrates the structure of an apparatus suitable for implementing the methods and / or technical solutions in the embodiments of this application. The apparatus 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0072] The following components are connected to I / O interface 505: input section 506 including keyboard, mouse, touch screen, microphone, infrared sensor, etc.; output section 507 including cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; storage section 508 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and communication section 509 including network interface card such as LAN (local area network) card, modem, etc. Communication section 509 performs communication processing via a network such as the Internet.

[0073] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, it performs the functions defined in the methods of this application.

[0074] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0075] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0076] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0077] Furthermore, the inclusion of a single word does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A method for optimizing a lightweight insulated protective gear recognition model for AR glasses, characterized in that, The methods and steps include the following: Obtain a training image dataset containing multiple viewpoints and multiple occlusion conditions, wherein the viewpoints include at least one of top view, eye view, top view and side view, and the occlusion conditions are labeled with occlusion levels; Using the training image dataset as input, a view-sensitive adaptive pruning process is performed based on the pre-trained object detection model to obtain a lightweight student model. The adaptive pruning process includes: calculating the channel responses of the convolutional layers or feature layers in the pre-trained object detection model under different viewpoints; constructing a view-sensitive weight matrix based on the channel responses, whereby the view-sensitive weight matrix characterizes the consistency of importance of each channel across multiple viewpoints; and sorting and selectively pruning the model channels according to the view-sensitive weight matrix. Following the adaptive pruning process, a knowledge distillation optimization process is performed, using the uncompressed teacher model to train the lightweight student model. The knowledge distillation optimization process introduces an occlusion guidance mechanism, dynamically adjusting the weights of the distillation loss based on the occlusion level of the training samples, so that the student model learns the feature responses of the teacher model more effectively under occlusion conditions. Output the final lightweight protective gear test model after pruning and distillation optimization.

2. The method for optimizing a lightweight insulating protective gear recognition model for AR glasses according to claim 1, characterized in that: The construction of the perspective-sensitive weight matrix specifically includes: For each channel, calculate its average activation value in each view group; Based on the average activation values ​​across all view groups, calculate the cross-view consistency score for this channel. This score is equal to the mean of the average activation values ​​across all view groups, divided by the sum of the standard deviation of the average activation values ​​across all view groups and a very small constant. The consistency score is weighted and fused with the overall importance index of the channel to obtain the final view sensitivity weight of each channel. This weight is equal to the first adjustable weight coefficient multiplied by the normalized overall importance index, plus the second adjustable weight coefficient multiplied by the normalized consistency score, where the sum of the first adjustable weight coefficient and the second adjustable weight coefficient is equal to 1.

3. The method for optimizing a lightweight insulating protective gear recognition model for AR glasses according to claim 1, characterized in that: The adaptive pruning process employs a staged pruning strategy, which includes at least two pruning stages. After each pruning stage, the model is briefly fine-tuned, and its performance is verified on a dedicated multi-view validation set. If the performance degradation under any view exceeds a preset threshold, the pruning operation of that stage is rolled back, and the pruning intensity of the corresponding layer is reduced.

4. The method for optimizing a lightweight insulating protective gear recognition model for AR glasses according to claim 1, characterized in that, Also includes: The distillation loss in the knowledge distillation optimization process includes at least feature map distillation loss and classification soft label distillation loss; the feature map distillation loss is the imitation loss of the student model to the key layer output feature map of the teacher model, which is obtained by calculating the L2 distance between the two feature maps; the classification soft label distillation loss is the imitation loss of the student model to the softened probability distribution of the classification head output of the teacher model, which is obtained by calculating the KL divergence between the two probability distributions.

5. The method for optimizing a lightweight insulating protective gear recognition model for AR glasses according to claim 4, characterized in that, When calculating the feature map distillation loss, a dynamic weight coefficient is applied to the feature map distillation loss term based on the occlusion level of the target in the current training sample. The higher the occlusion level, the larger the dynamic weight coefficient.

6. The method for optimizing a lightweight insulating protective gear recognition model for AR glasses according to claim 2, characterized in that, Also includes: The overall importance metric for the channel is determined based on the channel's L1 norm or the channel's scaling factor in the batch normalization layer.

7. A lightweight insulated protective gear recognition model optimization device for AR glasses, characterized in that, include: The training set acquisition unit is used to acquire a training image dataset containing multiple viewpoints and multiple occlusion conditions, wherein the viewpoints include at least one of top view, eye view, top view and side view, and the occlusion conditions are labeled with occlusion levels. The model pruning module is used to take the training image dataset as input and perform a view-sensitive adaptive pruning process based on the pre-trained object detection model to obtain a lightweight student model. The adaptive pruning process includes: calculating the channel responses of the convolutional layers or feature layers in the pre-trained object detection model under different viewpoints; constructing a view-sensitive weight matrix based on the channel responses, whereby the view-sensitive weight matrix characterizes the consistency of importance of each channel under multiple viewpoints; and sorting and selectively pruning the model channels according to the view-sensitive weight matrix. The model distillation module is used to perform a knowledge distillation optimization process after the adaptive pruning process, and to train the lightweight student model using the uncompressed teacher model; wherein, the knowledge distillation optimization process introduces an occlusion guidance mechanism, which dynamically adjusts the weight of the distillation loss according to the occlusion level of the training samples, so that the student model learns the feature response of the teacher model more effectively under occlusion conditions. The output module is used to output the final lightweight protective gear test model after pruning and distillation optimization.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable medium having computer program instructions stored thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images