A Closed-Set Authentication Method for Desktop Icons in Embedded NPUs

Abstract

This invention belongs to the fields of computer vision, deep learning, embedded systems, and human-computer interaction, and specifically provides a desktop icon closed-set authentication method for embedded NPUs. It includes the following steps: S1: Automatic generation of training data based on programmatic synthesis; S2: Heterogeneous dual-model joint training and structure optimization; S3: Deployment engineering method for embedded NPUs; S4: Application registration and first-level verification based on an enhanced averaging mechanism; S5: A second-level fusion mechanism based on text anchors and neighborhood topology. This invention decouples icon localization from closed-set authentication, using a single-class detector to locate candidate icon regions and a metric learning embedder and registration library to perform application identity comparison, enabling the system to expand through registration when new applications are added; the enhanced averaging registration mechanism reduces the random bias caused by single registration samples, improving the matching stability of icons under perturbations such as brightness, scaling, and background changes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, embedded systems and human-computer interaction, specifically to a desktop icon closed set authentication method for embedded NPUs. Background Technology

[0002] With the rapid evolution of desktop automated testing, accessibility development, and intelligent cockpit systems, accurate understanding of desktop screen content has become a critical technical aspect. Icons, as the most important semantic units in the operating system's graphical user interface (GUI), directly impact the execution logic of automated tasks through accurate identification. Existing technologies typically face three major challenges in addressing this issue: high computational resource requirements, poor closed-set verification capabilities, and insufficient robustness to visual iteration.

[0003] In embedded environments, chips equipped with dedicated NPUs, such as the Rockchip RK3588, possess strong tensor computation capabilities. However, limitations in their hardware architecture regarding bit width (e.g., INT8 quantization) and operator support (e.g., dynamic shapes, complex activation functions) make it difficult to directly migrate traditional large cloud models. For example, visual language model (VLM) solutions such as Microsoft OmniParser, which emerged in 2024, can achieve natural language descriptions of screen content with a large number of parameters (>1.5B), but the model size exceeding 1GB results in significant latency on embedded devices and cannot solve the closed-set authentication problem of "whether the icon belongs to a specific application in the registry".

[0004] Furthermore, application icons undergo frequent visual iterations throughout their lifecycle. These visual changes make matching mechanisms that rely solely on visual embedding vectors highly susceptible to failure, with similarity scores potentially dropping from a high confidence level of over 0.9 to below 0.55, leading to a large number of missed or false positives.

[0005] Currently, the main shortcomings of existing technologies in the field of desktop icon recognition can be summarized as follows: First, existing large-parameter visual language model solutions fall under the category of open-set semantic understanding, lacking the ability to perform one-to-one closed-set comparisons with specific registration databases, and cannot run independently and with low latency in a network-free environment.

[0006] Secondly, traditional image retrieval techniques (such as YOLO detection combined with ArcFace feature extraction) are mostly optimized for GPU servers and do not consider the quantization constraints of embedded NPUs. When these models are deployed to embedded NPUs to perform INT8 quantization, feature vector drift often occurs due to operator incompatibility or loss of quantization accuracy, with measured similarity reductions reaching 3% to 5%.

[0007] Furthermore, methods based solely on template matching (such as SSIM and ORB feature points) are extremely sensitive to minute deformations, color shifts, rotations, and size changes in icons. Finally, existing technologies all ignore the contextual signals surrounding the icon (such as text labels and positional topology), which tend to remain highly stable when the icon's visual features undergo drastic changes. Summary of the Invention

[0008] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0009] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution: a desktop icon closed-set authentication method for embedded NPUs, comprising the following steps: S1: Automatic generation of training data based on procedural synthesis: Construct a library of at least 100 icon assets with alpha channels, and adjust the background to a standard 640×640 pixel image. During the synthesis process, perform the following random transformations: geometric transformation, random rotation, spatial conflict detection, and alpha channel blending. S2: Heterogeneous Dual-Model Joint Training and Structure Optimization: A heterogeneous dual-model joint training strategy is adopted, which includes two parts: a detector for icon localization and an embedder for identity representation. The detector adopts the YOLOv8n detector, which is used to quickly locate all icon candidate areas from the desktop screenshot. Its input is a 640×640 RGB desktop screenshot, and the output is the position of the icon candidate box and its single-class "icon" confidence. The system enables the single_cls=True mode to treat all desktop icons as the same category. The embedder uses a MobileNetV2 backbone network combined with an ArcFace metric learning head to encode the identity features of a single icon image cropped by the detector. Its input is a 128×128 icon image block, and its output is a 256-dimensional L2 normalized feature vector, which is used for subsequent cosine similarity matching with the registration library. S3: Deployment engineering approach for embedded NPUs: Detector operator truncation: Identify and extract 6 convolution output nodes in YOLOv8n, and construct a subgraph model containing only static shape operators such as convolution; RKNN quantization configuration: An asymmetric quantization strategy is adopted, with the mean parameter mean_values=[[0,0,0]] and the standard deviation parameter std_values=[[255,255,255]]. During RKNN inference, the input pixels are normalized in the manner of normalized=(pixel-mean) / std. Setting mean_values ​​to [[0,0,0]] indicates that the mean subtraction operation is not performed. S4: Application Registration and Level 1 Validation Based on Enhanced Averaging Mechanism Icon registration is performed using an enhanced averaging method: the registration program performs 8 random data enhancements on the input icon, each time randomly combining rotation, brightness adjustment, random scaling, optional Gaussian blur, and background blending operations; Level 1 Validation Process: First, the input desktop screenshot is subjected to icon detection, and the detection box is cropped into 128×128 icon blocks. The embedding vector is then extracted by the embedder. Then, Registration characteristics of each application in the registry Calculate the cosine similarity for each match and take the maximum value as the first-level matching score. , ; S5: A two-level fusion mechanism based on text anchors and neighborhood topology: Text anchor point validation: Its input is the bounding box of the target icon obtained in the first-level validation stage. And the names of the primary candidate applications; Neighborhood location topology verification: Its input is the detection results of the target icon and its neighboring icons in the current screenshot, as well as the corresponding application topology descriptor stored in the registration phase; Secondary fusion stage: embedding the primary visual data into the score. Text anchor point score and neighborhood topology score The scores are then combined using a weighted method to obtain a comprehensive score. .

[0010] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, wherein in step S1: The geometric transformation includes scaling the randomly selected PNG icon material itself. First, the i-th icon is randomly selected from the icon library, and then its target size s is randomly determined, where s∈[32,96] pixels. The icon is then scaled to the target size using bilinear interpolation. The background image has been uniformly adjusted to 640×640 pixels in the previous step, and the icon is then superimposed only on the background with this fixed resolution. Random rotation includes: performing random rotation on the scaled icon foreground block, with the rotation angle θ controlled within the range of [-15°, +15°], using bilinear interpolation during rotation, and using BORDER_CONSTANT transparent fill to handle the blank boundary after rotation; Spatial conflict detection includes: after an icon is randomly placed on the background coordinates (x, y), calculating the intersection-union ratio (IUU) of the current candidate icon box with the already placed icon boxes one by one, and using 0.3 as a threshold for deduplication; if the threshold is exceeded, the placement position is resampled, and the process is repeated up to 50 times. Alpha channel blending: Blending the foreground pixels of rotated and scaled icons. pixels corresponding to the background image Perform pixel-by-pixel alpha fusion to obtain the final synthesized pixels. The calculation formula is as follows: , where α is the normalized value of the icon's alpha channel, with a value range of [0,1]. The fusion objects are "icon foreground" and "background wallpaper". The fusion result is a 640×640 composite training image after superimposing the icons.

[0011] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPU described in this invention, in step S2, the embedder first maps the icons to a 256-dimensional unit hyperspherical feature space, and then ArcFace introduces an interval constraint in the angle space. ArcFace adds a fixed interval m=0.5rad to the angle corresponding to the real category, and performs classification training in combination with a scaling factor s=30.0, thereby compressing the feature distance between icons of the same type and increasing the feature interval between icons of different types.

[0012] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, the six convolutional output nodes in S3 are as follows: / model.22 / cv2.0 / cv2.0.2 / Conv_output_0, corresponding to the shape [1,64,80,80], represents large-scale boxDFL output; / model.22 / cv3.0 / cv3.0.2 / Conv_output_0, with a shape of [1,1,80,80], represents large-scale classlogit output; / model.22 / cv2.1 / cv2.1.2 / Conv_output_0, with a shape of [1,64,40,40], represents the mid-scale boxDFL output; / model.22 / cv3.1 / cv3.1.2 / Conv_output_0, with a shape of [1,1,40,40], represents the mid-scale classlogit output; / model.22 / cv2.2 / cv2.2.2 / Conv_output_0, with a corresponding shape of [1,64,20,20], represents small-scale boxDFL output; / model.22 / cv3.2 / cv3.2.2 / Conv_output_0 corresponds to the shape [1,1,20,20], which represents the small-scale classlogit output.

[0013] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPU described in this invention, the normalization process for input pixels in S3 is as follows: configuring std_values ​​as [[255,255,255]] indicates that normalization by dividing by 255 is performed on each of the three color channels. This setting is mathematically consistent with the / 255 operation of ToTensor() in the PyTorch training phase.

[0014] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, the S4 enhanced averaging method specifically involves, for each enhancement result, inputting it into the embedder to extract a 256-dimensional L2 normalized embedding vector. The eight vectors are all derived from the feature representations of the same application icon under different perturbation conditions. They collectively reflect the stable identity semantics of the application icon under rotation, brightness, scaling, and background changes. Subsequently, the mean of the eight embedding vectors is calculated to obtain... Perform L2 normalization again to obtain the final registered features. .

[0015] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, in step S4, a first-level confidence level is output based on the score: TIER-1: When When the high-confidence matching result is reached, the secondary fusion verification is skipped. TIER-2: When At that time, it was assumed that there was some uncertainty in the visual signal, triggering a second-level fusion verification. TIER-3: When At that time, it was assumed that the icon may have undergone significant visual shift or redesign, triggering a low-score recovery check.

[0016] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, the specific method for text anchor point verification in S5 is as follows: First, a text search area is defined according to the icon frame size, including the lower label area, the upper label area, and the right label area; then, the image is cropped from the corresponding area and fed into a lightweight OCR model to extract candidate text. After normalizing the OCR results by modifying the modifiers, they are compared with the text aliases set (text_aliases) pre-stored in the registry for this application. Character-level edit distance similarity and character set Jaccard similarity are calculated respectively, and the maximum value is taken as the text anchor score. If no valid text is extracted, then... The output of this signal is a text consistency score between the target icon and the candidate application.

[0017] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPUs described in this invention, the specific method for neighborhood position topology verification in S5 is as follows: taking the center of the target icon as the origin, select K=4 icons with the closest Euclidean distance as neighbors; calculate the relative position description components for each neighbor, including horizontal offset, vertical offset, and distance. and azimuth And combine it with the embedded feature vector of the neighbor. This forms the current local topological descriptor. Then, the current topological descriptor and the registered topological descriptor are compared using the Hungarian algorithm or a greedy approximation algorithm to establish optimal neighbor correspondences. For each pair of matching neighbors, the positional similarity and neighbor embedding similarity are jointly calculated, and a weighted neighborhood topological score is obtained. If there are not enough neighbors, calculate based on the actual number of neighbors; if there are no neighbors, then... The output of this signal is a score showing the consistency between the target icon and the candidate application in terms of spatial layout.

[0018] As a preferred embodiment of the desktop icon closed-set authentication method for embedded NPU described in this invention, in the second-level fusion stage of S5, when a certain auxiliary signal is missing, its weight is transferred to the visual embedded signal to maintain weight normalization. If the first-level authentication is TIER-2, then according to... Whether the value exceeds 0.75 or 0.55, output "high", "low", or "unknown" respectively; if the first-level verification is TIER-3, then calculate the auxiliary signal for the entire registry library, and... and Output the restored matching result.

[0019] Compared with existing technologies, the present invention has the following advantages: It decouples icon localization from closed-set authentication, utilizes a single-class detector to locate icon candidate regions, and employs a metric learning embedder and a registration library to perform application identity comparison. This allows the system to expand through registration when new applications are added, without retraining the detection model. Furthermore, it reduces random bias caused by single registration samples by enhancing the average registration mechanism, improving the matching stability of icons under disturbances such as brightness, scaling, and background changes. When the first-level visual embedding confidence is insufficient, it introduces text anchors and neighborhood position topology as second-level fusion verification signals, which can compensate for visual embedding drift in scenarios such as icon theme switching, color changes, or visual redesign, reducing the risk of misidentification and missed identification. Finally, through detector post-processing dynamic operator truncation, lightweight embedding backbone selection suitable for quantization deployment, and quantization pre-processing parameter alignment, the detection and verification process is more suitable for offline deployment with embedded NPUs, and can output hierarchical confidence levels and interpretable intermediate scores, facilitating reliable decision-making by upper-layer systems based on different application scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart of a desktop icon closed set authentication method for embedded NPU according to the present invention; Figure 2 This is a flowchart of the embedder identity feature encoding method for a desktop icon closed set authentication method for embedded NPU according to the present invention. Figure 3 This is a schematic diagram of the truncation of the detector head operator in a desktop icon closed set authentication method for embedded NPU according to the present invention; Figure 4 This is a flowchart of a three-level cascaded decision fusion process for a desktop icon closed-set authentication method for embedded NPUs according to the present invention. Figure 5 This is a schematic diagram illustrating the construction of a neighborhood topology descriptor for a desktop icon closed-set authentication method for embedded NPUs according to the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0024] The core objective of this invention is to introduce a two-level fusion verification mechanism based on neighborhood topology relationships. This mechanism aims to accurately verify the application's identity within a three-level cascaded decision framework by extracting the text anchor signals (Signal A) and neighborhood topology descriptors (Signal B) surrounding the icon, even when visual features "drift." The overall technical framework consists of four collaborative schemes: a synthetic data generation scheme, a dual-model architecture scheme, a deployment engineering optimization scheme, and a multimodal fusion verification scheme.

[0025] At the model level, this invention constructs a heterogeneous dual-model system: a lightweight single-class YOLOv8n detector focused on icon localization, and an embedding network based on the MobileNetV2 backbone network and the ArcFace metric learning algorithm. At the deployment level, this invention proposes a dynamic operator truncation method for the NPU toolchain, preserving the convolutional feature map output of the static shape. At the verification logic level, this invention designs a three-level cascaded decision system, providing the final identity determination through a weighted fusion of visual confidence, text labels, and local topological descriptors.

[0026] For details, please refer to Figure 1 A desktop icon closed set authentication method for embedded NPUs includes the following steps: Phase 1: Automatic generation of training data based on procedural synthesis: Construct a library of at least 100 icon assets with alpha channels and adjust the background to a standard 640×640 pixel image.

[0027] During the synthesis process, the system performs the following random transformations: Geometric transformation: The system scales the randomly selected PNG icon itself, rather than scaling the entire background image. Specifically, the system first randomly selects the i-th icon from the icon library, then randomly determines its target size s, where s∈[32,96] pixels, and uses bilinear interpolation to scale the icon to the target size. The background image has already been uniformly adjusted to 640×640 pixels in the previous step, and the icons are only superimposed on this fixed resolution background. Therefore, the entire composite result always maintains a 640×640 resolution.

[0028] Random Rotation: Perform a random rotation on the scaled icon foreground block, with the rotation angle θ controlled within the range of [-15°, +15°]. Bilinear interpolation is used during rotation, and the blank borders after rotation are handled with BORDER_CONSTANT transparent fill to avoid black borders, jagged edges, or image truncation. Since the rotation operation only applies to the icon foreground block, and the background canvas remains fixed at 640×640 pixels, the output composite image resolution remains 640×640.

[0029] Spatial conflict detection: After an icon is randomly placed on the background at coordinates (x, y), the system calculates the intersection-over-union (IoU) ratio between the current candidate icon bounding box and each already placed icon bounding box, using 0.3 as a threshold for deduplication. If the IoU value exceeds this threshold, the placement position is resampled, with a maximum of 50 retries. This method aims to avoid large-area overlap between multiple icons, ensuring that the bounding boxes of each icon in the synthetic training samples are clearly distinguishable, thereby improving the quality of YOLO annotation and the training effect of subsequent detectors.

[0030] Alpha channel blending: Blending the foreground pixels of rotated and scaled icons. pixels corresponding to the background image Perform pixel-by-pixel alpha fusion to obtain the final synthesized pixels. The calculation formula is as follows: Where α is the normalized value of the icon's alpha channel, ranging from [0,1]. The fusion objects are the "icon foreground" and the "background wallpaper," and the fusion result is a 640×640 composite training image with the icons superimposed.

[0031] Phase Two: Joint Training and Structure Optimization of Heterogeneous Dual Models: Please refer to... Figure 2 The embedder encodes the icon image patches cropped by the detector into L2-normalized identity feature vectors, and then performs cosine similarity matching between these identity feature vectors and the registration database. This stage employs a heterogeneous dual-model joint training strategy, consisting of a detector for icon localization and an embedder for identity representation.

[0032] The YOLOv8n detector's function is to quickly locate all candidate icon regions from a desktop screenshot. Its input is a 640x640 RGB desktop screenshot, and its output is the position of the candidate icon box and its single-category "icon" confidence score. The system enables single_cls=True mode, treating all desktop icons as the same category, allowing the detector to focus on determining "whether it is an icon and its location" rather than undertaking specific application identity classification tasks.

[0033] The embedder uses a MobileNetV2 backbone network combined with an ArcFace metric learning head. Its function is to encode the identity features of individual icon images cropped by the detector. Its input is a 128×128 icon image block, and its output is a 256-dimensional L2 normalized feature vector, which is used for subsequent cosine similarity matching with the registry.

[0034] The reason for using this combination of models is that this invention aims to simultaneously solve two problems: "Where is the icon?" and "Who is represented by this icon?". The former is suitable for a lightweight object detection model, while the latter is suitable for a metric learning embedding model. Decoupling detection and authentication improves the deployment efficiency of the system on the embedded NPU and facilitates the dynamic addition of new applications within the closed-set registry framework without retraining the detector.

[0035] In terms of training principles, the embedder first maps the icons to a 256-dimensional unit hyperspherical feature space, and then ArcFace introduces margin constraints in the angle space. Specifically, ArcFace adds a fixed margin m=0.5rad to the angles corresponding to the true categories and performs classification training in conjunction with a scaling factor s=30.0, thereby compressing the feature distance between icons of the same category and increasing the feature margin between icons of different categories. For the closed-set authentication scenario of this invention, this approach helps improve the discriminative power of cosine similarity matching between registration icons and query icons, enabling the system to maintain stable recognition even when icons experience perturbations such as brightness changes, color shifts, or slight rotations.

[0036] Furthermore, this invention explicitly selects MobileNetV2 instead of MobileNetV3-like structures containing SE attention modules because operations such as Sigmoid in the SE module are prone to introducing accuracy loss after RKNN INT8 asymmetric quantization, causing embedding vector drift; while the MobileNetV2 structure is more conducive to embedded NPU quantization deployment and can guarantee the quality of quantized features.

[0037] Phase 3: Deployment Engineering Methods for Embedded NPUs Detector operator truncation: See also Figure 3The standard YOLOv8 ONNX model contains complex Reshape and Concat operators in the Detect header. This invention adopts a truncation strategy: it identifies and extracts six convolution output nodes from YOLOv8n to construct a subgraph model containing only static shape operators such as convolution.

[0038] Specifically, the six convolutional output nodes mentioned above are as follows: / model.22 / cv2.0 / cv2.0.2 / Conv_output_0, corresponding to the shape [1,64,80,80], represents large-scale boxDFL output; / model.22 / cv3.0 / cv3.0.2 / Conv_output_0, with a shape of [1,1,80,80], represents large-scale classlogit output; / model.22 / cv2.1 / cv2.1.2 / Conv_output_0, with a shape of [1,64,40,40], represents the mid-scale boxDFL output; / model.22 / cv3.1 / cv3.1.2 / Conv_output_0, with a shape of [1,1,40,40], represents the mid-scale classlogit output; / model.22 / cv2.2 / cv2.2.2 / Conv_output_0, with a corresponding shape of [1,64,20,20], represents small-scale boxDFL output; / model.22 / cv3.2 / cv3.2.2 / Conv_output_0 corresponds to the shape [1,1,20,20], which represents the small-scale classlogit output.

[0039] By truncating the dynamic Shape operators such as Reshape and Concat after the Detect head, and retaining only the static Shape convolution output nodes mentioned above, a quantizable subgraph model adapted to the RKNN toolchain can be constructed.

[0040] RKNN Quantization Configuration: During INT8 quantization, this invention performs RKNN transformation separately for the aforementioned derived detector ONNX model and embedder ONNX model. This invention employs an asymmetric quantization strategy, configuring the mean parameter mean_values=[[0,0,0]] and the standard deviation parameter std_values=[[255,255,255]]. During RKNN inference, the input pixels are normalized according to the formula normalized=(pixel-mean) / std. Configuring mean_values ​​as [[0,0,0]] indicates that the mean subtraction operation is not performed; configuring std_values ​​as [[255,255,255]] indicates that normalization by dividing by 255 is performed on each of the three color channels. This setting is mathematically consistent with the / 255 operation of ToTensor() during the PyTorch training phase, thereby eliminating the difference between pre-training processing and pre-deployment processing.

[0041] Phase 4: Application Registration and Level 1 Validation Based on Enhanced Averaging Mechanism Icon registration (enhanced averaging method): This embodiment uses... The augmentation averaging method is used. The registration program performs eight random data augmentations on the input icon, each time randomly combining rotation (±15°), brightness adjustment, random scaling, optional Gaussian blur, and background blending. For each augmentation result, an embedding vector of 256 dimensions is extracted from the embedding unit. The above eight vectors all originate from the feature representations of the same app icon under different perturbation conditions, collectively reflecting the stable identity semantics of the app icon under rotation, brightness, scaling, and background changes. Subsequently, the mean of the eight embedding vectors is calculated to obtain... Perform L2 normalization again to obtain the final registered features. This registration feature is essentially the central feature of the application icon under various perturbation conditions. It can reduce the random bias caused by single-sample registration and improve the robustness of subsequent matching to visual perturbations and minor UI iterations.

[0042] Level 1 Validation Process: The level 1 validation stage focuses on the similarity matching between the query icon features and features in the registry. Specifically, the system first performs icon detection on the input desktop screenshot, then crops the detection box into 128×128 icon blocks, and extracts the query embedding vector using an embedder. Subsequently, Registration characteristics of each application in the registry Calculate the cosine similarity for each match and take the maximum value as the first-level matching score. ,Right now: .

[0043] The system outputs a first-level confidence level based on this score: TIER-1 (High): When When the high-confidence matching result is reached, the secondary fusion verification is skipped. TIER-2 (Ambiguous): When At that time, it was assumed that there was some uncertainty in the visual signal, triggering a second-level fusion verification. TIER-3 (Low): When At that time, it was assumed that the icon may have undergone significant visual shift or redesign, triggering a low-score recovery check.

[0044] Phase 5: A two-level fusion mechanism based on text anchors and neighborhood topology: Please refer to [link / reference]. Figure 4 This stage is an auxiliary verification mechanism triggered on the basis of the first-level visual embedding verification. When the first-level matching result is in the fuzzy range or low score range, the system further utilizes the relatively stable text information and spatial topological relationship around the icon to perform a second-level fusion judgment, so as to improve the robustness of identity verification in the context of icon UI iteration.

[0045] Signal A: Text Anchor Validation: Its input is the bounding box of the target icon obtained in the first-level validation phase. The system first defines the text search area based on the icon frame size, including the bottom label area, the top label area, and the right label area; then it crops the image from the corresponding area and feeds it into a lightweight OCR model to extract candidate text. After performing normalization processing on the OCR results, including removing spaces, converting to lowercase, and removing modifiers such as beta and lite, the results are compared with the text aliases set (text_aliases) pre-stored in the registry for this application. Character-level edit distance similarity and character set Jaccard similarity are calculated, and the maximum value is taken as the text anchor point score. If no valid text is extracted, then... The output of this signal is a textual consistency score between the target icon and the candidate application.

[0046] Signal B: Neighborhood Topology Verification: Please refer to [link / reference needed] Figure 5 The input consists of the target icon and its neighboring icons detected in the current screenshot, as well as the corresponding application topology descriptor stored during the registration phase. The system selects K=4 icons with the closest Euclidean distance as neighbors, using the center of the target icon as the origin; for each neighbor, it calculates the relative position descriptor components, including horizontal offset, vertical offset, and distance. and azimuth And combine it with the embedded feature vector of the neighbor. This forms the current local topological descriptor. Then, the current topological descriptor and the registered topological descriptor are compared using the Hungarian algorithm or a greedy approximation algorithm to establish optimal neighbor correspondences. For each pair of matching neighbors, the positional similarity and neighbor embedding similarity are jointly calculated, and a weighted neighborhood topological score is obtained. If there are not enough neighbors, calculate based on the actual number of neighbors; if there are no neighbors, then... The output of this signal is a score showing the consistency between the target icon and the candidate application in terms of spatial layout.

[0047] In the secondary fusion stage, the system embeds the primary visual score. Text anchor point score and neighborhood topology score The scores are then combined using a weighted method to obtain a comprehensive score. When an auxiliary signal is missing, its weight is transferred to the visual embedding signal to maintain weight normalization. If the first-level validation is TIER-2, then according to... Whether the value exceeds 0.75 or 0.55, output "high", "low", or "unknown" respectively; if the first-level verification is TIER-3, then calculate the auxiliary signal for the entire registry library, and... and Output the restored matching result.

[0048] The synergy between this stage and previous stages is as follows: the first stage provides high-quality icon training samples; the second stage trains the detector and embedder; the fourth stage constructs registration features and topological descriptors; and the fifth stage uses these existing results for correction and recovery when primary visual judgment is insufficient. Its technical effects are: in normal scenarios, the system still achieves low-latency recognition using primary embedding matching; when UI iterations occur such as icon theme changes, color inversions, or brand redesigns, visual embedding drift can be compensated for through text labels and spatial topological relationships, thereby improving recognition accuracy and authorization stability. In actual testing, during moderate UI iterations, a certain icon... After introducing topological signals The score was improved to 0.806 and crossed the high confidence threshold. In intensity iteration scenarios, the low-score recovery path can significantly improve the recognition success rate by combining text anchors and neighborhood topology.

[0049] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A desktop icon closed-set authentication method for embedded NPUs, characterized in that, Includes the following steps: S1: Automatic generation of training data based on procedural synthesis: Construct a library of at least 100 icon assets with alpha channels, and adjust the background to a standard 640×640 pixel image. During the synthesis process, perform the following random transformations: geometric transformation, random rotation, spatial conflict detection, and alpha channel blending. S2: Heterogeneous Dual-Model Joint Training and Structure Optimization: A heterogeneous dual-model joint training strategy is adopted, which includes two parts: a detector for icon localization and an embedder for identity representation. The detector adopts the YOLOv8n detector, which is used to quickly locate all icon candidate areas from the desktop screenshot. Its input is a 640×640 RGB desktop screenshot, and the output is the position of the icon candidate box and its single-category "icon" confidence. The system enables the single_cls=True mode to treat all desktop icons as the same category. The embedder uses a MobileNetV2 backbone network combined with an ArcFace metric learning head to encode the identity features of a single icon image cropped by the detector. Its input is a 128×128 icon image block, and its output is a 256-dimensional L2 normalized feature vector, which is used for subsequent cosine similarity matching with the registration library. S3: Deployment engineering approach for embedded NPUs: Detector operator truncation: Identify and extract 6 convolution output nodes in YOLOv8n, and construct a subgraph model containing only static shape operators such as convolution; RKNN quantization configuration: An asymmetric quantization strategy is adopted, with the mean parameter mean_values=[[0,0,0]] and the standard deviation parameter std_values=[[255,255,255]]. During RKNN inference, the input pixels are normalized in the manner of normalized=(pixel-mean) / std. Setting mean_values ​​to [[0,0,0]] indicates that the mean subtraction operation is not performed. S4: Application Registration and Level 1 Validation Based on Enhanced Averaging Mechanism Icon registration is performed using an enhanced averaging method: the registration program performs 8 random data enhancements on the input icon, each time randomly combining rotation, brightness adjustment, random scaling, optional Gaussian blur, and background blending operations; Level 1 Validation Process: First, the input desktop screenshot is subjected to icon detection, and the detection box is cropped into 128×128 icon blocks. The embedding vector is then extracted by the embedder. Then, Registration characteristics of each application in the registry Calculate the cosine similarity for each match and take the maximum value as the first-level matching score. , ; S5: A two-level fusion mechanism based on text anchors and neighborhood topology: Text anchor point validation: Its input is the bounding box of the target icon obtained in the first-level validation stage. And the names of the primary candidate applications; Neighborhood location topology verification: Its input is the detection results of the target icon and its neighboring icons in the current screenshot, as well as the corresponding application topology descriptor stored in the registration phase; Secondary fusion stage: embedding the primary visual data into the score. Text anchor point score and neighborhood topology score The scores are combined using a weighted method to obtain a comprehensive score. .

2. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, In S1: The geometric transformation includes scaling the randomly selected PNG icon material itself. First, the i-th icon is randomly selected from the icon library, and then its target size s is randomly determined, where s∈[32,96] pixels. The icon is then scaled to the target size using bilinear interpolation. The background image has been uniformly adjusted to 640×640 pixels in the previous step, and the icon is then superimposed only on the background with this fixed resolution. Random rotation includes: performing random rotation on the scaled icon foreground block, with the rotation angle θ controlled within the range of [-15°, +15°], using bilinear interpolation during rotation, and using BORDER_CONSTANT transparent fill to handle the blank boundary after rotation; Spatial conflict detection includes: after an icon is randomly placed on the background coordinates (x, y), calculating the intersection-union ratio (IUU) of the current candidate icon box with the already placed icon boxes one by one, and using 0.3 as a threshold for deduplication; if the threshold is exceeded, the placement position is resampled, and the process is repeated up to 50 times. Alpha channel blending: Blending the foreground pixels of rotated and scaled icons. pixels corresponding to the background image Perform pixel-by-pixel alpha fusion to obtain the final synthesized pixels. The calculation formula is as follows: , where α is the normalized value of the icon's alpha channel, with a value range of [0,1]. The fusion objects are "icon foreground" and "background wallpaper". The fusion result is a 640×640 composite training image after superimposing the icons.

3. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, In step S2, the embedder first maps the icon to a 256-dimensional unit hyperspherical feature space, and then ArcFace introduces an interval constraint in the angle space. ArcFace adds a fixed interval m=0.5rad to the angle corresponding to the real category, and performs classification training in combination with a scaling factor s=30.0, thereby compressing the feature distance between icons of the same type and increasing the feature interval between icons of different types.

4. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, The six convolutional output nodes in S3 are as follows: / model.22 / cv2.0 / cv2.0.2 / Conv_output_0, corresponding to the shape [1,64,80,80], represents large-scale boxDFL output; / model.22 / cv3.0 / cv3.0.2 / Conv_output_0, with a shape of [1,1,80,80], represents large-scale classlogit output; / model.22 / cv2.1 / cv2.1.2 / Conv_output_0, with a shape of [1,64,40,40], represents the mid-scale boxDFL output; / model.22 / cv3.1 / cv3.1.2 / Conv_output_0, with a shape of [1,1,40,40], represents the mid-scale classlogit output; / model.22 / cv2.2 / cv2.2.2 / Conv_output_0, with a shape of [1,64,20,20], represents small-scale boxDFL output; / model.22 / cv3.2 / cv3.2.2 / Conv_output_0 corresponds to the shape [1,1,20,20], which represents the small-scale classlogit output.

5. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, The method for normalizing the input pixels in S3 is to configure std_values ​​as [[255,255,255]], which means that normalization by dividing by 255 is performed on each of the three color channels. This setting is mathematically consistent with the / 255 operation of ToTensor() in the PyTorch training phase.

6. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, The S4 enhancement averaging method specifically involves inputting each enhancement result into an embedder to extract a 256-dimensional L2-normalized embedding vector. The eight vectors are all derived from the feature representations of the same application icon under different perturbation conditions. They collectively reflect the stable identity semantics of the application icon under rotation, brightness, scaling, and background changes. Subsequently, the mean of the eight embedding vectors is calculated to obtain... Perform L2 normalization again to obtain the final registered features. .

7. The desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, In S4, a first-level confidence level is output based on this score: TIER-1: When When the high-confidence matching result is reached, the secondary fusion verification is skipped. TIER-2: When At that time, it was assumed that there was some uncertainty in the visual signal, triggering a second-level fusion verification. TIER-3: When At that time, it was assumed that the icon may have undergone significant visual shift or redesign, triggering a low-score recovery check.

8. The desktop icon closed-set authentication method for embedded NPU according to claim 1, characterized in that, The specific method for text anchor point verification in S5 is as follows: First, define the text search area based on the icon frame size, including the lower label area, the upper label area, and the right label area; then, crop the image from the corresponding area and feed it into a lightweight OCR model to extract candidate text. After normalizing the OCR results by modifying the modifiers, they are compared with the text aliases set (text_aliases) pre-stored in the registry for this application. Character-level edit distance similarity and character set Jaccard similarity are calculated respectively, and the maximum value is taken as the text anchor score. If no valid text is extracted, then... The output of this signal is the text consistency score between the target icon and the candidate application.

9. A desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, The specific method for neighborhood location topology verification in S5 is as follows: taking the center of the target icon as the origin, select K=4 icons with the closest Euclidean distance as neighbors; calculate the relative position description components for each neighbor, including horizontal offset, vertical offset, and distance. and azimuth And combined with the embedded feature vector of the neighbor. This forms the current local topological descriptor. Then, the current topological descriptor and the registered topological descriptor are compared using the Hungarian algorithm or a greedy approximation algorithm to establish optimal neighbor correspondences. For each pair of matching neighbors, the positional similarity and neighbor embedding similarity are jointly calculated, and a weighted neighborhood topological score is obtained. If there are not enough neighbors, calculate based on the actual number of neighbors; if there are no neighbors, then... The output of this signal is a score showing the consistency between the target icon and the candidate application in terms of spatial layout.

10. A desktop icon closed-set authentication method for embedded NPUs according to claim 1, characterized in that, In the second-level fusion stage of S5, when a certain auxiliary signal is missing, its weight is transferred to the visual embedding signal to maintain weight normalization. If the first-level verification is TIER-2, then according to... Whether the value exceeds 0.75 or 0.55, output "high", "low", or "unknown" respectively; if the first-level verification is TIER-3, then calculate the auxiliary signal for the entire registry library, and... and Output the restored matching result.

Images