Head architecture for deep neural network (DNN)

By introducing a new head architecture in DNN, this architecture handles local features through feature division, integration and aggregation, solving the shortcomings of the existing DNN head structure in dealing with local feature dependencies, and achieving more efficient and accurate feature extraction.

CN119731666APending Publication Date: 2025-03-28INTEL CORP
0 Cites 0 Cited by

Patent Information

Application Number
CN202280098542.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-08-26
Publication Date
2025-03-28

AI Technical Summary

Technical Problem

The head structure of existing DNNs is insufficient in dealing with local feature dependencies, resulting in limited model performance, especially in resource-constrained environments.

Method used

A new head architecture is adopted that model local to global feature dependencies through grouped feature division, integration, and aggregation. The specific method includes dividing the output feature map into multiple feature groups, and processing these feature groups in parallel in the head module, generating local tensors and forming a global vector through aggregation operation.

Benefits of technology

This header architecture improves DNN efficiency and accuracy, reduces computational complexity, power consumption and resource requirements, thereby performing better in various types of backbone networks.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A header of the DNN receives the OFM from a backbone network of the DNN. The head may divide the OFM into sets of features having the same size. The head may also generate a local tensor from the set of features. To generate a local tensor from the feature group, the header may also divide the feature group into two subgroups, for example based on a segmentation factor. The spatial size of the subgroup depends on the segmentation factor. A subset may be converted into an attention tensor. Another subset may be converted to a value tensor, which may have the same size as the attention tensor. The attention tensor and the value tensor are mixed to produce a local tensor. The local tensors of all feature groups may be aggregated to form a global vector, which may be fed into a classifier to output one or more classifications determined by the DNN.
Need to check novelty before this filing date? Find Prior Art