Neural Network Importance Coefficients for Text Task Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for obtaining task embeddings in text processing tasks require high computational costs and storage overhead, and the accuracy of similarity evaluation between tasks needs improvement.
Innovation Solution
A neural network with multiple network modules and importance coefficients is trained using text samples from text processing tasks, allowing for the determination of embedding features with lower computational and storage costs, and enabling more accurate task similarity assessment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current methods are used to obtain task embeddings, then task similarity can be evaluated, but computational costs and storage overhead are high
Solution Approach 1:
The patent segments the task embedding process into two parts: (1) training a shared neural network model on multiple text processing tasks to learn task-specific importance coefficients, and (2) using these coefficients to efficiently compute embeddings for new tasks. This segmentation allows the expensive training phase to be performed once shared across tasks, while inference becomes computationally efficient.
Solution Approach 2:
The patent creates a universal neural network model that can handle multiple text processing tasks simultaneously. The model learns task-specific importance coefficients that are reusable across different tasks, making the computational resources invested in training applicable to multiple tasks rather than requiring separate training for each task.
2Measurement precision
If current methods are used to obtain task embeddings, then task similarity can be evaluated, but storage overhead is high
Solution Approach 1:
The patent extracts only the essential task-specific information into compact importance coefficients rather than storing complete task embeddings. These coefficients capture the critical task characteristics needed for similarity evaluation while occupying minimal storage space, separating the essential task identity from the full embedding representation.
3Use of energy by moving object
If a neural network with importance coefficients is trained for text processing tasks, then computational costs and storage overhead are reduced, but the complexity of the method increases
Solution Approach 1:
The patent introduces importance coefficients as additional learnable parameters in the neural network that control the contribution of different network modules to task embeddings. While this increases the number of parameters during training, it enables efficient embedding computation at inference time by simply applying these coefficients to pre-computed network outputs, reducing overall computational cost.
Data Source
AI summary
A method, apparatus, device, and medium for determining the similarity of text processing tasks is provided. The method includes: determining a first task, a second task, and a neural network, the neural network includes a plurality of network modules and a plurality of importance coefficients corresponding to the plurality of network modules, and the importance coefficients are used to scale output values of a corresponding network module; respectively performing a target operation using the first task and the second task as a target task to obtain an embedding feature of the first task and an embedding feature of the second task; and determining the task similarity between the first task and the second task based on the embedding features. The target operation includes: training using text samples and obtaining a plurality of trained importance coefficients; and determining an embedding feature of the target task based on trained importance coefficients.