Hierarchical Sequence Labeling with Context Attention

The patent implements a hierarchical sequence labelling architecture where multiple labelling layers are nested within each other. Each layer processes sequences and passes results to deeper layers, creating a nested structure that resembles nested dolls. This allows the model to capture structured information at multiple levels of abstraction, with deeper layers accessing context from shallower layers, thereby resolving the contradiction between model depth and information retention.

The patent introduces an additional dimensional aspect to the sequence labelling model by incorporating hierarchical layers that operate at different levels of abstraction. Instead of a single flat processing layer, the model adds a vertical dimension with multiple labelling layers (L0, L1, L2, etc.), each capable of capturing different levels of structural information. This dimensional expansion enables the model to retain structured information while maintaining manageable complexity at each individual layer.

The patent segments the sequence labelling task into multiple independent but interconnected layers. Each labelling layer (L0, L1, L2, etc.) performs a specific labelling function and can be trained and processed separately. This segmentation allows the complex hierarchical task to be broken down into manageable components, reducing the overall computational complexity while maintaining high precision through the coordinated output of all layers.

The hierarchical architecture performs preliminary labelling actions at shallower layers before passing results to deeper layers. Each layer performs its labelling function in advance, creating intermediate results that are then used by subsequent layers. This preliminary action approach allows deeper layers to benefit from pre-processed information without having to recompute everything, thereby reducing computational complexity while maintaining extraction precision.

The patent extracts and isolates the contextual attention mechanism as a separate, specialized component within the hierarchical architecture. Rather than embedding complex attention computations throughout all layers, the attention mechanism is extracted and applied selectively at specific points where contextual information from multi-turn conversations needs to be integrated. This extraction approach maintains label inference accuracy while reducing overall computational resource requirements.

The contextual attention mechanism serves as an intermediary between the input sequences and the labelling layers. It mediates the flow of information by selectively attending to relevant contextual elements from multi-turn conversations and passing this refined information to the hierarchical labelling structure. This intermediary role allows the system to achieve high label inference accuracy without requiring all layers to process the full complexity of the original input, thereby reducing computational and memory requirements.