Attributed Sequence Embedding via Dual Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for sequence embedding and distance metric learning fail to effectively account for the dependencies between attribute and sequence data in attributed sequences, leading to suboptimal feature representations in applications like fraud detection and health care informatics.
Innovation Solution
A machine learning system combining a feedforward neural network for attribute data and a recurrent neural network, specifically a Long Short-Term Memory (LSTM) network, to generate a fixed-length feature representation that encodes dependencies within and between attribute and sequence data, using various coupling arrangements and objective functions for training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional sequence embedding methods are used, then the system can process sequential data, but it fails to capture dependencies between attribute and sequence data, resulting in suboptimal feature representations
Solution Approach 1:
The patent combines attribute data processing and sequence data processing into a unified neural network model. The attribute network and sequence network are integrated through shared layers and joint training, enabling the system to simultaneously capture attribute dependencies and sequence dependencies while producing a unified feature representation that reflects both types of relationships.
Solution Approach 2:
The neural network model is designed to perform multiple functions: processing fixed-length attribute data, processing variable-length sequence data, and capturing their interdependencies. The shared layers and common output space enable the system to handle both data types uniformly while maintaining their distinct characteristics and relationships.
2Measurement precision
If the system models all three types of dependencies (item, attribute, and attribute-sequence), then feature representation quality improves, but system complexity increases
Solution Approach 1:
The patent segments the processing task into distinct components: an attribute network for processing attribute data, a sequence network for processing sequential data, and shared layers for capturing their interactions. This modular segmentation allows each component to specialize in specific dependency types while working together to model all three types of dependencies comprehensively.
Solution Approach 2:
The model employs a nested architecture where attribute embeddings and sequence embeddings are integrated through shared layers. The attribute network and sequence network are nested within a common framework that allows their representations to interact and combine, enabling the system to capture complex interdependencies without requiring a completely separate architecture for each dependency type.
Data Source
AI summary
Machine learning systems and methods for embedding attributed sequence data. The attributed sequence data includes an attribute data part having a fixed number of attribute data elements and a sequence data part having a variable number of sequence data elements. An attribute network module includes a feedforward neural network configured to convert the attribute data part to an encoded attribute vector having a first number of attribute features. A sequence network module includes a recurrent neural network configured to convert the sequence data part to an encoded sequence vector having a second number of sequence features. In use, the machine learning system learns and outputs a fixed-length feature representation of input attributed sequence data which encodes dependencies between different attribute data elements, dependencies between different sequence data elements, and dependencies between attribute data elements and sequence data elements within the attributed sequence data.