Cross-Modal Retrieval Using Word-Overlap Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analytics software for time series data lacks user-friendly explanations and requires complex queries, making it difficult for human users to interpret machine learning results and search historical data effectively.
Innovation Solution
A computer processing system that employs a neural network with joint time series and text encoders, trained using an unsupervised method with a loss function and word-overlap-based spectral clustering, to retrieve similar data pairs and provide explanations in natural language, enabling users to search historical time series with descriptive queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning systems are used to perform classification tasks on time series data, then analysis accuracy is improved, but the results are returned without explanations making them difficult for users to interpret
Solution Approach 1:
The patent introduces text descriptions as an intermediary between the machine learning classification system and the user. These text descriptions explain the classification results in natural language, making the output interpretable while preserving the accuracy benefits of machine learning. The system generates explanations that bridge the gap between complex algorithmic results and human understanding.
2Productivity
If traditional example segments are used as search queries for historical time series data, then search functionality is provided, but the queries are not descriptive enough for effective searching
Solution Approach 1:
The patent transforms the search query parameter from simple example segments to rich text descriptions. By changing the nature of the query input from minimal examples to descriptive natural language, the system enables more effective and intuitive searching of historical time series data while maintaining retrieval functionality.
Solution Approach 2:
The system uses text descriptions as an intermediary between user intent and historical data retrieval. Instead of directly comparing example segments, the patent translates descriptive queries into meaningful search criteria, acting as a mediator that enhances search effectiveness while keeping the interface user-friendly.
3Measurement precision
If database query languages such as SQL are used to express complex search criteria, then precise querying capability is achieved, but the language is not comprehensible for average users
Solution Approach 1:
The patent creates a simplified copy or alternative interface to SQL-based querying. Instead of requiring users to learn complex query languages, the system provides natural language text descriptions that copy the functionality of precise SQL queries but in a form that average users can understand and use. This maintains query precision while dramatically improving accessibility.
Data Source
AI summary
A system for cross-modal data retrieval is provided that includes a neural network having a time series encoder and text encoder which are jointly trained using an unsupervised training method which is based on a loss function. The loss function jointly evaluates a similarity of feature vectors of training sets of two different modalities of time series and free-form text comments and a compatibility of the time series and the free-form text comments with a word-overlap-based spectral clustering method configured to compute pseudo labels for the unsupervised training method. The computer processing system further includes a database for storing the training sets with feature vectors extracted from encodings of the training sets. The encodings are obtained by encoding a training set of the time series using the time series encoder and encoding a training set of the free-form text comments using the text encoder.