AI Search Query Tagging with CRF Cross-Checking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI models for query tagging are domain-specific and produce suboptimal, potentially hallucinated labeled data, which is labor-intensive and of questionable quality, hindering the training of reliable sequence tagging models.
Innovation Solution
A Conditional Random Field (CRF) model, referred to as Partial2full, is designed to convert partially labeled queries from AI models into fully labeled queries, leveraging additional AI models for cross-checking and generating high-quality training data, and integrating with LLMs and knowledge graphs to optimize search query tagging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If LLMs are used to generate labeled data, then the quantity of training data increases, but the quality and reliability of the labeled data deteriorates due to hallucinations
Solution Approach 1:
The system implements a feedback mechanism where the CRF model evaluates and corrects the labels generated by LLMs. The CRF model processes the sequence of tokens and assigns labels based on contextual relationships, providing feedback on the accuracy of LLM-generated labels and correcting hallucinations through iterative refinement.
Solution Approach 2:
The CRF model acts as an intermediary between the LLM and the final training data. It receives the partial labels from the LLM, processes them through its own labeling mechanism, and outputs refined labels that combine the generative capability of LLMs with the precision of the CRF model.
2Measurement precision
If domain-specific AI models are used for tagging, then the accuracy for specific domains improves, but the ability to handle complete query sequences deteriorates
Solution Approach 1:
The CRF model is designed as a universal tagging system that can handle multiple domains and complete query sequences simultaneously. It processes the entire sequence of tokens from the query, maintaining contextual awareness across different domains while providing accurate labels for each token, thus achieving both domain-specific accuracy and sequence-level completeness.
Solution Approach 2:
The system merges the strengths of domain-specific models by integrating multiple AI models (LLMs and CRF models) into a unified pipeline. The LLM provides domain-specific knowledge and initial labeling, while the CRF model integrates this information across the complete sequence, combining specialized accuracy with comprehensive sequence handling.
3Reliability
If manual tagging by domain experts is used, then the quality of labeled data improves, but the time and labor required increases significantly
Solution Approach 1:
The system enables self-service labeling where the CRF model automatically processes and labels query sequences without requiring manual intervention from domain experts. The model uses its trained parameters to autonomously assign labels based on the input sequence, eliminating the time-consuming manual tagging process while maintaining high labeling quality.
Solution Approach 2:
The patent replaces the mechanical process of manual tagging by domain experts with an automated computational system. The CRF model substitutes human judgment with algorithmic processing, using statistical patterns and contextual relationships to assign labels automatically, thereby eliminating labor-intensive operations while preserving labeling accuracy.
Data Source
AI summary
Disclosed are systems and methods that provide a decision-intelligence (DI)-based, computerized framework for compiling and leveraging reliable sequence taggings for input queries related to executed searches. The disclosed framework can compile a trained computer model to fulfill partially labeled queries tagged by AI models as fully labeled queries. The disclosed framework can further leverage other AI models (e.g., deep neural networks, knowledge graphs, and the like), so that cross-checks can be performed between different models to guarantee high quality of labeled tokens. Thus, the framework can automatically generate and implement reliable training data to train a sequence tagging model for search query understanding. Thus, the search engine operating on such tagging model can provide improved results.