Crosslingual Topic Inference via Segment-Level Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for inferring crosslingual topics in non-parallel corpora at the document level lack precision due to the neglect of segment-level alignments, which are smaller than documents.
Innovation Solution
A topic inferring apparatus that considers segment-level alignments by storing multilingual document sets, managing segment-level alignments, and using probabilistic methods to generate and determine topic distributions across languages, including Dirichlet and Pitman-Yor processes, to improve the accuracy of crosslingual topic inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If document-level alignment is used for crosslingual topic inference, then the method is simple to implement, but the precision of topic inference is low
Solution Approach 1:
The patent segments documents into smaller units (sentences or phrases) and performs alignment at the segment level rather than document level. This segmentation allows for more precise matching of corresponding parts between source and target languages, thereby improving topic inference precision while maintaining implementation simplicity through modular processing.
2Measurement precision
If segment-level alignments are considered, then the precision of topic inference is improved, but the complexity of the system increases
Solution Approach 1:
By segmenting documents into smaller units and processing alignments at the segment level, the system achieves higher precision without proportionally increasing complexity. The segmented approach allows for more targeted and efficient computation compared to processing entire documents.
Solution Approach 2:
The patent performs preliminary segmentation and alignment of documents into segments before topic inference. This preliminary action organizes the data structure in advance, making subsequent topic inference more efficient and reducing the computational complexity during the main inference process.
3Measurement precision
If pre-defined alignments are required, then the topic assignment accuracy is high, but the preprocessing effort and time increase
Solution Approach 1:
The system performs automatic segment-level alignment without requiring pre-defined alignments. The alignment is self-generated through computational methods that match segments between languages based on their content and structure, eliminating the need for manual preprocessing while maintaining high topic assignment accuracy.
Solution Approach 2:
The patent automatically performs the alignment action as a preliminary step before topic inference, but does so through automated algorithms rather than manual processes. This preliminary alignment action is integrated into the overall processing pipeline, reducing the perceived preprocessing time and effort.
Data Source
AI summary
In order to solve a problem that the level of precision in inferring a crosslingual topic of documents or words in a non-parallel corpus associated in the document level is not high, a topic inferring apparatus includes: a word distribution information storage unit in which word distribution information is stored in association with each of two or more languages; a document topic distribution generating unit that acquires document topic distribution information of a multilingual document set; a segment topic distribution generating unit that acquires segment topic distribution information of each segment, using the document topic distribution information; and a word topic determining unit that determines, for each word contained in two or more documents contained in the multilingual document set, a topic of each word using the segment topic distribution information. Accordingly, it is possible to improve the level of precision in inferring a topic.