Content Management System Duplicate Topic Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Legacy linear documents contain duplicated topics when imported into a Component Content Management System (CCMS), requiring an efficient method to identify and exclude duplicate content to avoid redundant data.

Innovation Solution

A system that calculates signatures for topics using a hash function and compares them with existing CCMS topics, along with analyzing outgoing relationships, to determine similarity and identify duplicates, employing locality-sensitive hashing to efficiently filter out duplicate topics during importation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If legacy linear documents are imported into CCMS, then the documentation is made available in the new system, but duplicate topics are introduced causing data redundancy

Engineering Contradiction:

Improveimport efficiencyVSAvoiddata redundancy

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system performs preliminary actions by calculating signatures of all topics in the legacy document before importing them into CCMS. These signatures are compared against existing CCMS topic signatures to identify duplicates in advance, preventing redundant data from being added to the system while maintaining efficient import processing

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If exact matching is used to identify duplicate topics, then accuracy is improved, but computational cost increases significantly

Engineering Contradiction:

Improveduplicate detection accuracyVSAvoidcomputational cost

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system changes the parameter of topic comparison by transforming topics into fixed-length signatures using hash functions. This transformation converts the complex exact matching problem into a simpler signature comparison task, maintaining high duplicate detection accuracy while dramatically reducing computational cost and processing time

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If all topics from legacy documents are imported, then complete documentation is achieved, but storage space is wasted on duplicates

Engineering Contradiction:

Improvedocumentation completenessVSAvoidstorage efficiency

Core Design Contradiction:

Quantity of substanceVSLoss of substance

Solution Approach 1:

The system uses signature copying and comparison to identify duplicate topics. By comparing the signature of each legacy topic against existing CCMS topic signatures, the system determines whether a topic is a duplicate without copying the actual content, thus maintaining documentation completeness while avoiding storage waste on redundant topics

Inventive Principle:

Principle #26Copying

Data Source

PatentUS10095781B2Reuse of documentation components when migrating into a content management system

Publication Date: 2018.10.09 RED HAT INC

AI summary

A method relates to receiving, by a processing device, a document comprising a first topic to be imported into a content management system, calculating a first signature of the first topic in view of content associated with the first topic, determining whether the first topic of the document is substantially similar to at least one of a plurality of topics stored in the content management system by comparing the first signature with a respective signature of the plurality of topics stored in the content management system, and in response to a determination that the first topic of the document is not substantially similar to any of the plurality of topics stored in the content management system, adding the first topic and the content associated with the first topic to the content management system.