Device and Method for Training a Classification Model for Harmful Websites
By converting text data into fixed-dimensional vectors and employing clustering techniques, the method addresses the issue of outliers and duplicates in data sets, enhancing the performance and accuracy of harmful content detection models through improved data quality and domain-specific fine-tuning.
Patent Information
- Application Number
- KR1020240138319
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2044-10-11
AI Technical Summary
Existing data sets used for harmful content detection models often contain outliers and duplicates, which degrade model performance by causing overfitting and reducing generalization accuracy.
The method involves converting text data into fixed-dimensional numeric vectors using Sentence Embedding, applying dimensionality reduction techniques, and utilizing clustering algorithms to detect and remove outliers and duplicates, followed by fine-tuning the Large Language Model (LLM) for specific domain expertise.
This approach improves data quality, enhances model performance, and enables accurate classification of harmful content by effectively removing anomalies and duplicates, thereby improving the efficiency and accuracy of harmful content detection systems.
Smart Images

Figure 112024110454140-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a technology for detecting and removing outliers and duplicate data from incomplete datasets by utilizing a Large Language Model (LM). Specifically, it involves using a Sentence Embedding technique that converts text data into fixed-dimensional numeric vectors, utilizing the converted vectors as input data for the LLM model, and thereby effectively identifying and processing outliers and duplicates in the data to improve data quality and optimize the learning performance of the model. Background Technology
[0002] Technological trends for blocking harmful content are evolving through various approaches and algorithms. Each method utilizes unique technologies to effectively identify and block harmful content, and these technologies are becoming increasingly sophisticated.
[0003] Content filtering is the most basic blocking method, operating primarily through keyword filtering and pattern matching. This method scans text data based on a predefined list of harmful keywords to identify and block specific patterns. However, because it relies on fixed rules, it lacks flexibility and may have limitations in recognizing complex expressions.
[0004] Machine learning-based approaches emerged to overcome these limitations. These methods are used to identify harmful content by training on large-scale text, image, and video data. Deep learning technologies, particularly CNNs and RNNs, are commonly used and demonstrate powerful performance in image and text analysis. These models are highly adaptable to new harmful content and can progressively improve filtering accuracy through training.
[0005] Natural Language Processing (NLP) technology also plays an important role in detecting harmful content. NLP analyzes text data to evaluate harmfulness, and algorithms such as TF-IDF, Word2Vec, and BERT are frequently used. Through this, not only obvious harmful content but also subtle harmfulness can be effectively detected.
[0006] Blacklist management is a method that pre-defines harmful URLs, IP addresses, and other targets to block them, and is primarily used in network firewalls or DNS filtering systems. Real-time monitoring enables the immediate blocking of harmful content by monitoring user activity on websites or applications. Algorithms such as Support Vector Machines (SVM), Random Forests, and CNNs are utilized to detect and respond to harmful content in real time.
[0007] While these various technologies are being applied integrally to more effectively block harmful content in the changing online environment, data collected from URL databases containing harmful content may include a significant number of abnormal data (outliers).
[0008] If used for model training, this data carries the risk of degrading model performance. Examples include incorrectly labeled data or unexpected error pages.
[0009] Furthermore, the presence of a large amount of duplicate identical or similar data can hinder model training. Duplicate data risks degrading generalization performance by causing the model to over-adapt to specific patterns.
[0010] Furthermore, the dataset used for training is incomplete, making it difficult for the harmful content detection model to make reliable predictions. Incomplete datasets hinder proper clustering and classification, which can consequently reduce the accuracy of harmful content detection. The problem to be solved
[0011] The present invention aims to improve the quality of training data and thereby improve the performance of the model.
[0012] The present invention aims to effectively detect and remove outliers and duplicate data within a dataset using high-dimensional embedding vectors by vectorizing text data (Sentence Embedding) using LLM and then performing clustering based on it.
[0013] The present invention aims to enable accurate classification of harmful content by grouping contextually similar data and clustering semantically related data together.
[0014] The present invention aims to generate a model specialized for harmful content by fine-tuning an LLM model to suit a specific domain (harmful content detection). means of solving the problem
[0015] A method for training a harmful site classification model according to one embodiment may include the steps of: extracting text data of a webpage; preprocessing the extracted text data; training the preprocessed text data through a Large Language Model (LLM) classifier; converting the trained text into a fixed-dimensional numeric vector through sentence embedding and cleaning the harmful content; and feeding back the cleaned data as input to the preprocessing step.
[0016] According to one embodiment, the step of converting the learned text into a fixed-dimensional numeric vector through sentence embedding and cleaning outlier data or duplicate data may include: receiving an incomplete dataset containing harmful content; grouping the data by vectorizing it through clustering the input dataset to generate a Latent Vector, which is a fixed-dimensional numeric vector; reducing the dimensionality of the data by applying a dimension compression algorithm to the generated Latent Vector; detecting outlier data from the dimensionally reduced data; and refining the harmful content data by performing under-sampling to remove duplicate data based on the detected outlier data.
[0017] The step of reducing the dimensionality of the data by applying a dimensionality compression algorithm to the generated Latent Vector according to one embodiment may include the step of reducing the dimensionality of the data by applying at least one algorithm among the t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm, PCA (Principal Component Analysis) algorithm, and UMAP (Uniform Manifold Approximation and Projection) algorithm.
[0018] The step of detecting outlier data in the dimensionally reduced data according to one embodiment may include the step of detecting outlier data by applying at least one algorithm among the Elliptic Envelope algorithm, the Local Outlier Factor (LOF) algorithm, and the Isolation Forest algorithm.
[0019] The step of refining the harmful content data by performing under-sampling to remove duplicate data based on the detected outlier data according to one embodiment may include the step of removing duplicate data by applying at least one algorithm among the Edited Nearest Neighbors (ENN) algorithm, Repeated Edited Nearest Neighbors (RENN) algorithm, AllKNN algorithm, and Condensed Nearest Neighbor (CNN).
[0020] A harmful site classification model learning device according to one embodiment may include a text data extraction unit that extracts text data of a webpage, a preprocessing unit that preprocesses the extracted text data, an LLM classifier learning unit that learns the preprocessed text data through a Large Language Model (LM) classifier, and a data cleaning unit that converts the learned text into a fixed-dimensional numeric vector through sentence embedding, cleans the harmful content, and feeds the cleaned data as input to the preprocessing step.
[0021] According to one embodiment, the data cleaning unit receives the learned text, which is an incomplete dataset containing harmful content, and clusters the input dataset to generate a Latent Vector, which is a fixed-dimensional numerical vector, thereby vectorizing and grouping the data.
[0022] According to one embodiment, the data cleaning unit can refine the harmful content data by applying a dimension compression algorithm to the latent vector generated from the learned text to reduce the dimension of the data, detecting outlier data in the dimensionally reduced data, and performing under-sampling to remove duplicate data based on the detected outlier data. Effects of the invention
[0023] According to one embodiment, the quality of the training data can be improved, and thereby the performance of the model can be improved.
[0024] According to one embodiment, text data is vectorized (Sentence Embedding) using LLM, and then clustering is performed based on this, thereby effectively detecting and removing outliers and duplicate data within the dataset through high-dimensional embedding vectors.
[0025] According to one embodiment, by grouping contextually similar data and clustering semantically related data together, it is possible to enable accurate classification of harmful content.
[0026] According to one embodiment, by fine-tuning an LLM model to fit a specific domain (detection of harmful content), a model specialized for harmful content can be created. Brief explanation of the drawing
[0027] Figure 1 is a diagram illustrating a method for learning a harmful site classification model according to one embodiment. FIG. 2 is a diagram that explains in more detail a harmful content cleaning method among the harmful site classification model learning methods according to one embodiment. Figure 3 is a diagram illustrating an example of converting text into a numeric vector by performing Sentence Embedding. Figure 4 is a diagram illustrating the similarity between text embeddings. Figure 5 is a diagram illustrating the process of detecting abnormal data after clustering. Figure 6 is a diagram illustrating a sample of detected abnormal data. FIG. 7 is a diagram illustrating a harmful site classification model learning device according to one embodiment. Specific details for implementing the invention
[0028] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.
[0029] Embodiments according to the concept of the present invention may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.
[0030] Terms such as "first" or "second" may be used to describe various components, but said components should not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.
[0031] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.
[0032] The terms used herein are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.
[0035] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.
[0036] Figure 1 is a diagram illustrating a method for learning a harmful site classification model according to one embodiment.
[0037] A harmful site classification model training method according to one embodiment may utilize an LLM (large-scale language model) to effectively block harmful content.
[0038] Harmful content datasets collected from URL databases contain outliers and duplicate data, which can hinder accurate classification when used for model training.
[0039] To address this, the present invention converts text data into numeric vectors through Sentence Embedding and clusters the dataset using LLM. By removing outliers and duplicate data from the clustered dataset, the quality of the data is improved and the training performance of the model is enhanced. In this process, more sophisticated detection of harmful content is possible by utilizing the contextual understanding capabilities of LLM, and the accuracy of the model can be further improved through Fine-Tuning. This significantly enhances the efficiency and accuracy of the harmful content blocking system.
[0040] More specifically, a method for training a harmful site classification model according to one embodiment collects URLs of web pages that may contain harmful content and can extract text data of the web pages through this (101). In the present invention, a web page is accessed through the collected URLs, and text data is extracted from the corresponding page. This text data is core data that may contain harmful content and is subsequently used for analysis and processing.
[0041] For example, URLs of web pages that may contain harmful content can be collected first. These URLs can be gathered using a web crawler or a specific list of URLs. The collected URLs are used to access the corresponding web page, and the HTML source of the web page is retrieved via an HTTP request.
[0042] To do this, you can use Python's requests library or web crawling tools such as Scrapy. Use an HTML parsing tool like BeautifulSoup to extract text data from the retrieved HTML source.
[0043] In this invention, to extract all text content within a webpage tag, <h1> at< / h1> <h6>tag, tag, Content such as tags can be selectively extracted. During the parsing process, unnecessary text or metadata, such as advertisements, navigation links, scripts, and style tags, are removed to leave only pure content text.
[0044] At this point, specific tags can be removed using BeautifulSoup's .decompose() method. The extracted text data undergoes additional refinement processes, such as converting HTML entities into plain text or cleaning up whitespace and newline characters. The refined text data is finally stored in a local file, database, or other data processing pipeline, and is subsequently used as training data for a harmful content detection model.
[0045] Next, the harmful site classification model training method according to one embodiment can preprocess the extracted text data (102). In this step, the collected text data is refined and processed into a required format. For example, unnecessary characters are removed, or the text is standardized to facilitate analysis.
[0046] The process of preprocessing text data consists of several steps, and each step can improve data quality and enhance the performance of harmful content detection models.
[0047] To this end, unnecessary characters can be removed from text data during the preprocessing stage.
[0048] This includes special characters, unnecessary spaces, numbers, email addresses, URLs, etc. For example, you can use regular expressions to remove email addresses and URLs. For special characters, use regular expressions to remove all characters except alphabets and spaces.
[0049] Next, if the cases are mixed, convert them to lowercase. This allows 'Apple' and 'apple' to be treated as the same word. In most programming languages, this can be done through a simple method.
[0050] Next, stop words can be removed. Stop words are words that do not carry significant meaning in text analysis, including 'and', 'the', and 'is'. Removing stop words allows you to focus on important information. You can apply predefined stop word lists using various Python libraries.
[0051] Stemming and lemmatization are processes that transform words into their base forms. Stemming extracts the root of a word and reduces the modified word to the same root. For example, it transforms 'running', 'ran', and 'runs' into 'run'. Lutematization transforms words into lemmats by considering their grammatical forms. This task can be performed using Python libraries.
[0052] Sentence tokenizing is the process of dividing text into sentence units, allowing long text to be processed by breaking it down into individual sentences.
[0053] Word tokenization is a process of dividing a sentence into individual words, which allows text data to be converted into a list of words for analysis.
[0054] For example, specific words can be removed through a defined list. This list may include words related to specific topics or technical terms that are not suitable for the purpose of analysis.
[0055] Meanwhile, text normalization is the process of converting text data into a consistent format, involving tasks such as unifying variations of the same word or organizing sentence structures. For example, it converts 'can't' to 'cannot' or 'I've' to 'I have'.
[0056] Preprocessed text data is stored for subsequent analysis and model training. The data can be stored in local files, databases, or other data processing systems. Typically, text data is stored in CSV, JSON, or SQL databases.
[0057] This preprocessing process removes noise from text data and enables harmful content detection models to learn effectively.
[0058] Next, the harmful site classification model training method according to one embodiment can train preprocessed text data through an LLM (large-scale language model) classifier (103).
[0059] In this process, data is trained using LLM to create a model capable of classifying harmful content. Since the model can understand the meaning and context of the text, more accurate classification is possible.
[0060] For example, preprocessed text data can be prepared in a format suitable for training an LLM model. The dataset consists of text and labels indicating whether the content is harmful, and the dataset is divided into training, validation, and testing sets, and generally 80% can be used for training, 10% for validation, and the remaining 10% for testing.
[0061] Since LLM models cannot process text data directly, the text must be converted into tokens. This process involves converting text into fixed-size numeric vectors. Tokenizers are used to convert text into tokens; for example, BERT uses the WordPiece tokenizer to convert "This is an example" into ["this", "is", "an", "example"].
[0062] The length of token lists can vary. Since LLM models require fixed-size inputs, short lists are padded to match their length, while long lists are truncated to the required length. This ensures that all inputs have the same length, providing the model with data in a consistent format.
[0063] The present invention uses a pre-trained model such as BERT or GPT, and can be fine-tuned for specific tasks such as harmful content detection by adding a classification layer on top of this model. For example, an output layer using a softmax activation function can be added on top of BERT to classify two classes (harmful / normal).
[0064] In the present invention, an LLM model can be trained using a preprocessed dataset. The model analyzes a given text and adjusts weights to classify whether each text is harmful or not. A loss function is used to minimize the difference between the model's prediction and the actual label, and the cross-entropy loss function and the AdamW optimizer are used. Training is repeated for several epochs to improve the model's performance.
[0065] The model's performance is evaluated using a validation dataset during training. The validation process checks how well the model predicts on new data, thereby preventing overfitting during training. Model performance can be evaluated using metrics such as accuracy, precision, recall, and F1-score.
[0066] The trained model evaluates its final performance using a test dataset. The test dataset consists of data not used during the training process and is used to verify the model's generalization performance. In this process, the model's accuracy and effectiveness in classifying harmful content can be measured.
[0067] Models whose performance has been verified through testing are deployed to actual harmful content detection systems. The models analyze text data in real-time and automatically classify, block, or generate warnings for harmful content. They can be used in various environments by calling them via an API server or deploying them to cloud services.
[0068] Next, the harmful site classification model training method according to one embodiment can clean harmful content (104). Based on the data trained through the LLM classifier, the task of detecting and removing harmful content is performed. In this step, abnormal data or duplicate data is effectively removed to improve the quality of the dataset and enable more accurate content blocking.
[0069] Through this process, a system can be established to systematically detect and remove harmful content.
[0070] The cleaning process for harmful content can be explained in more detail through Fig. 2.
[0071] FIG. 2 is a diagram that explains in more detail a harmful content cleaning method among the harmful site classification model learning methods according to one embodiment.
[0072] An incomplete dataset containing harmful content is input (201).
[0073] We use a dataset composed of various data such as text, images, and videos. This dataset may contain harmful content, such as violent images, pornography, and hate speech. For example, assuming we input text data consisting of 100,000 posts collected from social media, there is a possibility that some posts may contain hate speech.
[0074] A Latent Vector, which is a fixed-dimensional numeric vector, can be generated by clustering the input dataset (202).
[0075] In step 202, the data is characterized and converted into a vector form. During this process, the dataset is grouped according to similar characteristics, and the Latent Vector is the result of compressing the data and converting it into a low-dimensional vector space. For text data, sentences or words are converted into numbers using embedding techniques (e.g., Word2Vec, BERT), and image data can be converted into vectors using a Convolutional Neural Network (CNN).
[0076] The dimensionality of the data can be reduced by applying a dimension compression algorithm to the generated Latent Vector (203).
[0077] Dimensionality reduction techniques (e.g., PCA, t-SNE) are primarily used. Dimensionality reduction improves computational efficiency and allows only important information to be retained. For example, a 300-dimensional latent vector can be reduced to 50 dimensions to allow similar data to be separated more clearly, while removing unnecessary information while preserving data patterns.
[0078] More specifically, to reduce the dimensionality of the data by applying a dimensionality compression algorithm to the generated Latent Vector, at least one of the following algorithms can be applied: t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm, PCA (Principal Component Analysis) algorithm, and UMAP (Uniform Manifold Approximation and Projection) algorithm.
[0079] Outlier data is detected in the dimensionality-reduced data (204).
[0080] In step 204, data that deviates from the general pattern is detected as outliers. This is likely to be harmful content exhibiting abnormal patterns. Outlier detection techniques such as Isolation Forest and LOF (Local Outlier Factor) can be used. For example, if text containing hate speech is distributed differently from other normal text, it can be detected as an outlier and classified as a candidate for harmful content.
[0081] For example, to detect outlier data in dimensionality-reduced data, at least one of the following algorithms can be applied: the Elliptic Envelope algorithm, the Local Outlier Factor (LOF) algorithm, and the Isolation Forest algorithm.
[0082] Finally, harmful content data can be refined based on the detected outlier data (205).
[0083] In step 205, data detected as outliers is analyzed to identify and remove harmful content. At this stage, filtered content is removed or modified from the dataset. For example, text containing hate speech is filtered out or violent images are deleted to form a refined dataset.
[0084] For example, duplicate data can be removed by applying at least one of the following algorithms: Edited Nearest Neighbors (ENN), Repeated Edited Nearest Neighbors (RENN), AllKNN, and Condensed Nearest Neighbor (CNN).
[0085] FIG. 3 is a drawing (300) illustrating an example of converting text into a numeric vector by performing Sentence Embedding.
[0086] Large-scale Language Models (LMs) cannot directly handle text-based data when processing input, so a process of converting text into numeric vectors is required. Sentence Embedding plays a crucial role in this conversion process. Sentence Embedding transforms text into fixed-dimensional numeric vectors, helping the model understand the meaning and structure of the text. Through this, LLM models can effectively process and learn from input text data.
[0087] The first step in generating sentence embeddings is using a tokenizer. A tokenizer is a tool that separates input text into words or sub-units. The process of dividing text into individual words or sub-words is called tokenization, and during this process, each word is converted into a unique numerical index. For example, the sentence "Hello world" is separated into two tokens, "Hello" and "world," and each token is mapped to a number according to a dictionary defined by the model. In this way, the tokenizer plays the role of separating text into individual tokens and converting them into numbers.
[0088] Tokenized data is subsequently converted into fixed-size vectors through Sentence Embedding. Sentence Embedding is a vector representation that captures the meaning of an entire sentence, implying the complex meanings of individual words or sentences. Various techniques are used to generate Sentence Embeddings, with representative examples including BERT, GPT, and Word2Vec. These techniques convert the meaning of sentences or text into vectors by reflecting word order and context. The converted vectors have a fixed size and become a format suitable for LLM models to receive as input.
[0089] The sentence embeddings generated in this way are ultimately used as input data for the LLM model. In other words, the LLM model accepts data converted from sentences or text into vectors and performs natural language processing tasks based on those vectors. Thanks to this conversion process, the LLM model acquires the ability to understand and analyze text data.
[0090] To summarize this process, since LLM models cannot directly process text data, the input text is first tokenized through a Tokenizer and converted into Sentence Embeddings, and the resulting vector can be used as input data for the LLM.
[0091] Figure 4 is a diagram (400) illustrating the similarity between text embeddings.
[0092] Tasks based on Large-Scale Language Models (LLMs) proceed in the direction of understanding the context of text. This means that texts with similar meanings are represented by similar vectors. Since vectors generated through embeddings possess high-dimensional structures, detailed information can be grasped more accurately. Because these embedding vectors reflect not merely the surface appearance of the text but also the meaning and context contained within it, they can process complex data containing diverse information.
[0093] By utilizing embedding values, grouping that reflects the semantic similarity of the data can be performed. This allows the model to more accurately identify relationships between texts, resulting in improved prediction accuracy.
[0094] It is difficult to fully understand text information regarding content with specific domains or special purposes using only foundational LLM models. For example, basic LLM may have limitations when processing certain types of data, such as harmful content. To compensate for this, fine-tuning is necessary. Fine-tuning is the process of further training a model for a specific purpose; by training the model with data such as harmful content, it enables the model to understand text in that field more clearly.
[0095] Through fine-tuning, the model gains a better understanding of the characteristics of specific content, enabling more sophisticated clustering. In the task of detecting or classifying harmful content, a fine-tuned model can better distinguish harmful text, thereby delivering practical performance.
[0096] Figure 5 is a diagram (500) illustrating the process of detecting abnormal data after clustering.
[0097] While LLM models are trained on large-scale datasets, additional fine-tuning is required to accurately classify data in specific domains, such as harmful content. Fine-tuning is the process of retraining a model with data from a specific field; by training the model with text data related to harmful content, it can be adjusted to be more sensitive to such content. During this process, text or image data is embedding and converted into a vector space, and the resulting vectors are visualized using clustering algorithms.
[0098] As shown in Fig. 5, high-dimensional data is represented in a 2D or 3D space through dimensionality reduction and visualization techniques such as t-SNE. The graphs shown in Fig. 5 illustrate different clustering stages. Embedding values obtained through a fine-tuned LLM model place data with similar characteristics in close proximity, with harmful content grouped into one cluster and non-harmful content into another. The colors of the graphs represent different clusters, allowing for the visual confirmation of relationships between the data. This process is also useful for classification tasks.
[0099] In the clustering process, outliers refer to data that is far from the center of a cluster or is not clearly distinguishable from other clusters. Comparing the intermediate clustering results with the final clustering results in the image, it can be seen that the distribution of each cluster has become clearer. However, there are still points located outside the cluster boundaries, and these data can be considered outliers. Outlier detection is performed using techniques such as Isolation Forest or LOF (Local Outlier Factor). These techniques identify outliers by detecting data points distributed differently from general patterns in a vector space.
[0100] Once outliers are identified through clustering, data cleaning can be performed based on this. This can be done to improve dataset quality by removing anomalous data, or by using under-sampling techniques to balance clusters by reducing data when there is an excessive amount of data in a specific cluster. Through this process, model performance can be improved, and incorrect classifications or predictions can be prevented.
[0101] Figure 6 is a drawing (600) illustrating a detected abnormal data sample.
[0102] According to the detected anomalous data samples, data with labeling errors, blocked pages, and error pages may be identified as outliers. These data exhibit forms different from typical web pages or content and primarily consist of unintended warning messages, connection failure notices, or data with incorrect labeling.
[0103] For example, the blocked page shown in the image displays a warning message indicating that a specific website is unsafe when a user attempts to access it. This may be due to security issues or harmful content, and access to such sites is blocked or restricted via warning messages. Additionally, error pages appear when normal access is impossible due to server errors or incorrect paths. Because these pages differ from standard web pages, they can be easily distinguished within the dataset.
[0104] Furthermore, data with labeling errors refers to cases where the actual content does not match the label assigned to it. For example, this includes instances where non-harmful content is incorrectly labeled as harmful, or conversely, where harmful content is incorrectly marked as safe. Since such errors can lead to inaccurate results during the model's training and prediction processes, they must be detected as outliers and removed or corrected.
[0105] FIG. 7 is a diagram illustrating a harmful site classification model learning device according to one embodiment.
[0106] The harmful site classification model training device performs the role of classifying and refining harmful content through the interaction of its various components.
[0107] First, the text data extraction unit (710) extracts text data from a webpage. This collects text information from a website containing harmful content and prepares data to be used for learning. For example, text such as site descriptions, titles, and user feedback can be extracted from a webpage containing gambling sites or malware to provide important information necessary to determine whether a site is harmful.
[0108] The preprocessing unit (720) performs the function of refining the extracted text data and converting it into a form suitable for analysis. This includes noise removal, tokenization, removal of duplicates, and processing of unnecessary punctuation. For example, HTML tags or unnecessary symbols are removed from text data extracted from a webpage, and the text is normalized to increase data consistency. The preprocessed data is converted into a form suitable for the LLM model to learn.
[0109] The LLM classifier training unit (730) is a step in which preprocessed text data is trained through a large-scale language model called LLM. The LLM has the ability to understand and classify various text data. The trained LLM model analyzes the text of a webpage to classify whether the site is harmful or not. For example, it can be used to classify a site as harmful by identifying text patterns such as gambling, adult content, and malicious software.
[0110] The data cleaning unit (740) can embed the learned text data to convert it into a fixed-dimensional numeric vector and then clean the harmful content. In this process, a clustering technique is used to group the dataset, and the quality of the data is improved through dimensionality reduction and outlier detection. The preprocessed text is converted into a fixed-dimensional numeric vector through embedding, which generates a representation in a vector space that reflects the semantic relationships of the text, allowing the model to identify the similarity of the data. The embedded vectors are grouped together by similar data through a clustering algorithm, thereby allowing harmful content and non-harmful content to be separated into separate groups.
[0111] To efficiently handle high-dimensional data, dimensionality reduction algorithms (such as PCA and t-SNE) are applied to reduce the data's dimensionality. Subsequently, noise is removed by detecting outlier data located outside cluster boundaries. During this process, detected outlier data is removed, while duplicate data is balanced and eliminated using under-sampling techniques. This enables more accurate and clearer cleaning of datasets containing harmful content.
[0112] Website data obtained from the trained LLM model is transformed into high-dimensional vectors through a sentence embedding process. Subsequently, the dimensionality of the data is reduced using dimensionality reduction techniques, and similar data is grouped using clustering algorithms. Normal website data and data containing harmful content are separated into distinct clusters, and anomalous data located outside the cluster boundaries is detected and removed, resulting in improved dataset quality.
[0113] The control unit (750) according to one embodiment can be interpreted as a Central Processing Unit (CPU) and can perform various operations and process data within the system.
[0114] In particular, the control unit (750) can read instructions from memory and interpret and execute the instructions, and can also perform arithmetic operations such as addition, subtraction, multiplication, and division.
[0115] Additionally, the control unit (750) can handle data storage and retrieval, and can also perform the function of reading data from memory and storing the result of performing operations back into memory.
[0116] Additionally, the control unit (750) can manage the execution flow of the program, and in particular, can control the flow of the program using commands such as conditional statements (if-else) or loop statements (for, while). Additionally, the control unit (750) may have a small, fast memory device called a register placed inside it, and this register may be used to temporarily store data or perform operations.
[0117] The control unit (750) can handle external events or exception situations and take appropriate measures, and can quickly access data and instructions by using a cache memory that is faster than the main memory.
[0118] In addition, the control unit (750) can use a system bus to communicate with memory or input / output devices and can provide various power management functions to minimize power consumption.
[0119] According to one embodiment, the quality of training data can be improved, thereby enhancing the performance of the model. Furthermore, by utilizing LLM to vectorize text data (Sentence Embedding) and performing clustering based on this, outliers and duplicate data within the dataset can be effectively detected and removed through high-dimensional embedding vectors. In addition, by grouping contextually similar data and clustering data with high semantic relevance, accurate classification of harmful content can be enabled. Ultimately, by fine-tuning the LLM model to suit a specific domain (harmful content detection), a model specialized for harmful content can be created.
[0121] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.
[0122] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.
[0123] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.
[0124] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.
[0125] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. < / h6>
Claims
Claim 1 A method for training a harmful site classification model comprising: a step of extracting text data from a webpage; a step of preprocessing the extracted text data; a step of training the preprocessed text data through a Large Language Model (LLM) classifier; a step of converting the trained text into a fixed-dimensional numeric vector through sentence embedding and cleaning outlier data or duplicate data, wherein the method comprises: a step of receiving an incomplete dataset containing harmful content; a step of vectorizing and grouping the data by clustering the input dataset to generate a Latent Vector, which is a fixed-dimensional numeric vector; a step of reducing the dimensionality of the data by applying a dimensionality compression algorithm to the generated Latent Vector; a step of detecting outlier data in the dimensionality-reduced data; a cleaning step comprising a step of refining the harmful content data by performing under-sampling to remove duplicate data based on the detected outlier data; and a step of feeding back the cleaned data as input to the preprocessing step. Claim 2 delete Claim 3 A method for training a harmful site classification model according to claim 1, wherein the step of reducing the dimensionality of data by applying a dimensionality compression algorithm to the generated Latent Vector comprises the step of reducing the dimensionality of data by applying at least one algorithm among the t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm, the PCA (Principal Component Analysis) algorithm, and the UMAP (Uniform Manifold Approximation and Projection) algorithm. Claim 4 A method for training a harmful site classification model according to claim 1, wherein the step of detecting outlier data in the dimensionally reduced data comprises the step of detecting outlier data by applying at least one algorithm among the Elliptic Envelope algorithm, the Local Outlier Factor (LOF) algorithm, and the Isolation Forest algorithm. Claim 5 A method for training a harmful site classification model according to claim 1, wherein the step of refining the harmful content data by performing under-sampling to remove duplicate data based on the detected outlier data comprises the step of removing duplicate data by applying at least one algorithm among the Edited Nearest Neighbours (ENN) algorithm, the Repeated Edited Nearest Neighbours (RENN) algorithm, the AllKNN algorithm, and the Condensed Nearest Neighbour (CNN). Claim 6 A harmful site classification model learning device comprising: a text data extraction unit for extracting text data of a webpage; a preprocessing unit for preprocessing the extracted text data; an LLM classifier learning unit for learning the preprocessed text data through a Large Language Model (LM) classifier; and a data cleaning unit for converting the learned text into a fixed-dimensional numeric vector through sentence embedding, cleaning harmful content, and feeding back the cleaned data as input to the preprocessing step, wherein the data cleaning unit receives the learned text, which is an incomplete dataset containing harmful content, and vectorizes and groups the data by clustering the input dataset to generate a Latent Vector, which is a fixed-dimensional numeric vector, and applies a dimension compression algorithm to the Latent Vector generated from the learned text to reduce the dimension of the data, detects outlier data in the dimensionally reduced data, and performs under-sampling to remove duplicate data based on the detected outlier data to refine the harmful content data. Claim 7 delete Claim 8 delete
Citation Information
Patent Citations
System and method for automatically verifying security events based on text mining
KR1020190070702A
Classification and control of text
KR1020210083299A
Data classification device and method using integrated data analysis learning
KR1020220097814A
System and method for learning language understanding algorithm
KR1020230030821A
Filtering system and method for classifying korean harmful text
KR1020230032554A
