Knowledge distillation vertical field detection method based on similarity maintenance

A detection method and a similarity technology, applied in the field of text-based position detection, can solve the problems that the recognition accuracy needs to be improved, the computing resources and computing time overhead are large, and achieve the effect of reducing overhead and improving performance.

Active Publication Date: 2021-11-19
NORTHEAST FORESTRY UNIVERSITY
View PDF12 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The present invention aims to solve the problem of large computing resources and computing time overhead in the existing position detection method, and the problem that the recognition accuracy of the existing method needs to be improved

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Knowledge distillation vertical field detection method based on similarity maintenance
  • Knowledge distillation vertical field detection method based on similarity maintenance
  • Knowledge distillation vertical field detection method based on similarity maintenance

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach 1

[0039] Specific implementation mode one: combine figure 1 To describe this embodiment,

[0040] This embodiment is a position detection method based on similarity-maintained knowledge distillation, which includes the following steps:

[0041] Obtain the position text to be detected, utilize Text-CNN network model to carry out position detection; The determination process of described Text-CNN network model comprises the following steps:

[0042] S1. Obtain instances of known positions and construct instance datasets:

[0043] Represents a data set with N instances, each instance x i contains a text s i , target t i and a stand label y i ;Each text consists of a sequence of words, s i ={w i0 ,w i1 ,...,w in}, each target also consists of a sequence of words, t i ={t i0 , t i1 ,...,t im}, n and m are respectively s i , t i The number of words in .

[0044] S2. Obtain the "Student" model based on knowledge distillation, that is, the Text-CNN network. The specifi...

Embodiment 1

[0089] Embodiment 1: Experiment of the present invention and baseline method:

[0090] (1) Dataset

[0091]Experiments are performed using two text stance detection datasets. One is the SemEval-2016 task 6 twitter position detection data set, and the other is the NLPCC-ICCPOL-2016 task 4 Chinese Weibo position detection data set. Each piece of data in the dataset is represented in the format of a triple ("stance", "target", "text"), where the "stance" label includes Favor, Against, and None.

[0092] For the English dataset, the training set contains 2914 English tweets with stance labels, and the test set contains 1249 tweets. There are 5 goals: "Atheism", "Climate Change is Concern(CC)", "Feminist Movement(FM)", "Hillary Clinton(HC)" and "Legalization of Abortion(LA)".

[0093] The Chinese dataset contains 4000 Chinese microblogs with stance labels, 3000 of which are used for training and 1000 for testing. There are also 5 targets: "IPhone SE", "Set off firecrackers in t...

Embodiment 2

[0118] Embodiment 2: parameter sensitivity experiment

[0119] To analyze the impact of hyperparameters, we conduct two parameter sensitivity experiments on the English Twitter stance detection dataset.

[0120] Furthermore, in the knowledge distillation method, a small dataset may not be enough for the "teacher" model to fully express its knowledge. In order to make up for the lack of data in the English Twitter corpus and obtain a larger data set, the EDA: Easy Data Augment method is used to expand the data set. Four methods of synonym replacement, random insertion, random swap and random deletion are used to process twitter text.

[0121] Synonym Replacement (SR): Refers to randomly selecting n words that are not stop words from a sentence, and replacing each of them with a randomly selected synonym.

[0122] Random Insertion (RI): Randomly select a word in a sentence that is not a stop word and insert a synonym of it anywhere in the sentence. Repeat n times.

[0123] R...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a knowledge distillation standing field detection method based on similarity maintenance, and belongs to the technical field of text emotion detection. The method aims to solve the problems that an existing vertical field detection method is large in computing resource and computing time expenditure and the recognition accuracy needs to be improved in the existing method. According to the method, the Text-CNN network model is utilized to carry out vertical field detection on a to-be-detected vertical field text; a Text-CNN network is determined based on a knowledge distillation mode, namely, a BERT model is used as a teacher model, and implicit knowledge learned from the teacher model is introduced into a student model text; and a loss function based on similarity maintenance is provided on the basis of a traditional knowledge distillation mode, so that the performance of the student model obtained on the basis of the knowledge distillation mode is further improved. The method is mainly used for text standing field detection.

Description

technical field [0001] The invention relates to a text-based position detection method, belonging to the technical field of text emotion detection. Background technique [0002] With the increasing popularity of mainstream social media, people can express their attitudes about almost everything at any time through online sites in the form of product reviews, blogs, tweets and Weibo. In recent years, automatic stance detection has attracted much attention due to its wide range of applications, especially in the fields of social media analysis, argument mining, truth discovery, and rumor detection. Stand detection is a fundamental research in textual opinion mining, which usually has two key inputs: (1) the target, and (2) the author's post or comment. Given two inputs, the goal of stance detection is to analyze the tendency of stance towards a specific goal expressed in the text, such as "for, against or neutral". Targets can be events, policies, social phenomena or product...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06F16/35G06N3/04G06N5/02
CPCG06F40/30G06F16/35G06N5/02G06N3/045
Inventor 李洋孙宇晴
Owner NORTHEAST FORESTRY UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap