Unlock instant, AI-driven research and patent intelligence for your innovation.

Internet data clearing method based on single public opinion event

A data clearing and Internet technology, applied in the field of data processing, can solve the problem of large noise in the results of public opinion processing

Pending Publication Date: 2021-11-16
西安康奈网络科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] For the processing of network public opinion data, the existing methods of clearing public opinion data are relatively traditional, pursuing simplicity and consistency. Considering the huge amount of data generated in the current Internet environment, there will still be a large amount of useless data after cleaning, which makes the data noise in the public opinion processing results is too big

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Internet data clearing method based on single public opinion event

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0062] The steps of segmenting the single-target public opinion data include: constructing a prefix dictionary based on the statistical dictionary, realizing efficient word map scanning, segmenting the sentences in the single-target public opinion data according to the prefix dictionary, and outputting all the segmentation results, according to The segmentation results generate a directed acyclic graph (DAG) composed of Chinese characters in the single target public opinion data, and use dynamic programming to find the maximum probability path in the directed acyclic graph, find out the maximum segmentation combination based on word frequency, and output Segment sentences into words.

[0063] The step of segmenting the sentences in the single-target public opinion data also includes:

[0064] If there are words not included in the prefix dictionary in the single-target public opinion data, the Viterbi algorithm is used for sentence segmentation based on the HMM model of the wo...

Embodiment 2

[0067] Public opinion incident 2: A public opinion incident of bullying students occurred in a school in a certain area

[0068] Step 1: Publicly collect relevant public opinion big data;

[0069] Step 2: Format public opinion data to generate formatted public opinion data with complete standards such as title, author, release time, content, source, etc.;

[0070] Step 3: Create a public opinion rule system, such as character subject: teacher, student; region: region name; event subject: school name; event: bullying, violent teaching, etc.; set the public opinion weight level as event subject, region, event, and character subject ;

[0071] Step 4: Use the segmentation algorithm to segment and combine public opinion data. Take the title of public opinion data as an example. Data 1: Campus bullying occurred in a certain school in a certain city. Use the segmentation algorithm to output sentences: certain city / some / school / occurrence / campus / bullying; Data 2: Campus bullying in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an internet data clearing method based on a single public opinion event, and relates to the technical field of data processing. Public opinion rule words including public opinion event statements, public opinion character statements, public opinion time statements, public opinion region statements and public opinion subject statements are set, the public opinion rule words are arranged according to weights, redundant fields or useless fields in the single target public opinion data are cleared according to the arrangement sequence, then the public opinion data are segmented, automatic matching and clearing of segmentation results are achieved in a graded mode, and invalid data are cleared. According to the Internet data clearing method based on the single public opinion event, the public opinion data is segmented, and the segmented public opinion data is automatically matched and cleared according to the weight.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method for clearing Internet data based on a single public opinion event. Background technique [0002] Public opinion is the abbreviation of "public opinion situation", which refers to the public opinion as the subject to social managers, enterprises, individuals and other organizations as the object around the occurrence, development and change of intermediary social events in a certain social space. Social attitudes generated and held by people and their political, social, and moral orientations. It is the sum of the beliefs, attitudes, opinions and emotions expressed by a large number of people about various phenomena and problems in society. [0003] Internet public opinion is the reflection of social public opinion in Internet space, and it is a direct reflection of social public opinion. Traditional social public opinion exists among the people, in the ideology...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/9535G06F16/215G06F40/216G06F40/289G06F40/242
CPCG06F16/9535G06F16/215G06F40/216G06F40/289G06F40/242
Inventor 罗箫
Owner 西安康奈网络科技有限公司