Unlock instant, AI-driven research and patent intelligence for your innovation.

Smoothing method and system

A smoothing and smoothing technology, applied in electrical digital data processing, natural language data processing, instruments, etc., can solve problems such as poor results

Active Publication Date: 2022-05-03
GUANGZHOU SHIYUAN ELECTRONICS CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Based on this, it is necessary to provide a smoothing method and system for the poor effect of traditional smoothing methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Smoothing method and system
  • Smoothing method and system
  • Smoothing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the associated drawings.

[0030] see figure 1 Shown is a flow chart of a smoothing method according to an embodiment of the present invention. The smoothing method in this embodiment comprises the following steps:

[0031] Step S110: Count the first occurrence times of the missing words in the target corpus, wherein the missing words are words whose occurrence times in the original corpus are 0.

[0032] In this step, since the missing word is a word that appears 0 times in the original corpus, in the process of assigning the smooth probability, it is impossible to distinguish between the error of the word itself in the missing data and the insufficient coverage of the corpus itself. Therefore, it is necessary to introduce Two case parameters to assign smooth probabilities. Therefore, in the target corpus, the number of occurr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to the technical field of natural language processing, in particular to a method and system for smoothing processing. The method includes the following steps: counting the first number of occurrences of missing words in the target corpus, wherein the number of occurrences of missing words in the original corpus is 0 words; calculate the normalized frequency index of the missing words according to the first occurrence times; calculate the smooth probability of the missing words according to the normalized frequency index and the remaining probability, and smooth the missing words according to the smoothing probability, where the remaining probability is from The sum of the occurrence probabilities of words that appear less than or equal to k times in the original corpus, where k is a positive integer. The above method and system can solve the problem of poor effect of the traditional smoothing processing method, distinguish between the possible errors in the missing words and the insufficient coverage of the corpus itself, smooth the missing words, reduce misjudgments, and enhance smoothing processing Effect.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a smoothing processing method and system. Background technique [0002] A language model is an abstract mathematical modeling of language based on the objective facts of language in the process of processing natural language. There will be missing data in the language model, and the missing data needs to be solved by a smoothing algorithm. The smoothing algorithm obtains the remaining probability for redistribution by hijacking the probability of words that have appeared, and assigns the probability that can be used for distribution to the missing words according to certain rules. The probability obtained by the distribution of missing words is called smooth probability. [0003] The inventor found the following problems in the traditional technology. Taking the Good Turing smoothing algorithm as an example, in the GoodTuring smoothing algorithm, the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/284
CPCG06F40/284
Inventor 李贤
Owner GUANGZHOU SHIYUAN ELECTRONICS CO LTD