Model training method and device, and method and device for realizing text processing

A technology for model training and text processing, applied in digital data processing, natural language data processing, instruments, etc., can solve problems such as accuracy dependence and error transmission, and achieve the effect of avoiding error transmission

Active Publication Date: 2020-05-15
BEIJING MININGLAMP SOFTWARE SYST CO LTD
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In related technologies, word segmentation and part-of-speech tagging are two separate tasks. In the pipeline structure, part-of-speech tagging is a downstream task of word segmentation, and its accuracy largely depends on the result of word segmentation. There is a problem of error transmission question

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model training method and device, and method and device for realizing text processing
  • Model training method and device, and method and device for realizing text processing
  • Model training method and device, and method and device for realizing text processing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the purpose, technical solution and advantages of the present invention more clear, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

[0049] The steps shown in the flowcharts of the figures may be performed in a computer system, such as a set of computer-executable instructions. Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0050] figure 1 It is a flow chart of the model training method of the embodiment of the present invention, such as figure 1 shown, including:

[0051] Step 101, complete word segmentation and part-of-speech tagging input data for the preset number, and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a model training method and device and a method and device for realizing text processing, and the method comprises the steps: for a preset number of input data which completesword segmentation and part-of-speech tagging, taking a character as a unit to identify a character contained in each chunk and the part-of-speech of each character; training the identified input datathrough a preset training model to obtain a text processing model for performing word segmentation and part-of-speech tagging on the to-be-processed text; and performing word segmentation and part-of-speech tagging on the to-be-processed text through the obtained text processing model. According to the embodiment of the invention, word segmentation and part-of-speech tagging are carried out at thesame time through text processing, and error transmission in the word segmentation and part-of-speech tagging process is avoided.

Description

technical field [0001] This article involves but is not limited to language processing technology, especially a model training method, device, and text processing method and device. Background technique [0002] Word segmentation and part-of-speech tagging play an important role in natural language processing; among them, word segmentation refers to identifying the composition of words in a sentence, and splitting the sentence into a sequence of words as units; part-of-speech tagging refers to identifying the part of speech of words in a sentence . [0003] At present, word segmentation includes dictionary-based word segmentation and statistics-based word segmentation; among them, dictionary-based word segmentation includes: matching the string to be matched with words in an established dictionary, and identifying words by matching entries ; Common dictionary-based word segmentation includes: forward maximum matching method, reverse maximum matching method and two-way match...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/289G06F40/30
Inventor 陈栋李嘉琛付骁弈
Owner BEIJING MININGLAMP SOFTWARE SYST CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap