Method and apparatus for determining language model
A language model and determination method technology, applied in natural language data processing, speech analysis, speech recognition and other directions, can solve problems such as low corpus performance
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0028] An embodiment of the present invention provides a system for determining a language model. figure 1 is a schematic diagram of a system for determining a language model according to an embodiment of the present invention. Such as figure 1 As shown, the language model determining system 100 includes: an input device 102 , a processor 104 and an output device 106 .
[0029] The input device 102 is configured to input a first corpus to the processor 104, wherein the first corpus is a language text selected in a preset context.
[0030] As the training set corpus of the language model to be used, corpus can be corpus from various sources in daily life, such as corpus in information annotation, corpus in web crawling, corpus in open source libraries, and corpus in a certain field provided by users. Effective corpus, etc., have a wide range of sources and a large amount of data. The corpus may correspond to a certain amount of task fields, wherein. The first corpus may be ...
Embodiment 2
[0054] According to an embodiment of the present invention, an embodiment of a method for determining a language model is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and , although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
[0055] figure 2 is a flowchart of a method for determining a language model according to an embodiment of the present invention, such as figure 2 As shown, the method includes the following steps:
[0056] Step S21, acquiring the first corpus, wherein the first corpus is the language text selected in the preset context.
[0057] In the above steps, the corpus is used as the training set corpus of the language model to be used, which can be corpus from various sources in daily life, such as corpu...
Embodiment 3
[0132] According to an embodiment of the present invention, a device for determining a language model for implementing the above method for determining a language model is also provided. Figure 4 is a schematic diagram of an apparatus for determining a language model according to an embodiment of the present invention. Such as Figure 4 As shown, the language model determining device 400 includes: a first acquiring module 402, a first training module 404 and a processing module 406, wherein.
[0133] The first acquiring module 402 is configured to acquire the first corpus, wherein the second corpus is a language text selected in a preset context.
[0134] The first training module 404 is configured to obtain a first language model by training the first corpus.
[0135] The processing module 406 is configured to use the first language model to screen the target text to obtain a second corpus, and obtain the second language model by training the second corpus, wherein the tar...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com