Text semantic coding method and system

A semantic coding and coding technology, applied in semantic analysis, natural language data processing, instruments, etc., can solve the problems of splicing unable to integrate the semantic coding of words, too large vocabulary, and poor word segmentation effect.

Active Publication Date: 2020-09-04
AISPEECH CO LTD
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0016] In order to at least solve the problem that the word segmentation algorithm in the existing technology may make mistakes, the vocabulary is too large and cannot contain all words, and the splicing cannot integrate the characteristics of the semantic coding of words, which makes the word segmentation effect poor.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text semantic coding method and system
  • Text semantic coding method and system
  • Text semantic coding method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0057] As an implementation, the interactive matching layer based on the multi-head attention mechanism includes at least: a matching interaction layer of word-word-word attention mechanism;

[0058] The word semantic code and the word semantic code are linearly transformed by the matching interaction layer of the word-word-word attention mechanism;

[0059] After the linear transformation, a preset number of zoomed dot product attention mechanisms are performed, the preset number of zoomed dot product results are spliced, and the spliced ​​zoomed dot product results are linearly transformed to obtain the word-word- The fusion result of the matching interaction layer of the word attention mechanism.

[0060] In this embodiment, the multi-head attention mechanism such as image 3 Shown:

[0061] In the multi-head attention mechanism, Query(V), Key(K), Value(V), where V=Q=x c , K=x w , the output of the top linear layer Linear in the figure is the semantic encod...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a text semantic coding method. The method comprises the steps: carrying out semantic encoding on a text through a character encoder and a word encoder, splicing results of character encoding of the text, splicing results of word encoding of the text, and obtaining character semantic encoding and word semantic encoding containing contexts; importing the character semantic codes and the word semantic codes obtained after splicing into an interactive matching layer based on a multi-head attention mechanism; and determining the fused word meaning code obtained by the interaction matching layer as the word meaning code of the text. The embodiment of the invention further provides a text semantic coding system. The embodiment of the invention provides anencoder based on multilayer word fusion. According to the encoder, after semantic encoding is carried out on characters and words, interaction is carried out on the obtained character and word semantic codes, then the interacted character and word semantic codes are fused through a gating unit trained in a self-adaptive mode, and deeper character and word semantic information is contained and serves as final text semantic representation.

Description

technical field [0001] The invention relates to the field of intelligent speech, in particular to a text semantic encoding method and system. Background technique [0002] With the continuous development of neural network models, more and more natural language processing tasks use neural network models. The text semantic encoder is the first step of the natural language processing task based on the neural network model, and the quality of the encoding effect directly affects the performance of the model. Existing neural network-based text semantic encoders usually segment the input text according to words (for example: credit | amount | want | how | just | can | increase), and then map different words to corresponding words according to the vocabulary id, and then further semantically encode the input text. [0003] In the process of realizing the present invention, the inventors have found that there are at least the following problems in the related art: [0004] Segmen...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06F40/289G06F40/126G06N3/04
CPCG06F40/30G06F40/289G06F40/126G06N3/045
Inventor 吴仁守
Owner AISPEECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap