Text data processing method, device and equipment

A text data and processing method technology, applied in the computer field, can solve the problems of large difference, large amount of calculation, low semantic level, etc., to achieve the effect of accurate output results and improved semantic level

Active Publication Date: 2019-07-19
ADVANCED NEW TECH CO LTD
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Whether it is a simple model corresponding to BERT or a complex model corresponding to BERT, which includes at least 12 network layers, the amount of calculation will be very large, and the model corresponding to BERT is based on character or character-level input. However, for some For text data composed of languages ​​(such as Chinese, etc.), if the text data is split into individual characters, the semantic level will be lower, so that the output results will differ greatly from the actual results. Therefore, it is necessary to provide a method that can improve Semantic level and a technical solution to improve the accuracy of output results

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text data processing method, device and equipment
  • Text data processing method, device and equipment
  • Text data processing method, device and equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0068] Such as figure 1 As shown, the embodiment of this specification provides a method for processing text data. The execution body of the method may be a terminal device or a server, etc., wherein the terminal device may be a mobile terminal device such as a mobile phone or a tablet computer, or a personal computer and other equipment. The server can be an independent server, or a server cluster composed of multiple servers, and the server can be a background server of a website (such as an online shopping website or a shopping application, etc.), or it can be a service (such as a financial server) business, etc.) servers, etc. This method can be used to provide a processing mechanism for text data, so as to improve the semantic level and the accuracy of output results. In order to improve the processing efficiency of text data, the execution subject of this embodiment may be described by taking a server as an example. For the case that the execution subject is a terminal...

Embodiment 2

[0094] Such as Figure 5 As shown, the embodiment of this specification provides a method for processing text data. The execution body of the method may be a terminal device or a server, etc., wherein the terminal device may be a mobile terminal device such as a mobile phone or a tablet computer, or a personal computer and other equipment. The server can be an independent server, or a server cluster composed of multiple servers, and the server can be a background server of a website (such as an online shopping website or a shopping application, etc.), or it can be a service (such as a financial server) business, etc.) servers, etc. This method can be used to provide a processing mechanism for text data, so as to improve the semantic level and the accuracy of output results. In order to improve the processing efficiency of text data, the execution subject of this embodiment may be described by taking a server as an example. For the case that the execution subject is a termina...

Embodiment 3

[0134] The above is the text data processing method provided by the embodiment of this specification. Based on the same idea, the embodiment of this specification also provides a text data processing device, such as Figure 6 shown.

[0135] The processing device of this text data comprises: target data acquisition module 601, first word segmentation module 602, first delimiter module 603 and result determination module 604, wherein:

[0136] Target data acquisition module 601, configured to acquire target text data for specified business;

[0137] The first word segmentation module 602 is used to perform word segmentation processing on the target text data, and determine the word segmentation position in the target text data;

[0138] A first delimiter module 603, configured to insert a predetermined delimiter at a word segmentation position in the target text data;

[0139] The result determination module 604 is configured to input the target text data inserted with the de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a text data processing method, a text data processing device and text data processing equipment. The method comprises the following steps: acquiring target text data aiming at an appointed service; performing word segmentation processing on the target text data, and determining word segmentation positions in the target text data; inserting a predeterminedseparator at a word segmentation position in the target text data; and inputting the target text data into which the separator is inserted into a predetermined language model to obtain result data related to the specified service.

Description

technical field [0001] This specification relates to the field of computer technology, in particular to a text data processing method, device and equipment. Background technique [0002] With the continuous development of terminal technology and network technology, online transactions have become an important transaction method at present, and online transactions involve different aspects, such as financial payment, information recommendation, logistics, transaction security, etc., so that you can get A large amount of data, and these data will contain a lot of text data, thus giving birth to a large number of work requirements for text processing. [0003] How to process text data has become an important problem that e-commerce platforms need to solve. It can be realized through the BERT (Bidirectional Encoder Representations from Transformers, a bidirectional encoder derived from Transformers) mechanism. In the process of processing text data using the model corresponding ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/27
CPCG06F40/289G06F40/30
Inventor 陈晓军刘佳崔恒斌
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap