Method and electronic device for generating dictionary format

Abstract

Embodiments of the invention provide a dictionary format generation method and an electronic device, which are used for processing special characters in a text and improving the accuracy of segmentinga field containing a special character string in text analysis. The method includes the steps of obtaining a plurality of textual information pieces from at least one data source, each textual information piece of the plurality of textual information pieces including a special character, and the special characters including numbers and / or letters; extracting at least one semantic segment relatedto the special characters in the plurality of textual information pieces, wherein each semantic segment in the at least one semantic segment includes the special characters and associated characters adjacent to the special characters and the number of characters of the associated characters is less than or equal to a preset number; and determining at least one dictionary format according to the atleast one semantic segment, the at least one dictionary format being used for representing distribution rules of the special characters in the corresponding semantic segment.

Description

technical field [0001] The invention relates to the technical field of computers, in particular to a dictionary format generation method and electronic equipment. Background technique [0002] With the rapid development of the mobile Internet, the amount of information it generates is increasing rapidly. How to extract the parts we are interested in from this information is exactly what Neuro-Linguistic Programming (NLP) needs to study. Especially for the entrance of the mobile Internet - the mobile phone has become a must for many Internet companies. Therefore, by correctly parsing these application texts, users can be provided with better services. [0003] Text parsing includes two stages: text segmentation and semantic analysis. For the application text information of various companies and enterprises on mobile phones, the general structure is relatively regular, the amount of text information is sufficient, and the frequency of template changes is relatively small, whi...

AI Technical Summary

Problems solved by technology

[0005] The first way is to build a huge dictionary library, which is technically and operationally difficult to achieve, requires a huge amount of time and manpower, and cannot be predicted in the future to meet the same format and only slightly modify some of the content emergence of new words

In addition, an excessively large dictionary will also lead to an increase in resource consumption in the text segmentation stage, and at the same time, the running speed and execution efficiency will be significantly reduced;

[0006] For the latter method of only selecting necessary fixed vocabulary, the entire sentence will be segmented very trivially, the integrity of the information will be destroyed to a certain extent, and it is not conducive to the processing in the later stage of semantic analysis

[0007] In summary, there is no better processing method for special characters in text in the prior art

Images