33results about How to "Improve word segmentation efficiency" patented technology

The invention discloses a Chinese word segmentation method based on two-way LSTM, CNN and CRF which improves and optimizes traditional Chinese word segmentation base on deep learning algorithm. The method comprises following specific steps: preprocessing the initial corpus, extracting corpus character feature information and pinyin feature information corresponding to characters; using the convolutional neural network to obtain pinyin feature information vector of the characters; using the word2vec model to obtain the character feature information vector of text; splicing pinyin feature vectors and character feature vectors to obtain context information vectors and put the context information vectors to a bidirectional LSTM neural network; decoding the output of the bidirectional LSTM using the linear chain condition random field to obtain the word segmentation sequence; decoding the word segmentation label sequence to obtain word segmentation results. The invention utilizes the deep neural network to extract text character features and pinyin features and combines the conditional random field decoding, can effectively extract Chinese text features and achieve good effect on Chinese word segmentation tasks.

The invention provides a multi-language based word segmentation method and apparatus. The method comprises: receiving a to-be-segmented text transmitted by a user, wherein the to-be-segmented text carries a statement separator; according to the statement separator, identifying the language type of each statement in the to-be-segmented text; according to the language type, searching for a corresponding word segmentation method in a pre-stored corresponding relationship between the language type and the word segmentation method; by adopting the word segmentation method corresponding to the language type, performing word segmentation on a statement of the corresponding language type; and outputting a word segmentation result of the to-be-segmented text to the user. According to the multi-language based word segmentation method, uniform word segmentation for applications or texts involving multiple languages can be performed, thereby improving the word segmentation efficiency.

The invention relates to a query method for indefinite words and phrases of evaluation documents based on inverted index, which relates to an index method in the field of data science and a word segmentation method in the field of NLP, and solves the query problem of indefinite words and phrases of evaluation documents. The invention comprises the following steps: 1, data preprocessing is carriedout on the document to be queried, word segmentation is carried out by using jieba word segmentation method, and word dictionary and word frequency information are obtained; 2, based on the inverted index principle of complete reconstruction strategy, an adaptive inverted table is established. 3, combine that information of the indefinite words and sentence to be searched, identifying the indefinite words and phrases position information in each word and phrases in the adaptive inverted table index, and indexing the paragraphs where the indefinite words and phrases are located, so as to complete the query function of the indefinite words and phrases in the evaluation documents. The basic idea of the invention is to divide the text data into words and establish an inverted index so as to realize fast searching for indefinite words and sentences, thereby realizing the inquiry function of evaluation documents. The application scenario is wide, so it has high socio-economic value.

Embodiments of the invention disclose a Chinese word segmentation method and apparatus. The method comprises the steps of dividing a text set into a plurality of short sentences and numbering the short sentences; for each Chinese character in the text set, obtaining a first short sentence number list corresponding to a current Chinese character, obtaining a second short sentence number list corresponding to an adjacent Chinese character adjacent to the right side of the current Chinese character, and calculating a degree of co-occurrence according to the first short sentence number list and the second short sentence number list; obtaining an adjacent character set corresponding to the current Chinese character, and calculating a relevant degree of adjacency according to the adjacent character set; determining whether a word consisting of the current Chinese character and the adjacent Chinese character is added into a candidate word set or not according to the degree of co-occurrence and the relevant degree of adjacency; and performing word segmentation on the text set according to the candidate word set. The method is small in calculation amount and high in accuracy when calculating the candidate word set, can effectively improve the accuracy of a word segmentation result and improve the efficiency of word segmentation, does not depend on a corpus dictionary, and can realize unsupervised candidate vocabulary extraction.

The invention discloses a method and system for intelligently understanding a user query intention, and the method comprises the steps: inputting a query statement, and carrying out the word segmentation processing through combining with a dictionary; performing part-of-speech tagging on the word segmentation result; performing named entity recognition on the words after the part-of-speech tagging; and carrying out grammatical analysis through a named entity identification result and a set grammatical rule to obtain a user query intention. According to the method, the input query statements are analyzed layer by layer according to the wording characteristics in the loan auditing industry, the query intention of the user is deeply understood, and the query efficiency is improved on the premise that the accuracy is ensured.

The invention discloses a method for constructing an emergency knowledge graph based on a Chinese word segmentation technology. The method specifically comprises the following steps: S1, inputting anemergency information text; S2, analyzing elements in the emergency information text in the step S1, extracting key data, and constructing an emergency knowledge base by utilizing the extracted key data; S3, performing word segmentation and judgment on the emergency information text input in the step S1 by adopting a multi-strategy combined Chinese word segmentation algorithm, and outputting a word segmentation result; S4, searching and matching the word segmentation result obtained in the step S3 in the emergency knowledge base by utilizing a retrieval engine, and outputting result data afterthe matching is successful; and S5, constructing an emergency knowledge graph according to the emergency service system in combination with the result data, and outputting graph result data. A scientific and comprehensive emergency knowledge graph is constructed according to an emergency business system, the data matching speed and the word segmentation precision are improved, the problems of lowretrieval efficiency and the like are solved, and the shared application service of emergency knowledge is realized.

The invention provides a word segmentation method supporting a large number of word banks, and a computer readable storage medium and a system. The method comprises the following steps: constructing adomain dictionary; constructing an offline word segmentation model based on a domain dictionary; for the original text to be subjected to word segmentation, performing word segmentation through an offline word segmentation model to obtain a first word segmentation result; carrying out to-be-searched word extraction on the original text to be subjected to word segmentation, then carrying out first-level index search and second-level index search in the domain dictionary based on the to-be-searched words, and finally screening second-level index results to extract candidate words; and recombining the candidate words and the first word segmentation result, constructing a directed graph of the original text based on a recombining result, and calculating an optimal word segmentation result based on a shortest path method. According to the method, the word segmentation result in the single field is combined with the big word search result, the directed graph is constructed based on the combination result, the problem of solving the optimal word segmentation scheme is converted into the problem of the optimal path to be quickly solved, and the method is very suitable for segmenting the big words.

The invention provides an intelligent matching system which comprises: a data acquisition module used for acquiring user registration input data and user behavior logs published on a network and a system platform; a recommended object modeling module which is used for extracting a keyword in the announcement information according to each piece of announcement information in user registration inutdata, obtaining all keywords interested by the user according to all announcement information concerned by the user in the user behavior log of the user for each user, and obtaining the interest degree of each keyword interested by the user according to the attention behavior of the user on each announcement information concerned by the user in the user behavior log of the user; and a recommendation algorithm module which is used for calculating the interest degree of each user in the announcement information according to the keyword extracted from the announcement information and the interestdegree of each user in the extracted keyword, and recommending the announcement information to a plurality of users with the highest interest degree. The intelligent matching system is high in information recommendation precision and efficiency.

The invention relates to the field of big data processing, in particular to an adaptive task scheduling method and system and a retrieval method comprising the adaptive task scheduling method. The adaptive task scheduling method comprises the steps that according to the resource utilization condition and the load change condition of each working node, the working nodes adjust the weights of the nodes every certain period; and a task scheduling node reads the weight condition of each working node, ranks the weights, and performs task scheduling according to the rank of the weights. According tothe method, the cluster heterogeneity and the current resource remaining situation of the working nodes are considered, the problem of relative imbalance of calculation task allocation of the workingnodes is solved, the overall calculation capability of the system is improved, the overall task completion time is shortened, and the performance is improved.

The invention discloses a Chinese word segmentation method based on a Hash algorithm, and relates to the field of natural language processing. The method comprises the following steps of S1, configuring a word segmentation device on a search engine and establishing a dictionary structure; s2, monitoring the return operation of the user, and obtaining the first character in an input box; s3, inputting the first character into a dictionary for primary searching and screening; s4, forming a tree by all words with the same first characters in the dictionary; s5, placing a second word in the word on a second layer of the tree, and creating a Hash index table; s6, carrying out Hash searching on the remaining characters; s7, after an IK reads the new lexicon, notifying the search engine to update; and S8, updating the dictionary information in the memory by the search engine. According to the invention, the Hash search is carried out on the first character by creating a dictionary storage mechanism, the dictionary structure and the algorithm of carrying out Hash search on the remaining characters via the tree result are established, and the search engine is updated by using IK word segmentation, so that the Chinese word segmentation efficiency is improved, the system complexity is reduced, and the index redundancy degree is reduced.

The invention relates to the technical field of Chinese text processing, in particular to a text word segmentation method and a text word segmentation device. The word segmentation method comprises the steps of obtaining a to-be-processed Chinese text, segmenting the Chinese text into a plurality of Chinese short texts; wherein each Chinese short text comprises a plurality of continuous Chinese characters representing a semantic meaning; according to the method, the length of the Chinese text can be reduced, interference of non-Chinese characters can be filtered out, the Chinese text subjectedto word segmentation is output on the basis of the multiple segmented Chinese short texts and the pre-trained Chinese word segmentation model, and the word segmentation efficiency of the Chinese textcan be improved.

The invention belongs to the technical field of Internet of Things and discloses a guiding system and method for electric power civil engineering foundation acceptance based on the Internet of Things.The geographic and meteorological information is collected by using a data layer, and a disaster analysis model and historical disaster information are inquired and sent to a basic service layer, sothat the meteorological monitoring and disaster analysis are completed, the information is automatically pushed. A business system layer monitors weather in real time, analyzes the disaster and givesan early warning, so that a user can check the meteorological disaster information at any time. The system is maintained by professionals through the Internet in real time, and the stability of the system is ensured. According to the present invention, the structure is reasonable, and the meteorological disasters can be effectively prevented.

The invention provides a word segmentation method and device. The method comprises the following steps: dividing a to-be-processed corpus into a plurality of corpus segments according to a preset granularity; inserting mask segments among the plurality of corpus segments, and inputting a to-be-predicted corpus containing the plurality of corpus segments and the mask segments into a pre-training language model; corpus information in the mask fragments adjacent to the corpus fragments is predicted through a pre-training language model; and performing word segmentation processing on the to-be-processed corpus based on the plurality of corpus segments and the corpus information to obtain a target word segmentation result. According to the method, the corpus information of the mask fragment can be predicted through the pre-training language model, so that word segmentation processing is completed through the corpus information obtained through prediction, word segmentation can be completed without the help of a dictionary or a word segmentation text, efficiency reduction caused by manual construction of the dictionary or the word segmentation text is avoided, and word segmentation efficiency is improved.

The invention relates to word segmentation technology and provides a word segmentation method and a word segmentation device aiming at the defects of large amount of computation and low identification accuracy of ambiguous words in the existing word segmentation methods. The word segmentation method comprises the steps of receiving the input word sequence and extracting at least a word segmentation package contained in the word sequence; and extracting and outputting the word string obtained in each extracted word segmentation package. The invention also provides the word segmentation device. In the technical scheme provided by the invention, the word sequence segmentation can be automatically finished in the input process; therefore, the technical scheme can greatly reduce the amount of computation of word segmentation operation and improve the word segmentation efficiency. In addition, the real segmentation intention of the user can be accurately reflected according to the segmentation on the word sequence conducted by the word segmentation packages, thus greatly improving the word segmentation accuracy of the word sequence.

The invention provides a coding method for clinical laboratory medicine text, and relates to the field of clinical laboratory medicine. The method comprises the following steps: analyzing and processing a clinical examination medical text to obtain an internal content structure of the clinical examination medical text, and carrying out structured coding on each structure of the clinical examination medical text; and calculating the similarity with each clinical examination medical term in a clinical examination medical term library before carrying out structured coding. Therefore, repeated and similar clinical examination medical terms can be effectively reduced; when the clinical examination medical terms are stored, a source structure based on segmented words is adopted, the segmented words serve as basic units of coding, and then different segmented words in a segmented word library are coded in a combined mode, so that the corresponding clinical examination medical terms are formed; a great storage space can be saved; word segmentation is performed by combining a word segmentation dictionary and a machine learning word segmentation device, so that the workload of manual auditing is reduced, and the word segmentation efficiency is improved; and three mapping modes of full mapping, basic mapping and main segmented word mapping are added, so that the universality is better.

The invention discloses a text word segmentation processing method and device, equipment and a medium. The method comprises the following steps: collecting a text to be subjected to word segmentation, the text to be subjected to word segmentation comprises a plurality of suspected words which are connected in series, and the suspected words are composed of pronunciation characters; all characters in the text to be subjected to word segmentation are sequentially traversed, redundant characters formed by continuous repetition in the suspected words are ignored in the traversing process, the redundant characters are converted into words in a dictionary tree diagram, the words are sequentially added into a result list, the dictionary tree diagram comprises a plurality of paths starting from a root node of the dictionary tree diagram and respectively reaching different tail end nodes, and the word sequence of the dictionary tree diagram is obtained; nodes through which each path passes store each character of the single word in sequence; and outputting the words in the result list in sequence as word segmentation results. According to the word segmentation device, word segmentation processing is carried out according to the tree diagram, abnormal repeated characters can be processed in the word segmentation process, redundant characters in the text to be subjected to word segmentation are ignored, and words contained in the text are extracted accurately and accurately.

The invention discloses a word segmentation method and device, electronic equipment and a storage medium. The word segmentation method comprises the steps: inputting a word segmentation word stock into a pre-stored baseline word segmentation model, and determining a preliminary word segmentation result of the word segmentation word stock based on the baseline word segmentation model; inputting thepreliminary word segmentation result into a pre-trained word segmentation model, and outputting a segmentation result of the preliminary word segmentation result based on the word segmentation model,the segmentation result comprising a segmentation unit, and the segmentation unit comprising a segmentation character and/or a segmentation character set; and combining the segmentation units according to a preset combination rule, and determining a final word segmentation result of the segmented word stock. According to the word segmentation method, the existing baseline word segmentation modelis not changed, and the convergence rate of the word segmentation model is ensured, and the word segmentation efficiency is improved, and the word segmentation result of the baseline word segmentationmodel is corrected, so that the accuracy of the word segmentation result is improved.