Word segmentation processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a word segmentation processing method and device, electronic equipment and a storage medium. The method comprises: obtaining target feature vectors corresponding to information to be subjected to word segmentation of multiple target words arranged in sequence; performing label prediction processing on the target feature vector to obtain first prediction results of the plurality of target words belonging to preset word segmentation labels; on the basis of a preset word segmentation tag, combining target words in a target word segmentation fragment corresponding to the information to be subjected to word segmentation, obtaining target word sequences corresponding to the target words, carrying out tag prediction processing on the target word sequences, and obtaining second prediction results that the target words belong to the preset word segmentation tag; according to the first prediction result and the second prediction result, determining target word segmentation labels corresponding to the target characters; and determining a word segmentation result of the to-be-segmented information according to the to-be-segmented information and the target word segmentation tag. By utilizing the scheme provided by the embodiment of the invention, the word segmentation effect and performance of the to-be-segmented information can be improved, and the word segmentation cost is reduced.

Description

Technical field [0001] The present disclosure relates to the field of Internet technology, and more particularly to a method, apparatus, electronic device, and storage medium. Background technique [0002] The Chinese word in natural language processing refers to the process of re-combining a consecutive word sequence in accordance with a certain specification. [0003] The relevant techniques are generally based on matching, statistical, depth learning, etc. However, the matching word algorithm (such as forward, reverse matching algorithm) is too dependent on the dictionary, the maintenance cost of the dictionary is high, the system resource consumption is large, and the matching word algorithm based on the word words and unregistered words. Treatment effect (such as the stability of the word boundary) is poor; based on the statistical word algorithm, the complexity is large, the word performance (for example, the vocabulary rate) is poor, and a large number of artificial labels...

AI Technical Summary

Problems solved by technology

[0004] The present disclosure provides a word segmentation processing method, device, electronic equipment, and storage medium to at least solve the problems of high access cost, poor word segmentation effect, and poor word segmentation performance in related technologies

Images