Long text cascade classification method, system and device and storage medium

Abstract

The invention relates to a long text cascade classification method which comprises the following steps: S1, preprocessing input long text data by using a sliding window mechanism, and segmenting the long text data into a plurality of intervals; s2, performing semantic encoding on the generated text in each interval to obtain a local semantic vector of each interval; s3, carrying out keyword extraction on the interval text, encoding the interval text into keyword vectors, carrying out vector splicing on the keyword vectors and the local semantic vectors, and obtaining an overall semantic vectorof the long text; s4, carrying out dimension reduction on the overall semantic vector, and using a classifier to carry out category label probability distribution calculation on the overall semanticvector after dimension reduction; and S5, according to the steps S1-S4, training a classification model of the long text corpus. According to the technical scheme provided by the invention, the performance of the classification model at the bottom layer can be improved to promote the development of other intelligent services in the vertical field, thereby improving the user experience and viscosity of intelligent products.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a long text cascade classification method, system, equipment and storage medium. Background technique [0002] With the continuous popularization of artificial intelligence technology, the development of industries in vertical fields is changing with each passing day, and new AI products are constantly emerging. [0003] For example, products such as case retrieval, regulatory retrieval, and intelligent consulting robots in the legal vertical field require the support of intelligent semantic analysis technology. However, intelligent semantic analysis technology includes a variety of algorithm models, such as element extraction models, relationship extraction models, and complex The event extraction models of these models are based on the classification model as the underlying support. The performance of the bottom layer is closely related to the a...

AI Technical Summary

Problems solved by technology

[0005] (1) Every word in the document is interactively calculated with all other words, resulting in high complexity, especially when the document length exceeds the threshold

[0006] (2) Different interval texts obtained by segmentation cannot interact across intervals, and a large amount of key information will be lost

[0007] These two shortcomings greatly limit the performance of existing classification models in vertical fields and the scope of application of upper-level models

Images