Judgment document-based bidirectional encoder characterization quantity model optimization method and device

Abstract

The invention relates to artificial intelligence, and provides a judgment document-based bidirectional encoder characterization quantity model optimization method and device. The method comprises thefollowing steps: determining an initial pre-training model corresponding to legal judgment document data according to an initial bidirectional encoder representation quantity model; obtaining a presetnumber of cause categories determined according to the legal judgment document data, and adding corresponding category labels to the cause categories; extracting a corresponding training data set from the legal judgment document data based on the category label, and performing data preprocessing on the training data set; and based on the preprocessed training data set, carrying out optimization training on the determined specific hyper-parameters of the initial pre-training model to obtain an optimized bidirectional encoder characterization quantity model. By the adoption of the method, natural language representation of the legal judgment document is achieved according to the optimized bidirectional encoder representation quantity model, and the application effect of the bidirectional encoder representation quantity model in the field of legal knowledge to which the judgment document belongs is improved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, in particular to a method and device for optimizing a two-way encoder characterization model based on referee documents. Background technique [0002] With the development of artificial intelligence technology and the popularization and application of natural language processing technology in people's work and life, as a major application in the field of natural language processing, the BERT model has become increasingly widely used. Among them, the BERT model represents the Bidirectional Encoder Representations from Transformers (Bidirectional Encoder Representations from Transformers) model, which aims to pre-train deep bidirectional representations by jointly adjusting the context in all layers. Its pre-training model based on large-scale corpus training is The downstream tasks of the model, such as sentence pair classification, single sentence classification, and seq...

AI Technical Summary

Problems solved by technology

[0004] At present, the cost of pre-training the BERT model is relatively high. Most model users cannot re-pre-train the BERT model based on the characteristic data of their application knowledge domain. They can only fine-tune the model. low and unreasonable task selection, etc., resulting in poor performance of the obtained model when applied in the corresponding knowledge field

Images