Financial supervision and inspection problem text classification method and device, equipment and storage medium

Abstract

The invention discloses a financial supervision and inspection problem text classification method and device, equipment and a storage medium. The method comprises the following steps: acquiring a financial supervision inspection question text; inputting the financial examination question text into the target classification model to obtain a risk result output by the target classification model; wherein the target classification model is obtained by pre-training on the basis of a BERT model, and a lexicon table of the BERT model is obtained by expanding a lexicon table corresponding to a SentencePiece word segmentation device. According to the scheme provided by the embodiment of the invention, on the basis of the Bert model, the word library table of the BERT model is expanded by introducing SentecePiece word segmentation, that is, the target classification model is obtained through easy-to-difficult training, so that the model fully learns more feature information in a financial supervision inspection problem text, and the purpose of improving the classification accuracy of the model is achieved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a method, device, equipment and storage medium for classifying texts of financial supervision and inspection issues. Background Art

[0002] At present, some businesses in the financial industry are in the process of transformation, the operating pressure of institutions is increasing, and loopholes in compliance are frequent. Under the tone of emphasizing compliance and preventing risks, supervision of the financial industry has always been strict, and large fines have frequently appeared. Financial institutions need to strengthen risk awareness and further improve their compliance management capabilities. To this end, financial institutions need to strengthen internal control management, conduct various on-site and off-site inspections for their own weak links, classify the problems found in internal inspections according to the severity of the problems, further clarify the classification of problem rectification and accountability, and achieve early detection and early disposal of internal problems in the institution, and promote the healthy and sustainable development of institutional operations based on compliance. How to help financial institutions quickly and accurately classify problems found in internal inspections, ensure that various businesses are carried out in accordance with regulatory requirements, and effectively reduce the risk of regulatory penalties is a difficult point in the internal control and compliance management of financial institutions.

[0003] During the supervision and inspection of the financial industry, detailed descriptions of the problems found during the inspection are made, and the severity of the problems is classified according to the descriptions of the problems. Different inspection scopes and contents during the actual supervision and inspection process will lead to different problems found during the inspection. The distribution range of problem data is large, and the length of texts describing the problem situation varies. A deep learning classification model that can take into account both long and short texts is needed. At the same time, if the problem description content is manually labeled, the scene coverage is incomplete, and manual labeling is difficult.

[0004] Due to their respective characteristics, CNN, RNN and LSTM deep learning neural network models cannot meet the requirements of classifying supervised inspection problems according to problem descriptions. Summary of the invention

[0005] The present invention provides a method, device, equipment and storage medium for classifying text of financial supervision and inspection problems, so as to realize deep learning classification of long and short texts at the same time. At the same time, the introduction of a vocabulary construction mechanism based on the SentencePiece word segmentation model can effectively reduce the model calculation complexity, improve the semantic understanding ability of model training, and can more effectively process and analyze problem text data.

[0006] According to one aspect of the present invention, a method for classifying text of financial supervision and inspection questions is provided, comprising:

[0007] Obtain the text of financial supervision and inspection questions;

[0008] Inputting the financial inspection question text into a target classification model to obtain a risk result output by the target classification model;

[0009] The target classification model is obtained based on the pre-training of the BERT model, and the vocabulary table of the BERT model is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter.

[0010] In a possible implementation, the training process of the target classification model includes:

[0011] Obtain a first question text sample;

[0012] Training the SentencePiece word segmenter based on the first question text to obtain a first word library table;

[0013] Updating a second vocabulary table of the BERT model based on the first vocabulary table, and performing an unsupervised training process on the BERT model based on a second question text sample;

[0014] When the BERT model training is completed, a target BERT model is obtained, and the target BERT model is fine-tuned based on the third question text sample to obtain the target classification model.

[0015] In a possible implementation, the training of the SentencePiece word segmenter based on the first question text to obtain a first word library table includes:

[0016] Configuring training parameters of the SentencePiece word segmenter, wherein the training parameters include at least one of a maximum vocabulary length, a vocabulary size, and a number of training rounds;

[0017] According to the training parameters, the first question text is segmented by the SentencePiece word segmenter to obtain a vocabulary to be used;

[0018] When the vocabulary to be used meets the preset size, the vocabulary to be used is determined as the first vocabulary table, wherein the vocabulary includes a plurality of sub-word units and numbers corresponding to the sub-word units.

[0019] In a possible implementation, performing an unsupervised training process on the BERT model based on the second question text sample includes:

[0020] Processing the second question text sample by word segmentation based on the longest matching algorithm to obtain at least two to-be-used word segmentations corresponding to the second question text sample;

[0021] Based on the pre-constructed masked language model, masking the to-be-used word segmentation corresponding to the second question text sample according to a preset ratio to obtain the to-be-used question text sample;

[0022] The BERT model is trained based on the question text sample to be used and the second question text sample to obtain the target BERT model.

[0023] In a possible implementation, fine-tuning the target BERT model based on the third question text sample to obtain the target classification model includes:

[0024] Wherein, the third question text sample includes a sample financial supervision and inspection question text and a corresponding classification label;

[0025] Adding a classification layer to the target BERT model to obtain a classification model to be trained, wherein the classification layer includes a fully connected layer and a softmax activation function;

[0026] The cross entropy loss function is used as the loss function, and the classification model to be trained is trained based on the sample financial supervision and inspection question text and the corresponding classification labels to obtain the target classification model.

[0027] In a possible implementation, the first question text sample is obtained through the following five types of files, including:

[0028] Policy documents related to financial supervision and inspection, financial supervision and inspection reports and case files, news reports and comments on the financial industry, professional books and papers on the financial industry, and work records and feedback from financial supervision and inspection practitioners.

[0029] In a possible implementation, inputting the financial inspection question text into a target classification model to obtain a risk result output by the target classification model includes:

[0030] Create a Web service application and run the target classification model in a Python environment;

[0031] The financial supervision and inspection question text input by the user is obtained based on the Web service application, and is input into the target classification model to obtain the risk classification level corresponding to the financial supervision and inspection question text.

[0032] According to another aspect of the present invention, there is provided a device for classifying texts of financial supervision and inspection questions, comprising:

[0033] A question text acquisition module is used to obtain the text of financial supervision and inspection questions;

[0034] A risk result determination module, used for inputting the financial inspection question text into a target classification model to obtain a risk result output by the target classification model;

[0035] The target classification model is obtained based on the pre-training of the BERT model, and the vocabulary table of the BERT model is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter.

[0036] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0037] at least one processor;

[0038] and a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the financial supervision and inspection question text classification method described in any embodiment of the present invention.

[0040] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the financial supervision and inspection question text classification method described in any embodiment of the present invention when executed.

[0041] The technical solution of the embodiment of the present invention includes: obtaining the text of financial supervision and inspection questions; inputting the text of financial inspection questions into the target classification model to obtain the risk results output by the target classification model; wherein the target classification model is obtained based on the BERT model pre-training, and the BERT model's vocabulary table is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter. The solution of the embodiment of the present invention, based on the Bert model, expands the BERT model's vocabulary table by introducing the SentencePiece word segmentation, that is, the target classification model is obtained by training from easy to difficult, and then the model is allowed to fully learn more feature information in the financial supervision and inspection question text, so as to achieve the purpose of improving the classification accuracy of the model.

[0042] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flowchart of a method for classifying texts of financial supervision and inspection questions provided in the first embodiment of the present invention;

[0045] Figure 2 A flowchart of a method for classifying text of financial supervision and inspection questions provided in Embodiment 2 of the present invention;

[0046] Figure 3 The process of pre-training the SentencePiece financial industry supervision and inspection question database provided in the second embodiment of the present invention;

[0047] Figure 4 The word segmentation process of the SentencePiece pre-training model provided in the second embodiment of the present invention;

[0048] Figure 5 The training provided in the second embodiment of the present invention is completed to obtain the QBERT model;

[0049] Figure 6 A classification diagram of a target classification model provided in the second embodiment of the present invention;

[0050] Figure 7 A schematic diagram of the structure of a text classification device for financial supervision and inspection issues provided in Embodiment 3 of the present invention;

[0051] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] Embodiment 1

[0055] Figure 1 This is a flowchart of a method for classifying text of financial supervision and inspection problems provided in the first embodiment of the present invention. This embodiment can be applied to the process of financial industry supervision and inspection to provide a detailed description of the problems found in the inspection and classify the severity of the problems according to the description of the problems. This method can be performed by a text classification device for financial supervision and inspection problems. The device can be implemented in the form of hardware and / or software, and the device can be configured in an electronic device. Figure 1 As shown, the method comprises the following steps:

[0056] S110. Obtain the text of financial supervision and inspection issues.

[0057] Among them, the financial supervision and inspection problem text may be the text whose severity needs to be determined.

[0058] Specifically, during the supervision and inspection of the financial industry, a detailed description of the problems found during the inspection can be made, and the severity of the problems can be classified according to the description of the problems. This helps financial institutions to quickly and accurately classify the problems found during internal inspections, ensure that the conduct of various businesses complies with regulatory requirements, and effectively reduce the risk of regulatory penalties.

[0059] S120, inputting the financial inspection question text into the target classification model to obtain the risk result output by the target classification model.

[0060] Among them, the target classification model can be a well-established model, and the target classification model can determine the corresponding risk classification results based on the text of the financial inspection problem. For example, the severity of the problem is classified into five levels: minor, general, concerned, major and serious. By clearly classifying the problem, financial institutions can be assisted to quickly take measures to correct the behavior, control risks and investigate responsibilities, further control the scope of the problem, and reduce the risk of regulatory penalties.

[0061] The target classification model is pre-trained based on the BERT model, and the BERT model's vocabulary table is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter. By introducing a vocabulary construction mechanism based on the SentencePiece word segmentation model, the model's computational complexity can be effectively reduced, the semantic understanding ability of model training can be improved, and problem text data can be processed and analyzed more effectively.

[0062] In an embodiment of the present invention, a financial inspection question text is input into a target classification model to obtain a risk result output by the target classification model, including: creating a Web service application and running the target classification model in a Python environment; obtaining the financial supervision and inspection question text input by the user based on the Web service application, and inputting it into the target classification model to obtain a risk classification level corresponding to the financial supervision and inspection question text.

[0063] Specifically, the target classification model can be loaded into Python, and a web application can be built to build a REST API service using the BERT model. The service accepts input data (the text of the financial supervision and inspection questions entered by the user), uses the target classification model for inference, and returns the result (risk classification level).

[0064] Exemplarily, the following steps may be included:

[0065] Model preparation: Save the target classification model. You can choose the format to save the model according to the framework, as long as you can ensure that the saved model can be loaded in Python.

[0066] Create a Web service: Create a Web service application to handle external HTTP requests. The application needs to contain application definition and routing. The application definition includes loading the target classification model and tokenizer, encoding the HTTP request content text, applying the target classification model to the encoded text for reasoning, extracting specific classification results, and finally returning the results.

[0067] Financial practitioners enter the description information of the problems found in the financial supervision and inspection in the business system, triggering the problem classification scenario. The application sends the entered problem description information (i.e. the text of the financial supervision and inspection problem) to the service corresponding to the created target classification model through an http request to obtain the problem classification results of the target classification model. For the problems found in the financial inspection classified as concern, major and serious by the model, the financial institution needs to take effective measures immediately to rectify the problems and avoid the risk of external regulatory penalties.

[0068] The technical solution of the embodiment of the present invention includes: obtaining the text of financial supervision and inspection questions; inputting the text of financial inspection questions into the target classification model to obtain the risk results output by the target classification model; wherein the target classification model is obtained based on the BERT model pre-training, and the BERT model's vocabulary table is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter. The solution of the embodiment of the present invention, based on the Bert model, expands the BERT model's vocabulary table by introducing the SentencePiece word segmentation, that is, the target classification model is obtained by training from easy to difficult, and then the model is allowed to fully learn more feature information in the financial supervision and inspection question text, so as to achieve the purpose of improving the classification accuracy of the model.

[0069] Embodiment 2

[0070] Figure 2 This is a flowchart of a method for classifying texts of financial supervision and inspection questions provided in the second embodiment of the present invention. Based on the above embodiment, this embodiment is further specific. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 2 As shown, the method includes:

[0071] S210: Obtain a first question text sample.

[0072] Among them, the first question text sample can be a text corresponding to the financial industry supervision and inspection issues.

[0073] In an embodiment of the present invention, the text of financial supervision and inspection questions is obtained through the following five types of files, including: policy documents related to financial supervision and inspection, financial supervision and inspection reports and case files, news reports and comments on the financial industry, professional books and papers on the financial industry, and work records and feedback from financial supervision and inspection practitioners.

[0074] Exemplarily, the first question text sample may be collected in the following manner:

[0075] Collect five types of text resources related to financial industry supervision and inspection issues: the first type is policy documents related to financial supervision and inspection, such as financial regulatory policies, regulations, and notices issued by the government. Such documents contain professional financial regulatory terms and expressions; the second type is financial supervision and inspection reports and case files. These texts record in detail the process of financial supervision and inspection, the problems found, and the results of handling, and are important materials for building a problem vocabulary; the third type is news reports and comments on the financial industry, covering the dynamics and problems of the financial industry; the fourth type is professional books and papers on the financial industry, which help to improve the professional line and accuracy of the vocabulary; the fifth type is the work records and feedback of financial supervision and inspection practitioners, which can directly reflect the actual problems encountered in supervision and inspection work. Ensure the instructions and diversity of the data so that the model can learn a rich vocabulary and expressions in the field of financial industry supervision and inspection.

[0076] S220 , training a SentencePiece word segmenter based on the first question text to obtain a first word library table.

[0077] The first word library table refers to the word library table obtained after training the SentencePiece word segmenter.

[0078] In some embodiments, before training the SentencePiece tokenizer based on the first question text, data preprocessing may be performed on the first question text and a preliminary vocabulary may be constructed.

[0079] Specifically, clean and preprocess the text data, organize the text materials related to the five types of financial supervision and inspection issues into a text file, one content description text occupies one line, each line of text is complete, the length of each line of text is the actual length of the text describing the problem, the content is semantically complete, and stored as a txt file, remove blank lines, pause words, low-frequency words, irrelevant words, special punctuation marks, retain the amount symbol, etc. Then, train the SentencePie ce word segmenter based on the preprocessed first question text to obtain the first word library table.

[0080] In an embodiment of the present invention, a SentencePiece word segmenter is trained based on a first question text to obtain a first word library table, including: configuring training parameters of the SentencePiece word segmenter; according to the training parameters, segmenting the first question text by the SentencePiece word segmenter to obtain a vocabulary table to be used; when the vocabulary table to be used meets a preset size, determining the vocabulary table to be used as the first word library table.

[0081] The training parameters include at least one of the maximum vocabulary length, the vocabulary size and the number of training rounds. The vocabulary to be used refers to the vocabulary obtained by segmenting a large amount of first question texts through the SentencePiece word segmenter, and the vocabulary includes multiple subword units and numbers corresponding to the subword units.

[0082] Specifically, the first question text may be repeatedly segmented by the SentencePiece segmenter to obtain a vocabulary list to be used, until the size of the current vocabulary list to be used meets the preset size, and the current vocabulary list to be used is used as the first vocabulary list. The preset size may be represented by the total number of subword units included in the vocabulary list.

[0083] In some embodiments, configuring the training parameters may be creating a training configuration file, such as a maximum vocabulary length, a vocabulary size, a number of training rounds, etc., for adjusting the parameters to control the number and length of the generated sub-word units.

[0084] SentencePiece model pre-training: Use the prepared first question text and pre-configured training parameters to start model training using the train() method of SentencePiece, perform word segmentation according to the frequency of subwords until the vocabulary reaches a predetermined size, and construct a large vocabulary table of preliminary word segmentation, namely the first vocabulary table. The vocabulary table includes the results, including all subword units and corresponding ID numbers.

[0085] like Figure 3 As shown, the SentencePiece pre-training financial industry supervision and inspection question word library process provided in the second embodiment of the present invention; Figure 4 As shown, the word segmentation process of the SentencePiece pre-training model provided in the second embodiment of the present invention.

[0086] S230: Update the second vocabulary table of the BERT model based on the first vocabulary table, and perform an unsupervised training process on the BERT model based on the second question text sample.

[0087] Among them, the second vocabulary table can be the most initial vocabulary table of the BERT model, and the second question text sample can be the same as the first question text sample or different, but both can be collected from text resources related to five categories of financial industry supervision and inspection issues.

[0088] Specifically, the generated first word library table can be added to the word library table of the Bert model to expand the initial word library table of the Bert model. Then, the BERT model is unsupervisedly trained based on the second question text sample.

[0089] In an embodiment of the present invention, performing an unsupervised training process on a BERT model based on a second question text sample may include: performing word segmentation processing on the second question text sample based on a longest matching algorithm to obtain at least two to-be-used word segmentations corresponding to the second question text sample; performing masking processing on the to-be-used word segmentations corresponding to the second question text sample according to a preset ratio based on a pre-constructed masked language model to obtain a to-be-used question text sample; and training a BERT model based on the to-be-used question text sample and the second question text sample to obtain a target BERT model.

[0090] Specifically, the longest matching algorithm is used to segment the second question text sample. An attempt is made to find the longest possible match in the dictionary to ensure the accuracy of the segmentation and the integrity of the semantics. Through this process, multiple segmentations to be used corresponding to the second question text sample can be obtained.

[0091] Furthermore, the pre-built Masked Language Model (MLM) is used to mask the words to be used at a preset ratio (usually about 15%). A certain ratio of words can be randomly selected and replaced with special symbols (such as [MASK]). After the above processing, the question text samples to be used are obtained. The purpose of this is to let the model learn to predict the masked words based on the context, thereby enhancing its ability to understand and generate language.

[0092] Finally, the BERT model is trained using the masked question text sample and the original second question text sample (as a label or reference answer). During the training process, the model will try to predict the masked word segmentation and adjust its parameters by comparing it with the original text. After multiple iterations of training, the model gradually learns the deep features and language rules of the text, and finally obtains the trained target BERT model, namely the target BERT model.

[0093] For example, we first use a large amount of financial industry inspection question description text to continue pre-training based on BERT-base-Chinese.

[0094] Then, prepare a large amount of unlabeled text data related to financial supervision and inspection, namely the second problem samples, as pre-training data. These data include compliance data such as financial policy documents, supervision and inspection reports, financial news, company reports, and data on problems found in financial inspections, which serve as training data for BERT model pre-training. Use the longest matching word segmentation algorithm to segment each piece of pre-training data.

[0095] Furthermore, based on the pre-built masked language model, the word segmentation mask to be used corresponding to the second question text sample is processed according to a preset ratio. That is, some words are randomly masked from the input series, and these masked words are predicted based on the context, that is, for the pre-trained text X, 85% of X remains unchanged, 15% of the text W is replaced with the '[M]' symbol, and the text W replaced with the '[M]' symbol is predicted based on the context.

[0096] Predict the next sentence and determine whether the two sentences are continuous, so that the BERT model can better understand the relationship between the sentences. Use different inspection problem description fragments as negative samples, and use the continuous text in the same inspection problem description as positive samples. BERT pre-training combines the above two tasks and performs them simultaneously. Then add the loss data calculated by the model to complete the pre-training of the financial industry inspection problem description text based on the BERT pre-training model. This can achieve classification tasks with two sentences as input. Figure 5 As shown, the training provided in the second embodiment of the present invention is completed to obtain the QBERT model.

[0097] S240. When the BERT model training is completed, a target BERT model is obtained, and the target BERT model is fine-tuned based on the third question text sample to obtain a target classification model.

[0098] Specifically, the target BERT model can be fine-tuned to obtain a target classification model.

[0099] In some embodiments, a model fine-tuning question description text dataset, i.e., a third question text sample, may be prepared. The third question text sample includes a question description and a corresponding question classification label. The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The longest matching word segmentation algorithm is used to segment the question description text in the training set, and the Chinese character sequence is divided into possible vocabulary units.

[0100] Use the transformers library to load the pre-trained target BERT model and the tokenizer corresponding to the BERT model to convert the training question text into words that the model can understand and add special tags required by BERT, such as [CLS] and [SEP].

[0101] Convert the input into a high-dimensional vector representation, create an attention mask and token type IDs for each input sequence, and the token type IDs are usually initialized to 0. Convert the label to a format acceptable to the model.

[0102] In an embodiment of the present invention, the target BERT model is fine-tuned based on the third question text sample to obtain a target classification model, including: adding a classification layer to the target BERT model to obtain a classification model to be trained, wherein the classification layer includes a fully connected layer and a softmax activation function; using the cross entropy loss function as the loss function, and training the classification model to be trained based on the sample financial supervision and inspection question text and the corresponding classification labels to obtain a target classification model.

[0103] The third question text sample includes a sample financial supervision and inspection question text and a corresponding classification label;

[0104] It should also be noted that a classification layer can be added on the basis of the target Bert model. The classification layer includes a fully connected layer and a softmax activation function. The cross entropy loss function is used as the fine-tuning model loss function, and the Adam optimizer is used to update the model weights and optimize the model parameters. Through this classification layer, the probability of the label corresponding to the question text can be obtained.

[0105] During the fine-tuning process, the validation set is used regularly to evaluate the model performance. After fine-tuning is completed, the model is finally evaluated using the test set and the fine-tuned model is saved for subsequent use. Figure 6 As shown, it is a classification diagram of the target classification model provided in the second embodiment of the present invention.

[0106] S250, inputting the financial inspection question text into the target classification model to obtain the risk result output by the target classification model.

[0107] The technical solution of the embodiment of the present invention classifies financial industry supervision and inspection issues by introducing the BERT model, replacing the traditional manual method and improving the efficiency of problem classification. The introduction of a vocabulary construction mechanism based on the SentencePiece word segmentation model can effectively reduce the model calculation complexity, improve the semantic understanding ability of model training, and more effectively process and analyze problem text data. Using the longest matching word segmentation algorithm to segment text data of problem descriptions found in financial inspections can accurately provide more accurate word segmentation results for the Bert model and improve the model prediction accuracy.

[0108] Embodiment 3

[0109] Figure 7 This is a schematic diagram of the structure of a text classification device for financial supervision and inspection issues provided by the third embodiment of the present invention. Figure 7 As shown, the device comprises:

[0110] The question text acquisition module 310 is used to acquire the text of the financial supervision and inspection question;

[0111] The risk result determination module 320 is used to input the financial inspection question text into the target classification model to obtain the risk result output by the target classification model;

[0112] Among them, the target classification model is obtained based on the pre-training of the BERT model, and the vocabulary table of the BERT model is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter.

[0113] The device also includes a target classification model training module, which includes:

[0114] A first question text sample acquisition submodule, used to acquire a first question text sample;

[0115] A first word library table determination submodule is used to train a SentencePiece word segmenter based on a first question text to obtain a first word library table;

[0116] An unsupervised training submodule, used for updating a second vocabulary table of the BERT model based on the first vocabulary table, and performing an unsupervised training process on the BERT model based on a second question text sample;

[0117] The fine-tuning submodule is used to obtain a target BERT model when the BERT model training is completed, and fine-tune the target BERT model based on the third question text sample to obtain a target classification model.

[0118] Optionally, the first vocabulary table determination submodule is specifically used for:

[0119] Configure the training parameters of the SentencePiece word segmenter, where the training parameters include at least one of the maximum word length, the vocabulary size, and the number of training rounds;

[0120] According to the training parameters, the first question text is segmented by the SentencePiece word segmenter to obtain a vocabulary to be used;

[0121] When the vocabulary list to be used meets the preset size, the vocabulary list to be used is determined as a first vocabulary list, wherein the vocabulary list includes a plurality of sub-word units and numbers corresponding to the sub-word units.

[0122] The optional unsupervised training submodule is specifically used to:

[0123] Processing the second question text sample by word segmentation based on the longest matching algorithm to obtain at least two to-be-used word segmentations corresponding to the second question text sample;

[0124] Based on the pre-constructed masked language model, masking the to-be-used word segmentation corresponding to the second question text sample according to a preset ratio to obtain the to-be-used question text sample;

[0125] The BERT model is trained based on the question text sample to be used and the second question text sample to obtain a target BERT model.

[0126] Optionally, the fine-tuning submodule is specifically used to:

[0127] The third question text sample includes a sample financial supervision and inspection question text and a corresponding classification label;

[0128] Add a classification layer to the target BERT model to obtain a classification model to be trained, where the classification layer includes a fully connected layer and a softmax activation function;

[0129] The cross entropy loss function is used as the loss function, and the classification model to be trained is trained based on the sample financial supervision and inspection question text and the corresponding classification labels to obtain the target classification model.

[0130] Optionally, the text of the financial supervision and inspection questions can be obtained through the following five types of documents, including:

[0131] Policy documents related to financial supervision and inspection, financial supervision and inspection reports and case files, news reports and comments on the financial industry, professional books and papers on the financial industry, and work records and feedback from financial supervision and inspection practitioners.

[0132] Optionally, the risk result determination module 320 is specifically used to:

[0133] Create a web service application and run the target classification model in the Python environment;

[0134] The financial supervision and inspection question text input by the user is obtained based on the Web service application, and is input into the target classification model to obtain the risk classification level corresponding to the financial supervision and inspection question text.

[0135] The financial supervision and inspection question text classification device provided in the embodiment of the present invention can execute the financial supervision and inspection question text classification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0136] Embodiment 4

[0137] Figure 8A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0138] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0139] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0140] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a text classification method for financial supervision and inspection issues.

[0141] In some embodiments, the method for classifying texts of financial supervisory inspection questions may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for classifying texts of financial supervisory inspection questions described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for classifying texts of financial supervisory inspection questions in any other appropriate manner (e.g., by means of firmware).

[0142] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0147] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for classifying texts of financial supervision and inspection problems, characterized in that: include: Obtain the text of financial supervision and inspection questions; Inputting the financial inspection question text into a target classification model to obtain a risk result output by the target classification model; The target classification model is obtained based on the pre-training of the BERT model, and the vocabulary table of the BERT model is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter.

2. The method according to claim 1, characterized in that The training process of the target classification model includes: Obtain a first question text sample; Training the SentencePiece word segmenter based on the first question text to obtain a first word library table; Updating a second vocabulary table of the BERT model based on the first vocabulary table, and performing an unsupervised training process on the BERT model based on a second question text sample; When the BERT model training is completed, a target BERT model is obtained, and the target BERT model is fine-tuned based on the third question text sample to obtain the target classification model.

3. The method according to claim 2, characterized in that The step of training the SentencePiece word segmenter based on the first question text to obtain a first word library table includes: Configuring training parameters of the SentencePiece word segmenter, wherein the training parameters include at least one of a maximum vocabulary length, a vocabulary size, and a number of training rounds; According to the training parameters, the first question text is segmented by the SentencePiece word segmenter to obtain a vocabulary to be used; When the vocabulary to be used meets the preset size, the vocabulary to be used is determined as the first vocabulary table, wherein the vocabulary includes a plurality of sub-word units and numbers corresponding to the sub-word units.

4. The method according to claim 2, characterized in that: The performing an unsupervised training process on the BERT model based on the second question text sample includes: Processing the second question text sample by word segmentation based on the longest matching algorithm to obtain at least two to-be-used word segmentations corresponding to the second question text sample; Based on the pre-constructed masked language model, masking the to-be-used word segmentation corresponding to the second question text sample according to a preset ratio to obtain the to-be-used question text sample; The BERT model is trained based on the question text sample to be used and the second question text sample to obtain the target BERT model.

5. The method according to claim 2, characterized in that: The target BERT model is fine-tuned based on the third question text sample to obtain the target classification model, include: Wherein, the third question text sample includes a sample financial supervision and inspection question text and a corresponding classification label; Adding a classification layer to the target BERT model to obtain a classification model to be trained, wherein the classification layer includes a fully connected layer and a softmax activation function; The cross entropy loss function is used as the loss function, and the classification model to be trained is trained based on the sample financial supervision and inspection question text and the corresponding classification labels to obtain the target classification model.

6. The method according to claim 1, characterized in that The first question text sample is obtained through the following five types of files, including: Policy documents related to financial supervision and inspection, financial supervision and inspection reports and case files, news reports and comments on the financial industry, professional books and papers on the financial industry, and work records and feedback from financial supervision and inspection practitioners.

7. The method according to claim 1, characterized in that The step of inputting the financial inspection question text into the target classification model to obtain the risk result output by the target classification model includes: Create a Web service application and run the target classification model in a Python environment; The financial supervision and inspection question text input by the user is obtained based on the Web service application, and is input into the target classification model to obtain the risk classification level corresponding to the financial supervision and inspection question text.

8. A text classification device for financial supervision and inspection problems, characterized in that: include: A question text acquisition module is used to obtain the text of financial supervision and inspection questions; A risk result determination module, used for inputting the financial inspection question text into a target classification model to obtain a risk result output by the target classification model; The target classification model is obtained based on the pre-training of the BERT model, and the vocabulary table of the BERT model is expanded based on the vocabulary table corresponding to the SentencePiece word segmenter.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the financial supervision and inspection question text classification method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the financial supervision and inspection question text classification method described in any one of claims 1-7 when executed.

Images