Method, device and system for extracting qualitative indicators of regulatory events

Abstract

The present invention discloses a method, device and system for extracting qualitative indicators of regulatory events, and relates to the field of computer technology. A specific implementation of the method includes: obtaining regulatory information containing text content; performing sentence splitting on the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type; semantically annotating the split sentences, thereby extracting a set of information phrases, wherein the information phrase set contains one or more information phrases associated with an information dimension; performing vectorization processing on each information phrase in the information phrase set to obtain an information vector set; and using sentiment classification processing for the information vector set to obtain a qualitative classification result. By matching qualitative indicators based on scenario judgment of regulatory events, not only is docking with the overall regulatory evaluation system achieved, but also the qualitative indicator extraction process can make full use of different scenario attributes, thereby greatly improving the algorithm effect and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and system for extracting qualitative indicators of regulatory events. Background Art

[0002] Financial supervision implements the inspection and supervision of the entire financial industry, including financial institutions and financial businesses. The sources of regulatory information for financial supervision are very diverse, including data with quantitative information such as financial business data and financial data, as well as qualitative information such as on-site inspection supervision, public opinion review, and government affairs events. The regulatory information is processed into regulatory indicators to evaluate the regulated objects. At present, for quantitative indicators with quantitative information such as business indicators and financial indicators, information processing systems can be used to process data in quantitative information. However, for qualitative indicators, such as regulatory cooperation and compliance, which require qualitative analysis, they are often judged manually.

[0003] The manual measurement of qualitative indicators makes it impossible for the regulatory indicator processing system of financial regulators to fully utilize the real-time and batch big data acquisition and processing capabilities brought by the information processing system. The manual measurement method itself has many defects. It cannot avoid the qualitative indicator error problem caused by human subjective judgment factors, and the information source of manual verification is relatively single. For example, the information source of public opinion review indicator evaluation in manual verification is limited to a single complaint and petition. Due to the limited capacity and efficiency of manual work, it is impossible to obtain public opinion from social networks, media and other massive Internet-based information sources in real time and batch, nor is it possible to realize real-time and batch processing of qualitative regulatory information. Once the capacity of the regulated object exceeds the manual workload, manual work cannot even meet the comprehensive financial industry regulatory needs. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a method, device and system for extracting qualitative indicators of regulatory events, which can collect regulatory information from multi-dimensional data sources in real time, use natural language processing to implement a machine learning algorithm for extracting qualitative regulatory indicators from massive regulatory information, and through event judgment for regulatory event scenarios, realize docking with the overall regulatory indicator system, and can be modularly, scalably and portablely integrated into the application of the regulatory indicator system. By providing semantic analysis, information extraction and association integration at the event level, data intelligence, expert knowledge base and regulatory indicators are combined in a tool-based manner, while providing intelligent collection and processing of qualitative regulatory information, the real-time, effectiveness and efficiency of financial supervision are improved.

[0005] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for extracting qualitative indicators of regulatory events is provided, which is characterized by comprising:

[0006] Obtain regulatory information including text content;

[0007] Perform sentence splitting on the text of the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method;

[0008] semantically annotating the split sentences, thereby extracting an information phrase set, wherein the information phrase set includes one or more information phrases, and each information phrase is associated with an information dimension;

[0009] Performing vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase; and

[0010] For the information vector set, sentiment classification processing is used to obtain a qualitative classification result.

[0011] Optionally, associating each statement with a corresponding regulatory event type through a machine learning approach includes: determining the corresponding regulatory event type through a machine learning-based scenario classification process.

[0012] Optionally, the scene classification process based on machine learning further includes: using a machine learning classification method based on artificial corpus annotated data to perform scene classification on each sentence.

[0013] Optionally, the machine learning classification method is a support vector machine classification algorithm, and the artificial corpus annotation data is maintained through an expert knowledge base, including predefined trigger word data for financial supervision.

[0014] Optionally, the regulatory event type is based on the classification of regulatory event scenarios, including: industry news, on-site inspections, public opinion reviews, and government affairs events. The text of the regulatory information achieves the matching of the qualitative classification results with qualitative regulatory indicators based on regulatory event scenarios through the classification of the regulatory event scenarios, so that comprehensive regulatory evaluation can be further achieved in combination with quantitative regulatory indicators.

[0015] Optionally, the semantic annotation includes: performing BIOES semantic annotation through Chinese named entity recognition.

[0016] Optionally, BIOES semantic annotation through Chinese named entity recognition is performed based on manual semantic annotation data maintained in an expert knowledge base, wherein the one or more information phrases in the annotated sentences have associated information phrase attribution indicators and information dimension indicators, each character in each information phrase corresponds to the information phrase attribution indicator of the information phrase, and the information dimension indicator of each information phrase indicates the information dimension of the information phrase.

[0017] Optionally, the vectorization processing is word vector encoding for incremental training using financial regulatory supplementary corpus.

[0018] Optionally, the vectorization process further includes: mapping each information phrase to a word vector having a vector dimension in a vector dimension set through the word vector encoding.

[0019] Optionally, each vector dimension in the vector dimension set is defined based on a regulatory event type of a statement to which the information phrase belongs, wherein information phrases corresponding to different regulatory event types have corresponding subsets in the vector dimension set.

[0020] Optionally, the method further comprises: before executing the vectorization process, preprocessing the information phrase set, the preprocessing comprising: word segmentation, stop word removal, and simplified and traditional character encoding conversion.

[0021] Optionally, the sentiment classification process is a classification process based on a deep neural network, comprising:

[0022] For each information vector in the information vector set, performing feature extraction to obtain a feature vector set of corresponding feature vectors;

[0023] Determine the aspect embedding vector for the specific qualitative aspect you want to extract;

[0024] Determine an attention weight vector using the determined aspect embedding vector together with each feature vector, and further determine a set of weighted feature vectors, wherein the attention weight vector indicates the respective importance of each information phrase in the sentence for the specific qualitative extraction target; and

[0025] The weighted feature vector set is used together with the feature vector set to determine a conditional probability distribution representing a qualitative classification result through a classification network, wherein the conditional probability distribution indicates whether the result tendency for the specific qualitative aspect is positive, negative or neutral.

[0026] Optionally, the feature extraction is implemented by a long short-term memory network, wherein for each input information vector, the long short-term memory network generates a corresponding implicit feature vector as output for processing by the next layer of the deep neural network.

[0027] Optionally, the classification network includes a fully connected layer and a Softmax function at its output, wherein the weighted feature vector set together with the feature vector set are processed through an activation function in the fully connected layer to obtain a result representation, and the result representation is converted into the conditional probability distribution through the Softmax function, wherein the activation function is a tanh function.

[0028] Optionally, the specific qualitative aspect is based on the regulatory event type, and the method further comprises: outputting the qualitative classification result, wherein the output comprises presenting the specific qualitative aspect of the statement and a result tendency for the specific qualitative aspect.

[0029] Optionally, the method further comprises: receiving feedback on the qualitative classification result, and performing adjustment based on the feedback.

[0030] Optionally, the adjustment includes:

[0031] receiving the feedback, wherein the feedback includes a result correctness indicator indicating whether a qualitative classification result of the sentence of the regulatory information is correct or not and a true result indicator;

[0032] In response to feedback indicating that the qualitative classification result is wrong, the corresponding true result is used to adjust parameters related to the sentiment classification process and training parameters.

[0033] Optionally, the parameters related to the sentiment classification process include one or more of the following:

[0034] LSTM network parameters;

[0035] Parameters of the aspect embedding vector; and

[0036] Parameters of the classification network.

[0037] According to a second aspect of an embodiment of the present invention, a device for extracting qualitative indicators of regulatory events is provided, characterized in that it includes: a regulatory information reading module, a regulatory information parsing module, a regulatory indicator extraction module and a regulatory indicator qualitative module; wherein,

[0038] The regulatory information reading module is used to obtain regulatory information containing text content, perform sentence splitting on the text of the regulatory information, and obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method;

[0039] The regulatory information parsing module is used to semantically annotate the split sentences, so as to extract an information phrase set, wherein the information phrase set includes one or more information phrases, and each information phrase is associated with an information dimension;

[0040] The regulatory indicator extraction module is used to perform vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase; and

[0041] The regulatory indicator qualitative module is used to obtain a qualitative classification result by using sentiment classification processing for the information vector set.

[0042] Optionally, the device further comprises a supervision indicator output module and a supervision indicator adjustment module; wherein,

[0043] The supervision indicator output module is used to output the qualitative classification result, wherein the output includes the specific qualitative aspect of the presentation statement and the result tendency for the specific qualitative aspect; and

[0044] The supervision indicator adjustment module is used to receive feedback on the qualitative classification result and perform adjustment based on the feedback.

[0045] According to a third aspect of an embodiment of the present invention, a system for extracting qualitative indicators of regulatory events is provided, comprising: a regulatory information base and a qualitative indicator extraction server that is communicatively connected to the regulatory information base using the method provided by the first aspect of an embodiment of the present invention.

[0046] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device for extracting qualitative indicators of regulatory events, including:

[0047] one or more processors;

[0048] a storage device for storing one or more programs,

[0049] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0050] According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method provided in the first aspect of the embodiment of the present invention is implemented.

[0051] An embodiment of the above invention has the following advantages or beneficial effects: the present invention can make full use of the advantages of various aspects of information by widely collecting regulatory information from relevant service databases, public media, public opinion and other data sources that may involve financial regulatory information. At the same time, it can not only combine the original dedicated information sources but also cover a wide range of network resources so as to actively reach the information sources that have not been involved in previous financial monitoring. Compared with the information collected by manual regulatory means, it can achieve truly comprehensive and multi-dimensional regulatory coverage. The qualitative indicator extraction technology of the present invention not only achieves docking with the overall regulatory evaluation system through qualitative indicator matching based on the scene judgment of regulatory events, but also integrates with the quantitative indicator evaluation in a unified evaluation framework; at the same time, the type matching of regulatory events performed on regulatory information also enables the qualitative indicator extraction process to make full use of different scene attributes, thereby greatly improving the extraction algorithm effect and efficiency. In addition, the qualitative indicator extraction of the present invention, by introducing sentiment classification based on deep learning, combines data intelligence, expert knowledge base and regulatory indicators in a tool-based manner to achieve high precision and high versatility.

[0052] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0054] Figure 1 It is a schematic diagram of the main process of the method for extracting qualitative indicators of regulatory events according to an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of a main process of obtaining a qualitative classification result by using sentiment classification processing according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram showing attention weights of different information phrases in an example sentence under an attention mechanism according to an embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of main modules of a device for extracting qualitative indicators of regulatory events according to an embodiment of the present invention;

[0058] Figure 5 It is an exemplary system architecture 500 to which the method for extracting qualitative indicators of regulatory events or the apparatus for extracting qualitative indicators of regulatory events according to an embodiment of the present invention can be applied;

[0059] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0061] According to one aspect of an embodiment of the present invention, a method for extracting qualitative indicators of regulatory events is provided.

[0062] The embodiment of the present invention is described in the context of a qualitative indicator extraction system for example financial supervision. The qualitative indicators used by financial supervision to evaluate the regulated objects are derived from the processing of non-quantitative information, which involves but is not limited to various documents, reports, archives, media messages, email communications, work files and other text records, and its sources are very wide and diverse. In the example qualitative indicator extraction system of the present invention, the regulatory information used for regulatory indicator extraction is maintained by the regulatory information library, which can be obtained from relevant service databases, public media, public opinion and other various data sources that may involve financial regulatory events. The regulatory information can be maintained in the regulatory information library in structured data forms and unstructured data forms. Because the sources of these regulatory information are different. For example, public opinion and public opinion can come from various network public platforms and social platforms including web sites, forums, blogs, Weibo, etc., and public opinion can also come from various complaints and petitions; on-site inspection information can involve various financial statements, documents and archives, original vouchers and rules and regulations reported during the on-site inspection of the regulated objects; government affairs events can include current news, new regulations and new policies, etc. Therefore, the forms of these regulatory information are extremely diverse, and the amount of information is huge. At the same time, the information that an information document may contain is also very complex. Information about the same regulated object is also very likely to be scattered in regulatory information from different sources and in various forms; and this information often has good timeliness and information content. If the corresponding regulatory events in these information can be extracted from the regulatory events, the qualitative indicators that can be identified by the regulatory system will be effectively realized. Real-time monitoring and prediction of regulatory events will be effectively achieved. However, due to the diversity of sources and forms of regulatory information, in order to provide effective qualitative indicator extraction information, the technology of the present invention identifies the regulatory event scenarios corresponding to the regulatory information through event judgment, thereby realizing the integration of the extracted qualitative indicators with the overall regulatory system through the division of regulatory events, and can be combined with quantitative indicators to coordinate regulatory evaluation. The technology of the present invention further utilizes regulatory event classification to promote the algorithm effect and efficiency of qualitative indicator extraction.

[0063] Throughout this application, "regulatory information" is used to refer to regulatory information entries maintained in a regulatory information repository. In this context, a "regulatory information" entry may correspond to, for example, a text entry of an on-site inspection report maintained in a regulatory information repository, a text entry of a social media post, or a text entry of a news article, etc. In various embodiments, text entries of "regulatory information" may be maintained in the regulatory information repository in various data formats. In some embodiments, each regulatory information has an associated information timestamp, a data entry timestamp, an acquisition source identifier, and one or more of other relevant information that facilitates information query. Among them, the information timestamp indicates the content release time or completion time of the corresponding regulatory information, for example: an on-site inspection report has an associated report completion date, a social media post has an associated post release time, and so on.

[0064] Figure 1 is a schematic diagram of the main process of the method for extracting qualitative indicators of regulatory events according to an embodiment of the present invention, such as Figure 1 As shown, the method for extracting qualitative indicators of regulatory events according to an embodiment of the present invention includes: step S101, step S102, step S103, step S104 and step S105.

[0065] Step S101: Obtain regulatory information including text content.

[0066] In some embodiments of the present invention, regulatory information is obtained from a regulatory information repository of a qualitative indicator extraction system. For example, the regulatory information repository may provide an application programming interface (API) to facilitate information query. In some example embodiments, the regulatory information required for indicator extraction may be obtained from the regulatory information repository in a query manner for a specific regulated object, a specific regulatory event, or other desired information. For example, the expected regulatory information release time may be limited for a specific regulated object and the source range of the regulatory information may be further limited to obtain regulatory information that meets the specific requirements. In other example embodiments, regulatory information may be received directly from the regulatory information repository by active batch transmission, wherein the frequency, capacity, and other transmission methods of batch transmission may be set. The acquired regulatory information may include data in various forms, including at least text content, such as the text of an on-site inspection report, the text of a news article, or the text of a social media post.

[0067] Step S102: Sentence splitting is performed on the text of the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method.

[0068] After obtaining the regulatory information from the regulatory information library, event classification will be performed first. In an example embodiment of the present invention, regulatory event classification is performed by a context judgment submodule. However, those skilled in the art will appreciate that the operation of regulatory event classification can be performed directly in the regulatory information reading module, thereby integrating its function with the function of the regulatory information reading module. In other embodiments, regulatory event classification may be performed by a separate module separate from the regulatory information reading module. By classifying regulatory information into different regulatory event types and performing subsequent processing based on the specific regulatory event type of each regulatory information, such as semantic annotation and sentiment analysis, the subsequent processing time can be greatly shortened and the efficiency of the algorithm can be improved.

[0069] In an example embodiment, regulatory information can be classified into one of the following types: industry news, on-site inspections, public opinion reviews, and government affairs events. Regulatory information classification can be achieved through various classification algorithms known in the field of natural language processing. In this example embodiment, regulatory event classification is achieved through a support vector machine (SVM) algorithm based on a manually annotated corpus data set, where the SVM algorithm classification process is based on an SVM classification model.

[0070] The modeling of the SVM classification model requires a predetermined sample data set. In some embodiments, the reference corpus is first annotated manually, and the trigger words in each sentence in the corpus are manually annotated, and the trigger words are annotated as corresponding to one of the above-mentioned regulatory event types, thereby obtaining an annotated sample data set. In some other embodiments, artificial corpus annotation can also be performed in other ways. After obtaining the annotated corpus data set through manual annotation, the corpus is vectorized to construct the feature vector required for classification. In some embodiments, the trigger words can be used as objects to be classified, and the feature vectors can be constructed according to the linguistic features (semantic, lexical, and syntactic features) of the trigger words. In some embodiments, the trigger word data related to financial supervision can be maintained in the expert knowledge base of the indicator extraction system, and the trigger word data can be obtained by calling the application programming interface (API) of the expert knowledge base. In an example embodiment, the construction of the feature vector is performed using a latent semantic analysis (LSA) model. Next, the SVM model training is performed on the sample data set by selecting appropriate kernel functions and classification model parameters to generate an SVM classification model. The SVM classification model can improve its classification reliability by continuously expanding the sample data set and iteratively performing the training process, so that its classification accuracy continuously approaches or reaches the desired accuracy range. In some embodiments, a radial basis function can be selected as a kernel function, and the model parameters can be determined by cross-validation.

[0071] The initial step of the classification process of regulatory information is similar to the initial step of the training process of the SVM model. In some embodiments, a trigger word can be used as an object to be classified, and a feature vector can be constructed according to the linguistic features (semantic, lexical, and syntactic features) of the trigger word. The constructed feature vector is used as the input to the SVM classification model, and the SVM model processes the input and maps it to the corresponding output, thereby obtaining a representation of the classification result of the regulatory information corresponding to the input (i.e., in the example embodiment, corresponding to one type of industry news, on-site inspection, public opinion review, and government affairs). In other embodiments, the constructed feature vector can be used for other classification algorithms, such as logistic regression or neural network classification algorithms. By dividing regulatory information into types of industry news, on-site inspection, public opinion review, and government affairs events, the extracted qualitative indicators can be integrated with the overall regulatory system, and the regulatory evaluation can be carried out in conjunction with quantitative indicators. At the same time, the classification of regulatory events can further promote the algorithm effect and efficiency of subsequent qualitative indicator extraction. In other embodiments, regulatory information can be divided into other different projects and different numbers of types according to different indicator requirements.

[0072] The scene judgment submodule performs regulatory event classification on the regulatory information and outputs an identifier of the event classification corresponding to the regulatory information. In some other embodiments, the scene judgment submodule can also provide a representation of the trigger words identified in the classification process and the corresponding linguistic features for use in subsequent processing. In an example embodiment, in a regulatory qualitative indicator extraction system, for example, through a qualitative indicator extraction database, the correlation between the regulatory information and the corresponding event classification and other related data is maintained. In some other embodiments, the data related to the regulatory information can also be directly stored as metadata in association with the regulatory information data, thereby facilitating other modules to directly call the regulatory information and its metadata, thereby realizing modular deployment and architecture expansion.

[0073] The text of regulatory information, such as the on-site inspection report, social media posting or news report article in the example implementation, usually contains one or more sentences. Before extracting qualitative indicators, the text needs to be split into individual sentences first. In the example implementation, sentence splitting is performed in the regulatory information reading module. In some other embodiments, sentence splitting can be performed in a submodule in the regulatory information reading module, or can be performed in a separate module separated from the regulatory information reading module. Sentence splitting can be implemented using various sentence splitting algorithms in natural language processing. In some embodiments, sentence splitting is implemented by an experience-based classification method. The experience-based classification method involves sentence end judgment combined with punctuation rules. In other embodiments, sentence splitting can be implemented in combination with a rule-based model, and the rule-based model can determine the grammatical features of the sentence and implement sentence end judgment in combination with punctuation rules. Further, the probability-based model can be further combined to predict the probability of the elements (words, punctuation, numbers, characters, etc.) in the sentence appearing as the end of the sentence, and the rule-based model is integrated to perform feature extraction, and the sentence end judgment of sentence splitting is implemented by a classifier. The sentence obtained by sentence splitting the regulatory information text is associated with the event type of the regulatory information. In an example implementation, in the regulatory qualitative indicator extraction system, for example, through the qualitative indicator extraction database, the association between the regulatory information statements and the corresponding event classifications and other related data is maintained. In some other implementations, the data related to the regulatory information statements can also be directly stored as metadata in association with the regulatory information statements.

[0074] Step S103: semantically annotate the split sentences, so as to extract an information phrase set, wherein the information phrase set includes one or more information phrases, and each information phrase is associated with an information dimension.

[0075] In the process of extracting indicators based on statements, it is first necessary to semantically annotate the split statements, thereby identifying the entities contained in the statements and their relationships. In an example implementation, the semantic annotation of the statements is implemented using Chinese named entity recognition (NER). An expert knowledge base is maintained in the indicator extraction system, and the identification of entities in the statements is realized by calling the API of the expert knowledge base. In the expert knowledge base, a NER database is maintained, in which the entities in the statements of the regulatory information text can be divided into multiple information dimensions including but not limited to industry, event, time, relationship, status, location, institution, etc., for the scenes of regulatory events involved in the regulatory qualitative indicator extraction system (e.g., industry news, on-site inspections, public opinion review, government affairs events) and objects (e.g., entity correspondences of expert data involved in the entity list of regulated objects, core dictionaries of regulatory events, and other regulatory customized indicator extractions, etc.), and the index of the identified entities is established. In some example implementations, the association between the identified entities and the information in the extracted related dimensions is maintained in the qualitative indicator extraction database to implement the indexing of entities and their associated information dimensions. In an example implementation, the learning process of NER recognition in an expert knowledge base is carried out using artificial semantic annotation data maintained in the expert knowledge base, and a set of overall information dimensions can be maintained by the expert knowledge base. The example artificial annotation data is annotated in a BIOES manner, and each character in the annotated sentence has an associated information phrase attribution indicator and an information dimension indicator. The information phrase attribution indicator is used to indicate the scope of each individual phrase, i.e., Begin, indicating the beginning; Intermediate, indicating the middle; End, indicating the end; Single, indicating a single character; Other, indicating others, used to mark irrelevant characters; the information dimension indicator indicates the corresponding information dimension of each individual phrase. As an example, a sentence split in a regulatory information text is "Shanghai Huangpu United Microfinance Co., Ltd. held an opening ceremony at the Shanghai Real Estate (Group) Building in Pudong", and its annotation result is [B-LOC, E-LOC, B-LOC, E-LOC, B-ORG, I-ORG, I-ORG, I-ORG, I-ORG, I-ORG, I-ORG, I-ORG, I-ORG, E-ORG, O, O, O, O, O, O]. Specifically, "LOC" indicates that the information dimension of the corresponding information phrase is location, and "ORG" indicates that the information dimension of the corresponding information phrase is institution. Thus, each sentence is identified as individual entities through NER, that is, character subsets, each of which contains one or more characters and is associated with a corresponding dimension.

[0076] In an example implementation, the semantic annotation process is an annotation process of event types determined based on scenario judgment. Specifically, each of the event types of industry news, on-site inspections, public opinion reviews, and government affairs events in the example implementation has a corresponding NER model, thereby achieving entity-dimension association for specific scenarios, which can fully utilize the text features related to different regulatory events and their scenarios, significantly shorten the processing time of semantic annotation while improving algorithm efficiency.

[0077] Step S104: Perform vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase.

[0078] In each sentence, the information phrase associated with the information dimension still has the characteristics of natural language. The algorithm of the data processing system has very limited efficiency in directly processing characters representing natural language, and consumes a lot of computing resources and storage space, which need to be converted into machine-readable code. By vectorizing each information phrase in the information phrase set, the information phrase can be mapped to an information vector under the vector space dimension, thereby converting it into a form that is easy to process using an algorithm. Before encoding the information phrase of a sentence by word vector, it is first necessary to perform preprocessing on the sentence. For example, preprocessing can include: segmenting the text of the sentence, removing stop words, and converting the simplified and traditional characters.

[0079] In the example implementation, the dimension of each information vector in the vector space is defined by the regulatory event type of the statement to which the information phrase belongs, which can be further combined with the information dimension of the statement. For example, an example information phrase can be attributed to the regulatory event type of public opinion review. For this regulatory event type, a corresponding information vector dimension set can be maintained in the expert knowledge base, where each information vector in this example includes five vector dimensions as shown below:

[0080] <Finance, Regulation, Public Opinion, Industry, Action>

[0081] The mapping established by statistical learning for each information phrase can be represented as an information vector based on the degree of relevance of the information phrase in each vector dimension. It can be understood that the definition of vector dimension is not limited to the above example, and there can be more or less dimensions and the attributes of each dimension can be different, and as statistical learning proceeds, the vector dimension can be continuously optimized.

[0082] In an example implementation, the mapping of information phrases to information vectors is achieved by encoding word vectors that are incrementally trained using the financial regulatory supplementary corpus. Incremental training can start by loading pre-trained word vectors, first preliminarily training the word vector encoding model to obtain a word vector encoding model with initial parameters. In the example qualitative indicator extraction system, regulatory training data is maintained through a regulatory information repository, which contains manually annotated financial regulatory supplementary corpus data. By further inputting the financial regulatory supplementary corpus data into the preliminarily trained word vector encoding model, a mapping model of targeted information phrases to information vectors for the desired application scenario can be obtained.

[0083] In further example implementations, the mapping model is further refined for different types of regulatory events in regulatory information. For example, continuing with the above example, the financial regulatory supplementary corpus data contains type-based corpus data subsets corresponding to the types of industry news, on-site inspections, public opinion reviews, and government affairs events. Incremental training is performed for each different type of corpus data subset, so that a mapping model for the corresponding regulatory event type can be obtained. Continuing with the example vector dimension just now. After the example information phrases "on-site inspection" and "field visit" are vectorized, the following example information vectors are obtained:

[0084] On-site inspection = <0.1, 0.8, 0.0, 0.0, 0.1>

[0085] Field visits = <0.1, 0.5, 0.1, 0.1, 0.2>.

[0086] Among them, the numerical value under each dimension in the information vector of each information phrase indicates the degree of relevance of the information attribute corresponding to the vector dimension in the information phrase. For example, the information phrase "on-site inspection" has a higher relevance under the "supervision" vector dimension, while the information phrase "field visit" also has a high relevance under this vector dimension, but the relevance is lower than that of "on-site inspection". Both have a low relevance under the "public opinion" vector dimension. After the information phrase is vectorized to obtain the information vector, it can be further processed by the machine learning model.

[0087] In an example implementation, the information vectors obtained by vectorization may be further normalized before being processed by machine learning. Normalization includes unifying the value ranges of each information vector so that they can be compared with each other. Normalization may also include the association of similar information vectors. Continuing with the above example, although the information vectors corresponding to the information phrases "on-site inspection" and "field visit" differ in numerical value, in the vector space, the distance between their corresponding information vectors is closer than that of other vectors, and thus, the two can be identified as similar information vectors and thus associated. Similar association of information vectors can promote the speed of subsequent machine learning, for example, it can help improve the convergence speed and accuracy of the algorithm.

[0088] Step S105: Use sentiment classification processing to obtain a qualitative classification result for the information vector set.

[0089] Next, we will refer to Figure 2 To describe the process of obtaining a qualitative classification result for the information vector set. Figure 2 is a schematic diagram of the main process of obtaining qualitative classification results using sentiment classification processing according to an embodiment of the present invention, such as Figure 2 As shown, the method for obtaining a qualitative classification result by using sentiment classification processing according to an embodiment of the present invention includes: step S201, step S202, step S203 and step S204.

[0090] Step S201: Perform feature extraction on each information vector in the information vector set to obtain a feature vector set of corresponding feature vectors.

[0091] The vectorized information phrase has a computer-readable representation. In order to realize the extraction of qualitative regulatory indicators, the technology of the present invention uses sentiment classification to classify regulatory information. In the example implementation, a deep neural network is used and specifically combined with a long short-term memory network to implement machine learning sentiment classification. The long short-term memory (LSTM) network can avoid the gradient vanishing or overshooting problems that may occur in the recurrent neural network (RNN). In this example implementation, an LSTM network is used for feature extraction, and the corresponding hidden vector can be obtained after the information vector in the information vector set is input into the trained LSTM network. The hidden vector after feature extraction can be linearized by the softmax function of the deep neural network to finally obtain a result vector with the same dimensional size as the number of categories of the final sentiment classification. In the example, the result vector has three dimensions, positive, negative and neutral.

[0092] Step S202: Determine an aspect embedding vector for a specific qualitative aspect that is desired to be extracted.

[0093] In order to realize the extraction of qualitative indicators based on regulatory events, the technology of the present invention needs to introduce aspect embedding vectors for the attributes of regulatory events in the qualitative analysis of sentences, wherein the sentiment classification results of the same sentence may be completely different for different aspect embedding vectors with different tendencies. The following is an example sentence that has undergone the preprocessing process described above to demonstrate the process of sentiment classification: "Kuailu Group promised to support platforms such as Jinlu and Dangtian, but Donghongqiao Microfinance Co., Ltd. has difficulties in repayment and its liquid assets are questionable, which should be explained immediately." After the following sentiment classification process, the corresponding sentiment classification results can be obtained for different aspects. For example, for the aspect "Kuailu Group", the sentiment classification result of the sentence can be "neutral", while for the aspect "Donghongqiao Microfinance Co., Ltd.", the sentiment classification result of the sentence can be "negative". Therefore, the determination of the aspect embedding vector is based on the qualitative aspects expected to be extracted, and different qualitative aspects correspond to corresponding aspect embedding vectors. In an example implementation, the aspect embedding vector trained based on the manually annotated reference corpus can be maintained by an expert knowledge base, and the parameters of the aspect embedding vector are continuously adjusted and optimized according to the supervision process in the subsequent recognition process.

[0094] Step S203: Use the determined aspect embedding vector together with each feature vector to determine an attention weight vector, and then determine a weighted feature vector set, wherein the attention weight vector indicates the corresponding importance of each information phrase in the sentence for the specific qualitative extraction target.

[0095] In order to enable sentiment analysis for different qualitative aspects to identify important parts of sentence elements based on the LSTM network, it is necessary to further add an attention mechanism to the machine learning model, so that important parts of sentences can be identified for different specific qualitative aspects. In an example implementation, each feature vector in the feature vector set can be used together with the aspect embedding vector to form the input of the activation function to obtain the activation output of the LTSM network. In some implementations, tanh can be selected as the activation function. In other implementations, sigmoid or other activation functions can also be selected as appropriate. Then, the obtained activation output is further input into the softmax function together with the corresponding feature vector to obtain the attention weight vector.

[0096] Reference below Figure 3 Let’s use example sentences to demonstrate the attention mechanism in sentiment classification. Figure 3The attention weights of different information phrases in the example sentence under the attention mechanism are shown. Specifically, the darker the color depth of the corresponding color block, the greater the attention weight of the corresponding information phrase. It can be understood that a sentence is split into two lines for the convenience of illustration, but in fact, the two lines of text together constitute the example sentence above. Figure 3 In the example, it can be seen that the information phrases "bottoming out", "repayment", "difficulties", "assets" and "doubtful" have higher attention weights than others, and these elements with higher attention weights actually correspond semantically to the aspects of the example "Kuailu Group" and "Donghongqiao Microfinance Co., Ltd." respectively. Thus, sentiment classification for different qualitative aspects can be effectively achieved through the attention mechanism. In the example implementation, the attention weight vector trained based on the manually annotated reference corpus can be maintained by the expert knowledge base, and the parameters of the attention weight vector are continuously adjusted and optimized according to the supervision process in the subsequent recognition process. After determining the attention weight vector, each feature vector in the feature vector set is weighted using the attention weight vector to obtain a weighted feature vector set.

[0097] Step S204: Determine, by means of a classification network, a conditional probability distribution representing a qualitative classification result using the weighted feature vector set together with the feature vector set, wherein the conditional probability distribution indicates whether the result tendency for the specific qualitative aspect is positive, negative or neutral.

[0098] The last layer in the deep neural network of the example implementation is a classification network, which includes a fully connected layer and a Softmax function at the output of the fully connected layer. In an example implementation, each weighted feature vector in the weighted feature vector set and each feature vector in the unweighted feature vector set are input into an activation function together to obtain a feature representation corresponding to the sentence. In some embodiments, tanh can be selected as the activation function. The feature representation is further converted into a result vector with the same dimensional size as the number of categories of the final sentiment classification through a linearization layer, wherein each dimension of the result vector has a real value. Finally, via the Softmax function, the result vector is converted into a conditional probability distribution, indicating which of the positive, negative and neutral tendencies of the sentence after qualitative classification for that particular qualitative aspect.

[0099] Continuing with the previous example, for the qualitative aspect "Kuailu Group", the attention weight vector of the example sentence is focused on the information phrase "bottoming out", and the corresponding sentiment classification result can be "neutral". For the aspect "Donghongqiao Microfinance Co., Ltd.", the attention weight vector of the example sentence is focused on the information phrases "difficulty in repayment" and "assets in doubt", and the corresponding sentiment classification result can be "negative".

[0100] In some embodiments, the example qualitative indicator extraction system can perform result output, and the output includes the specific qualitative aspect of the presentation statement and the result tendency for the specific qualitative aspect. The following Table 1 shows an example of the output result obtained with reference to the above example statement:

[0101] main body Donghongqiao Microfinance Co., Ltd. Public opinion Negative Public opinion keywords Difficulties in redemption, assets in doubt Related Subject Kuailu Group

[0102] Table 1

[0103] For example, the above sentiment analysis result information and the corresponding information phrase keywords can be displayed in the result output; in addition, the result output can include the output of qualitative indicators for the overall regulatory evaluation system. Continuing with the above example, the sentence is identified as a "public opinion review" regulatory event type after scene judgment. Therefore, in the regulatory evaluation system, for the "public opinion review" qualitative indicator associated with "Donghongqiao Microfinance Co., Ltd.", its "negative" classification result will result in a deduction in the subsequent evaluation. At the same time, "Kuailu Group" can be maintained as an associated word of the "public opinion review" qualitative indicator in the regulatory evaluation system.

[0104] In some further embodiments, after the execution result output, the example qualitative indicator extraction system can receive feedback on the qualitative classification result, the feedback including a result correctness indicator indicating whether the qualitative classification result of the sentence of the regulatory information is correct or not and a true result indicator. Based on the feedback, the deep neural network used for sentiment classification can be adjusted.

[0105] In some example embodiments, the adjustment of the data is for feedback that the qualitative classification result of the sentence indicating the supervision information is incorrect, and the corresponding real result is used to adjust the parameters and training parameters related to the sentiment classification process. The parameters and training parameters related to the classification process in the example qualitative indicator extraction system are jointly maintained as a model parameter file. Before the qualitative indicator extraction system processes the real data, the classification model is first trained using the model training data to obtain an initialized model parameter file. After that, the adjustment process is used to process the qualitative indicator extraction system classification model on the real data, and the feedback result of the manual review result is obtained. The supervised training process using the fine-tuning network to use the feedback result focuses on specific fine-tuning of historical data in the historical error record data. The parameter adjustment performed by the fine-tuning network mainly involves the parameters of the classification network, the parameters of the long short-term memory network, and the parameters of the aspect embedding vector. At the same time, the fine-tuning network also adjusts the parameters involved in the training process, such as the attenuation step, the attenuation coefficient, the sample capacity of the test iteration, and the length of the iteration interval. The fine-tuning process is continuously executed in an iterative manner, and the expected classification accuracy range is continuously approached or reached based on the real data feedback.

[0106] According to a second aspect of an embodiment of the present invention, a device for extracting qualitative indicators of regulatory events is provided.

[0107] Figure 4 is a schematic diagram of the main modules of the apparatus for extracting qualitative indicators of regulatory events according to an embodiment of the present invention, such as Figure 4 As shown, the device 400 for extracting qualitative indicators of regulatory events includes:

[0108] A regulatory information reading module 401 is configured to obtain regulatory information including text content, and to perform sentence segmentation on the text of the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method;

[0109] The supervision information parsing module 402 performs semantic annotation on the split sentences, thereby extracting an information phrase set, wherein the information phrase set includes one or more information phrases, and each information phrase is associated with an information dimension;

[0110] The supervision indicator extraction module 403 performs vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase;

[0111] A supervision indicator qualitative module 404 uses sentiment classification processing to obtain a qualitative classification result for the information vector set; and

[0112] Optionally, the supervision indicator output module 405 outputs the qualitative classification result, wherein the output includes the specific qualitative aspect of the presentation statement and the result tendency for the specific qualitative aspect; and

[0113] Optionally, the supervision indicator adjustment module 406 receives feedback on the qualitative classification result and performs adjustment based on the feedback.

[0114] According to a third aspect of an embodiment of the present invention, a system for extracting qualitative indicators of regulatory events is provided, comprising: a regulatory information database and a qualitative indicator extraction server using the method provided by the first aspect of an embodiment of the present invention.

[0115] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device for extracting qualitative indicators of regulatory events, including:

[0116] one or more processors;

[0117] a storage device for storing one or more programs,

[0118] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0119] According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method provided in the first aspect of the embodiment of the present invention is implemented.

[0120] Figure 5 An exemplary system architecture 500 is shown to which the method for extracting qualitative indicators of regulatory events or the apparatus for extracting qualitative indicators of regulatory events according to an embodiment of the present invention may be applied.

[0121] like Figure 5 As shown, system architecture 500 may include a supervisory information repository 501, a network 502, and a server 503. Network 502 is used to provide a medium for a communication link between supervisory information repository 501 and server 503. Network 502 may include various connection types, such as wired, wireless communication links, or optical fiber cables.

[0122] The user can use the supervision information base 501 to interact with the server 503 through the network 502 to receive or send messages, etc. The supervision information base 501 can have various database architectures and be deployed in various ways using various servers or computing devices.

[0123] The server 503 may be a system for implementing the extraction of qualitative indicators of regulatory events, for example, in response to a user's query, providing a qualitative indicator extraction result of the queried regulatory event.

[0124] It should be understood that Figure 5 The number of regulatory information repositories, networks, and servers in the embodiment is only illustrative. Any number of regulatory information repositories, networks, and servers may be provided according to implementation requirements.

[0125] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present invention. Figure 6 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0126] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0127] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0128] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0129] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments of the present invention can be implemented by software or hardware. The modules described can also be set in a processor, for example, can be described as: a processor includes: the regulatory information reading module, used to obtain regulatory information containing text content, perform sentence splitting on the text of the regulatory information, and obtain one or more sentences, wherein each sentence is associated with the corresponding regulatory event type through a machine learning method; the regulatory information parsing module, used to semantically annotate the split sentences, thereby extracting an information phrase set, wherein the information phrase set contains one or more information phrases, and each information phrase is associated with an information dimension; the regulatory indicator extraction module, used to perform vectorization processing on each information phrase in the information phrase set, and obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase; and the regulatory indicator qualitative module, used for obtaining a qualitative classification result for the information vector set using sentiment classification processing. Among them, the names of these modules do not constitute a limitation on the module itself under certain circumstances.

[0132] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining regulatory information containing text content; performing sentence splitting on the text of the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method; semantically annotating the split sentences to extract an information phrase set, wherein the information phrase set contains one or more information phrases, and each information phrase is associated with an information dimension; performing vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase; and using sentiment classification processing to obtain a qualitative classification result for the information vector set.

[0133] The present invention can make full use of the advantages of various information by widely collecting regulatory information from relevant service databases, public media, public opinion and other data sources that may involve financial regulatory information. At the same time, it can not only combine the original dedicated information sources but also cover a wide range of network resources so as to actively reach information sources that have not been involved in previous financial monitoring. Compared with the information collected by manual regulatory means, it can achieve truly comprehensive and multi-dimensional regulatory coverage. The qualitative indicator extraction technology of the present invention not only achieves docking with the overall regulatory evaluation system through qualitative indicator matching based on scenario judgment of regulatory events, but also integrates with quantitative indicator evaluation in a unified evaluation framework; at the same time, the type matching of regulatory events performed on regulatory information also enables the qualitative indicator extraction process to make full use of different scenario attributes, thereby greatly improving the algorithm effect and efficiency of extraction. In addition, the qualitative indicator extraction of the present invention, by introducing sentiment classification based on deep learning, combines data intelligence, expert knowledge base and regulatory indicators in a tool-based manner to achieve high precision and high versatility.

[0134] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting qualitative indicators of regulatory events, characterized in that: include: Obtain regulatory information including text content; Perform sentence splitting on the text of the regulatory information to obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method; Semantically annotating the split sentences, thereby extracting a set of information phrases, wherein the set of information phrases includes one or more information phrases, and each information phrase is associated with an information dimension; wherein the semantic annotation process is an annotation process based on the event type determined by scene judgment; Performing vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase, and the dimension of each information vector in the vector space is defined based on the regulatory event type of the statement to which the information phrase belongs; and For the information vector set, using sentiment classification processing to obtain a qualitative classification result; The sentence splitting of the text of the regulatory information to obtain one or more sentences includes: integrating a rule-based model to extract features, and implementing sentence end judgment of sentence splitting through a classifier; The vectorization processing further includes: performing normalization processing on each information vector in the information vector set, wherein the normalization processing includes classifying mutually similar vectors in the information vectors and associating them as similar information vectors; The sentiment classification process is a classification process based on a deep neural network, including: for each information vector in the information vector set, performing feature extraction to obtain a feature vector set of corresponding feature vectors; determining an aspect embedding vector for a specific qualitative aspect that is desired to be extracted; using the determined aspect embedding vector together with each feature vector to determine an attention weight vector, and then determining a weighted feature vector set, wherein the attention weight vector indicates the corresponding importance of each information phrase in the sentence for the specific qualitative extraction target; and using the weighted feature vector set together with the feature vector set to determine, through a classification network, a conditional probability distribution representing a qualitative classification result, wherein the conditional probability distribution indicates whether the result for the specific qualitative aspect tends to be positive, negative, or neutral.

2. The method according to claim 1, characterized in that Associating each statement with a corresponding regulatory event type through a machine learning approach includes: determining the corresponding regulatory event type through a machine learning-based scenario classification process.

3. The method according to claim 2, characterized in that The machine learning-based scene classification process further includes: using a machine learning classification method based on artificial corpus annotated data to perform scene classification on each sentence.

4. The method according to claim 3, characterized in that The machine learning classification method is a support vector machine classification algorithm, and the artificial corpus annotation data is maintained through an expert knowledge base, including predefined trigger word data for financial supervision.

5. The method according to claim 4, characterized in that The regulatory event type is a classification based on regulatory event scenarios, including: industry news, on-site inspections, public opinion reviews, and government affairs events. The text of the regulatory information achieves matching of the qualitative classification results with qualitative regulatory indicators based on regulatory event scenarios through the classification of the regulatory event scenarios, so that comprehensive regulatory evaluation can be further achieved in combination with quantitative regulatory indicators.

6. The method according to claim 1, characterized in that The semantic annotation includes: performing BIOES semantic annotation through Chinese named entity recognition.

7. The method according to claim 6, characterized in that BIOES semantic annotation through Chinese named entity recognition is performed based on manual semantic annotation data maintained in an expert knowledge base, wherein the one or more information phrases in the annotated sentence have associated information phrase attribution indicators and information dimension indicators, each character in each information phrase corresponds to the information phrase attribution indicator of the information phrase, and the information dimension indicator of each information phrase indicates the information dimension of the information phrase.

8. The method according to claim 1, characterized in that The vectorization processing is to use the financial regulatory supplementary corpus to perform incremental training on word vector encoding.

9. The method according to claim 8, characterized in that The vectorization process further includes: mapping each information phrase to a word vector having a vector dimension in a vector dimension set through the word vector encoding.

10. The method according to claim 9, characterized in that Information phrases corresponding to different regulatory event types have corresponding subsets of the set of vector dimensions.

11. The method according to claim 1, characterized in that: Further including: Before executing the vectorization process, the information phrase set is preprocessed, and the preprocessing includes: word segmentation, stop word removal, and character encoding conversion between simplified and traditional Chinese characters.

12. The method according to claim 1, characterized in that The feature extraction is achieved through a long short-term memory network, wherein for each input information vector, the long short-term memory network generates a corresponding implicit feature vector as output for processing by the next layer of the deep neural network.

13. The method according to claim 1, characterized in that The classification network includes a fully connected layer and a Softmax function at its output, wherein the weighted feature vector set together with the feature vector set are processed through an activation function in the fully connected layer to obtain a result representation, and the result representation is converted into the conditional probability distribution through the Softmax function, wherein the activation function is a tanh function.

14. The method according to claim 1, characterized in that The specific qualitative aspect is based on the regulatory event type, and the method further includes: outputting the qualitative classification result, wherein the output includes presenting the specific qualitative aspect of the statement and a result tendency for the specific qualitative aspect.

15. The method according to claim 1, characterized in that Further including: Feedback is received for the qualitative classification results, and adjustments are performed based on the feedback.

16. The method according to claim 15, characterized in that The adjustments include: receiving the feedback, wherein the feedback includes a result correctness indicator indicating whether a qualitative classification result of the sentence of the regulatory information is correct or not and a true result indicator; In response to feedback indicating that the qualitative classification result is wrong, the corresponding true result is used to adjust parameters related to the sentiment classification process and training parameters.

17. The method according to claim 16, characterized in that Parameters related to the sentiment classification process include one or more of the following: LSTM network parameters; Parameters of the aspect embedding vector; and Parameters of the classification network.

18. A device for extracting qualitative indicators of regulatory events, characterized in that: include: Regulatory information reading module, regulatory information parsing module, regulatory indicator extraction module and regulatory indicator qualitative module; among them, The regulatory information reading module is used to obtain regulatory information containing text content, perform sentence splitting on the text of the regulatory information, and obtain one or more sentences, wherein each sentence is associated with a corresponding regulatory event type through a machine learning method; The regulatory information parsing module is used to semantically annotate the split sentences, so as to extract a set of information phrases, wherein the set of information phrases includes one or more information phrases, and each information phrase is associated with an information dimension; wherein the semantic annotation process is a annotation process based on the event type determined by scene judgment; The regulatory indicator extraction module is used to perform vectorization processing on each information phrase in the information phrase set to obtain an information vector set, wherein each information vector in the information vector set corresponds to each vectorized information phrase, and the dimension of each information vector in the vector space is defined based on the regulatory event type of the statement to which the information phrase belongs; and The regulatory indicator qualitative module is used to obtain a qualitative classification result by using sentiment classification processing for the information vector set; The regulatory information reading module is specifically used to: integrate a rule-based model to extract features and implement sentence end judgment for sentence splitting through a classifier; The supervision indicator extraction module is also used to: perform normalization processing on each information vector in the information vector set, wherein the normalization processing includes classifying mutually similar vectors in the information vectors and associating them as similar information vectors; The regulatory indicator qualitative module is specifically used to: perform feature extraction on each information vector in the information vector set to obtain a feature vector set of corresponding feature vectors; determine an aspect embedding vector for a specific qualitative aspect that is desired to be extracted; use the determined aspect embedding vector together with each feature vector to determine an attention weight vector, and then determine a weighted feature vector set, wherein the attention weight vector indicates the corresponding degree of importance of each information phrase in the sentence for the specific qualitative extraction target; and use the weighted feature vector set together with the feature vector set to determine a conditional probability distribution representing a qualitative classification result through a classification network, wherein the conditional probability distribution indicates whether the result for the specific qualitative aspect tends to be positive, negative, or neutral.

19. The device according to claim 18, characterized in that It further includes a regulatory indicator output module and a regulatory indicator adjustment module; wherein, The supervision indicator output module is used to output the qualitative classification result, wherein the output includes the specific qualitative aspect of the presentation statement and the result tendency for the specific qualitative aspect; and The supervision indicator adjustment module is used to receive feedback on the qualitative classification result and perform adjustment based on the feedback.

20. An electronic device for extracting qualitative indicators of regulatory events, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1 to 17.

21. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images