An opinion risk reasoning method, device and equipment and readable storage medium

Abstract

This application provides a method, apparatus, device, and readable storage medium for reasoning about public opinion risks. It obtains public opinion information by annotating real-time acquired internet information using a pre-trained deep learning model, and then performs reasoning based on this information to derive public opinion risk information. The public opinion risk information and the original information are then stored together. When search keywords are input, no interactive matching is required, and relevant information can be displayed quickly and accurately. The public opinion information derived through corpus reasoning is public opinion risk information and does not alter the original text content, ensuring the accuracy of the public opinion risk information. Therefore, this application is suitable for relatively complex needs, significantly improving the accuracy of judging public opinion risks through machine reading comprehension and the response speed of real-time question answering.

Description

Technical Field

[0001] This application relates to the technical field of computer technology, and in particular to a reasoning method, apparatus, device, and readable storage medium for public opinion risk. Background Technology

[0002] Machine Reading Comprehension (MRC) is a technology that uses algorithms to enable computers to understand the semantics of articles and answer related questions. It's a task that tests a machine's ability to understand natural language by having it answer questions based on a given context. Because both the articles and questions are in human language form, MRC falls under the category of Natural Language Processing (NLP) and is one of its newest and most popular research areas. There are four common tasks in MRC applications: cloze tests, multiple choice, span extraction, and free-response questions.

[0003] Current technologies for solving machine reading comprehension tasks primarily employ free-response question-and-answer methods. Free-response question-and-answer involves providing a passage C and a question Q, where the correct answer 'a' may not always be a subsequence of passage C. or The free-response task predicts the correct answer 'a' to question Q by maximizing the conditional probability P(a|C,Q). Free-response question answering is the most challenging of the four tasks and also the one of the most interesting and closely watched by industry. The answer format in free-response tasks is very flexible, effectively testing natural language understanding. However, by organizing answers to better suit human reading comprehension, the original text is altered, sometimes resulting in significant differences from the original. Furthermore, the free-response method encodes research data and questions during answer organization and then performs interactive matching, thus failing to meet the requirements for real-time question-and-answer response speed. Especially in the field of public opinion risk information technology, the accuracy requirements are extremely high; even slight changes to the meaning of the original text can cause problems in subsequent applications. The inability to meet real-time question-and-answer response speed is also detrimental to user experience.

[0004] Therefore, how to improve the accuracy of judging public opinion risks through machine reading comprehension and the response speed of real-time question answering are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned problems, this application provides a method, apparatus, device, and readable storage medium for reasoning about public opinion risks, aiming to improve the accuracy of judging public opinion risks through machine reading comprehension and the response speed of real-time question answering. The embodiments of this application disclose the following technical solutions:

[0006] Firstly, this application provides a method for reasoning about public opinion risk, the method comprising:

[0007] Real-time acquisition of text corpora from the Internet;

[0008] The text corpus is input into a pre-trained deep learning model, the text corpus is annotated to obtain public opinion information, and the public opinion information is used for corpus reasoning to obtain public opinion risk information.

[0009] Store the public opinion risk information and the corresponding text corpus;

[0010] In response to user search commands, relevant public opinion risk information and corresponding text corpora are retrieved based on the user's input.

[0011] Optionally, the deep learning model is specifically trained through the following steps:

[0012] Input a sample text corpus of length n; where n is a positive integer;

[0013] The sample text corpus data is word-folded to obtain vectors of shallow semantic encoding information and positional information about the sample text corpus.

[0014] The vector is fed into the first encoding layer of the deep learning model, and the semantic encoding result of the sample text corpus data is obtained through the pre-trained model BERT; the pre-trained model BERT is composed of multiple layers of self-attention network layers and feedforward neural network layers stacked together.

[0015] The semantic encoding result is passed through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted sample text location information;

[0016] The sample matrix result value of the sample text corpus data is obtained by calculating the matrix; the sample matrix result value represents whether the sample text in the sample text location information is the public opinion risk information;

[0017] Determine whether the difference between the sample matrix result value and the preset result value of manual annotation is less than a preset deviation value;

[0018] If the difference is not less than the preset deviation value, then the gradient descent algorithm is used to update the iterative model parameters;

[0019] If the difference is less than the preset deviation value, then model training is stopped.

[0020] Optionally, corpus reasoning can be performed on the public opinion information to obtain public opinion risk information, including:

[0021] The deep learning model is used to obtain a matrix of the number of events and the extracted text location information. The matrix is ​​then calculated to obtain a matrix result value, and it is determined whether the matrix result value is greater than a preset matrix result value.

[0022] If the matrix result value is greater than the preset matrix result value, then the public opinion information is inferred to be public opinion risk information;

[0023] If the matrix result value is less than or equal to the preset matrix result value, it is inferred that the public opinion information is not public opinion risk information.

[0024] Optionally, the real-time acquisition of text corpora from the Internet includes:

[0025] Build a distributed crawler architecture to obtain the latest text corpus from various financial news websites, company announcements, and government announcements in real time.

[0026] Optionally, the reasoning method for public opinion risk also includes:

[0027] The publication time of the text corpus and the relevant company of the text corpus are stored.

[0028] Secondly, this application provides a reasoning device for public opinion risk, the device comprising:

[0029] The acquisition module is used to acquire text corpora from the Internet in real time;

[0030] The public opinion risk assessment module is used to input the text corpus into a pre-trained deep learning model, annotate the text corpus to obtain public opinion information, and perform corpus inference on the public opinion information to obtain public opinion risk information.

[0031] The storage module is used to store the public opinion risk information and the corresponding text corpus.

[0032] The retrieval module is used to respond to user retrieval commands and retrieve relevant public opinion risk information and corresponding text corpora based on the content entered by the user.

[0033] Optionally, the reasoning device for public opinion risk further includes a model training module, used for:

[0034] Input a sample text corpus of length n; where n is a positive integer;

[0035] The sample text corpus data is word-folded to obtain vectors of shallow semantic encoding information and positional information about the sample text corpus.

[0036] The vector is fed into the first encoding layer of the deep learning model, and the semantic encoding result of the sample text corpus data is obtained through multiple self-attention network layers and feedforward network layers.

[0037] The semantic encoding result is passed through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted sample text location information;

[0038] The sample matrix result value of the sample text corpus data is obtained by calculating the matrix; the sample matrix result value represents whether the sample text in the sample text location information is the public opinion risk information;

[0039] Determine whether the difference between the sample matrix result value and the preset result value of manual annotation is less than a preset deviation value;

[0040] If the difference is not less than the preset deviation value, then the gradient descent algorithm is used to update the iterative model parameters;

[0041] If the difference is less than the preset deviation value, then model training is stopped.

[0042] Optionally, the public opinion risk assessment module includes:

[0043] The inference unit is used to obtain a matrix of the number of events and the extracted text location information through the deep learning model, calculate the matrix result value, and determine whether the matrix result value is greater than a preset matrix result value. The matrix result value represents whether the text in the text location information is the public opinion risk information. If the matrix result value is greater than the preset matrix result value, it is inferred that the public opinion information is public opinion risk information. If the matrix result value is less than or equal to the preset matrix result value, it is inferred that the public opinion information is not public opinion risk information.

[0044] Thirdly, this application provides a reasoning device for public opinion risk, including: a memory and a processor;

[0045] The memory is used to store programs;

[0046] The processor is used to execute the program and implement each step of the above-mentioned reasoning method for public opinion risk.

[0047] Fourthly, this application provides a reasoning storage medium for public opinion risk, on which a computer program is stored. When the computer program is executed by a processor, it implements the various steps of the aforementioned reasoning method for public opinion risk.

[0048] Compared with the prior art, this application has the following beneficial effects:

[0049] This application provides a method, apparatus, device, and readable storage medium for inferring public opinion risk. It obtains public opinion information by annotating real-time acquired internet information using a pre-trained deep learning model. Then, it performs corpus inference on this public opinion information to obtain public opinion risk information. After obtaining the public opinion risk information, its corresponding original information is stored. Because the public opinion risk information and the original information are stored together, when users input search keywords that involve the original information content, no interactive matching is required, and relevant information can be displayed quickly and accurately, improving response speed. Furthermore, this application infers from public opinion information. To ensure the accuracy of the results, the deep learning model only performs corpus inference on the public opinion information during processing. The public opinion information obtained through corpus inference is public opinion risk information and does not change the original text content, ensuring the accuracy of the public opinion risk information. Therefore, this application is suitable for relatively complex needs and significantly improves the accuracy of judging public opinion risk through machine reading comprehension and the response speed of real-time question answering. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a public opinion risk reasoning method provided in this application embodiment;

[0052] Figure 2 A flowchart illustrating a method for training a deep learning model, as provided in an embodiment of this application;

[0053] Figure 3 A flowchart illustrating another method for reasoning about public opinion risks provided in this application embodiment;

[0054] Figure 4 This is a schematic diagram of a public opinion risk inference device provided in an embodiment of this application. Detailed Implementation

[0055] As described earlier, current technologies for solving machine reading comprehension tasks primarily rely on free-response question-and-answer methods. However, while these methods organize answers to better suit human reading comprehension, they alter the original text, sometimes resulting in significant discrepancies. Furthermore, the question-and-answer approach encodes research data and questions during answer organization and then performs interactive matching, thus failing to meet the demand for real-time question-and-answer response speeds. This is particularly critical in the field of public opinion risk information, where the accuracy of information is paramount. Even slight alterations to the original text can cause problems for subsequent applications, and the inability to meet real-time question-and-answer response speeds is detrimental to user experience. Therefore, improving the accuracy of machine reading comprehension in assessing public opinion risks and the speed of real-time question-and-answer responses is a pressing technical challenge for those skilled in the art.

[0056] In view of this, this application provides a method, apparatus, device, and readable storage medium for reasoning about public opinion risks. It obtains public opinion information by annotating real-time acquired internet information using a pre-trained deep learning model, then performs corpus reasoning on this information to obtain public opinion risk information, and finally stores the corresponding original information. Because the public opinion risk information and the original information are stored together, when users input search keywords that involve the original information content, no interactive matching is required, and relevant information can be displayed quickly and accurately. Furthermore, this application reasons about public opinion information; to ensure the accuracy of the results, the deep learning model only performs corpus reasoning on the public opinion information during processing. The public opinion information obtained through corpus reasoning is public opinion risk information and does not change the original text content, thus ensuring the accuracy of the public opinion risk information. Therefore, this application is suitable for relatively complex needs and significantly improves the accuracy of judging public opinion risks through machine reading comprehension and the response speed of real-time question answering.

[0057] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0058] See Figure 1 The figure illustrates a method for reasoning about public opinion risk provided in an embodiment of this application, and its flowchart is shown below. Figure 1 As shown, it includes:

[0059] S101: Real-time acquisition of text corpora from the Internet.

[0060] In this embodiment, automated collection of internet data enables the construction of a web crawler architecture to quickly obtain real-time information. By analyzing text corpora from the internet, the timeliness of the acquired data can be guaranteed, thus allowing the technical solution of this application to quickly extract the information most desired by the user from the latest information.

[0061] S102: Input the text corpus into a pre-trained deep learning model, annotate the text corpus to obtain public opinion information, and perform corpus reasoning on the public opinion information to obtain public opinion risk information.

[0062] As an example, in this embodiment of the application, the corpus annotation is to annotate the text corpus as real financial news. First, the company entity in the news and the type of risk event related to the company are identified. Then, the text in the article that can prove the occurrence of the risk event is annotated. The annotated content can be continuous sentences or disjointed paragraphs.

[0063] Machine reading comprehension models based on deep learning can accurately analyze each sentence, process the latest data in real time, achieve high accuracy, eliminate unimportant parts in massive amounts of data, and perform fine-grained data retrieval, greatly improving accuracy.

[0064] S103: Store public opinion risk information and the corresponding text corpus.

[0065] In this embodiment of the application, the public opinion risk information and the corresponding text corpus are stored offline. Elastic Search is used as the storage tool to save the public opinion risk information and the corresponding text corpus together.

[0066] Elastic Search is a distributed technology for searching and analyzing massive amounts of data. It can be used as a search engine in various scenarios, including e-commerce websites, portals, and enterprise IT systems, and can also be used for near real-time data analysis of massive datasets. Elastic Search features distributed file storage, where every field is indexed and searchable, and a distributed real-time analytics search engine with near real-time, second-level response times for massive datasets.

[0067] S104: In response to the user's search command, retrieve relevant public opinion risk information and corresponding text corpus based on the content entered by the user.

[0068] In this embodiment, the user inputs a risk factor, calls the ES (Elastic Search) interface, retrieves relevant public opinion risk information and corresponding text corpus based on the user's input risk factor, and displays it on a visual operation interface for the user to view.

[0069] This application provides a method for reasoning about public opinion risks. It obtains public opinion information by annotating real-time internet information using a pre-trained deep learning model, then performs reasoning on this information to derive public opinion risk information. After obtaining the risk information, its corresponding original information is stored. Because the risk information and original information are stored together, when a user inputs search keywords that involve the original information content, no interactive matching is required, and relevant information can be displayed quickly and accurately. Furthermore, this application reasons from public opinion information; to ensure the accuracy of the results, the deep learning model only performs reasoning on the public opinion information. The public opinion information derived through reasoning is the risk information, and the original content is not altered, thus ensuring the accuracy of the risk information. Therefore, this application is suitable for relatively complex needs, significantly improving the accuracy of judging public opinion risks through machine reading comprehension and the response speed of real-time question answering.

[0070] See Figure 2 The figure shown is a flowchart illustrating a method for training a deep learning model according to an embodiment of this application. Figure 2 As shown, it includes:

[0071] S201: Input a sample text corpus of length n.

[0072] Where n is a positive integer, which can be set by the user.

[0073] In this embodiment of the application, the input sample text corpus is a sequence of text with a length of n, in the form of T:[t1,t2,...tn].

[0074] S202: Perform word folding on the sample text corpus data to obtain vectors of shallow semantic encoding information and positional information about the sample text corpus.

[0075] In this embodiment, the sample text corpus data is word-folded. After word folding, a vector of shallow semantic encoding information and positional information about the sample text corpus is obtained. This vector is in the form of E:[e1,e2,...en].

[0076] S203: Pass the vector into the first encoding layer of the deep learning model, and obtain the semantic encoding result of the sample text corpus data through the pre-trained model BERT.

[0077] The vector E is fed into the first encoding layer of the deep learning model. By training the BERT model, the semantic encoding result of the sample text corpus data is obtained. The semantic encoding result is in the form of C:[c1,c2,...cn].

[0078] BERT is a pre-trained model composed of multiple layers of self-attention network and feedforward neural network. The calculation method of the attention mechanism of the multiple self-attention network is Equation (1), the feedforward neural network is Equation (2), and the final calculation result is Equation (3).

[0079]

[0080] FFN(X)=max(0,XW1+b1)W2+b2 (2)

[0081] C i+1 =FFN(LayerNorm(Attentioin(W i q C i W i k C i W i v C i ))) (3)

[0082] Where dk is the dimension of the hidden layer of the model, W and b are trainable parameters of the model, C0 is the word vector E, the final calculation result is C, and Q, K, and V are the products of different trainable parameters W and C.

[0083] S204: Pass the semantic encoding results through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted text location information.

[0084] Then, through the second encoding layer of the deep learning model, an m*n*n dimension matrix P is obtained, where m is the number of events and n is the length of the sequence, which is used to represent the extracted text position information.

[0085] The calculation method in the second encoding layer of the deep learning model is as follows: (3)-(5)

[0086] Q1 = W q1 C+b q1 (3)

[0087] Q2 = W q2 C+b q2 (4)

[0088] P = Q1 T Q2 (5)

[0089] Where W and b are both trainable parameters of the model, the final calculation result is C, and the final m*n*n dimension matrix is ​​P.

[0090] S205: Calculate the matrix to obtain the sample matrix result value of the sample text corpus data.

[0091] The sample matrix result value represents whether the text in the text location information is public opinion risk information.

[0092] After model calculation, matrix P is obtained. Matrix P can be used to decode the specific extracted text paragraphs. Matrix P(k,i,j) represents the score of whether the text at positions i to j in the article can be used as evidence for event k. If P(k,i,j)>0.5, it means that the part from i to j in the article is the desired result and can be used as evidence for the occurrence of event k. Otherwise, it is discarded.

[0093] S206: Determine whether the difference between the sample matrix result value and the preset result value of manual annotation is less than a preset deviation value. If the difference is less than the preset deviation value, proceed to step S208; if the difference is not less than the preset deviation value, proceed to step S207.

[0094] The F1 score is calculated based on the results and used as the basis for evaluating the model's effectiveness.

[0095] The calculation method for evaluating model performance is as follows:

[0096]

[0097]

[0098]

[0099] Where p is precision, r is recall, TP is the number of true positive samples (the number of correctly predicted samples), FP is the number of false positive samples (the number of incorrect predictions), FN is the number of false negative samples (the number of samples that should have been predicted as positive but were not predicted by the model), and F1 is a metric for evaluating the model, ranging from 0 to 1, with the closer to 1 indicating a better model performance.

[0100] If the F1 value does not meet expectations, that is, the matrix result value is not less than the preset deviation value compared with the preset result value of manual annotation, then proceed to step S207.

[0101] S207: Update the iterative model parameters using the gradient descent algorithm.

[0102] The model parameters are updated iteratively using the loss function of the model as follows, until the difference between the sample matrix result value and the preset result value of manual annotation is less than the preset deviation value:

[0103] loss = y true log y pred +(1-y true log(1-y) pred (9)

[0104] Where y pred为 The sample matrix result value of the model, y true为 Preset result values ​​for manual annotation.

[0105] S208: Stop model training.

[0106] This application provides a deep learning model method for reasoning about public opinion risk. The method is trained with a large amount of sample text data, and the data results of each training are compared with preset results values ​​labeled by humans. The gradient descent algorithm is used to update and iterate the model parameters to achieve the standard, thereby improving the accuracy of the deep learning model.

[0107] See Figure 3 The figure illustrates another method for reasoning about public opinion risks provided in this application embodiment, and its flowchart is shown below. Figure 3 As shown, it includes:

[0108] S301: Build a distributed crawler architecture to obtain the latest text corpus from various financial news websites, company announcements, and government announcements in real time.

[0109] Building a distributed web crawler architecture combines multiple hosts to work together on a single crawling task, significantly improving crawling efficiency. This ensures the timeliness of information and increases the speed of acquiring the latest text corpora.

[0110] S302: Input the text corpus into a pre-trained deep learning model, and perform corpus annotation on the text corpus to obtain public opinion information.

[0111] S303: Obtain a matrix of the number of events and the extracted text location information through a deep learning model, calculate the matrix result value, and determine whether the matrix result value is greater than the preset matrix result value.

[0112] The articles collected by the crawler are used as input, and the model calculates to obtain matrix P. The specific extracted text paragraphs can be decoded through matrix P. The matrix result value represents the score of whether the text at positions i to j in the article can be used as the argument for event k.

[0113] If the matrix result value is greater than the preset matrix result value, then the public opinion information is inferred to be public opinion risk information.

[0114] If the matrix result value is less than or equal to the preset matrix result value, it is inferred that the public opinion information is not public opinion risk information.

[0115] S304: Store public opinion risk information and corresponding text corpora; store the publication time of the text corpora and the relevant companies.

[0116] The public opinion risk information, the corresponding text corpus, the publication time of the text corpus, and the related companies are stored offline. Elastic Search is used as the storage tool, and the public opinion risk information, the corresponding text corpus, the publication time of the text corpus, and the related companies are provided as an index for the retrieval part.

[0117] S305: In response to a user's search command, retrieve relevant public opinion risk information and corresponding text corpus based on the user's input.

[0118] When a user inputs a risk factor, the Elastic Search (ES) interface is invoked. Based on the user's input risk factor, relevant public opinion risk information and corresponding text corpora are retrieved and displayed on a visual interface for the user to view.

[0119] This application provides a method for inferring public opinion risk. It establishes a distributed crawler architecture to acquire real-time text corpora, such as the latest text corpora from various financial news websites, company announcements, and government announcements. This improves the timeliness of the text corpora. Public opinion risk information is calculated using a pre-trained deep learning model. Because the public opinion risk information is stored together with the original information, when users input search keywords that involve the original information content, no interactive matching is required; relevant information can be displayed quickly and accurately. Furthermore, this application infers from public opinion information. To ensure the accuracy of the results, the deep learning model only performs corpus inference on the public opinion information during processing. The public opinion information inferred from the corpus is public opinion risk information and does not change the original text content, thus ensuring the accuracy of the public opinion risk information. Therefore, this application is suitable for relatively complex needs, significantly improving the accuracy of judging public opinion risk through machine reading comprehension and the response speed of real-time question answering.

[0120] Furthermore, this application also provides a corresponding reasoning device for public opinion risk. See [link / reference] Figure 4 The figure shows a schematic diagram of a public opinion risk reasoning device provided in an embodiment of this application. As can be seen from the figure, the public opinion risk reasoning device includes at least the following modules: an acquisition module, a public opinion risk judgment module, a storage module, and a retrieval module.

[0121] The acquisition module 401 is used to acquire text corpora from the Internet in real time.

[0122] The public opinion risk assessment module 402 is used to input text corpus into a pre-trained deep learning model, annotate the text corpus to obtain public opinion information, and perform corpus inference on the public opinion information to obtain public opinion risk information.

[0123] Storage module 403 is used to store public opinion risk information and the corresponding text corpus.

[0124] The retrieval module 404 is used to respond to the user's retrieval command and retrieve relevant public opinion risk information and corresponding text corpus based on the content entered by the user.

[0125] Optionally, the public opinion risk inference device also includes a model training module for:

[0126] Input a sample text corpus of length n, where n is a positive integer; perform word folding on the sample text corpus to obtain a vector of shallow semantic encoding information and positional information about the sample text corpus;

[0127] The vector is fed into the first encoding layer of the deep learning model, and the semantic encoding result of the sample text corpus data is obtained through the pre-trained model BERT; the pre-trained model BERT is composed of multiple layers of self-attention network layers and feedforward neural network layers stacked together.

[0128] The semantic encoding results are passed through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted text location information. The matrix is ​​then used to calculate the matrix result value of the sample text corpus data. The matrix result value represents whether the text in the text location information is public opinion risk information.

[0129] Check whether the calculated matrix result value is less than the preset deviation value of the manually labeled result value;

[0130] If the deviation is not less than the preset deviation value, then the gradient descent algorithm is used to update the iterative model parameters;

[0131] If the deviation is less than the preset value, model training will stop.

[0132] Optional, the public opinion risk assessment module includes:

[0133] The inference unit is used to obtain a matrix of the number of events and the extracted text location information through a deep learning model, calculate the matrix result value, and determine whether the matrix result value is greater than a preset matrix result value. If the matrix result value is greater than the preset matrix result value, the public opinion information is inferred to be public opinion risk information; if the matrix result value is less than or equal to the preset matrix result value, the public opinion information is inferred to be public opinion risk information.

[0134] Optionally, the acquisition module 401 is specifically used to build a distributed crawler architecture to acquire the latest text corpus from various financial news websites, company announcements, and government announcements in real time.

[0135] Optionally, the storage module 403 is also used to store the publication time of the text corpus and the related companies of the text corpus.

[0136] This application provides a reasoning device for public opinion risk. It obtains public opinion information by annotating real-time internet information using a pre-trained deep learning model, then performs reasoning on this information to derive public opinion risk information. After obtaining the risk information, the corresponding original information is stored. Because the risk information and the original information are stored together, when users input search keywords that involve the original information content, no interactive matching is required, and relevant information can be displayed quickly and accurately. Furthermore, this application performs reasoning on public opinion information. To ensure the accuracy of the results, the deep learning model only performs reasoning on the public opinion information during processing. The public opinion information derived through reasoning is the risk information and does not alter the original content, thus guaranteeing the accuracy of the risk information. Therefore, this application is suitable for relatively complex needs, significantly improving the accuracy of judging public opinion risk through machine reading comprehension and the response speed of real-time question answering.

[0137] This application also provides corresponding devices and computer-readable storage media for implementing the solutions provided in this application.

[0138] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the various steps of the public opinion risk reasoning described in any embodiment of this application.

[0139] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.

[0140] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0141] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0142] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0143] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A reasoning method for public opinion risk, characterized in that, The method includes: Real-time acquisition of text corpora from the Internet; The text corpus is input into a pre-trained deep learning model, the text corpus is annotated to obtain public opinion information, and the public opinion information is used for corpus reasoning to obtain public opinion risk information. Store the public opinion risk information and the corresponding text corpus; In response to user search commands, relevant public opinion risk information and corresponding text corpora are retrieved based on the user's input. The deep learning model is specifically trained through the following steps: Input a sample text corpus of length n; where n is a positive integer; The sample text corpus data is word-folded to obtain vectors of shallow semantic encoding information and positional information about the sample text corpus. The vector is fed into the first encoding layer of the deep learning model, and the semantic encoding result of the sample text corpus data is obtained through the pre-trained model BERT; the pre-trained model BERT is composed of multiple layers of self-attention network layers and feedforward neural network layers stacked together. The semantic encoding result is passed through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted sample text location information; The sample matrix result value of the sample text corpus data is obtained by calculating the matrix; the sample matrix result value represents whether the sample text in the sample text location information is the public opinion risk information; Determine whether the difference between the sample matrix result value and the preset result value of manual annotation is less than a preset deviation value; If the difference is not less than the preset deviation value, then the gradient descent algorithm is used to update the iterative model parameters; If the difference is less than the preset deviation value, then model training is stopped.

2. The method according to claim 1, characterized in that, The process of performing corpus reasoning on the public opinion information to obtain public opinion risk information includes: The deep learning model generates a matrix containing the number of events and the extracted text location information. The matrix is ​​then calculated to obtain a matrix result value, and it is determined whether the matrix result value is greater than a preset matrix result value. The matrix result value represents whether the text in the text location information is the public opinion risk information. If the matrix result value is greater than the preset matrix result value, then the public opinion information is inferred to be public opinion risk information; If the matrix result value is less than or equal to the preset matrix result value, it is inferred that the public opinion information is not public opinion risk information.

3. The method according to claim 1, characterized in that, The real-time acquisition of text corpora from the Internet includes: Build a distributed crawler architecture to obtain the latest text corpus from various financial news websites, company announcements, and government announcements in real time.

4. The method according to claim 1, characterized in that, The method further includes: The publication time of the text corpus and the relevant company of the text corpus are stored.

5. A reasoning device for public opinion risk, characterized in that, The device includes: The acquisition module is used to acquire text corpora from the Internet in real time; The public opinion risk assessment module is used to input the text corpus into a pre-trained deep learning model, annotate the text corpus to obtain public opinion information, and perform corpus inference on the public opinion information to obtain public opinion risk information. The storage module is used to store the public opinion risk information and the corresponding text corpus. The retrieval module is used to respond to user retrieval commands and retrieve relevant public opinion risk information and corresponding text corpora based on the content entered by the user. The model training module is specifically used for: Input a sample text corpus of length n; where n is a positive integer; The sample text corpus data is word-folded to obtain vectors of shallow semantic encoding information and positional information about the sample text corpus. The vector is fed into the first encoding layer of the deep learning model, and the semantic encoding result of the sample text corpus data is obtained through the pre-trained model BERT; the pre-trained model BERT is composed of multiple layers of self-attention network layers and feedforward neural network layers stacked together. The semantic encoding result is passed through the second encoding layer of the deep learning model to obtain a matrix of the number of events and the extracted sample text location information; The sample matrix result value of the sample text corpus data is obtained by calculating the matrix; the sample matrix result value represents whether the sample text in the sample text location information is the public opinion risk information; Determine whether the difference between the sample matrix result value and the preset result value of manual annotation is less than a preset deviation value; If the difference is not less than the preset deviation value, then the gradient descent algorithm is used to update the iterative model parameters; If the difference is less than the preset deviation value, then model training is stopped.

6. The apparatus according to claim 5, characterized in that, The public opinion risk assessment module includes: The inference unit is used to obtain a matrix of the number of events and the extracted text location information through the deep learning model, calculate the matrix result value, and determine whether the matrix result value is greater than a preset matrix result value. The matrix result value represents whether the text in the text location information is the public opinion risk information. If the matrix result value is greater than the preset matrix result value, it is inferred that the public opinion information is public opinion risk information. If the matrix result value is less than or equal to the preset matrix result value, it is inferred that the public opinion information is not public opinion risk information.

7. A reasoning device for public opinion risk, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the method as described in any one of claims 1 to 4.

8. A reasoning and storage medium for public opinion risk, wherein a computer program is stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images