An opinion perception method for blockchain on-chain off-chain data fusion

Abstract

The application provides a kind of public opinion perception method for blockchain on-chain off-chain data fusion, comprising: on-chain off-chain data extraction module, data cleaning module, text sentiment analysis module, text classification module, on-chain off-chain data fusion module. Collect data by deploying blockchain full node, export Ethereum blockchain data to CSV file using Ethereum-ETL tool, then use it as data source for public opinion perception analysis, realize real-time acquisition of the latest public opinion information on Ethereum. At the same time, import the results of public opinion analysis into on-chain off-chain fusion middleware, realize off-chain synchronization to obtain related information of the same public opinion, realize the role of public opinion perception supervision on blockchain. The monitoring and analysis of blockchain public opinion data is an important part of blockchain supervision, and the data generation, publication and dissemination process of blockchain data are significantly different from general Internet data. It realizes the supervision of public opinion on blockchain, and realizes the fusion of on-chain and off-chain data for public opinion perception.

Description

Technical Field

[0001] This invention relates to the field of blockchain on-chain and off-chain data fusion technology, and in particular to a public opinion perception method for blockchain on-chain and off-chain data fusion. Background Technology

[0002] Effective regulation of blockchain is a necessary condition for the healthy development of the blockchain ecosystem, but effective technical means are currently lacking.

[0003] The anonymity, decentralization, openness, and diversity of blockchain systems present a series of technical challenges to effective regulation. How to clean transaction information generated by smart contracts within a blockchain system, and how to detect illegal information within the blockchain system, are all technical problems that need to be researched and solved. Furthermore, the ability to perceive public sentiment on the blockchain remains a technological gap. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for public opinion perception by fusing on-chain and off-chain data in blockchain. Monitoring and analyzing blockchain public opinion data is an important aspect of blockchain regulation. Furthermore, blockchain data differs significantly from general internet data, with substantial differences in its data generation, publication, and dissemination processes. This method achieves both public opinion regulation of the blockchain and the fusion of on-chain and off-chain data for public opinion perception.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for public opinion perception by fusing on-chain and off-chain data in blockchain, comprising the following steps:

[0006] Step 1: On-chain negative public sentiment perception;

[0007] Step 2: Monitoring off-chain public opinion hotspots;

[0008] Step 3: Integration of on-chain and off-chain public opinion data.

[0009] In a preferred embodiment, step 1 specifically includes:

[0010] Step 11: Obtain the text dataset from transactions on Ethereum;

[0011] Step 12: Clean the transaction data;

[0012] Step 13: Perform sentiment analysis on the input text;

[0013] Step 14: Send negative public opinion information to the middleware and the off-chain public opinion hotspot detection module.

[0014] In a preferred embodiment, step 11 specifically includes:

[0015] Step 111: Deploy an Ethereum full node; Install the execution client Geth and the consensus client Prysm on your computer; After starting the Geth client, Geth will automatically begin synchronizing historical data;

[0016] Step 112: Extract Ethereum full node data; When the Ethereum full node starts, geth.ipc will appear. Use this file as the interface to extract Ethereum transaction data.

[0017] In a preferred embodiment, step 12 specifically includes:

[0018] Step 121: Filter the input of each transaction and delete transactions generated by calling the smart contract or transactions with empty additional information;

[0019] Step 122: Convert the input in the transaction from hexadecimal to UTF-8, and delete the transactions with garbled characters after conversion.

[0020] In a preferred embodiment, step 13 specifically includes:

[0021] Step 131: Input the text into the model and map each word of the text into a low-dimensional vector.

[0022] e i =W wrd v i

[0023] Embedding matrix Where V is a fixed-size vocabulary, d w It is the size of the word embedding; v i It is a vector of size |V|, located at index e. i The value at position 1 is 1, and the values ​​at other positions are 0;

[0024] Step 132: Use BiLSTM to obtain high-level features from low-dimensional vectors;

[0025] Step 133: Obtain high-level features of the text by considering its overall context.

[0026]

[0027] An attention mechanism is used to generate a weight vector, and the word-level features at each time step are multiplied by the weight vector and merged into a sentence-level feature vector.

[0028] M = tanh(H)

[0029] α = softmax(ω) T M)

[0030] r = Hα T

[0031] h * =tanh(r)

[0032] Where H is the matrix composed of the output vectors generated by BiLSTM, T is the length of the sentence, r is the weighted matrix formed by H; ω is the training parameter vector; tanh and softmax are both activation functions;

[0033] Step 134: Use sentence-level feature vectors for relation classification

[0034]

[0035]

[0036] The classification uses a softmax classifier to predict the label of sentence S from a set of discrete classes Y.

[0037] In a preferred embodiment, step 14 specifically includes: storing the transaction information of the input containing text with negative sentiment analysis in the middleware on-chain negative public opinion information database, and sending the relevant public opinion hotspots to the off-chain network public opinion detection module.

[0038] In a preferred embodiment, step 2 specifically includes:

[0039] Step 21: Based on the results of the on-chain public opinion sentiment analysis module, search for relevant public opinion information on the Internet, and then use a web crawler to crawl the relevant information and save it to the local database;

[0040] Step 22: Clean the hot data and perform text preprocessing;

[0041] Step 23: Use the TextCNN text classification model to distinguish the types of public opinion hotspots;

[0042] Step 24: Record the classification results of public opinion hotspots and send them to the middleware.

[0043] In a preferred embodiment, step 22 specifically includes:

[0044] Step 221: Organize the initial crawled public opinion hotspot data, removing irrelevant symbols such as spaces and line breaks.

[0045] res = res.strip()

[0046] Step 222: Adjust the garbled text data according to the encoding format. Common Chinese encoding methods include UTF-8, GBK, and ISO-8859-1.

[0047] res=requests.get(url,headers=headers)

[0048] res.encoding = 'utf-8'

[0049] res = res.text

[0050] Step 223: Text Preprocessing

[0051] The word segmentation uses Jieba to segment Chinese words. Before segmentation, punctuation is removed, then word segmentation is performed, stop words are removed, and word frequencies are counted. The segmentation results are saved to the fenci column of the dataframe, and the word frequency statistics are saved to words_dict. Then, based on the words in words_dict, a words_id mapping table is generated, which can be directly constructed using a Python dictionary.

[0052] The constructed dataset is vectorized. In order for TextCNN to be used, the text needs to be truncated and padded. Finally, the data is converted into NumPy format.

[0053] In a preferred embodiment, step 23 specifically includes:

[0054] Step 231: Concatenate word vectors to form a sentence matrix; assuming the longest segment of text has n words, sentences with fewer than n words are padded with m words, and each word is a k-dimensional vector; the final word vector matrix is ​​n*k; represented as:

[0055]

[0056] Step 232: One-dimensional convolution operation, which includes a filter. It is applied to a window containing the word h to generate a new feature; for example, feature c i From the word x i:i+h-1 Window generation:

[0057] c i =f(w•x) i:i+h-1 +b)

[0058] h is the window height, i.e., the height of the filter. It is a bias term, w is the weight, and f is a non-linear function; this filter is applied to the sentence {x} 1:h ,x 2:h+1 ,…,x n-h+1:n Generate feature maps:

[0059] c = [c1, c2, ..., c n-h+1 ]

[0060] Step 233: Apply the max timeout pooling operation to the feature map and set the maximum value. As a corresponding feature-specific filter, its idea is to capture the most important feature for each feature map, that is, the feature with the highest value.

[0061] Step 234: Use multiple filters with different window sizes to obtain multiple features; these features form a feature map and are passed to a fully connected softmax layer, the output of which is a probability distribution over the labels.

[0062] In a preferred embodiment, step 3 specifically includes:

[0063] Step 31: Save the transaction information containing negative sentiment text returned by the on-chain sentiment analysis module to the middleware's on-chain negative sentiment information database;

[0064] Step 32: Save the internet data related to negative on-chain public opinion searched by the off-chain network public opinion monitoring module to the middleware's off-chain public opinion hotspot information database;

[0065] Step 33: Display the latest saved negative public opinion information on the blockchain and related public opinion information on the internet off the blockchain; and issue an alert for negative public opinion information on the blockchain, while displaying related public opinion information off the blockchain.

[0066] Compared with existing technologies, this invention has the following beneficial effects: By deploying full blockchain nodes to collect data, and using Ethereum-ETL tools to export Ethereum blockchain data to a CSV file, this data can be used as a data source for public opinion perception analysis, enabling real-time acquisition of the latest public opinion information on Ethereum. Simultaneously, the results of the public opinion analysis are imported into an on-chain / off-chain fusion middleware, allowing for the synchronous acquisition of relevant information on the same public opinion off-chain, thus achieving the function of monitoring public opinion perception on the blockchain. Monitoring and analyzing blockchain public opinion data is an important aspect of blockchain regulation. Furthermore, blockchain data differs significantly from general internet data, with substantial differences in its data generation, publication, and dissemination processes. This invention achieves both public opinion regulation of the blockchain and the fusion of on-chain and off-chain data for public opinion perception. Attached Figure Description

[0067] Figure 1 This is a system functional block diagram of a preferred embodiment of the present invention;

[0068] Figure 2 This is a system functional flowchart of a preferred embodiment of the present invention;

[0069] Figure 3 This is a diagram of a BiLSTM model with an attention mechanism according to a preferred embodiment of the present invention.

[0070] Figure 4 This is a model example of a preferred embodiment of the present invention. Detailed Implementation

[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0072] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0073] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0074] A method for public opinion perception through the fusion of on-chain and off-chain data in blockchain is presented. The system includes: an on-chain and off-chain data extraction module, a data cleaning module, a text sentiment analysis module, a text classification module, and an on-chain and off-chain data fusion module. This ensures the detection of public opinion on the blockchain and simultaneously achieves public opinion perception through the fusion of on-chain and off-chain data, providing a more comprehensive view of public opinion information on the blockchain. (Reference) Figure 1-4 ,:

[0075] Step 1: On-chain negative public opinion perception

[0076] 1. Obtain the text dataset from Ethereum transactions.

[0077] Step 1: Deploy an Ethereum full node

[0078] Install the execution client Geth and the consensus client Prysm on your computer.

[0079] After launching the Geth client, Geth will automatically begin synchronizing historical data.

[0080] Step 2: Extract Ethereum full node data

[0081] When an Ethereum full node starts, a geth.ipc file will be generated. This file serves as the interface for extracting Ethereum transaction data.

[0082] Cleaning transaction data

[0083] Step 1: Filter the input of each transaction and delete transactions generated by calling smart contracts or transactions with empty additional information.

[0084] Step 2: Convert the input data in the transaction from hexadecimal to UTF-8, and delete the transactions with garbled characters after conversion.

[0085] Sentiment analysis of input text

[0086] Step 1: Input the text into the model and map each word of the text into a low-dimensional vector.

[0087] e i =W wrd v i

[0088] Embedding matrix Where V is a fixed-size vocabulary, d w It refers to the size of the word embedding. i It is a vector of size |V|, located at index e. i The value at position 1 is 1, while the values ​​at other positions are 0.

[0089] Step 2: Use BiLSTM to obtain high-level features from low-dimensional vectors.

[0090] Step 3: Obtain high-level features of the text by considering its overall context.

[0091]

[0092] An attention mechanism is used to generate a weight vector, and the word-level features at each time step are multiplied by the weight vector and merged into a sentence-level feature vector.

[0093] M = tanh(H)

[0094] α = softmax(ω) T M)

[0095] r = Hα T

[0096] h * =tanh(r)

[0097] Where H is the matrix composed of the output vectors generated by BiLSTM, T is the length of the sentence, and r is the weighted matrix formed by H. ω is the training parameter vector. tanh and softmax are both activation functions.

[0098] Step 4: Use sentence-level feature vectors for relation classification

[0099]

[0100]

[0101] The classification uses a softmax classifier to predict the label of sentence S from a set of discrete classes Y.

[0102] Send negative public opinion information to the middleware and off-chain public opinion hotspot detection module

[0103] Step 1: Store the transaction information containing text with negative sentiment analysis in the middleware's on-chain negative sentiment information database, and at the same time send the relevant sentiment hotspots to the off-chain network sentiment detection module.

[0104] Step Two: Off-Chain Public Opinion Hotspot Monitoring

[0105] Crawling relevant public opinion hotspot data from the Internet

[0106] Step 1: Based on the results of the on-chain public opinion sentiment analysis module, search for relevant public opinion information on the Internet, and then use a web crawler to crawl the relevant information and save it to a local database.

[0107] Cleaning hot data and performing text preprocessing

[0108] Step 1: Organize the initial crawled public opinion hotspot data, removing irrelevant symbols such as spaces and line breaks.

[0109] res = res.strip()

[0110] Step 2: Adjust the encoding of the garbled text data according to its format. Common Chinese encoding methods include UTF-8, GBK, and ISO-8859-1. (Taking UTF-8 as an example)

[0111] res=requests.get(url,headers=headers)

[0112] res.encoding = 'utf-8'

[0113] res = res.text

[0114] Step 3: Text Preprocessing

[0115] The Chinese text is segmented using Jieba. Before segmentation, punctuation is removed, followed by segmentation, then stop words are removed, and word frequencies are calculated. The segmentation results are saved to the `fenci` column of the dataframe, and the word frequency statistics are saved to `words_dict`. Then, based on the words in `words_dict`, a `words_id` mapping table is generated, which can be directly constructed using a Python dictionary.

[0116] The constructed dataset is vectorized. In order for TextCNN to be used, the text needs to be truncated and padded. Finally, the data is converted into NumPy format.

[0117] 3. Use the TextCNN text classification model to distinguish the types of public opinion hotspots.

[0118] Step 1: Concatenate word vectors to form a sentence matrix. Let the longest segment of text have n words, and sentences with fewer than n words be padded with m words. Each word is a k-dimensional vector. The final word vector matrix is ​​n*k. This can be represented as:

[0119]

[0120] Step 2: One-dimensional convolution operation, which includes a filter. It is applied to a window of word h to generate a new feature. For example, feature c i From the word x i:i+h-1 Window generation:

[0121] c i =f(w·x) i:i+h-1 +b)

[0122] h is the window height, i.e., the height of the filter. is a bias term, w is the weight, and f is a non-linear function. This filter is applied to the sentence {x}. 1:h ,x 2:h+1 ,…,x n-h+1:n Generate feature maps:

[0123] c = [c1, c2, ..., c n-h+1 ]

[0124] Step 3: Apply the max timeout pooling operation to the feature map and set the maximum value. This serves as a special feature filter. The idea is to capture the most important features for each feature map, i.e., the features with the highest values.

[0125] Step 4: Use multiple filters (with different window sizes) to obtain multiple features. These features form a feature map and are passed to a fully connected softmax layer, whose output is a probability distribution over the labels.

[0126] 4. Record the classification results of public opinion hotspots and send them to the middleware.

[0127] Step 3: Integration of On-Chain and Off-Chain Public Opinion Data

[0128] Transaction information containing public opinion text on the storage chain

[0129] Step 1: Save the transaction information containing negative sentiment text returned by the on-chain sentiment analysis module to the middleware's on-chain negative sentiment information database.

[0130] Internet public opinion hotspot data from storage chain

[0131] Step 1: Save the internet data related to negative on-chain public opinion searched by the off-chain network public opinion monitoring module to the middleware's off-chain public opinion hotspot information database.

[0132] Integrating and displaying on-chain and off-chain public opinion data

[0133] Step 1: Display the latest saved negative public opinion information on the blockchain and related public opinion information on the off-chain internet.

[0134] Step 2: Issue alerts for negative public opinion on the blockchain and simultaneously display relevant public opinion information off the blockchain.

Claims

1. A method for public opinion perception by fusing on-chain and off-chain data in a blockchain, characterized in that... Includes the following steps: Step 1: On-chain negative public sentiment perception; Step 2: Monitoring off-chain public opinion hotspots; Step 3: Integration of on-chain and off-chain public opinion data; Step 1 specifically includes: Step 11: Obtain the text dataset from transactions on Ethereum; Step 12: Clean the transaction data; Step 13: Perform sentiment analysis on the input text; Step 14: Send negative public opinion information to the middleware and off-chain public opinion hotspot detection module; Step 13 specifically includes: Step 131: Input the text into the model and map each word of the text into a low-dimensional vector. = Embedding matrix ∈ Where V is a fixed-size vocabulary, It is the size of the word embedding; It is a vector of size |V|, and it is at index The value at position 1 is 1, and the values ​​at other positions are 0; Step 132: Use BiLSTM to obtain high-level features from low-dimensional vectors; Step 133: Obtain high-level features of the text by considering its overall context. = [ ] An attention mechanism is used to generate a weight vector, and the word-level features at each time step are multiplied by the weight vector and merged into a sentence-level feature vector. M = tanh(H) r = H = tanh(r) Where H is a matrix composed of the output vectors generated by BiLSTM, T is the length of the sentence, and r is a weighted matrix formed by H; It is the parameter vector for training; tanh and softmax are both activation functions. Step 134: Use sentence-level feature vectors for relation classification (y|S) = softmax( + ) = arg (y|S) The classification uses a softmax classifier to predict the label of sentence S from a set of discrete classes Y. ; Step 2 specifically includes: Step 21: Based on the results of the on-chain public opinion sentiment analysis module, search for relevant public opinion information on the Internet, and then use a web crawler to crawl the relevant information and save it to the local database; Step 22: Clean the hot data and perform text preprocessing; Step 23: Use the TextCNN text classification model to distinguish the types of public opinion hotspots; Step 24: Record the classification results of public opinion hotspots and send them to the middleware; Step 3 specifically includes: Step 31: Save the transaction information containing negative sentiment text returned by the on-chain sentiment analysis module to the middleware's on-chain negative sentiment information database; Step 32: Save the internet data related to negative on-chain public opinion searched by the off-chain network public opinion monitoring module to the middleware's off-chain public opinion hotspot information database; Step 33: Display the latest saved negative public opinion information on the blockchain and related public opinion information on the internet off the blockchain; and issue an alert for negative public opinion information on the blockchain, while displaying related public opinion information off the blockchain.

2. The method for public opinion perception by fusing on-chain and off-chain data in blockchain according to claim 1, characterized in that, Step 11 specifically includes: Step 111: Deploy an Ethereum full node; Install the execution client Geth and the consensus client Prysm on your computer; After starting the Geth client, Geth will automatically begin synchronizing historical data; Step 112: Extract Ethereum full node data; When the Ethereum full node starts, geth.ipc will appear. Use this file as the interface to extract Ethereum transaction data.

3. The method for public opinion perception by fusing on-chain and off-chain data in blockchain according to claim 1, characterized in that, Step 12 specifically includes: Step 121: Filter the input of each transaction and delete transactions generated by calling the smart contract or transactions with empty additional information; Step 122: Convert the input in the transaction from hexadecimal to UTF-8, and delete the transactions with garbled characters after conversion.

4. The method for public opinion perception by fusing on-chain and off-chain data in blockchain according to claim 1, characterized in that, Step 14 specifically includes: storing the transaction information of the input containing text with negative sentiment analysis in the middleware on-chain negative public opinion information database, and sending the relevant public opinion hotspots to the off-chain network public opinion detection module.

5. The method for public opinion perception by fusing on-chain and off-chain data in blockchain according to claim 1, characterized in that, Step 22 specifically includes: Step 221: Organize the initial crawled public opinion hotspot data, removing irrelevant symbols such as spaces and line breaks. res = res.strip() Step 222: Adjust the garbled text data according to the encoding format. Common Chinese encoding methods include UTF-8, GBK, and ISO-8859-1. res = requests.get(url, headers=headers) res.encoding = 'utf-8' res = res.text Step 223: Text Preprocessing The word segmentation uses Jieba to segment Chinese words. Before segmentation, punctuation is removed, then word segmentation is performed, stop words are removed, and word frequencies are counted. The segmentation results are saved to the fenci column of the dataframe, and the word frequency statistics are saved to words_dict. Then, based on the words in words_dict, a words_id mapping table is generated, which can be directly constructed using a Python dictionary. The constructed dataset is vectorized. In order for TextCNN to be used, the text needs to be truncated and padded. Finally, the data is converted into NumPy format.

6. The method for public opinion perception by fusing on-chain and off-chain data in blockchain according to claim 1, characterized in that, Step 23 specifically includes: Step 231: Concatenate word vectors to form a sentence matrix; assuming the longest segment of text has n words, sentences with fewer than n words are padded with m words, and each word is a k-dimensional vector; the final word vector matrix is ​​n * k; represented as: = ... Step 232: One-dimensional convolution operation, which includes a filter w∈ It is applied to a window containing the word 'h' to generate a new feature; feature From words Window generation: = f(w ∙ + b) h is the window height, i.e., the height of the filter; b It is a bias term, w is the weight, and f is a non-linear function; this filter is applied to the sentence { , ,…, Generate feature maps: c=[ , , ..., ] Step 233: Apply the max timeout pooling operation to the feature map and set the maximum value. = max{c} serves as a special feature filter corresponding to this; its idea is to capture the most important feature for each feature map, that is, the feature with the highest value; Step 234: Use multiple filters with different window sizes to obtain multiple features; these features form a feature map and are passed to a fully connected softmax layer, the output of which is a probability distribution over the labels.

Images