A Chinese Comment Sentiment Analysis Method Based on BERT-MO-CNN-BILSTM

Abstract

The present invention provides a Chinese comment sentiment analysis method based on BERT-MO-CNN-BILSTM, belonging to the field of sentiment analysis. The content includes: obtaining Chinese comment texts and sentiment classification, and preprocessing the text data; converting the preprocessed text into a word embedding matrix through ChineseBERT; optimizing the hyperparameters of the multi-scale convolutional tandem bidirectional long short-term memory neural network through a mine exploitation optimization algorithm; inputting the word embedding matrix into the neural network model with optimized hyperparameters, and training to obtain a BERT-MO-CNN-BILSTM Chinese comment sentiment analysis model; after preprocessing the text data to be sentiment classified and generating a word embedding matrix, inputting it into the trained model to obtain sentiment classification. The method of the present invention can improve the accuracy of Chinese comment sentiment analysis and provide a basis for enterprises to understand user needs and product improvement.

Description

Technical Field

[0001] The present invention relates to the technical field of text sentiment analysis, and more specifically, to a method for Chinese comment sentiment analysis based on BERT-MO-CNN-BILSTM. Background Art

[0002] Text sentiment analysis is a computational study of the opinions and emotions expressed in text. With the increasing maturity of the e-commerce model and the continuous expansion of the scale of online shopping, the number of online reviews after consumers purchase goods is also increasing day by day. The rich emotional information contained in these reviews will not only become an important basis for potential consumers to judge the quality of goods, but also help manufacturers fully understand user needs and thus improve products. Therefore, the research on online review text sentiment analysis has important practical significance.

[0003] Compared with ordinary text, online reviews have the problems of less vocabulary and sparse features, which pose higher requirements for text semantic feature representation. In recent years, with the development of machine learning methods, deep learning technology has been widely used in the field of text sentiment analysis. However, a single deep learning module has problems such as insufficient comprehensive feature extraction, inaccurate semantic representation, and low prediction accuracy. Therefore, many researchers mix different deep learning, especially neural network models, to improve the performance of the model. At the same time, the performance of the neural network model is affected by hyperparameters. Inappropriate hyperparameters may directly lead to diagnostic errors and affect the diagnostic accuracy. Therefore, optimizing the neural network based on an optimization algorithm to find the optimal match of hyperparameters has also become a research hotspot. However, there is still room for further improvement in the optimization accuracy and computational efficiency of existing intelligent optimization algorithms. Especially for the optimization of complex neural network models, finding a local optimum within a limited time is often more practically significant than finding a global optimum. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for Chinese comment sentiment analysis based on BERT-MO-CNN-BILSTM, which can solve the problems of inaccurate semantic representation of traditional static word vectors, insufficient feature extraction capabilities of single models and single scales, and insufficient efficiency and accuracy of traditional optimization algorithms, enabling the model to have better sentiment classification effects and providing reliable technical support for improving the online review sentiment classification effect.

[0005] The present invention is implemented as follows:

[0006] The technical solution for achieving the purpose of the present invention is: a method for Chinese comment sentiment analysis based on BERT-MO-CNN-BILSTM, including the following steps:

[0007] Step 1: Obtain Chinese review texts and their sentiment classifications, and preprocess the Chinese review texts, including text cleaning, stop word removal, and word segmentation;

[0008] Step 2: Convert the preprocessed text into a word embedding matrix through ChineseBERT, and convert the sentiment classification into a one-hot encoding;

[0009] Step 3: Set the parameters required for the Mine Optimization Algorithm (MO);

[0010] Step 4: Determine the network structure of the multi-scale convolutional concatenated bidirectional long short-term memory neural network, and optimize the hyperparameters of the multi-scale convolutional concatenated bidirectional long short-term memory neural network (CNN-BILSTM) through 6 stages of the Mine Optimization Algorithm;

[0011] Step 5: Input the word embedding matrix into the neural network model with optimized hyperparameters, and train to obtain a Chinese review text sentiment analysis model based on BERT-MO-CNN-BILSTM;

[0012] Step 6: Preprocess the text data to be sentiment classified, generate a word embedding matrix through ChineseBERT, and then input it into the trained Chinese text review sentiment analysis model to obtain the sentiment classification.

[0013] Furthermore, in Step 3, the parameters required for the Mine Optimization Algorithm (MO) include the number of mines K, the proportion of high-quality mines Kp, where 0 < Kp < 1, the mining point base Ma, the mining center search range factor Tp, where 0 < Tp < 0.2, the maximum moving step ratio Dt, where 0.01 < Dt < 0.2, and the maximum number of iterations T.

[0014] Furthermore, in Step 4, the Mine Optimization Algorithm includes 6 stages:

[0015] (1) Mine initialization stage, randomly generate K mines within the solution domain, each mine has a mining center, and the initial value of the position of the mine mining center K i The expression is:

[0016] K i =lb+ (ub - lb) × rand (0, 1)

[0017] where, K i represents the position of the i th mine mining center, i = 1, 2, …, K, lb is the lower limit of the search domain in each dimension, and ub is the upper limit of the search domain in each dimension.K i , lb and ub are all N-dimensional vectors, where N represents the dimension of the problem to be solved. rand (0, 1) represents an N-dimensional vector composed of random numbers between 0 and 1.

[0018] (2) In the stage of selecting high-quality mines, calculate the fitness values of the K mining centers, and select the E mines with the optimal fitness values of the mining centers as high-quality mines:

[0019] E = round(K × Kp)

[0020] Among them, E is the number of high-quality mines; Kp is the proportion of high-quality mines; the round() function represents rounding the value in the parentheses to an integer.

[0021] (3) In the mining stage, for the E high-quality mines, randomly generate different numbers of mining points within the mining center search range factor Tp around each mining center of the mine, and the search range range decreases as the number of iterations increases:

[0022] range = Tp × (ub - lb) × (1 - t / T)

[0023] Among them, t is the number of the current iteration, and T is the maximum number of iterations;

[0024] The number of mining points is related to the fitness value ranking of the high-quality mine mining center and the number of iterations. The number of mining points generated around each high-quality mine mining center can be expressed as:

[0025] S a = round (Ma × ( E -a + 1) × (1 - t / T))

[0026] Among them, S a is the number of mining points generated around the a th high-quality mine, a is the ranking of the fitness value of the high-quality mine, and the lower the fitness value, the higher the ranking ，a = 1, 2,..., E, and Ma is the mining point base;

[0027] Then the position of each generated mining point is:

[0028] K aj = KA a+range ×( 2 × rand (0, 1)-1)

[0029] Among them, K aj represents the position of the a th mining point generated around the j th high-quality mine exploitation center, j=1,2,…, S a , KA a is the position of the a th high-quality mine exploitation center, K aj and KA a are both N-dimensional vectors;

[0030] (4) In the stage of evaluating mining groups, first, for E high-quality mines, form a mining group by combining the exploitation center of the high-quality mine with the mining points generated by the high-quality mine. For non-high-quality mines, the mining group consists of the exploitation center of the mine itself;

[0031] Secondly, calculate the fitness value of each mining point in each mining group, and combine the fitness value of the exploitation center of each mining group to obtain the average fitness value FT i and the average position KS i , that is:

[0032]

[0033]

[0034] Among them, F() is the fitness function;

[0035] Define the mining group with the average fitness value FT i optimal as the optimal mining group, and define the mining groups with the average fitness values FT i worst and second-worst as the worst mining group and the second-worst mining group;

[0036] Thirdly, through the comparison of fitness values, obtain the optimal fitness value FKD min and the corresponding position, the worst fitness value FKD max , the optimal position and the second-optimal position in the optimal mining group, and the average fitness value FT max 、The optimal fitness value in each mining group ( FKT min ) i and its location ；

[0037] (5)In the prospecting stage, the mining center prospecting is carried out according to the following steps:

[0038] Change the mining center location of each mine K i , to the location where the optimal fitness value ( FKT min ) i in the mining group where the mining center is located;

[0039] For the optimal mining group, as well as the global optimal fitness value FKD min in the mining group where it is located, no more prospecting is carried out, that is, its mining center location remains unchanged;

[0040] For non-optimal mining groups, and non-global optimal fitness values FKD min in the mining group where it is located, select a high-quality mine mining center as the target with a certain probability, and prospect from the mining center of this mining group in the direction of the high-quality mine mining center. The probability of a certain high-quality mine being selected is related to the fitness value of its mining center, and the probability can be expressed as:

[0041]

[0042] Where R (a) represents the probability of the a-th high-quality mine being selected, ( FKT min ) a represents the optimal fitness value of the a-th high-quality mine mining group, and the moving distance D i of the mining center is related to the fitness value and the number of iterations. Suppose the i -th mining center selects the a -th high-quality mine, then D i can be expressed as:

[0043] D i =D t ×(ub - lb)× F ia × T ia ×( K a -K i )

[0044] Among them, D t is the maximum moving step ratio, F ia is the fitness value influence term, T ia is the iteration number influence term, and they are respectively:

[0045] F ia = (( FKT min ) i - ( FKT min ) a ) / ( FKD max -FKD min )

[0046] T ia = 1 - e (-rand(0,1)×t / tmax)

[0047] Then the initial position of the mining center in the next iteration will be updated to:

[0048] K i t+1 =K i t +D i t

[0049] where the superscript represents the iteration number;

[0050] (6) In the human resource allocation stage, for the worst mining group and the second-worst mining group, the mining personnel will be allocated. Among them, for the worst mining group, its mining center will be allocated to the sub-optimal position in the optimal mining group; for the second-worst mining group, the original mining center will be deleted, and a point will be randomly generated within the solution domain as the new mining center. The random generation method is the same as the method for mine initialization;

[0051] Delete all mining points, determine whether the iteration ends. If the iteration ends, output the optimal parameters. If the iteration does not end, return to stage (2) and repeat the calculation from stage (2) to stage (6).

[0052] Further, in step four, the multi-scale convolutional concatenated bidirectional long short-term memory neural network includes an input layer, a multi-scale convolutional layer, a pooling layer, a merging layer, a BILSTM layer, a fully connected layer, and an output layer; among them, the multi-scale convolutional layer and the pooling layer perform convolution and pooling operations on convolutional kernels of different sizes respectively, and then merge the results and input the merged results into the BILSTM layer.

[0053] Further, in step four, the hyperparameters include the number of filters in the convolutional layer, the sizes of the multi-scale convolutional kernels, the number of hidden units in the BILSTM, the epoch size, and the training batch size.

[0054] The beneficial effects of the present invention are as follows: The present invention provides a Chinese comment sentiment analysis method of BERT-MO-CNN-BILSTM. Among them, the dynamic word vectors of the text can be obtained through the Chinese version ChineseBERT of BERT. The multi-scale convolutional neural network (CNN) can obtain local information of different scales through convolutional kernels of different sizes. The long short-term memory neural network (BILSTM) can fully explore global and temporal information. And the mining optimization algorithm (MO) is a novel meta-heuristic optimization algorithm originally proposed in the present invention. By optimizing the hyperparameters of the model through this MO algorithm, the model can have better classification effects, providing reliable technical support for improving the online comment sentiment classification effect. Description of the Drawings

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of a Chinese comment sentiment analysis method of BERT-MO-CNN-BILSTM provided by the embodiment of the present invention;

[0057] Figure 2 It is a detailed flowchart of the MO optimization algorithm provided by the embodiment of the present invention;

[0058] Figure 3 It is a graph showing the change of the fitness value of different optimization algorithms with the number of iterations provided by the embodiment of the present invention. Detailed Embodiments

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in combination with the implementation cases and drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0060] The following uses a specific Chinese comment sentiment analysis case as an example to illustrate the method of the present invention.

[0061] Such as Figure 1 , a Chinese comment sentiment analysis method based on BERT-MO-CNN-BILSTM, comprising the following steps:

[0062] Step 1: Obtain Chinese comment texts and sentiment classifications, and preprocess the Chinese comment texts, including text cleaning, stop word removal, and word segmentation;

[0063] Step 2: Convert the preprocessed text into a word embedding matrix through ChineseBERT, and convert the sentiment classification into a one-hot encoding;

[0064] Step 3: Set the parameters required for the Mine Optimization Algorithm (MO);

[0065] Step 4: Determine the network structure of the multi-scale convolutional concatenated bidirectional long short-term memory neural network, and optimize the hyperparameters of the multi-scale convolutional concatenated bidirectional long short-term memory neural network (CNN-BILSTM) through 6 stages of the Mine Optimization Algorithm;

[0066] Step 5: Input the word embedding matrix into the neural network model with optimized hyperparameters, and train to obtain a Chinese comment text sentiment analysis model based on BERT-MO-CNN-BILSTM;

[0067] Step 6: Preprocess the text data to be sentiment classified, generate a word embedding matrix through ChineseBERT, and then input it into the trained Chinese text comment sentiment analysis model to obtain a sentiment classification.

[0068] Further, in step three, the parameters required for the Mine Optimization Algorithm (MO) include the number of mines K, the proportion of high-quality mines Kp, where 0 < Kp < 1, the mining point base Ma, the mining center search range factor Tp, where 0 < Tp < 0.2, the maximum moving step ratio Dt, where 0.01 < Dt < 0.2, and the maximum number of iterations T.

[0069] Further, in step four, the Mine Optimization Algorithm includes six stages:

[0070] (1) Mine initialization stage: Randomly generate K mines within the solution domain. Each mine has a mining center, and the initial value of the position of the mine's mining center K i The expression is:

[0071] K i =lb+ (ub - lb) × rand (0, 1)

[0072] where K i represents the position of the mining center of the i -th mine, i i = 1, 2, …, K, lb is the lower limit of the search domain in each dimension, ub is the upper limit of the search domain in each dimension, K i , lb, and ub are all N-dimensional vectors, where N represents the dimension of the problem to be solved, rand (0, 1) represents an N-dimensional vector composed of random numbers between 0 and 1;

[0073] (2) Selecting high-quality mines stage: Calculate the fitness values of the mining centers of K mines, and select the E mines with the best fitness values of the mining centers as high-quality mines:

[0074] E = round(K × Kp)

[0075] where E is the number of high-quality mines; Kp is the proportion of high-quality mines; the round() function represents rounding the value in the parentheses to an integer;

[0076] (3) Mining stage: For the E high-quality mines, randomly generate different numbers of mining points within the mining center search range factor Tp around each mine's mining center. The search range range decreases as the number of iterations increases:

[0077] range = Tp × (ub - lb) × (1 - t t / T)

[0078] where tis the number of current iterations, and T is the maximum number of iterations; range Decreasing with the increase of the number of iterations can enable a relatively large local exploration ability in the early stage of the search and a strong local convergence ability in the later stage of the search;

[0079] The number of mining points is related to the fitness value ranking of high-quality mine exploitation centers and the number of iterations. The number of mining points generated around each high-quality mine exploitation center can be expressed as:

[0080] S a = round (Ma × ( E -a + 1) × (1 - t / T))

[0081] Where, S a is the number of mining points generated around the a th high-quality mine, a is the ranking of the fitness value of the high-quality mine, and the lower the fitness value, the higher the ranking ，a = 1, 2, …, E, and Ma is the base number of exploitation points; It can be seen from the S a expression that ， the better the fitness value of the high-quality mine, the more mining points are generated around it, which is more conducive to finding local or global optimal solutions;

[0082] Then the position of each generated mining point is:

[0083] K aj = KA a +range × ( 2 × rand (0, 1) - 1)

[0084] Where, K aj represents the position of the a th mining point generated around the j th high-quality mine exploitation center, j=1,2,…, S a , KA a is the position of the a th high-quality mine exploitation center, K aj and KA a are both N-dimensional vectors;

[0085] (4) In the evaluation stage of mining groups, first, for E high-quality mines, a mining group is formed by the mining center of the high-quality mine and the mining points generated by the high-quality mine. For non-high-quality mines, the mining group is composed of the mining center of the mine itself;

[0086] Second, calculate the fitness value of each mining point in each mining group, and combine the fitness value of the mining center of each mining group to obtain the average fitness value of each mining group FT i and the average position KS i , that is:

[0087]

[0088]

[0089] Among them, F() is the fitness function;

[0090] Define the mining group with the average fitness value FT i as the optimal mining group, and define the mining groups with the average fitness value FT i that are the worst and the second-worst as the worst mining group and the second-worst mining group;

[0091] Third, through the comparison of fitness values, obtain the optimal fitness value among all mining centers and mining points FKD min and the corresponding position, the worst fitness value FKD max , the optimal position and the sub-optimal position in the optimal mining group, the average fitness value of the worst mining group FT max 、 The optimal fitness value in each mining group ( FKT min ) i and its location ；

[0092] Through the evaluation of mining groups, each mining center can quickly locate the optimal position for local search;

[0093] (5) In the prospecting stage, the mining center conducts prospecting according to the following steps:

[0094] Change the mining center position of each mine K i , to the position where the optimal fitness value ( FKT min ) i in the mining group where the mining center is located;

[0095] For the optimal mining group and the global optimal fitness value FKD min For the mining group where it is located, prospecting is no longer carried out, that is, the position of its mining center remains unchanged;

[0096] For the non-optimal mining group and the non-global optimal fitness value FKD min For the mining group where it is located, a high-quality mine mining center is selected as the target with a certain probability, and prospecting is carried out from the mining center of this mining group in the direction of the high-quality mine mining center. The probability of a certain high-quality mine being selected is related to the fitness value of its mining center. The probability can be expressed as:

[0097]

[0098] Among them R (a) represents the probability of the a-th high-quality mine being selected, ( FKT min ) a represents the optimal fitness value of the a-th high-quality mine mining group, and the moving distance of the mining center D i is related to the fitness value and the number of iterations. Suppose the i th mining center selects the a th high-quality mine, then D i can be expressed as:

[0099] D i =D t ×(ub - lb)× F ia × T ia ×( K a -K i )

[0100] Among them, D t is the maximum moving step ratio, F ia is the fitness value influence term, T ia is the iteration number influence term, and they are respectively:

[0101] F ia = (( FKT min ) i - ( FKT min ) a ) / ( FKDmax -FKD min )

[0102] T ia = 1 - e (-rand(0,1)×t / tmax)

[0103] Then the position of the mining center will be updated to the initial position in the next iteration as follows:

[0104] K i t+1 =K i t +D i t

[0105] where the superscript represents the number of iterations;

[0106] Through prospecting, the positions of each mining center can be further optimized;

[0107] (6) In the human resource allocation stage, for the worst - performing mining group and the second - worst - performing mining group, the mining personnel will be allocated. Specifically, for the worst - performing mining group, its mining center will be allocated to the second - best position in the best - performing mining group; for the second - worst - performing mining group, the original mining center will be deleted, and a point will be randomly generated within the solution domain as the new mining center. The random generation method is the same as the method for mine initialization;

[0108] Through human resource allocation, while strengthening the local search ability, the ability to jump out of the local optimum is improved;

[0109] Delete all mining points, determine whether the iteration ends. If the iteration ends, output the optimal parameters. If the iteration does not end, then return to the key stage (2) and repeat the calculation from stage (2) to stage (6).

[0110] Furthermore, in step four, the multi - scale convolutional concatenated bidirectional long short - term memory neural network includes an input layer, a multi - scale convolutional layer and a pooling layer, a merging layer, a BILSTM layer, a fully - connected layer, and an output layer; among them, the multi - scale convolutional layer and the pooling layer perform convolution and pooling operations on different - sized convolutional kernels respectively and then merge the results, and input the merged results into the BILSTM layer.

[0111] Furthermore, in step four, the hyperparameters include the number of filters in the convolutional layer, the sizes of the multi - scale convolutional kernels, the number of hidden units in the BILSTM, the number of epochs, and the training batch size.

[0112] The experimental data comes from computer sales review data on an e-commerce platform. There are 4,000 review texts in total, which are divided into two sentiment categories: positive and negative, with 2,000 comments in each category. In this embodiment, 70% of the total number of samples are selected as training samples, 10% as verification samples, and 20% as test samples.

[0113] The specific implementation process is as follows:

[0114] like Figure 1 , a BERT-MO-CNN-BILSTM Chinese review sentiment analysis method, including the following steps:

[0115] Step 1: Obtain Chinese review text and sentiment classification, and preprocess the Chinese review text, including text cleaning, stop word removal, and word segmentation;

[0116] Step 2: Convert the preprocessed text into a word embedding matrix through ChineseBERT, and convert the sentiment classification into one-hot encoding;

[0117] Step 3: Set the parameters required for the mining optimization algorithm (MO), including the number of mines K=5, the proportion of high-quality mines Kp=0.4, the base number of mining points Ma=2, the mining center search range factor Tp=0.1, the maximum moving step ratio Dt=0.05, and the maximum number of iterations T=10;

[0118] Step 4: Determine the network structure of the multi-scale convolutional bidirectional long short-term memory neural network, which includes the input layer, multi-scale convolutional layer and pooling layer, merging layer, BILSTM layer, fully connected layer and output layer; optimize the hyperparameters of the multi-scale convolutional bidirectional long short-term memory neural network (CNN-BILSTM) through the six stages of the mining optimization algorithm. The mining optimization algorithm process is as follows: Figure 2 As shown in the figure, the upper limit of the number of filters in the hyperparameters, the number of hidden units in BILSTM, the size of multi-scale convolution kernels, the round size epoch, and the training batch size is lb=[16,16,3,4,5,3,16], and the lower limit is ub=[256,256,6,8,10,16,256];

[0119] Step 5: Input the word embedding matrix into the CNN-BILSTM model with optimized hyperparameters, and train the Chinese review text sentiment analysis model based on BERT-MO-CNN-BILSTM;

[0120] Step 6: Preprocess the text data to be sentimentally classified, generate a word embedding matrix through ChineseBERT, and then input it into the trained Chinese text comment sentiment analysis model to obtain sentiment classification.

[0121] To verify the advantages of the method of the present invention, after preprocessing the review text, generating the word embedding matrix by ChineseBERT, and performing one-hot encoding on the sentiment classification, the bidirectional long short-term memory neural network (BILSTM), multi-scale convolutional neural network (CNN), bidirectional long short-term memory neural network concatenated with multi-scale convolutional neural network (BILSTM-CNN), multi-scale convolutional neural network concatenated with bidirectional long short-term memory neural network (CNN-BILSTM), multi-scale convolutional neural network optimized by the aurora optimization algorithm concatenated with bidirectional long short-term memory neural network (PLO-CNN-BILSTM), multi-scale convolutional neural network optimized by the frost ice optimization algorithm concatenated with bidirectional long short-term memory neural network (RIME-CNN-BILSTM), and MO-CNN-BILSTM proposed by the present invention were respectively used for calculation, and the accuracy, precision, recall rate, and F1 value were used to evaluate the classification results of each method. The evaluation results are shown in Table 1. At the same time, taking the negative of the accuracy as the fitness function, the convergence processes of the PLO, RIME, and MO methods when optimizing the hyperparameters of CNN-BILSTM are as Figure 3 shown. It can be seen that the MO-CNN-BILSTM method proposed by the present invention has achieved the best performance in each index, and the convergence process is better than that of the PLO and RIME optimization algorithms.

[0122] Table 1 Comparison of evaluation indexes of each model

[0123]

[0124] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A BERT-MO-CNN-BILSTM Chinese review sentiment analysis method, characterized in that: The following steps are involved: Step 1: Obtain Chinese review text and sentiment classification, and preprocess the Chinese review text, including text cleaning, stop word removal, and word segmentation; Step 2: Convert the preprocessed text into a word embedding matrix through ChineseBERT, and convert the sentiment classification into one-hot encoding; Step 3: Set the parameters required for the mining optimization algorithm; Step 4: Determine the network structure of the multi-scale convolutional bidirectional long short-term memory neural network, and optimize the hyperparameters of the multi-scale convolutional bidirectional long short-term memory neural network through the six stages of the mining optimization algorithm; The mining optimization algorithm includes 6 stages: (1) In the mine initialization stage, K mines are randomly generated in the solution domain. Each mine has a mining center. The initial value K of the mining center position is i The expression is: K i =lb+(ub-lb)×rand(0,1) Among them, K i represents the location of the mining center of the i-th mine, i = 1, 2, ..., K, lb is the lower limit of the search domain in each dimension, ub is the upper limit of the search domain in each dimension, K i , lb, ub are all N-dimensional vectors, where N represents the dimension of the problem to be solved, and rand(0,1) represents an N-dimensional vector composed of random numbers between 0 and 1; (2) In the stage of selecting high-quality mines, the fitness values ​​of the mining centers of K mines are calculated, and the E mines with the best fitness values ​​of the mining centers are selected as high-quality mines: E = round(K × Kp) Among them, E is the number of high-quality mines; Kp is the proportion of high-quality mines; the round() function represents rounding the value in the brackets to an integer; (3) Mining stage: For E high-quality mines, different numbers of mining points are randomly generated within the mining center search range factor Tp around the mining center of each mine. The search range range decreases as the number of iterations increases: range=Tp×(ub-lb)×(1-t / T) Among them, t is the number of current iterations, and T is the maximum number of iterations; The number of mining points is related to the fitness value ranking and iteration number of the high-quality mine mining center. The number of mining points generated around each high-quality mine mining center is expressed as: S a =round(Ma×(E-a+1)×(1-t / T) Among them, S a is the number of mining points generated around the a-th high-quality mine, a is the ranking of the fitness value of the high-quality mine, the lower the fitness value, the higher the ranking, a=1,2,…,E, Ma is the base number of mining points; The location of each mining point generated is: K aj =KA a +range×(2×rand(0,1)-1) Among them, K aj Represents the location of the jth mining point generated around the ath high-quality mine mining center, j = 1, 2, ..., S a , K.A. a is the location of the ath high-quality mine mining center, K aj and KA a are all N-dimensional vectors; (4) Mining group evaluation stage: First, for E high-quality mines, the mining center of the high-quality mine and the mining points generated by the high-quality mine are combined into a mining group. For non-high-quality mines, the mining group is composed of the mining center of the mine itself; Secondly, calculate the fitness value of each mining point in each mining group, and combine the fitness value of the mining center of each mining group to obtain the average fitness value FT of each mining group. i and the average position KS i ,Right now: Where F() is the fitness function; The average fitness value FT i The optimal mining group is defined as the optimal mining group, and the average fitness value FT i The worst and second worst mining groups are defined as the worst mining group and second worst mining group; Again, by comparing the fitness values, we can get the optimal fitness value FKD among all mining centers and mining points. min And the corresponding position, the worst fitness value FKD max , the average fitness value FT of the optimal position and suboptimal position in the optimal mining group and the worst mining group max , the optimal fitness value (FKT) in each mining group min ) i and its location; (5) During the prospecting phase, the mining center conducts prospecting in the following steps: The mining center position K of each mine i , changed to the optimal fitness value (FKT min ) i Location; For the optimal mining group, and the global optimal fitness value FKD min The mining group where it is located will no longer conduct prospecting, that is, the location of its mining center will remain unchanged; For non-optimal mining groups and non-globally optimal fitness values ​​FKD min The mining group in which the mining group is located selects a high-quality mine mining center as the target according to probability, and conducts prospecting from the mining center of the mining group to the mining center of the high-quality mine. The probability of a high-quality mine being selected is related to the fitness value of its mining center, and the probability is expressed as: Where R(a) represents the probability of the ath high-quality mine being selected, (FKT min ) a represents the optimal fitness value of the a-th high-quality mine mining group, and the moving distance D of the mining center i It is related to the fitness value and the number of iterations. If the i-th mining center selects the a-th high-quality mine, then D i It is expressed as: D i =D t ×(ub-lb)×F ia ×T ia ×(K a -K i ) Among them, D t is the maximum moving step ratio, F ia is the fitness value influencing term, T ia are the influencing terms of the number of iterations, which are: FAVORITE ia =((FKT min ) i -(FKT min ) a ) / (FKD max -FKD min ) T ia =1-e (-rand(0,1)×t / T) Then the initial position of the mining center in the next iteration will be updated to: K i t+1 =K i t +D i t The superscript represents the number of iterations; (6) Human resource deployment stage: for the worst mining group and the second-worst mining group, the mining personnel will be deployed. For the worst mining group, its mining center will be deployed to the second-best position in the best mining group; for the second-worst mining group, the original mining center will be deleted, and a point will be randomly generated in the solution domain as the new mining center. The random generation method is consistent with the mine initialization method. Delete all mining points and determine whether the iteration is over. If the iteration is over, output the optimal parameters. If the iteration is not over, return to stage (2) and repeat the calculation of stages (2) to (6). Step 5: Input the word embedding matrix into the neural network model with optimized hyperparameters, and train the Chinese review text sentiment analysis model based on BERT-MO-CNN-BILSTM; Step 6: Preprocess the text data to be sentimentally classified, generate a word embedding matrix through ChineseBERT, and then input it into the trained Chinese review text sentiment analysis model to obtain sentiment classification.

2. The Chinese review sentiment analysis method of BERT-MO-CNN-BILSTM as claimed in claim 1, characterized in that: In step 3, the parameters required for the mine exploitation optimization algorithm include the number of mines K, the proportion of high-quality mines Kp, where 0 < Kp < 1, the exploitation point base Ma, the exploitation center search range factor Tp, where 0 < Tp < 0.2, the maximum moving step ratio Dt, where 0.01 < Dt < 0.2, and the maximum number of iterations T.

3. The Chinese review sentiment analysis method of BERT-MO-CNN-BILSTM as claimed in claim 1, characterized in that: In step 4, the multi-scale convolutional concatenated bidirectional long short-term memory neural network includes an input layer, a multi-scale convolutional layer and a pooling layer, a merging layer, a BILSTM layer, a fully connected layer and an output layer; among them, the multi-scale convolutional layer and the pooling layer need to perform convolution and pooling operations on convolutional kernels of different sizes respectively and then merge them, and input the merged result into the BILSTM layer.

4. The Chinese review sentiment analysis method of a BERT-MO-CNN-BILSTM as claimed in claim 1, characterized in that: In step 4, the hyperparameters include the number of filters in the convolutional layer, the convolutional kernel sizes of multiple scales, the number of hidden units in the BILSTM, the epoch size and the training batch size.

Images