Multi-feature fusion entity sentiment analysis method and system based on BERT

Abstract

This application discloses a multi-feature fusion entity sentiment analysis method and system based on BERT, the method comprising the following steps: annotating text to obtain entity sentiment data; training a model based on the entity sentiment data to obtain an entity sentiment classification model; and using the entity sentiment classification model to perform entity sentiment analysis on the predicted text. Through this application, the entity sentiment classification performance can be improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology. More specifically, the present invention relates to a BERT-based multi-feature fusion entity sentiment analysis method and system. Background Art

[0002] With the rapid rise and continuous development of numerous internet platforms, people have become accustomed to sharing their thoughts and feelings on these platforms. Gaining insights into user needs and experiences through user-generated content has become a common requirement for many service providers in various fields. Entity sentiment analysis, one of the most popular applications in natural language processing, provides a detailed and effective solution to this need.

[0003] Compared to traditional sentiment classification techniques, entity sentiment classification focuses on more fine-grained, specific entities. The sentiment of specific entities can provide features for different tasks, making the application scenarios more diverse and flexible.

[0004] Currently, sentence-level sentiment analysis methods based on the BERT pre-trained model can achieve relatively advanced performance. They usually consider a single feature (entity semantic feature) or use a more complex model to fuse a certain feature. However, they still have the following shortcomings:

[0005] 1. In the entity sentiment analysis task, the performance was not as good as expected due to the failure to utilize more entity-related features;

[0006] 2. The model does not comprehensively consider the relationship between entity semantic information and entity location information, sentence semantic information, and entity context features. The model that only uses entity semantic features leads to low model performance.

[0007] 3. The model using multi-feature fusion has a complex feature fusion method, high computational overhead, little performance improvement, and high model complexity. Summary of the Invention

[0008] The embodiment of the present application provides a multi-feature fusion entity sentiment analysis method based on BERT to at least solve the problem of subjective factors in related technologies.

[0009] The present invention provides a multi-feature fusion entity sentiment analysis method based on BERT, which includes the following steps:

[0010] Annotation step: annotate the text to obtain entity sentiment data;

[0011] Training step: performing model training based on the entity emotion data to obtain an entity emotion classification model;

[0012] Prediction step: Use the entity sentiment classification model to perform entity sentiment analysis on the predicted text.

[0013] As a further improvement of the present invention, the marking step specifically includes the following steps:

[0014] Construction steps: Build the annotation platform;

[0015] Input step: input the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

[0016] As a further improvement of the present invention, the construction step specifically includes the following steps:

[0017] Rule definition step: define annotation rules;

[0018] Label definition steps: define the annotation label;

[0019] Instance given step: given a labeled instance.

[0020] As a further improvement of the present invention, the training step specifically includes the following steps:

[0021] Verification step: verifying the data format of the entity emotion data and dividing it to obtain a training data set;

[0022] Marking step: marking the position information of the training data set;

[0023] Feature acquisition step: inputting the training data set containing the labeled information into the BERT pre-training model to obtain the hidden layer output feature vector;

[0024] Feature fusion step: obtaining a fused feature vector based on the hidden layer output feature vector;

[0025] Parameter acquisition step: The fused feature vector is probabilized through softmax to obtain model parameters.

[0026] As a further improvement of the present invention, the position information mark in the marking step includes a whole sentence classification identifier mark, an entity start position mark, and an entity end position mark.

[0027] As a further improvement of the present invention, the feature fusion step specifically includes the following steps:

[0028] Combining step: combining the hidden layer output feature vectors by adding and taking the arithmetic average and then inputting them into a feedforward neural network to calculate the whole sentence classification vector, the entity context vector, the entity context vector and the entity vector;

[0029] Splicing step: splicing and fusing the whole sentence classification vector, entity context vector, entity context vector and entity vector to obtain the fused feature vector.

[0030] As a further improvement of the present invention, back-propagation gradient descent is used to adjust the model parameters.

[0031] Based on the same inventive concept, the present invention also discloses a BERT-based multi-feature fusion entity sentiment analysis system based on any of the inventions.

[0032] The BERT-based multi-feature fusion entity sentiment analysis system includes:

[0033] Annotation module, which annotates text and obtains entity sentiment data;

[0034] A training module performs model training based on the entity emotion data to obtain an entity emotion classification model;

[0035] The prediction module uses the entity sentiment classification model to perform entity sentiment analysis on the predicted text.

[0036] As a further improvement of the present invention, the marking module specifically includes:

[0037] Build units and annotation platforms;

[0038] An input unit inputs the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

[0039] As a further improvement of the present invention, the building block specifically includes:

[0040] Rule definition unit, defines the annotation rules;

[0041] Label definition unit, defines the annotation label;

[0042] Instance Given a unit, given annotated instance.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. We propose a BERT-based multi-feature fusion entity sentiment analysis method. By appropriately integrating entity semantic information, entity location information, sentence semantic information, and entity context feature information, we improve the performance of the entity sentiment classification task model.

[0045] 2. In the entity sentiment analysis task, combining multiple feature vectors for entity sentiment analysis effectively improves performance;

[0046] 3. Use multi-feature fusion, the feature fusion method is simple and the computational complexity is reduced.

[0047] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 This is an overall flow chart of a multi-feature fusion entity sentiment analysis method based on BERT provided by an embodiment of the present invention;

[0050] Figure 2 This is an overall flow chart of the technical solution disclosed in this embodiment;

[0051] Figure 3 yes Figure 1 The overall flow chart of step S1 disclosed;

[0052] Figure 4 yes Figure 3 The overall flow chart of step S11 disclosed;

[0053] Figure 5 yes Figure 1 The overall flow chart of step S2 disclosed;

[0054] Figure 6 Schematic diagram of the entity sentiment classification model disclosed in this embodiment;

[0055] Figure 8 This is a structural framework diagram of a multi-feature fusion entity sentiment analysis system based on BERT provided in this embodiment;

[0056] Figure 9 is a framework diagram of a computer device according to an embodiment of the present invention.

[0057] In the above picture:

[0058] 1. Labeling module; 2. Training module; 3. Prediction module; 11. Construction unit; 12. Input unit; 111. Rule definition unit; 112. Label definition unit; 113. Instance given unit; 21. Verification unit; 22. Labeling unit; 23. Feature acquisition unit; 24. Feature fusion unit; 241. Combination unit; 242. Splicing unit; 25. Parameter acquisition unit; 80. Bus; 81. Processor; 82. Memory; 83. Communication interface. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0060] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0061] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0062] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0063] The present invention is described in detail below with reference to the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in this field based on these embodiments are all within the scope of protection of the present invention.

[0064] Before elaborating on various embodiments of the present invention, the core inventive concept of the present invention is summarized and elaborated in detail through the following several embodiments.

[0065] The present invention can perform multi-feature fusion entity sentiment analysis based on BERT, and improve the model performance by appropriately fusing entity semantic information, entity location information, sentence semantic information and entity context feature information.

[0066] Example 1:

[0067] Reference Figures 1 to 7 As shown, this example reveals a specific implementation method of a multi-feature fusion entity sentiment analysis method based on BERT (hereinafter referred to as the "method").

[0068] Specifically, refer to Figure 1 and Figure 2As shown, the method disclosed in this embodiment mainly includes three parts: entity and entity sentiment labeling, model training, and model prediction, including the following steps:

[0069] Step S1: annotate the text and obtain entity sentiment data.

[0070] Specifically, in some of these embodiments, reference is made to Figure 3 The step S1 shown specifically includes the following steps:

[0071] S11. Build a labeling platform;

[0072] S12: Input the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

[0073] Specifically, in some of these embodiments, reference is made to Figure 4 The step S11 shown specifically includes the following steps:

[0074] S111. Define labeling rules;

[0075] S112, defining a label;

[0076] S113. A labeling instance is given.

[0077] Specifically, we use the annotation platform to annotate the entity sentiment task corpus. For example, in the sentence "optimistic about the long-term development potential of Minglu Technology", the annotated entity "Minglu Technology" is annotated and the sentiment of the annotated entity "Minglu Technology" is positive.

[0078] Then refer to Figure 5 and Figure 6 As shown, step S2 is executed to perform model training based on the entity emotion data to obtain an entity emotion classification model.

[0079] Specifically, in some embodiments, step S2 includes the following steps:

[0080] S21, performing data format verification on the entity emotion data, and dividing and obtaining a training data set;

[0081] S22, marking the position information of the training data set;

[0082] S23, inputting the training data set containing the labeled information into the BERT pre-training model to obtain the hidden layer output feature vector;

[0083] S24, obtaining a fused feature vector based on the hidden layer output feature vector;

[0084] S25. Probabilize the fused feature vector through softmax to obtain model parameters.

[0085] Specifically, data is input into the model and model training is performed. First, the position information of the given text containing entities is marked. The position information marking includes the whole sentence classification identifier mark, the entity start position mark, and the entity end position mark. The specific position marking process is: insert the whole sentence classification identifier symbol "[CLS]" at the beginning of the sentence, and insert "[EL]" and "[ER]" to mark the start and end positions of the entity respectively. Let length be the length of the sentence, i and j are the start and end positions of the entity in the sentence after all identifiers are inserted. Taking the sentence "optimistic about the long-term development potential of Minglu Technology" as an example, the sentence after text position information marking should be "[CLS] optimistic about the long-term development potential of [EL] Minglu Technology [ER]".

[0086] Specifically, in step S23, the text containing the marked information is input into the BERT pre-trained model, and the BERT pre-trained model is used to perform fine-tuning on the given corpus for the single sentence task, obtaining the output feature vector of the last hidden layer (denoted as H). The fine-tuned BERT model has stronger representation capabilities and better performance.

[0087] Specifically, refer to Figure 7 The step S24 shown specifically includes the following steps:

[0088] S241, combining the hidden layer output feature vectors by adding and taking the arithmetic average and then inputting them into a feedforward neural network to calculate the whole sentence classification vector, the entity context vector, the entity context vector, and the entity vector;

[0089] S242: Concatenate and fuse the whole sentence classification vector, the entity context vector, the entity context vector, and the entity vector to obtain the fused feature vector.

[0090] Specifically, the hidden layer output feature vectors are combined by adding multiple vectors and taking the arithmetic average and then inputting them into the Feed Forward Neural Network to calculate the whole sentence classification vector H cls , entity context vector H L , entity context vector H R and entity vector H entity .

[0091] Specifically, according to the index, the hidden vector corresponding to the “[CLS]” tag is recorded as H o , the entity vector from left to right is denoted as H i ~H j , the entity vectors from left to right are denoted as H1~H i-2 , the entity context vector from left to right is denoted as Hj+2 ~H length-1 In order of calculation, the formulas are as follows:

[0092] Whole sentence classification vector H cls :H cls =W cls [tanh(H o )]+b cls

[0093] Entity vector H entity :

[0094] Entity context vector H L :

[0095] Entity context vector H R :

[0096] Among them, W is the parameter matrix in the feedforward neural network, the matrix size is dxd, b is the bias matrix, the size is dx1, W cls ∈R d×d , W entity ∈R d×d , b cls ∈R d×1 , b entity ∈R d×1 .

[0097] Specifically, the calculated whole sentence classification vector H cls , entity vector H entity , entity context vector H L and entity context vector H R The features are spliced ​​and then input into the new feedforward neural network for feature fusion to obtain the fused feature vector.

[0098] Specifically, the vector that integrates multiple features is finally probabilized through softmax to obtain the final prediction result. The calculation formula is as follows:

[0099] The fused feature vector: H merge =W merge [concat(H cls ,H L ,H entity ,H R )]+b merge

[0100] Probabilistic eigenvector: p = softmax(H merge )

[0101] Among them, W merge∈R L×4d , b merge ∈R L×1 , L is the number of entity classification labels.

[0102] Specifically, after all feature vectors are probabilized, the label at the position with the maximum probability value is the "prediction" result of the current model parameters for the sentence. In addition, the cross-entropy loss function is used to calculate the loss between this result and the correct label, and back-propagation gradient descent is used to adjust the model parameters.

[0103] Then, step S3 is executed to perform entity sentiment analysis on the predicted text using the entity sentiment classification model.

[0104] A multi-feature fusion entity sentiment analysis method based on BERT disclosed in an embodiment of the present application improves the performance of the entity sentiment classification task model by appropriately fusing entity semantic information, entity position information, sentence semantic information and entity context feature information; in the entity sentiment analysis task, entity sentiment analysis is performed by combining multiple feature vectors to effectively improve performance; by using multi-feature fusion, the feature fusion method is simple and the computational complexity is reduced.

[0105] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent.

[0106] Example 2:

[0107] In combination with the BERT-based multi-feature fusion entity sentiment analysis method disclosed in Example 1, this embodiment discloses a specific implementation example of a BERT-based multi-feature fusion entity sentiment analysis system (hereinafter referred to as the "system").

[0108] Reference Figure 8 As shown, the system includes:

[0109] Annotation module 1 annotates the text and obtains entity sentiment data;

[0110] Training module 2, performing model training based on the entity emotion data to obtain an entity emotion classification model;

[0111] Prediction module 3 uses the entity sentiment classification model to perform entity sentiment analysis on the predicted text.

[0112] Specifically, in some embodiments, the marking module 1 specifically includes:

[0113] Construction unit 11, constructing a labeling platform;

[0114] The input unit 12 inputs the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

[0115] Specifically, in some embodiments, the construction unit 11 specifically includes:

[0116] A rule definition unit 111 defines a marking rule;

[0117] The label definition unit 112 defines the annotation label;

[0118] The instance giving unit 113 gives a labeled instance.

[0119] Specifically, in some embodiments, the training module 2 specifically includes:

[0120] A verification unit 21 performs data format verification on the entity emotion data and divides and obtains a training data set;

[0121] a marking unit 22 for marking position information on the training data set;

[0122] The feature acquisition unit 23 inputs the training data set containing the labeled information into the BERT pre-training model to obtain the hidden layer output feature vector;

[0123] A feature fusion unit 24 is configured to obtain a fused feature vector based on the hidden layer output feature vector;

[0124] The parameter acquisition unit 25 performs probabilistic transformation on the fused feature vector through softmax to obtain model parameters.

[0125] Specifically, in some embodiments, the feature fusion unit 24 specifically includes:

[0126] A combining unit 241 combines the hidden layer output feature vectors by adding and taking an arithmetic average and then inputting the combined feature vectors into a feedforward neural network to calculate a whole sentence classification vector, an entity context vector, an entity context vector, and an entity vector;

[0127] The concatenation unit 242 concatenates and fuses the whole sentence classification vector, the entity context vector, the entity context vector, and the entity vector to obtain the fused feature vector.

[0128] The technical solutions of the multi-feature fusion entity sentiment analysis system based on BERT disclosed in this embodiment and the multi-feature fusion entity sentiment analysis method based on BERT disclosed in Example 1 for the remaining identical parts please refer to Example 1 and will not be repeated here.

[0129] Embodiment three:

[0130] Combine Figure 9 As shown, this embodiment discloses a specific implementation of a computer device. The computer device may include a processor 81 and a memory 82 storing computer program instructions.

[0131] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0132] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these.Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0133] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0134] The processor 81 reads and executes the computer program instructions stored in the memory 82 to implement any one of the BERT-based multi-feature fusion entity sentiment analysis methods in the above embodiments.

[0135] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. Figure 9 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0136] The communication interface 83 is used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication interface 83 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0137] The bus 80 includes hardware, software, or both, and couples the components of the computer device to each other. The bus 80 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, the bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfinityBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Area Network (VESA) bus, or a SATA bus. Bus, referred to as VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 80 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0138] The computer device can realize multi-feature fusion entity sentiment analysis based on BERT, thereby achieving the combination of Figure 1 Described method.

[0139] In addition, in conjunction with the BERT-based multi-feature fusion entity sentiment analysis method in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when executed by a processor, the computer program instructions implement any of the BERT-based multi-feature fusion entity sentiment analysis methods in the above embodiments.

[0140] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] In summary, the beneficial effect of the present invention is that a multi-feature fusion entity sentiment analysis method based on BERT is proposed, which improves the performance of the entity sentiment classification task model by fusing entity semantic information, entity position information, sentence semantic information and entity context feature information in an appropriate way; in the entity sentiment analysis task, entity sentiment analysis is performed by combining multiple feature vectors to effectively improve the performance; by using multi-feature fusion, the feature fusion method is simple and the computational complexity is reduced.

[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A multi-feature fusion entity sentiment analysis method based on BERT, characterized by , including the following steps: Annotation step: annotate the text to obtain entity sentiment data; Training step: performing model training based on the entity emotion data to obtain an entity emotion classification model; Prediction step: using the entity sentiment classification model to perform entity sentiment analysis on the predicted text; The training step specifically includes the following steps: Verification step: verifying the data format of the entity emotion data and dividing and obtaining a training data set; Marking step: marking the position information of the training data set; Feature acquisition step: inputting the training data set containing the labeled information into the BERT pre-training model to obtain the hidden layer output feature vector; Feature fusion step: obtaining a fused feature vector based on the hidden layer output feature vector; Parameter acquisition step: probabilize the fused feature vector through softmax to obtain model parameters; The feature fusion step specifically includes the following steps: Combination step: combining the hidden layer output feature vectors by adding and taking arithmetic average and then inputting them into a feedforward neural network to calculate the whole sentence classification vector, entity context vector, entity context vector and entity vector; Specifically, the calculation formulas of the whole sentence classification vector, the entity context vector, the entity context vector and the entity vector are respectively as follows: H cls =W cls [tanh(H o )]+b cls Among them, H cls is the whole sentence classification vector, H L is the entity context vector, H R is the entity context vector, H entity is the entity vector, H o To mark the corresponding hidden vector, the entity vector is represented from left to right as H i ~H j , the entity above vectors are denoted from left to right as H1~H i-2 , the entity context vector from left to right is denoted as H j+2 ~H length-1 , according to the calculation order, W is the parameter matrix in the feedforward neural network, the matrix size is dxd, b is the bias matrix, the size is dx1, W cls ∈R d×d , W entity ∈R d×d , b cls ∈R d×1 , b entity ∈R d×1 ; Splicing step: splicing and fusing the whole sentence classification vector, entity context vector, entity context vector and entity vector to obtain the fused feature vector; Specifically, the fused feature vector is: H merge =W merge [concat(H cls ,H L ,H entity ,H R )]+b merge Among them, b merge ∈R L×1 , L is the number of entity classification labels; The parameter acquisition step specifically includes: The fused feature vector is probabilized by softmax to obtain the model parameters. The calculation formula is as follows: p=softmax(H merge ) Among them, W merge ∈R L×4d , p is the probability value of the fused feature vector.

2. The multi-feature fusion entity sentiment analysis method based on BERT as claimed in claim 1, characterized in that: The marking step specifically includes the following steps: Construction steps: Build the annotation platform; Input step: input the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

3. The multi-feature fusion entity sentiment analysis method based on BERT as claimed in claim 2, characterized in that: The construction step specifically includes the following steps: Rule definition step: define annotation rules; Label definition steps: define the annotation label; Instance given step: given labeled instance.

4. The multi-feature fusion entity sentiment analysis method based on BERT as claimed in claim 1, characterized in that: The position information mark in the marking step includes a whole sentence classification identifier mark, an entity start position mark, and an entity end position mark.

5. The multi-feature fusion entity sentiment analysis method based on BERT as claimed in claim 1, characterized in that: The model parameters are adjusted using back-propagation gradient descent.

6. A multi-feature fusion entity sentiment analysis system based on BERT, characterized by ,include: Annotation module, which annotates text and obtains entity sentiment data; A training module performs model training based on the entity emotion data to obtain an entity emotion classification model; A prediction module, using the entity sentiment classification model to perform entity sentiment analysis on the predicted text; The training module specifically includes the following units: Verification unit: performs data format verification on the entity emotion data, and divides and obtains a training data set; Marking unit: marking the position information of the training data set; Feature acquisition unit: input the training data set containing the label information into the BERT pre-training model to obtain the hidden layer output feature vector; Feature fusion unit: obtaining a fused feature vector based on the hidden layer output feature vector; Parameter acquisition unit: probabilizes the fused feature vector through softmax to obtain model parameters; Wherein, the feature fusion unit specifically includes: Combining unit: combining the hidden layer output feature vectors by adding and taking arithmetic average and then inputting them into a feedforward neural network to calculate the whole sentence classification vector, entity context vector, entity context vector and entity vector; Specifically, the calculation formulas of the whole sentence classification vector, the entity context vector, the entity context vector and the entity vector are respectively as follows: H cls =W cls [tanh(H o )]+b cls Among them, H cls is the whole sentence classification vector, H L is the entity context vector, H R is the entity context vector, H entity is the entity vector, H o To mark the corresponding hidden vector, the entity vector is represented from left to right as H i ~H j , the entity above vectors are denoted from left to right as H1~H i-2 , the entity context vector from left to right is denoted as H j+2 ~H length-1 , according to the calculation order, W is the parameter matrix in the feedforward neural network, the matrix size is dxd, b is the bias matrix, the size is dx1, W cls ∈R d×d , W entity ∈R d×d , b cls ∈R d×1 , b entity ∈R d×1 ; A concatenation unit: concatenates and fuses the whole sentence classification vector, the entity context vector, the entity context vector and the entity vector to obtain the fused feature vector; Specifically, the fused feature vector is: H merge =W merge [concat(H cls ,H L ,H entity ,H R )]+b merge Among them, b merge ∈R L×1 , L is the number of entity classification labels; Wherein, the parameter acquisition unit specifically includes: The fused feature vector is probabilized by softmax to obtain the model parameters. The calculation formula is as follows: p=softmax(H merge ) Among them, W merge ∈R L×4d , p is the probability value of the fused feature vector.

7. The BERT-based multi-feature fusion entity sentiment analysis system according to claim 6, characterized in that: The marking module specifically includes: Build units and annotation platforms; An input unit inputs the text into the annotation platform to perform artificial entity and entity emotion annotation to obtain the entity emotion data.

8. The BERT-based multi-feature fusion entity sentiment analysis system as claimed in claim 7, characterized in that: The construction unit specifically includes: A rule definition unit defines annotation rules; A label definition unit defines a label. Instance Given a unit, given annotated instance.

Images