Evaluation word and evaluation object extraction method, system and device and readable storage medium

Abstract

The embodiment of the invention relates to the technical field of sentiment analysis, and provides an evaluation word and evaluation object extraction method, system and device and a readable storage medium, and the method comprises the steps: obtaining a comment text, and carrying out word segmentation and word vector conversion preprocessing on the comment text in a word embedding layer through a natural language preprocessing technology, converting the comment text into a numerical vector which can be processed; performing feature extraction on an input word vector through a parallel bidirectional IndGRU layer and a dynamic routing layer; and performing sequence labeling by using a CRF layer, and intercepting an original input text through a labeling sequence to obtain an evaluation word and an evaluation object. According to the technical scheme, the feature is expressed by using the vector in the capsule feature aggregation model, the vector has the length and the direction, feature information capable of being expressed is richer, and the extraction accuracy of the evaluation words and the evaluation objects is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of sentiment analysis, and in particular to a method, system, device and readable storage medium for extracting evaluation words and evaluation objects. Background Art

[0002] At present, the evaluation word and evaluation object extraction method based on syntactic rules requires the support of certain expert knowledge and is easily affected by the completeness and correctness of the defined rules.

[0003] The traditional machine learning evaluation word and evaluation object extraction method requires the use of feature engineering to construct features. Common features include part of speech, position features, or grammatical features. The workload of feature engineering is large, and traditional machine learning model methods mainly use scalar features and cannot make good use of the correlation between abstract features.

[0004] Therefore, it is a technical problem that needs to be solved urgently by those skilled in the art to propose a technical solution that can effectively solve the problem that traditional machine learning models cannot make good use of the correlation between abstract features. Summary of the invention

[0005] The purpose of this application is to solve the problem that the traditional machine learning model method uses scalar features and cannot make good use of the correlation between abstract features. This application provides a method for automatically extracting evaluation words and evaluation objects based on capsule feature aggregation. The capsule feature aggregation model uses vectors to express features. The vector has length and direction, and can express richer feature information, effectively improving the accuracy of extracting evaluation words and evaluation objects. The technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for extracting evaluation words and evaluation objects, the method comprising the following steps:

[0007] Obtain the comment text, perform word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer using natural language preprocessing technology, and convert the comment text into a processable numerical vector;

[0008] Feature extraction of input word vectors is performed through parallel bidirectional IndGRU layers and dynamic routing layers;

[0009] The CRF layer is used for sequence labeling. The original input text is intercepted through the labeled sequence to obtain the evaluation words and evaluation objects.

[0010] Optionally, in creating development branches and design branches on the model design management platform based on the project R&D activity process, the creation of development branches and design branches is based on the production date of the project R&D activity process, and without the project R&D activity process, the development branches and design branches cannot be pulled and the model cannot be modified.

[0011] Furthermore, the word embedding layer includes: receiving a text sentence as input, pre-training a word vector with a length of 100 through a word2vector method, and converting the text sentence into a vector matrix through the pre-trained word vector.

[0012] Furthermore, the parallel bidirectional IndGRU layer includes: replacing part of the matrix of the hidden state in the original GRU unit by Hadamard product to obtain the hidden layer output in the IndGRU unit, and the calculation process is:

[0013] r t =sigmoid(W r x t +U r ⊙h t-1 +b r ) (1)

[0014] z t =sigmoid(W z x t +U z ⊙h t-1 +b z ) (2)

[0015]

[0016] Among them, U r ,U z ,U t is a one-dimensional vector, and “⊙” represents the Hadamard product of the corresponding elements in the vector;

[0017] The relationship between words in the text is mutual. To obtain information after the current time step, a bidirectional IndGRU is used. BiIndGRU uses two separate IndGRU network structures to obtain information from the previous time step and the next time step respectively. is the forward IndGRU unit hidden state output, For the backward IndGRU hidden layer output, use the concatenation vector is the hidden state output of BiIndGRU at time step t.

[0018] Furthermore, the parallel bidirectional IndGRU layer uses several BiIndGRUs to extract features in parallel at the same time, adds a Dropout operation, and constructs capsule features through IndGRU. The calculation formula is as follows:

[0019]

[0020] BiIndGRU i represents the data operation function of the i-th bidirectional independent IndGRU, represents the hidden layer features of the i-th bidirectional independent IndGRU at time t, and represents the initial feature capsule.

[0021] Furthermore, the dynamic routing layer includes a dynamic routing algorithm, which is an algorithm for information transmission between two adjacent layers of capsules in a capsule network;

[0022] In the dynamic routing algorithm, the initial capsule and the output capsule are linked in a fully connected form, and the calculation method is as follows:

[0023]

[0024] in, represents the prediction vector, which is obtained through a fully connected layer, θ j are the parameters of the fully connected layer, It is the initial capsule;

[0025] c ij For the initial capsule and output capsule The coupling coefficient between the initial capsule and the output capsule indicates that the initial capsule transmits some information to the output capsule. Middle, output capsule It is the linear weighted sum of the prediction vectors of all initial capsules for it, calculated as follows:

[0026]

[0027] b ij For the initial capsule Connect to output capsule The prior probability of the initial value is 0, and the coupling coefficient c ij By b ij The coupling coefficient between capsules is calculated using the Leaky-Softmax function as follows:

[0028]

[0029] Where m is the number of initial capsules.

[0030] Furthermore, in the capsule network, the capsule vector represents a certain type of feature, and its length represents the probability of the existence of a certain type of feature. Represents the output capsule, which represents the feature vector of a certain type of feature. It is obtained by the Squashing function. The Squashing function is used to reduce the length of short vectors to greater than 0 and long vectors to less than 1. The calculation method is as follows:

[0031]

[0032] Update b by taking the dot product of the predicted capsule vector and the final output capsule vector ij , calculated as follows:

[0033]

[0034] The final output v of the dynamic routing layer at time t t It is obtained by concatenating the output capsules and is calculated as follows:

[0035]

[0036] Furthermore, the CRF layer includes: inputting the output of the dynamic routing layer into the CRF through the fully connected layer for labeling the text sequence, and obtaining the evaluation words and evaluation objects in the comment text through the labeled sequence labels.

[0037] In a second aspect, the present invention provides a system for extracting evaluation words and evaluation objects, the system comprising: a preprocessing module, a feature extraction module, and an output module, wherein:

[0038] Preprocessing module: used to obtain comment text, perform word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer through natural language preprocessing technology, and convert the comment text into a processable numerical vector;

[0039] Feature extraction module: used to extract features from input word vectors through parallel bidirectional IndGRU layers and dynamic routing layers;

[0040] Output module: used to perform sequence labeling using the CRF layer, intercept the original input text through the labeled sequence, and obtain evaluation words and evaluation objects.

[0041] In a third aspect, the present invention provides an electronic device, including a processor and a memory;

[0042] The memory is used to store operation instructions;

[0043] The processor is used to execute the above-mentioned evaluation word and evaluation object extraction method by calling the operation instruction.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for extracting evaluation words and evaluation objects described above is implemented.

[0045] The evaluation word and evaluation object extraction scheme disclosed in the present application obtains the review text, performs word segmentation and word vector conversion preprocessing on the review text in the word embedding layer through natural language preprocessing technology, and converts the review text into a processable numerical vector; extracts features from the input word vector through the parallel bidirectional IndGRU layer and the dynamic routing layer; uses the CRF layer for sequence annotation, intercepts the original input text through the annotation sequence, and obtains the evaluation words and evaluation objects. The capsule feature aggregation model uses vectors to express features. The vector has length and direction, and can express richer feature information, effectively improving the extraction accuracy of evaluation words and evaluation objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0047] Figure 1 A flowchart of a method for extracting evaluation words and evaluation objects provided in an embodiment of the present application;

[0048] Figure 2 A structural diagram of a method for extracting evaluation words and evaluation objects provided in an embodiment of the present application;

[0049] Figure 3 A structural diagram of a method for extracting evaluation words and evaluation objects provided in an embodiment of the present application;

[0050] Figure 4 A structural diagram of a method for extracting evaluation words and evaluation objects provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the structure of an evaluation word and evaluation object extraction system provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0054] It will be understood by those skilled in the art that, unless otherwise stated, the singular forms "a", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0055] In order to make the objectives, technical solutions and advantages of the present application more clear, the specific implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0056] Before introducing the technical solution of this application, a brief introduction to the application scenario of this application solution is given first:

[0057] Sentiment analysis is the analysis and mining of sentiment information in text data, and the evaluation text contains two words that are crucial to sentiment analysis: evaluation words and evaluation objects. In product reviews, evaluation words are words that express the reviewer's views or emotional tendencies on the product or a certain aspect of the product, and evaluation objects are words that describe the product or a certain aspect of the product. Evaluation words can directly affect the judgment of the reviewer's emotional tendencies, and evaluation object words contain specific fine-grained information about the reviewer's comments. Sometimes it is necessary to combine the unique characteristic attributes of the evaluation entity reflected by the evaluation words and the evaluation object to determine the emotional tendency of the evaluation text. The evaluation word and evaluation object extraction method can be applied to the fine-grained sentiment analysis of system reviews. The extracted evaluation word or evaluation object tuple can be used as input data for fine-grained information analysis of the review text.

[0058] The core of this application is to provide a method for extracting evaluation words and evaluation objects. Figure 1 FIG. 1 is a flow chart of the method for extracting evaluation words and evaluation objects provided in this embodiment. Figure 2 The structure diagram of the evaluation word and evaluation object extraction method provided in this embodiment is shown. In order to better understand the method, Figure 1 and Figure 2 The technical solution of this embodiment is described in detail:

[0059] Step S101, obtaining a comment text, performing word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer using natural language preprocessing technology, and converting the comment text into a processable numerical vector;

[0060] Step S102, extracting features from the input word vector through a parallel bidirectional IndGRU layer and a dynamic routing layer;

[0061] Step S103: Use the CRF layer to perform sequence annotation, intercept the original input text through the annotation sequence, and obtain evaluation words and evaluation objects.

[0062] Among them, the problem of extracting evaluation words and evaluation objects is a sequence labeling problem, which mainly focuses on the position of words in the sentence. This paper adopts the BIO (Begin, Inside, Outside) labeling scheme. Five labels {BT, IT, BO, IO, O} are defined, which represent {the beginning of the evaluation object, the inside of the evaluation object, the beginning of the evaluation word, the inside of the evaluation word, other words}, respectively. A brief example is shown in Table 1. "Love" is an evaluation word, the label is BO, operating is the beginning of the evaluation object, the label is BT, and system is a word in the evaluation object, the label is IT.

[0063] sentence I love the operating system Label O BO O BT IT

[0064] Table 1

[0065] This method takes the system's comment text as input, uses the existing natural language preprocessing method, performs word segmentation and word vector conversion preprocessing on the comment text in the word embedding layer, converts the comment text into processable numerical vectors, and then extracts features from the input word vectors through the parallel bidirectional IndGRU layer and the dynamic routing layer. Finally, the CRF method is used for sequence labeling, and finally the original input text is intercepted by the labeled sequence to obtain the final evaluation words and evaluation objects. This method belongs to a pre-trained model, in which the neural network parameters are obtained by training on the labeled data set.

[0066] Embodiment 1:

[0067] Structure and function of each module

[0068] (1) Word Embedding Layer

[0069] This step mainly processes the natural language of product reviews into numerical data that can be calculated by computers; the word2vector method is used to pre-train word vectors with a length of 100. The word embedding layer accepts text sentences as input and uses pre-trained word vectors to convert the input text into a vector matrix.

[0070] (2) Parallel Bidirectional IndGRU Layer

[0071] GRU is a recurrent neural network that controls the flow of information between time steps through internal gate units to prevent gradient vanishing and gradient explosion problems in long-term memory. GRU has two gate control units: update gate and reset gate. The values ​​of both gates are calculated by the hidden layer state at the previous moment and the input at the current moment. This method uses Hadamard product to replace part of the matrix calculation of the hidden state in the original GRU unit. The calculation process of the hidden layer output in the final IndGRU unit is as follows:

[0072] r t =sigmoid(W r x t +U r ⊙h t-1 +b r ) (1)

[0073] z t =sigmoid(W z x t +U z ⊙h t-1 +b z ) (2)

[0074]

[0075] Among them U r ,U z ,U t They are all one-dimensional vectors, and "⊙" represents the Hadamard product of the corresponding elements in the vector. Ordinary IndGRU only saves information before the current time step. The relationship between words in the text is mutual, so it is necessary to obtain information after the current time step at the same time, and a bidirectional IndGRU is needed. Figure 3 As shown, BiIndGRU uses two separate IndGRU network structures to obtain the information of the previous time step and the next time step respectively. is the forward IndGRU unit hidden state output, For the backward IndGRU hidden layer output, use the concatenation vector is the hidden state output of BiIndGRU at time step t.

[0076] The parallel bidirectional IndGRU layer uses multiple BiIndGRUs to extract features in parallel at the same time, and adds a Dropout operation to improve the generalization effect of the model. The calculation formula for constructing capsule features through IndGRU is as follows:

[0077]

[0078] BiIndGRU irepresents the data operation function of the i-th bidirectional independent IndGRU, represents the hidden layer features of the i-th bidirectional independent IndGRU at time t, and represents the initial feature capsule.

[0079] (3) Dynamic Routing Layer

[0080] The dynamic routing algorithm is an algorithm for information transmission between two adjacent layers of capsules in the capsule network. The dynamic routing algorithm can realize the aggregation of capsule features. The algorithm process is as follows Figure 4 As shown, represents the i-th initial capsule obtained at time t, represents the j-th output capsule after aggregation, Indicates the initial capsule Output Capsule The generated prediction capsule. The algorithm mainly consists of two stages: prediction vector The calculation phase and final output capsule Calculation phase.

[0081] In the dynamic routing algorithm, the initial capsule and the output capsule are linked in a fully connected form, and the prediction vector It is obtained through a fully connected layer and is calculated as follows:

[0082]

[0083] Among them, θ j is the parameter of the fully connected layer. Let c ij For the initial capsule and output capsule The coupling coefficient between them determines how much information the initial capsule transmits to the output capsule. It is the linear weighted sum of the prediction vectors of all initial capsules, calculated as follows:

[0084]

[0085] Assume b ij For the initial capsule Connect to output capsule The prior probability of the initial value is 0, then the coupling coefficient c ij It is through b ij In order to filter out the interference of background information in the text, the Leaky-Softmax function is used to calculate the coupling coefficient between capsules. The calculation method is as follows:

[0086]

[0087] Where m is the number of initial capsules. In the capsule network, the capsule vector represents a certain type of feature, and its length can represent the probability of the existence of a certain type of feature. Let Indicates output capsule The feature vector representing a certain type of feature, The nonlinear mapping function proposed by the Squashing function ensures that the length of short vectors is reduced to near 0 and the length of long vectors is reduced to slightly less than 1. The calculation method is as follows:

[0088]

[0089] In the capsule network, the direction of the capsule vector represents a certain type of feature instance. In order to make the predicted capsule vector convey more information to the output capsule with a direction similar to it, b is updated by the dot product of the predicted capsule vector and the final output capsule vector. ij , calculated as follows:

[0090]

[0091] In the dynamic routing algorithm, it is necessary to dynamically update c iteratively through the steps shown in formulas (7)-(10) ij The value of strengthens the connection strength between capsules with similar directions, so that more predicted capsule information can be transferred to the output capsule vectors with similar directions.

[0092] The final output v of the dynamic routing layer at time t t It is obtained by concatenating the output capsules and is calculated as follows:

[0093]

[0094] The biggest difference between the dynamic routing algorithm used in this embodiment and the original dynamic routing algorithm is that the input and output vectors of the capsule vector are not compressed by the Squashing function. This model is a composite model, and the dynamic routing algorithm is only used for the aggregation part of the features. Squashing may cause data information loss, so the Squashing function is only used in the calculation process of the coupling coefficient between capsules.

[0095] (4) CRF layer

[0096] CRF can use the relationship between the labels of sequence elements to obtain the globally optimal label sequence. This embodiment uses a fully connected layer to input the output of the dynamic routing layer into CRF to achieve the final labeling of the text sequence; finally, the evaluation words and evaluation objects in the comment text can be obtained through sequence label screening. This embodiment includes three platforms: development and operation integration, model design management, and code hosting. Among them, the development and operation integration platform serves as a carrier platform for the project R&D activity process, and is responsible for the full life cycle management of project R&D activities, including the creation of project R&D activity processes and changes in project R&D activity stages (start-demand analysis-development-testing-online-end).

[0097] Embodiment 2:

[0098] The dynamic routing layer can use the Reversed Dynamic Routing strategy to update c ij :

[0099] In common routing policies, c ij It determines how much information the initial capsule transmits to each output capsule. In the reverse routing strategy, c ij Determine how much information the output capsule receives from the initial capsule. The main difference between the normal routing strategy and the reverse dynamic routing strategy is that b ij Get c ij The calculation method of c is in the reverse dynamic routing strategy. ij The calculation method is as follows:

[0100]

[0101] Where m is the number of initial capsules.

[0102] The dynamic routing layer can also use the modulus of the vector to modify the connection strength c between capsules ij Since the capsule vector length in the capsule network can represent the probability of feature existence, the coupling coefficient between capsules can be corrected by the length of the initial capsule vector. The calculation method is as follows:

[0103]

[0104] a i =||u i || (13)

[0105] c ij =a i * leaky-softmax(b ij ) (14)

[0106] where u i Indicates the initial capsule The eigenvector obtained by the Squashing function, a i Represents vector u i Length.

[0107] According to the above embodiments, the technical solution of the present application is further verified by listing examples.

[0108] Example 1: Extracting evaluation words and evaluation objects from system user comments

[0109] (1) Collect system user comment text data, manually annotate the data, annotate the evaluation words and evaluation objects of the comment sentences, construct sample data for training, and use different numbers to represent different sequence labels: {the beginning of the evaluation object, the inside of the evaluation object, the beginning of the evaluation word, the inside of the evaluation word, other words} = {1, 2, 3, 4, 0};

[0110] For example: Comment text: The teachers in this institution are very enthusiastic

[0111] Evaluation object and evaluation words: <Teacher, very enthusiastic>

[0112] Text label sequence: 0000123440

[0113] (2) Train the evaluation word and evaluation object extraction model based on capsule feature aggregation on the training data; obtain the weight parameters of the neural network such as GRU and dynamic routing layer in the model;

[0114] (3) If a user submits a new comment in the system, the new comment text is input into the trained model to obtain the evaluation words and evaluation text of the text;

[0115] For example, the review text: This institution has a good environment

[0116] After the model calculation, the sequence result is: 00001234

[0117] The evaluation object and evaluation words are extracted from the review text through sequence tags: <environment, very good>

[0118] (4) The system can use evaluation words and evaluation object data to perform statistical analysis.

[0119] In the embodiment of the present application, by obtaining the comment text, the comment text is preprocessed by segmentation and word vector conversion in the word embedding layer through natural language preprocessing technology, and the comment text is converted into a processable numerical vector; the input word vector is extracted by the parallel bidirectional IndGRU layer and the dynamic routing layer; the CRF layer is used for sequence annotation, and the original input text is intercepted by the annotation sequence to obtain the evaluation words and evaluation objects. The capsule feature aggregation model uses vectors to express features. The vector has length and direction, and can express richer feature information, which effectively improves the extraction accuracy of evaluation words and evaluation objects.

[0120] based on Figure 1 The embodiment shown provides a method for extracting evaluation words and evaluation objects. Figure 5 A system for extracting evaluation words and evaluation objects provided by an embodiment of the present application is shown. Figure 5 As shown, the system can mainly include: 201 pre-processing module, 202 feature extraction module, 203 output module, wherein,

[0121] 201 Preprocessing module: used to obtain comment text, perform word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer through natural language preprocessing technology, and convert the comment text into a processable numerical vector;

[0122] 202 feature extraction module: used to extract features from input word vectors through a parallel bidirectional IndGRU layer and a dynamic routing layer;

[0123] 203 output module: used for sequence labeling using the CRF layer, intercepting the original input text through the labeling sequence, and obtaining evaluation words and evaluation objects.

[0124] In the embodiment of the present application, by obtaining the comment text, the comment text is preprocessed by segmentation and word vector conversion in the word embedding layer through natural language preprocessing technology, and the comment text is converted into a processable numerical vector; the input word vector is extracted by the parallel bidirectional IndGRU layer and the dynamic routing layer; the CRF layer is used for sequence annotation, and the original input text is intercepted by the annotation sequence to obtain the evaluation words and evaluation objects. The capsule feature aggregation model uses vectors to express features. The vector has length and direction, and can express richer feature information, which effectively improves the extraction accuracy of evaluation words and evaluation objects.

[0125] It can be understood that the above modules of the evaluation word and evaluation object extraction system in this embodiment have the function of realizing Figure 1The functions of the corresponding steps of the method in the embodiment shown in . The function can be implemented by hardware, or by hardware executing the corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of the above modules, please refer to Figure 1 The corresponding description of the method in the embodiment shown in will not be repeated here.

[0126] An embodiment of the present application provides an electronic device, including a processor and a memory;

[0127] A memory, used for storing operation instructions;

[0128] The processor is used to execute the evaluation word and evaluation object extraction method provided in any embodiment of the present application by calling an operation instruction.

[0129] As an example, Figure 6 A schematic diagram of the structure of an electronic device applicable to the embodiment of the present application is shown. Figure 6 As shown, the electronic device 2000 includes: a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, such as through a bus 2002. Optionally, the electronic device 2000 may also include a transceiver 2004. It should be noted that in actual applications, the transceiver 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.

[0130] The processor 2001 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 2004 may include a receiver and a transmitter. The transceiver 2004 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0131] Processor 2001 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0132] The bus 2002 may include a path to transmit information between the above components. The bus 2002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0133] The memory 2003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0134] Optionally, the memory 2003 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement the evaluation word and evaluation object extraction method provided in any embodiment of the present application.

[0135] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method and will not be described in detail here.

[0136] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the evaluation word and evaluation object extraction method shown in the above method embodiment is implemented.

[0137] The computer-readable storage medium provided in the embodiments of the present application is applicable to any embodiment of the above method and will not be described in detail here.

[0138] The evaluation word and evaluation object extraction scheme provided in the embodiment of the present application obtains the review text, performs word segmentation and word vector conversion preprocessing on the review text in the word embedding layer through natural language preprocessing technology, and converts the review text into a processable numerical vector; extracts features from the input word vector through the parallel bidirectional IndGRU layer and the dynamic routing layer; uses the CRF layer for sequence annotation, intercepts the original input text through the annotation sequence, and obtains the evaluation words and evaluation objects. The capsule feature aggregation model uses vectors to express features. The vector has length and direction, and can express richer feature information, effectively improving the extraction accuracy of evaluation words and evaluation objects.

[0139] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the specific system can be divided into different functional modules to complete all or part of the functions described above.

[0140] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0141] In several embodiments covered by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, or it can be an electrical, mechanical or other form of connection.

[0142] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0143] In addition, each functional module in each embodiment of the present application can be implemented in the form of hardware or in the form of software functional modules. If these functional modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or part of the prior art that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer program product is stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, a computer instruction can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)).

[0144] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for extracting evaluation words and evaluation objects, characterized in that: The following steps are involved: Obtain the comment text, perform word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer using natural language preprocessing technology, and convert the comment text into a processable numerical vector; Feature extraction of input word vectors is performed through parallel bidirectional IndGRU layers and dynamic routing layers; The CRF layer is used for sequence labeling. The original input text is intercepted through the labeled sequence to obtain the evaluation words and evaluation objects.

2. The method for extracting evaluation words and evaluation objects according to claim 1, characterized in that: The word embedding layer includes: receiving a text sentence as an input, pre-training a word vector with a length of 100 through a word2vector method, and converting the text sentence into a vector matrix through the pre-trained word vector.

3. The method for extracting evaluation words and evaluation objects according to claim 1, characterized in that: The parallel bidirectional IndGRU layer includes: replacing part of the matrix of the hidden state in the original GRU unit by Hadamard product to obtain the hidden layer output in the IndGRU unit, the calculation process is: r t =sigmoid(W r x t +U r ⊙h t-1 +b r ) (1) z t =sigmoid(W z x t +U z ⊙h t-1 +b z ) (2) Among them, U r ,U z ,U t is a one-dimensional vector, "⊙" represents the Hadamard product of the corresponding elements in the vector; The relationship between words in the text is mutual. At the same time, we obtain information after the current time step. We use bidirectional IndGRU. BiIndGRU uses two separate IndGRU network structures to obtain information from the previous time step and the next time step respectively. is the forward IndGRU unit hidden state output, For the backward IndGRU hidden layer output, use the concatenation vector is the hidden state output of BiIndGRU at time step t.

4. The method for extracting evaluation words and evaluation objects according to claim 3, characterized in that: The method comprises: The parallel bidirectional IndGRU layer uses several BiIndGRUs to extract features in parallel at the same time, adds a Dropout operation, and constructs capsule features through IndGRU. The calculation formula is as follows: BiIndGRU i represents the data operation function of the i-th bidirectional independent IndGRU, represents the hidden layer features of the i-th bidirectional independent IndGRU at time t, and represents the initial feature capsule.

5. The method for extracting evaluation words and evaluation objects according to claim 1, characterized in that: The dynamic routing layer includes a dynamic routing algorithm, which is an algorithm for information transmission between two adjacent layers of capsules in a capsule network; In the dynamic routing algorithm, the initial capsule and the output capsule are linked in a fully connected form, and the calculation method is as follows: in, represents the prediction vector, which is obtained through a fully connected layer, θ j are the parameters of the fully connected layer, It is the initial capsule; c ij For the initial capsule and output capsule The coupling coefficient between the initial capsule and the output capsule indicates that the initial capsule transmits some information to the output capsule. Middle, output capsule It is the linear weighted sum of the prediction vectors of all initial capsules for it, calculated as follows: b ij For the initial capsule Connect to output capsule The prior probability of the initial value is 0, and the coupling coefficient c ij By b ij The coupling coefficient between capsules is calculated using the Leaky-Softmax function as follows: Where m is the number of initial capsules.

6. The method for extracting evaluation words and evaluation objects according to claim 5, characterized in that: In the capsule network, the capsule vector represents a certain type of feature, and its length represents the probability of the existence of a certain type of feature. Represents the output capsule, which represents the feature vector of a certain type of feature. It is obtained by the Squashing function. The Squashing function is used to reduce the length of short vectors to greater than 0 and long vectors to less than 1. The calculation method is as follows: Update b by taking the dot product of the predicted capsule vector and the final output capsule vector ij , calculated as follows: The final output v of the dynamic routing layer at time t t It is obtained by concatenating the output capsules and is calculated as follows:

7. The method for extracting evaluation words and evaluation objects according to claim 1, characterized in that: The CRF layer includes: inputting the output of the dynamic routing layer into the CRF through the fully connected layer for labeling the text sequence, and filtering and obtaining the evaluation words and evaluation objects in the review text through the labeled sequence labels.

8. A system for extracting evaluation words and evaluation objects, characterized in that: The system includes: a preprocessing module, a feature extraction module, and an output module, wherein: Preprocessing module: used to obtain comment text, perform word segmentation and word vector conversion preprocessing on the comment text at the word embedding layer through natural language preprocessing technology, and convert the comment text into a processable numerical vector; Feature extraction module: used to extract features from input word vectors through parallel bidirectional IndGRU layers and dynamic routing layers; Output module: used to perform sequence labeling using the CRF layer, intercept the original input text through the labeled sequence, and obtain evaluation words and evaluation objects.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store operation instructions; The processor is used to execute the method according to any one of claims 1 to 7 by calling the operation instruction.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images