Phrase extraction model training, phrase extraction method, device, equipment and medium

Abstract

The application discloses a phrase extraction model training method and device, a phrase extraction method and device, computer equipment and a storage medium. The method comprises the following steps: inputting a training phrase feature into a feature recognition module to obtain a first probability value, and inputting a training phrase vector into a phrase classification module to obtain a second probability value; determining a predicted phrase label corresponding to the training phrase according to the first probability value and the second probability value, and determining a predicted loss value according to a target phrase label and the predicted phrase label; when the predicted loss value does not reach a preset convergence condition, iteratively updating initial parameters in a preset extraction model until the predicted loss value reaches the convergence condition, and then recording the preset extraction model after the convergence as a phrase extraction model. The application improves the efficiency and accuracy of phrase extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of classification models, and in particular to a phrase extraction model training method and device, a phrase extraction method and device, equipment and a medium. BACKGROUND

[0002] With the development of science and technology, natural language processing technology has also developed rapidly. Natural language processing technology is used in many fields. For example, keyword extraction, phrase extraction, or text translation processing.

[0003] In the prior art, a supervised model or an unsupervised model is generally trained by a large amount of data, and then phrase extraction is performed by the supervised model or the unsupervised model. However, both models need to be manually screened to verify the extracted phrases after phrase extraction. As a result, the phrase extraction efficiency and accuracy are low. SUMMARY

[0004] The present application provides a phrase extraction model training method, a phrase extraction method, a device, a computer device and a storage medium to solve the problem of low efficiency and accuracy of phrase extraction in the prior art.

[0005] A phrase extraction model training method comprises:

[0006] At least one training phrase group is obtained. One of the training phrase groups comprises a training phrase, a target phrase label, a training phrase feature, and a training phrase vector.

[0007] A preset extraction model comprising initial parameters is obtained. The preset extraction model comprises a feature recognition module and a phrase classification module.

[0008] The training phrase feature is input into the feature recognition module to obtain a first probability value, and the training phrase vector is input into the phrase classification module to obtain a second probability value.

[0009] A predicted phrase label corresponding to the training phrase is determined according to the first probability value and the second probability value, and a predicted loss value is determined according to the target phrase label and the predicted phrase label.

[0010] When the predicted loss value does not reach a preset convergence condition, the initial parameters in the preset extraction model are iteratively updated until the predicted loss value reaches the convergence condition. The preset extraction model after convergence is recorded as a phrase extraction model.

[0011] A phrase extraction method comprises:

[0012] receive a phrase extraction instruction containing a text to be extracted, and perform word segmentation processing on the text to be extracted to obtain at least one to-be-recognized phrase;

[0013] determine a to-be-recognized feature and a to-be-recognized vector corresponding to the to-be-recognized phrase, and input the to-be-recognized feature and the to-be-recognized vector into a phrase extraction model; the phrase extraction model is generated according to the above phrase extraction model training method;

[0014] determine a feature probability value corresponding to the to-be-recognized feature through a feature recognition module in the phrase extraction model, and determine a classification probability value corresponding to the to-be-recognized vector through a phrase classification module in the phrase extraction model;

[0015] determine a phrase probability value according to the feature probability value and the classification probability value corresponding to the same to-be-recognized phrase, and compare the phrase probability value with a preset probability threshold;

[0016] take the to-be-recognized phrase corresponding to the phrase probability value greater than or equal to the preset probability threshold as a target phrase of the text to be extracted.

[0017] A phrase extraction model training device comprises:

[0018] a training phrase acquisition module configured to acquire at least one training phrase group; one training phrase group comprises a training phrase, a target phrase label, a training phrase feature, and a training phrase vector;

[0019] a model acquisition module configured to acquire a preset extraction model containing initial parameters; the preset extraction model comprises a feature recognition module and a phrase classification module;

[0020] a first probability value determination module configured to input the training phrase feature into the feature recognition module to obtain a first probability value, and input the training phrase vector into the phrase classification module to obtain a second probability value;

[0021] a loss value determination module configured to determine a predicted phrase label corresponding to the training phrase according to the first probability value and the second probability value, and determine a predicted loss value according to the target phrase label and the predicted phrase label;

[0022] a model update module configured to iteratively update the initial parameters in the preset extraction model when the predicted loss value does not reach a preset convergence condition, until the predicted loss value reaches the convergence condition, and record the preset extraction model after convergence as a phrase extraction model.

[0023] A phrase extraction device comprises:

[0024] The word segmentation processing module is configured to receive a phrase extraction instruction containing a text to be extracted, and perform word segmentation processing on the text to be extracted to obtain at least one to-be-recognized phrase.

[0025] The feature determination module is configured to determine a to-be-recognized feature and a to-be-recognized vector corresponding to the to-be-recognized phrase, and input the to-be-recognized feature and the to-be-recognized vector into the phrase extraction model.

[0026] The second probability value determination module is configured to determine a feature probability value corresponding to the to-be-recognized feature by using the feature recognition module in the phrase extraction model, and determine a classification probability value corresponding to the to-be-recognized vector by using the phrase classification module in the phrase extraction model.

[0027] The probability value comparison module is configured to determine a phrase probability value according to the feature probability value and the classification probability value corresponding to the same to-be-recognized phrase, and compare the phrase probability value with a preset probability threshold.

[0028] The target phrase determination module is configured to determine a to-be-recognized phrase corresponding to a phrase probability value greater than or equal to the preset probability threshold as a target phrase of the text to be extracted.

[0029] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned phrase extraction model training method or the above-mentioned phrase extraction method when executing the computer program.

[0030] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned phrase extraction model training method or the above-mentioned phrase extraction method.

[0031] The above-mentioned phrase extraction model training method, phrase extraction method, device, computer device, and storage medium. The method can focus on training discrete features of a phrase (i.e., the above-mentioned training phrase feature) through a feature recognition module and focus on training vector features of a phrase (i.e., the above-mentioned training phrase vector) through a phrase classification module by setting the feature recognition module and the phrase classification module in a preset extraction model. Thus, the method realizes the judgment of whether a training phrase is a real phrase based on discrete feature analysis and vector feature analysis. In this way, the efficiency and accuracy of phrase extraction are improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0033] Figure 1 is an application environment diagram of the phrase extraction model training method or the phrase extraction method in an embodiment of the present application;

[0034] Figure 2 is a flowchart of the phrase extraction model training method in an embodiment of the present application;

[0035] Figure 3 is a flowchart of the phrase extraction method in an embodiment of the present application;

[0036] Figure 4 is a principle block diagram of the phrase extraction model training device in an embodiment of the present application;

[0037] Figure 5 is a principle block diagram of the phrase extraction device in an embodiment of the present application;

[0038] Figure 6 is a schematic diagram of the computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions of the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative labor under the premise of the present application, all belong to the scope of protection of the present application.

[0040] The phrase extraction model training method provided by the embodiments of the present application can be applied in the application environment as shown in Figure 1 . Specifically, the phrase extraction model training method is applied in a phrase extraction model training system, and the phrase extraction model training system includes an application environment as shown in Figure 1The illustrated client and server communicate through a network to solve the problem of low efficiency and accuracy of phrase extraction in the prior art. The client, also known as the user end, is a program that provides local services for the client corresponding to the server. The client can be installed on various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, but is not limited to these. The server can be implemented by a standalone server or a server cluster composed of multiple servers. The server can be a standalone server or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, etc. Basic cloud computing services.

[0041] In an embodiment, as shown in Figure 2 A phrase extraction model training method is provided. The method is applied to the server in Figure 1 for example, which includes the following steps:

[0042] S10: Obtain at least one training phrase group; each training phrase group includes a training phrase, a target phrase label, training phrase features, and a training phrase vector.

[0043] It can be understood that the training phrase can be collected from different websites or extracted from corpus text (such as books or artificial intelligence dialogues). The target phrase label is used to represent whether the training phrase is a real phrase. That is, the target phrase label includes a label representing that the training phrase is a real phrase, and also includes a label representing that the training phrase is not a real phrase (such as the training phrase being a word or a complete sentence). The target phrase label can be obtained by manual annotation in advance. The training phrase features are discrete features of the training phrase. For example, the training phrase features can include the word frequency of the training phrase, the number of times the training phrase appears in quotes, the number of times the training phrase appears in parentheses, whether the first word of the training phrase is a stop word or whether the last word is a stop word, etc. The training phrase vector is a vector representation of the training phrase. The training phrase vector can be generated by converting the training phrase into a vector through a text processing model (such as a bert model).

[0044] Further, in this embodiment, the phrase label, training phrase features, and training phrase vector in the training phrase group are associated with the training phrase in the training phrase group.

[0045] S20: Obtain a preset extraction model containing initial parameters; the preset extraction model includes a feature recognition module and a phrase classification module.

[0046] It can be understood that the preset extraction model is a model for predicting whether a training phrase is a real phrase. In the preset extraction model, a feature recognition module and a phrase classification module are set. The feature recognition module is a logistic regression module, which is used to determine whether a training phrase is a real phrase according to training phrase features, and output a probability value of the training phrase being a real phrase. The phrase classification module can be constructed using a textcnn model, which is used to determine whether a training phrase is a real phrase according to a training phrase vector, and output a probability value of the training phrase being a real phrase.

[0047] S30: input the training phrase features into the feature recognition module to obtain a first probability value, and input the training phrase vector into the phrase classification module to obtain a second probability value.

[0048] It can be understood that the first probability value represents the probability that the feature recognition module determines that the training phrase is a real phrase based on the training phrase features. The second probability value represents the probability that the phrase classification module determines that the training phrase is a real phrase based on the training phrase vector.

[0049] Specifically, after obtaining the preset extraction model, the training phrase features and the training phrase vector of the training phrase can be input into the preset extraction model. The feature recognition module in the preset extraction model determines the phrase feature distribution of the training phrase features, so as to determine the first probability value of the training phrase corresponding to the training phrase features being a real phrase according to the phrase feature distribution. The phrase classification module in the preset extraction model determines whether the training phrase vector belongs to the phrase vector of a real phrase, so as to output the second probability value of the training phrase corresponding to the training phrase vector being a real phrase.

[0050] S40: determine the predicted phrase label corresponding to the training phrase according to the first probability value and the second probability value, and determine the predicted loss value according to the target phrase label and the predicted phrase label.

[0051] It can be understood that the predicted phrase label represents the label that the preset extraction model predicts whether the training phrase is a real phrase. The predicted phrase label can include a label representing that the training phrase is a real phrase, and also include a label representing that the training phrase is not a real phrase. The predicted loss value represents the degree of difference between the target phrase label and the predicted phrase label.

[0052] Specifically, after the first probability value and the second probability value are obtained, the sum of the first probability value and the second probability value corresponding to the same training phrase can be determined as a probability value sum. The probability value sum is compared with a preset probability value threshold (the preset probability value threshold can be set according to training requirements, for example, the preset probability value threshold is set to 0.7, etc.), and a comparison result is obtained, so that the predicted phrase label corresponding to the training phrase is determined according to the comparison result. After the predicted phrase label is determined, the predicted loss value can be determined according to the target phrase label and the predicted phrase label corresponding to the same training phrase. The predicted loss value can be determined according to the target phrase label and the predicted phrase label by using a loss function such as a cross-entropy loss function or an absolute value loss function.

[0053] The comparison result can be a result representing that the probability value sum is greater than or equal to the preset probability value threshold, or a result representing that the probability value sum is less than the preset probability value threshold. Further, when the comparison result represents that the probability value sum is greater than or equal to the preset probability value threshold, the predicted phrase label is a label representing that the training phrase is a real phrase. When the comparison result represents that the probability value sum is less than the preset probability value threshold, the predicted phrase label is a label representing that the training phrase is not a real phrase.

[0054] S50: When the predicted loss value does not reach a preset convergence condition, the initial parameters in the preset extraction model are iteratively updated until the predicted loss value reaches the convergence condition, and the preset extraction model after convergence is recorded as a phrase extraction model.

[0055] It can be understood that the convergence condition can be a condition that the predicted loss value is less than a set threshold, that is, when the predicted loss value is less than the set threshold, the training is stopped; the convergence condition can also be a condition that the predicted loss value is very small and cannot be further reduced after 10000 calculations, that is, when the predicted loss value is very small and cannot be reduced after 10000 calculations, the training is stopped, and the preset extraction model after convergence is recorded as a phrase extraction model.

[0056] Further, after determining the predicted phrase label corresponding to the training phrase according to the first probability value and the second probability value, and determining the predicted loss value according to the target phrase label and the predicted phrase label, if the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset extraction model are adjusted according to the predicted loss value. The training phrase feature and the training phrase vector corresponding to the training phrase are re-input into the preset extraction model with the adjusted initial parameters, so that another training phrase is selected when the predicted loss value of the training phrase reaches the preset convergence condition, and the steps S30 and S40 are performed to obtain the predicted loss value corresponding to the training phrase, and the initial parameters of the preset extraction model are adjusted again according to the predicted loss value when the predicted loss value does not reach the preset convergence condition, so that the predicted loss value of the training phrase reaches the preset convergence condition.

[0057] In this way, after the preset extraction model is trained by all the training phrases, the result output by the preset extraction model can be continuously close to the accurate result, and the phrase recognition accuracy is higher and higher, until the predicted loss values of all the training phrases reach the preset convergence condition, and the preset extraction model after convergence is recorded as the phrase extraction model.

[0058] In the embodiment, by setting the feature recognition module and the phrase classification module in the preset extraction model, the feature recognition module can focus on the discrete features (i.e., the training phrase features) of the training phrase, and the phrase classification module can focus on the vector features (i.e., the training phrase vectors) of the training phrase. Thus, the judgment of whether the training phrase is a real phrase based on the discrete feature analysis and the vector feature analysis is realized, and the efficiency and accuracy of the phrase extraction are improved.

[0059] In an embodiment, before the step S10, i.e., before the at least one training phrase group is obtained, the method further includes:

[0060] (1) obtaining a sample corpus set, and performing word segmentation processing on the sample corpus in the sample corpus set to obtain at least one candidate phrase.

[0061] It can be understood that the sample corpus can be collected from different websites by a crawler technology, or obtained by performing text recognition on a text image obtained by shooting a book. Then, all the collected sample corpora are constructed into a sample corpus set.

[0062] Specifically, after obtaining the sample corpus, the sample corpus in the sample corpus can be segmented by the Jieba segmentation method to obtain candidate phrases in the sample corpus. The number of words contained in the candidate phrases extracted from the sample corpus is less than a preset number (the preset number can be set to 5, 6 or 7). For example, when the preset number is set to 6, after the sample corpus is segmented by the Jieba segmentation method, the words obtained by segmentation are integrated to obtain candidate phrases. In the process of integrating words into candidate phrases, it is ensured that the length of the words contained in each candidate phrase cannot exceed 6.

[0063] (2) obtaining positive sample phrases from a preset third-party platform; one of the positive sample phrases corresponds to a positive sample label.

[0064] Understandably, the preset third-party platform can be a third-party website or a unit for storing positive sample phrases after manually collecting positive sample phrases. The positive sample phrase refers to a real phrase. That is, there is no possibility of words or sentences in the positive sample phrase. The positive sample label indicates that the positive sample phrase is a real phrase. The positive sample label can be obtained by manual annotation.

[0065] (3) determining negative sample phrases from all the candidate phrases according to the positive sample phrases and the candidate phrases, and setting negative sample labels corresponding to the negative sample phrases.

[0066] Specifically, after obtaining the positive sample phrases from the preset third-party platform, the difference set between the positive sample phrases and all the candidate phrases can be determined as the negative sample phrases. That is, the candidate phrases and the positive sample phrases are matched. If the candidate phrase does not match any positive sample phrase, the candidate phrase is considered as a negative sample phrase. If the candidate phrase matches any positive sample phrase, the candidate phrase is considered as a positive sample phrase. Understandably, all candidate phrases are considered as a set, and all positive sample phrases are considered as another set. Then, the candidate phrases belonging to the candidate phrase set but not belonging to the positive sample set are considered as negative sample phrases. The candidate phrases belonging to the candidate phrase set and also belonging to the positive sample set are considered as positive sample phrases. The negative sample label represents that the negative sample phrase is not a real phrase.

[0067] (4) generating the training phrase group according to the positive sample phrases, the negative sample phrases, the positive sample labels corresponding to the positive sample phrases, and the negative sample labels corresponding to the negative sample phrases.

[0068] Specifically, after determining the positive sample phrases, negative sample phrases, the positive sample labels corresponding to the positive sample phrases, and the negative sample labels corresponding to the negative sample phrases, the positive sample phrases can be regarded as training phrases, and the positive sample labels can be regarded as the target phrase labels corresponding to the positive sample phrases. Alternatively, the negative sample phrases can be regarded as training phrases, and the negative sample labels can be regarded as the target phrase labels corresponding to the negative sample phrases. Thus, multiple training phrase groups can be constructed using all the positive and negative sample phrases according to the above method.

[0069] In one embodiment, generating the training phrase set based on the positive sample phrase, the negative sample phrase, the positive sample label corresponding to the positive sample phrase, and the negative sample label corresponding to the negative sample phrase includes:

[0070] (1) Perform discrete feature extraction on the positive sample phrase and the negative sample phrase to obtain the training phrase features corresponding to the positive sample phrase and the training phrase features corresponding to the negative sample phrase.

[0071] Understandably, the above description indicates that training phrase features may include word frequency, the number of times the training phrase appears in quotation marks, the number of times the training phrase appears in parentheses, and whether the first or last word of the training phrase is a stop word. Therefore, discrete feature statistics can be performed on positive and negative sample phrases. For example, since both positive and negative sample phrases can be obtained from website crawling or books, statistics can be performed on all websites or all books. For instance, the number of times a positive or negative sample phrase appears in parentheses in an article, the number of times a positive or negative sample phrase appears in quotation marks in an article, or stop word detection can be performed on positive or negative sample phrases (e.g., pre-constructing a stop word set, and then matching the first or last word of the positive or negative sample phrase with the stop words in the stop word set to determine whether the first or last word of the positive or negative sample phrase is a stop word).

[0072] (2) Perform vector transformation on the positive sample phrase and the negative sample phrase to obtain the training phrase vector corresponding to the positive sample phrase and the training phrase vector corresponding to the negative sample phrase.

[0073] Specifically, after obtaining the positive and negative sample phrases, they can be segmented into word segments to obtain segmented positive and negative sample phrases. These segmented phrases are then input into a text processing model (such as the TextCNN model). The text processing model determines the word embedding vector, sentence block tensor, and positional encoding tensor of each word segment in the positive sample phrase. Based on these parameters, the training phrase vector corresponding to the positive sample phrase is determined. Similarly, the text processing model determines the word embedding vector, sentence block tensor, and positional encoding tensor of each word segment in the negative sample phrase. Based on these parameters, the training phrase vector corresponding to the negative sample phrase is determined.

[0074] (3) Construct the training phrase group based on the positive sample phrase, the positive sample label corresponding to the positive sample phrase, the training phrase feature and the training phrase vector, and construct the training phrase group based on the negative sample phrase, the negative sample label corresponding to the negative sample phrase, the training phrase feature and the training phrase vector.

[0075] Specifically, after determining the training phrase vectors corresponding to positive and negative sample phrases, the positive sample phrase can be considered as a training phrase, and the positive sample label as the target phrase label corresponding to that positive sample phrase. Then, the target phrase label, training phrase features, and training phrase vector corresponding to the same positive sample phrase are associated with that positive sample phrase, thus obtaining a training phrase group. Similarly, negative sample phrases can be considered as training phrases, and negative sample labels as the target phrase labels corresponding to that negative sample phrase. Then, the target phrase label, training phrase features, and training phrase vector corresponding to the same negative sample phrase are associated with that negative sample phrase, thus obtaining a training phrase group. In this way, multiple training phrase groups can be constructed using the above method.

[0076] In one embodiment, step S30, namely, inputting the training phrase features into the feature recognition module to obtain a first probability value, includes:

[0077] (1) The feature recognition module determines the phrase feature distribution corresponding to the training phrase features.

[0078] Specifically, after obtaining the preset extraction model, the training phrase features are input into the feature recognition module of the preset extraction model. A distribution coordinate system is pre-constructed in this feature recognition module, which contains a boundary line. The feature recognition module then determines the phrase feature distribution corresponding to the training phrase features, that is, all training phrase features are set in this distribution coordinate system. The position of the training phrase features in this distribution coordinate system is the phrase feature distribution.

[0079] (2) Determine the first probability value based on the phrase feature distribution.

[0080] Specifically, after determining the phrase feature distribution corresponding to the training phrase features through the feature recognition module, a first probability value can be determined based on the phrase feature distribution. That is, as mentioned above, the feature recognition module pre-constructs a distribution coordinate system, and a dividing line exists within this system. The more training phrase features that are above and far from this dividing line, the higher the probability that the training phrase is a true phrase. In this case, the first probability value can be 0.8 or 0.9. The more training phrase features that are below and far from this dividing line, the higher the probability that the training phrase is not a true phrase. In this case, the first probability value can be 0.1 or 0.2. If all training phrase features are near the dividing line, the training phrase may or may not be a true phrase. In this case, the first probability value can be around 0.5. Thus, the positional relationship between each training phrase feature and the dividing line on the distribution coordinate system can be determined based on the phrase feature distribution, thereby obtaining the first probability value.

[0081] In one embodiment, inputting the training phrase vector into the phrase classification module to obtain a second probability value includes:

[0082] (1) The training phrase vectors are convolved by the convolutional network layer in the phrase classification module to obtain convolutional feature vectors.

[0083] Specifically, after obtaining the preset extraction model, the training phrase vectors are convolved by the convolution kernels (the number and size of the convolution kernels are not limited in this embodiment) of the convolutional network layer in the phrase classification module of the preset extraction model, thereby extracting the features in the training phrase vectors and obtaining the convolutional feature vectors.

[0084] (2) The convolutional feature vector is pooled through the pooling network layer in the phrase classification module to obtain the pooled feature vector.

[0085] Specifically, after convolutional processing of the training phrase vectors through the convolutional network layer in the phrase classification module to obtain convolutional feature vectors, the convolutional feature vectors are input into a pooling network layer (which can be an average pooling layer or a max pooling layer). The pooling network layer then performs pooling processing on the convolutional feature vectors to obtain pooled feature vectors.

[0086] (3) The second probability value is determined by the fully connected layer in the phrase classification module based on the pooling feature vector.

[0087] Specifically, after pooling the convolutional feature vector through the pooling network layer in the phrase classification module to obtain the pooled feature vector, the convolutional feature vector is input into the fully connected layer in the phrase classification module. The fully connected layer classifies the training phrase based on the pooled feature vector, that is, determines the probability value of the training phrase belonging to the real phrase, i.e., the second probability value.

[0088] In one embodiment, such as Figure 3 As shown, a phrase extraction method is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included:

[0089] S01: Receive a phrase extraction instruction containing the text to be extracted, and perform word segmentation on the text to be extracted to obtain at least one phrase to be identified.

[0090] Understandably, phrase extraction instructions can be sent directly by the user through the client, or they can be automatically generated after the user uploads the text to be extracted through the client. The text to be extracted can be any Chinese text. For example, the text to be extracted can be text from a book, or it can be dialogue text in a smart customer service scenario.

[0091] Specifically, after receiving a phrase extraction instruction containing the text to be extracted, the text to be extracted can be segmented using the Jieba word segmentation method to obtain all the phrases to be identified in the text. The number of words in each phrase extracted from the text is less than a preset number (the preset number can be set to 5, 6, or 7). For example, when the preset number is set to 6, after segmenting the text using the Jieba word segmentation method, all the segmented words are integrated to obtain the phrases to be identified. During the integration of words into phrases to be identified, it is ensured that the length of each phrase does not exceed 6 characters. Furthermore, the segmentation of the text using the Jieba word segmentation method in this embodiment is only an example. That is, any method capable of segmenting the text to be processed can be applied to this embodiment. For example, word dictionaries or neural network models can also be used to complete the segmentation process of the text to be processed.

[0092] S02: Determine the features and vectors to be identified corresponding to the phrase to be identified, and input the features and vectors to be identified into the phrase extraction model; the phrase extraction model is generated according to the above-mentioned phrase extraction model training method.

[0093] Understandably, the features to be identified are discrete features of the phrase to be identified. For example, the features to be identified may include the word frequency of the phrase, the number of times the phrase appears in quotation marks, the number of times the phrase appears in parentheses, whether the first word or the last word of the phrase is a stop word, etc. The vector to be identified is the vector representation of the phrase to be identified, which can be generated by transforming the training phrases using a text processing model (such as the BERT model). The phrase extraction model is generated according to steps S10 to S50 above.

[0094] S03: Determine the feature probability value corresponding to the feature to be identified through the feature recognition module in the phrase extraction model, and determine the classification probability value corresponding to the vector to be identified through the phrase classification module in the phrase extraction model.

[0095] Understandably, the feature probability value represents the probability that the feature recognition module determines the phrase to be recognized as a true phrase based on the features to be recognized. The classification probability value represents the probability that the phrase classification module determines the phrase to be recognized as a true phrase based on the vector to be recognized.

[0096] Specifically, after determining the features and vectors corresponding to the phrases to be identified, and inputting these features and vectors into the phrase extraction model, the feature recognition module within the phrase extraction model determines the phrase feature distribution of the features to be identified. Based on this phrase feature distribution, the feature probability value of the phrase corresponding to the features to be identified as a true phrase is determined. The phrase classification module within the phrase extraction model determines whether the vector to be identified belongs to the phrase vectors of true phrases, thereby outputting the classification probability value of the phrase corresponding to the vector to be identified as a true phrase.

[0097] S04: Determine the phrase probability value based on the feature probability value and the classification probability value corresponding to the same phrase to be identified, and compare the phrase probability value with a preset probability threshold.

[0098] In essence, the phrase probability value represents the probability that the phrase to be identified is a true phrase, determined by the phrase extraction model. The preset probability threshold can be set according to requirements; for example, it can be set to 0.8, 0.9, or 0.95.

[0099] Specifically, after determining the feature probability value and classification probability value corresponding to the phrase to be identified, the phrase probability value can be determined based on the feature probability value and classification probability value corresponding to the same phrase to be identified, and then compared with a preset probability threshold. The phrase probability value can be determined using the following expression:

[0100] y = w1x1 + w2x2

[0101] Where y is the phrase probability value; w1 is the first weight value corresponding to the feature probability value; x1 is the feature probability value; w2 is the second weight value corresponding to the classification probability value; and x2 is the classification probability value. Further, the first and second weight values ​​can be randomly set, and the sum of the first and second weight values ​​is 1. For example, both the first and second weight values ​​can be set to 0.5. Alternatively, the first weight value can be set to 0.3, and the second weight value can be set to 0.7.

[0102] S05: The phrase to be identified corresponding to the phrase probability value that is greater than or equal to the preset probability threshold is taken as the target phrase of the text to be extracted.

[0103] Specifically, after comparing the phrase probability value with a preset probability threshold, if the phrase probability value is greater than or equal to the preset probability threshold, the phrase to be identified corresponding to the phrase probability value greater than or equal to the preset probability threshold is recorded as the target phrase of the text to be extracted. If the phrase probability value is less than the preset probability threshold, the phrase to be identified corresponding to the phrase probability value less than the preset probability threshold is not the target phrase of the text to be extracted.

[0104] Furthermore, if the phrase probability value is less than a preset probability threshold, all phrases with probability values ​​less than the preset threshold can be recorded as phrases to be combined. All phrases to be combined are segmented to obtain several words. These words are then recombine (the recombination must follow the original text order). For example, if a phrase to be combined contains 6 words, it can be segmented to obtain 6 words, which can then be recombined into new phrases of 5 or 4 words. As another example, assuming there are two or more adjacent phrases to be combined in the text to be extracted, the words in the adjacent phrases can be recombine to form one or more new phrases. Then, steps S02 to S05 above are used to determine whether the new phrase is a real phrase. If the new phrase is a real phrase, it is recorded as the target phrase. Repeating this process ensures the completeness of real phrase extraction from the text to be extracted, thereby improving the accuracy of phrase extraction.

[0105] In this embodiment, the phrase extraction model trained in the above embodiments verifies whether the phrase to be identified is a real phrase from two aspects: discrete features (i.e., the features to be identified) and vector features (i.e., the vectors to be identified), thereby completing the phrase extraction process of the text to be extracted. This improves the efficiency and accuracy of phrase extraction.

[0106] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0107] In one embodiment, a phrase extraction model training device is provided, which corresponds one-to-one with the phrase extraction model training method described in the above embodiments. For example... Figure 4 As shown, the phrase extraction model training device includes a training phrase acquisition module 10, a model acquisition module 20, a first probability value determination module 30, a loss value determination module 40, and a model update module 50. Detailed descriptions of each functional module are as follows:

[0108] The training phrase acquisition module 10 is used to acquire at least one training phrase group; a training phrase group includes a training phrase, a target phrase label, training phrase features, and a training phrase vector.

[0109] The model acquisition module 20 is used to acquire a preset extraction model containing initial parameters; the preset extraction model includes a feature recognition module and a phrase classification module.

[0110] The first probability value determination module 30 is used to input the training phrase features into the feature recognition module to obtain a first probability value, and to input the training phrase vector into the phrase classification module to obtain a second probability value;

[0111] The loss value determination module 40 is used to determine the predicted phrase label corresponding to the training phrase based on the first probability value and the second probability value, and to determine the predicted loss value based on the target phrase label and the predicted phrase label;

[0112] The model update module 50 is used to iteratively update the initial parameters in the preset extraction model when the predicted loss value does not reach the preset convergence condition, until the predicted loss value reaches the convergence condition, and then record the converged preset extraction model as a phrase extraction model.

[0113] Preferably, the phrase extraction model training device further includes:

[0114] The sample corpus acquisition module is used to acquire a sample corpus set and perform word segmentation on the sample corpus in the sample corpus set to obtain at least one candidate phrase;

[0115] The phrase crawling module is used to obtain positive sample phrases from a preset third-party platform using web crawling technology; each positive sample phrase corresponds to a positive sample tag;

[0116] A negative sample filtering module is used to determine negative sample phrases from all the candidate phrases based on the positive sample phrases and the candidate phrases, and to set the negative sample labels corresponding to the negative sample phrases;

[0117] The training statement construction module is used to generate the training phrase group based on the positive sample phrase, the negative sample phrase, the positive sample label corresponding to the positive sample phrase, and the negative sample label corresponding to the negative sample phrase.

[0118] Preferably, the training statement construction module includes:

[0119] The feature filtering unit is used to perform discrete feature extraction on the positive sample phrases and the negative sample phrases to obtain the training phrase features corresponding to the positive sample phrases and the training phrase features corresponding to the negative sample phrases.

[0120] A vector transformation unit is used to perform vector transformation on the positive sample phrase and the negative sample phrase to obtain the training phrase vector corresponding to the positive sample phrase and the training phrase vector corresponding to the negative sample phrase.

[0121] The training phrase construction unit is used to construct the training phrase group based on the positive sample phrase, the positive sample label corresponding to the positive sample phrase, the training phrase feature and the training phrase vector, and to construct the training phrase group based on the negative sample phrase, the negative sample label corresponding to the negative sample phrase, the training phrase feature and the training phrase vector.

[0122] Preferably, the first probability value determination module 30 includes:

[0123] A feature distribution determination unit is used to determine the phrase feature distribution corresponding to the training phrase features through the feature recognition module;

[0124] The first probability value determination unit is used to determine the first probability value based on the phrase feature distribution.

[0125] Preferably, the first probability value determination module 30 further includes:

[0126] The vector convolution unit is used to perform convolution processing on the training phrase vector through the convolutional network layer in the phrase classification module to obtain the convolutional feature vector;

[0127] The vector pooling unit is used to perform pooling processing on the convolutional feature vector through the pooling network layer in the phrase classification module to obtain a pooled feature vector.

[0128] The second probability value determination unit is used to determine the second probability value based on the pooling feature vector through the fully connected layer in the phrase classification module.

[0129] Specific limitations regarding the phrase extraction model training device can be found in the limitations of the phrase extraction model training method described above, and will not be repeated here. Each module in the aforementioned phrase extraction model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0130] In one embodiment, such as Figure 5 As shown, a phrase extraction device is provided, comprising:

[0131] The word segmentation processing module 01 is used to receive a phrase extraction instruction containing the text to be extracted, and to perform word segmentation processing on the text to be extracted to obtain at least one phrase to be identified.

[0132] Feature determination module 02 is used to determine the features to be identified and the vector to be identified corresponding to the phrase to be identified, and input the features to be identified and the vector to be identified into the phrase extraction model; the phrase extraction model is generated according to the above-mentioned phrase extraction model training method;

[0133] The second probability value determination module 03 is used to determine the feature probability value corresponding to the feature to be identified by the feature recognition module in the phrase extraction model, and to determine the classification probability value corresponding to the vector to be identified by the phrase classification module in the phrase extraction model.

[0134] The probability value comparison module 04 is used to determine the phrase probability value based on the feature probability value and the classification probability value corresponding to the same phrase to be identified, and to compare the phrase probability value with a preset probability threshold.

[0135] The target phrase determination module 05 is used to identify the phrase to be identified as the target phrase of the text to be extracted if the phrase probability value is greater than or equal to the preset probability threshold.

[0136] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data used in the phrase extraction model training method or phrase extraction method described in the above embodiments. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a phrase extraction model training method, or when the computer program is executed by the processor, it implements a phrase extraction method.

[0137] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the phrase extraction model training method in the above embodiments, or when the processor executes the computer program, it implements the phrase extraction method in the above embodiments.

[0138] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the phrase extraction model training method in the above embodiments, or when the computer program is executed by a processor, it implements the phrase extraction method in the above embodiments.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0141] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for training a phrase extraction model, characterized in that, include: Obtain at least one training phrase group; A training phrase set includes training phrases, target phrase labels, training phrase features, and training phrase vectors; The training phrase features are the discrete features of the training phrases, and the training phrase vectors are the vector representations of the training phrases. Obtain a preset extraction model containing initial parameters; the preset extraction model includes a feature recognition module and a phrase classification module; The training phrase features are input into the feature recognition module, and the phrase feature distribution of the training phrase features is determined by the feature recognition module in the preset extraction model. The first probability value output by the feature recognition module based on the phrase feature distribution is obtained. The training phrase vector is input into the phrase classification module, and the second probability value output by the phrase classification module after determining whether the training phrase vector belongs to a real phrase is obtained. The predicted phrase label corresponding to the training phrase is determined based on the first probability value and the second probability value, and the prediction loss value is determined based on the target phrase label and the predicted phrase label. When the predicted loss value does not reach the preset convergence condition, the initial parameters in the preset extraction model are iteratively updated until the predicted loss value reaches the convergence condition. Then, the preset extraction model after convergence is recorded as a phrase extraction model. The step of inputting the training phrase vector into the phrase classification module to obtain the second probability value includes: The training phrase vectors are convolved by the convolutional network layer in the phrase classification module to obtain convolutional feature vectors. The convolutional feature vector is pooled by the pooling network layer in the phrase classification module to obtain a pooled feature vector. The second probability value is determined by the fully connected layer in the phrase classification module based on the pooling feature vector.

2. The phrase extraction model training method as described in claim 1, characterized in that, Before obtaining at least one training phrase group, the method further includes: Obtain a sample corpus set, and perform word segmentation on the sample corpus set to obtain at least one candidate phrase; Positive sample phrases are obtained from a pre-defined third-party platform; each positive sample phrase corresponds to a positive sample tag. Based on the positive sample phrases and the candidate phrases, negative sample phrases are determined from all the candidate phrases, and negative sample labels are set for the negative sample phrases. The training phrase group is generated based on the positive sample phrase, the negative sample phrase, the positive sample label corresponding to the positive sample phrase, and the negative sample label corresponding to the negative sample phrase.

3. The phrase extraction model training method as described in claim 2, characterized in that, The step of generating the training phrase group based on the positive sample phrase, the negative sample phrase, the positive sample label corresponding to the positive sample phrase, and the negative sample label corresponding to the negative sample phrase includes: Discrete feature extraction is performed on the positive sample phrases and the negative sample phrases to obtain the training phrase features corresponding to the positive sample phrases and the training phrase features corresponding to the negative sample phrases; The positive sample phrases and the negative sample phrases are transformed into vectors to obtain the training phrase vectors corresponding to the positive sample phrases and the training phrase vectors corresponding to the negative sample phrases. The training phrase group is constructed based on the positive sample phrase, the positive sample label corresponding to the positive sample phrase, the training phrase feature, and the training phrase vector; and the training phrase group is constructed based on the negative sample phrase, the negative sample label corresponding to the negative sample phrase, the training phrase feature, and the training phrase vector.

4. The phrase extraction model training method as described in claim 1, characterized in that, The step of inputting the training phrase features into the feature recognition module to obtain a first probability value includes: The feature recognition module determines the phrase feature distribution corresponding to the training phrase features. The first probability value is determined based on the phrase feature distribution.

5. A phrase extraction method, characterized in that, include: Receive a phrase extraction instruction containing the text to be extracted, and perform word segmentation on the text to be extracted to obtain at least one phrase to be identified; The features to be identified and the vector to be identified corresponding to the phrase to be identified are determined, and the features to be identified and the vector to be identified are input into the phrase extraction model; the phrase extraction model is generated according to the phrase extraction model training method as described in any one of claims 1 to 4; The feature probability value corresponding to the feature to be identified is determined by the feature recognition module in the phrase extraction model, and the classification probability value corresponding to the vector to be identified is determined by the phrase classification module in the phrase extraction model. A phrase probability value is determined based on the feature probability value and the classification probability value corresponding to the same phrase to be identified, and the phrase probability value is compared with a preset probability threshold. The phrases whose probability values ​​are greater than or equal to the preset probability threshold are taken as the target phrases of the text to be extracted.

6. A phrase extraction model training device, characterized in that, include: The training phrase acquisition module is used to acquire at least one training phrase group; A training phrase group includes a training phrase, a target phrase label, training phrase features, and a training phrase vector; the training phrase features are discrete features of the training phrase, and the training phrase vector is a vector representation of the training phrase. The model acquisition module is used to acquire a preset extraction model containing initial parameters; the preset extraction model includes a feature recognition module and a phrase classification module. The first probability value determination module is used to input the training phrase features into the feature recognition module, determine the phrase feature distribution of the training phrase features through the feature recognition module in the preset extraction model, obtain the first probability value output by the feature recognition module according to the phrase feature distribution, and input the training phrase vector into the phrase classification module, obtain the second probability value output by the phrase classification module after determining whether the training phrase vector belongs to the real phrase. The loss value determination module is used to determine the predicted phrase label corresponding to the training phrase based on the first probability value and the second probability value, and to determine the predicted loss value based on the target phrase label and the predicted phrase label; The model update module is used to iteratively update the initial parameters in the preset extraction model when the predicted loss value does not reach the preset convergence condition, until the predicted loss value reaches the convergence condition, and then record the converged preset extraction model as a phrase extraction model. The step of inputting the training phrase vector into the phrase classification module to obtain the second probability value includes: The training phrase vectors are convolved by the convolutional network layer in the phrase classification module to obtain convolutional feature vectors. The convolutional feature vector is pooled by the pooling network layer in the phrase classification module to obtain a pooled feature vector. The second probability value is determined by the fully connected layer in the phrase classification module based on the pooling feature vector.

7. A phrase extraction device, characterized in that, include: The word segmentation module is used to receive a phrase extraction instruction containing the text to be extracted, and to perform word segmentation on the text to be extracted to obtain at least one phrase to be identified. A feature determination module is used to determine the features to be identified and the vector to be identified corresponding to the phrase to be identified, and input the features to be identified and the vector to be identified into the phrase extraction model; the phrase extraction model is generated according to the phrase extraction model training method as described in any one of claims 1 to 4; The second probability value determination module is used to determine the feature probability value corresponding to the feature to be identified by the feature recognition module in the phrase extraction model, and to determine the classification probability value corresponding to the vector to be identified by the phrase classification module in the phrase extraction model. The probability value comparison module is used to determine the phrase probability value based on the feature probability value and the classification probability value corresponding to the same phrase to be identified, and to compare the phrase probability value with a preset probability threshold. The target phrase determination module is used to identify phrases whose probability values ​​are greater than or equal to the preset probability threshold as the target phrases of the text to be extracted.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the phrase extraction model training method as described in any one of claims 1 to 4, and when the processor executes the computer program, it implements the phrase extraction method as described in claim 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the phrase extraction model training method as described in any one of claims 1 to 4, and when the computer program is executed by the processor, it implements the phrase extraction method as described in claim 5.

Images