Text processing method and device and related product

Abstract

The invention discloses a text processing method and device and a related product. The method comprises the steps of obtaining m initial keywords to be processed; the m initial keywords are obtained on the basis of a plurality of original texts associated with a target object, the target object indicates a target application or a target event, the m initial keywords are matched with a target corpus constructed on the basis of a target text set, and n initial keywords with successful matching results with the target corpus are obtained and serve as first keywords, the target text is a common text of a target object, the similarity between each first keyword and a corresponding target directional text is determined to obtain n similarities, and the target directional text indicates a text in the same analysis direction as the target object; and sorting the n first keywords according to a sorting result of the n similarities from high to low, and taking the first q first keywords as target keywords. Therefore, the noise of the extracted keywords is reduced, and the accuracy of subsequent analysis results can be improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a text processing method, apparatus and related products. Background Technology

[0002] Currently, different platforms allow sharing and commenting on specific applications or events. Taking games as an example, users can share their gaming experiences on different platforms or comment on content posted by other users.

[0003] To perform analysis on a specific application or event, it's generally necessary to extract keywords from a large amount of sharing or comments related to that application or event, enabling a more intuitive and clear analysis. However, the keywords extracted by the text processing methods offered in related technologies may contain significant noise, leading to low accuracy in subsequent analysis results for that specific application or event. Summary of the Invention

[0004] This application provides a text processing method, apparatus, and related products, which aim to reduce noise in extracted keywords and thereby improve the accuracy of analysis results for specific applications or events.

[0005] The first aspect of this application provides a text processing method, including:

[0006] Obtain m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with the target object; the target object indicates the target application or target event; m is an integer greater than or equal to 2;

[0007] The m initial keywords are matched with the target corpus to obtain n first keywords; the target corpus is constructed based on the target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords that successfully match the target corpus; n is a positive integer less than or equal to m;

[0008] Determine the similarity between each primary keyword and its corresponding target-oriented text to obtain n similarity scores; the target-oriented text indicates text that is in the same direction of analysis as the target object.

[0009] Sort the n primary keywords according to their similarity scores from highest to lowest, and select the top q primary keywords as target keywords; q is an integer greater than or equal to 1.

[0010] A second aspect of this application provides a text processing apparatus, comprising:

[0011] The acquisition module is used to acquire m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with the target object; the target object indicates the target application or target event; m is an integer greater than or equal to 2;

[0012] The matching module is used to match m initial keywords with the target corpus to obtain n first keywords; the target corpus is constructed based on the target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords that have successfully matched with the target corpus; n is a positive integer less than or equal to m;

[0013] The determination module is used to determine the similarity between each primary keyword and its corresponding target-oriented text, resulting in n similarity scores; the target-oriented text indicates text that is in the same direction of analysis as the target object.

[0014] The processing module is used to sort the n primary keywords according to the n similarity results from high to low, and to take the top q primary keywords as target keywords; q is an integer greater than or equal to 1.

[0015] A third aspect of this application provides an electronic device, the device comprising a processor and a memory:

[0016] The memory is used to store program code and transmit the program code to the processor;

[0017] The processor is used to execute the steps of the text processing method provided in the first aspect according to the instructions in the program code.

[0018] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the steps of the text processing method provided in the first aspect.

[0019] The fifth aspect of this application provides a computer program product, including a computer program or instructions that, when executed, implement the steps of the text processing method provided in the first aspect.

[0020] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0021] The text processing method provided in this application embodiment obtains m initial keywords to be processed; wherein the m initial keywords are obtained based on multiple original texts associated with a target object, the target object refers to the target application or target event, and m is an integer greater than or equal to 2; then the m initial keywords are matched with a target corpus constructed based on the target text set to obtain n initial keywords that have a successful matching result with the target corpus as first keywords, wherein the target text is the commonly used text of the target object, and n is a positive integer less than or equal to m; further, the similarity between each first keyword and the corresponding target-oriented text is determined to obtain n similarity scores, wherein the target-oriented text refers to text with the same analysis direction as the target object; finally, the n first keywords are sorted according to the sorting result of the n similarity scores from high to low, and the first q first keywords are used as target keywords, where q is an integer greater than or equal to 1.

[0022] Thus, by matching the initial keywords with a target corpus of commonly used texts based on the target object, some keywords irrelevant to the target object can be removed, reducing the noise of the extracted keywords. Furthermore, by determining the similarity between the first keyword and the target-oriented texts with the same analytical direction as the target object, target keywords with the same analytical direction as the target object can be selected from multiple first keywords based on similarity. This improves the relevance between the target keywords and the target object, reduces the noise of the extracted keywords, and improves the accuracy of subsequent analysis results. Attached Figure Description

[0023] Figure 1 A schematic diagram illustrating a text processing method provided in an embodiment of this application;

[0024] Figure 2 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0025] Figure 3 A schematic diagram illustrating a clustering result provided in an embodiment of this application;

[0026] Figure 4 A highlighted schematic diagram of a node provided in an embodiment of this application;

[0027] Figure 5 A schematic diagram of nodes provided in an embodiment of this application;

[0028] Figure 6 A trend change graph of target keywords provided in an embodiment of this application;

[0029] Figure 7 A schematic diagram illustrating a text processing method provided in an embodiment of this application;

[0030] Figure 8 A structural schematic diagram of a text processing device provided by an embodiment of the present application;

[0031] Figure 9 A structural schematic diagram of a server in an embodiment of the present application;

[0032] Figure 10 A structural schematic diagram of a terminal device in an embodiment of the present application. Detailed implementation manners

[0033] In the related art, two text processing methods are provided for extracting keywords from text.

[0034] The Chinese word segmentation framework is a powerful Python library for segmenting Chinese text. The Chinese word segmentation framework adopts the maximum probability path algorithm implemented based on the prefix dictionary, searches all possible segmentation combinations through dynamic programming, and then uses the backtracking method to find the maximum probability path, so as to achieve accurate and efficient word segmentation.

[0035] The Chinese word segmentation framework provides three word segmentation modes: accurate mode, full mode, and search engine mode. The accurate mode attempts to cut the sentence most accurately and is suitable for text analysis; the full mode scans out all the words that can form words in the sentence, with very high speed, but cannot solve the ambiguity problem; the search engine mode, based on the accurate mode, further segments long words to improve the recall rate and is suitable for search engine word segmentation.

[0036] In addition, the Chinese word segmentation framework also supports custom dictionaries, and users can add custom words according to needs. It also provides a word tagging function that can tag the word nature of each word.

[0037] Although the text processing method based on the Chinese word segmentation framework can extract keywords from a large amount of text, due to the lack of semantic information guidance, the keywords extracted based on the Chinese word segmentation framework are likely to introduce some invalid noise words, such as: "then", "like this"; at the same time, due to the lack of semantic information guidance, the Chinese word segmentation framework is also likely to split a word. For example, it may split "for example" into two words "such as" and "say".

[0038] The topic model can automatically extract topics (i.e., keywords) from a large amount of text data, so as to detect various events related to these topics. For example, the topic model can be the Latent Dirichlet Allocation (LDA) model. The advantage of this method is that it can effectively handle the noise in text data, but at the same time it also faces challenges of high computational complexity and limited real-time performance.

[0039] LDA (Limited Data Generation) is a probabilistic generative model that assumes documents are generated from a mixture of multiple topics, and each topic is generated from a mixture of multiple words. LDA can identify important topics from a document collection and detect the evolution of these topics. The advantages of LDA are its ability to handle large-scale text datasets and its adaptability to the emergence of new topics and the disappearance of old ones.

[0040] However, LDA models primarily focus on identifying and detecting topic evolution from large amounts of text data. Text processing methods based on LDA models require developers to pre-specify the number of topics, which can become very difficult in dynamic topic analysis scenarios. For example, during keyword extraction, the number of topics might increase from 10 to 100, but the developers have only pre-specified 10 topics. In this case, LDA-based text processing might be unable to identify these 100 topics, leading to inaccurate keyword extraction, or even the extraction of keywords irrelevant to the target audience, increasing noise.

[0041] It should be understood that keywords extracted by text processing methods based on Chinese word segmentation frameworks or by text processing methods based on topic models may contain a lot of noise, resulting in low accuracy of subsequent analysis results for specific applications or events.

[0042] In view of this, embodiments of this application provide a text processing method, apparatus, and related products. The method includes: obtaining m initial keywords to be processed; wherein the m initial keywords are obtained based on multiple original texts associated with a target object, the target object indicating a target application or target event, and m is an integer greater than or equal to 2; then matching the m initial keywords with a target corpus constructed based on a target text set to obtain n initial keywords that successfully match the target corpus as first keywords, wherein the target text is commonly used text of the target object, and n is a positive integer less than or equal to m; further determining the similarity between each first keyword and the corresponding target-oriented text to obtain n similarity scores, wherein the target-oriented text indicates text with the same analysis direction as the target object; finally, sorting the n first keywords according to the sorting results of the n similarity scores from high to low, and using the first q first keywords as target keywords, where q is an integer greater than or equal to 1.

[0043] Thus, by matching the initial keywords with a target corpus of commonly used texts based on the target object, some keywords irrelevant to the target object can be removed, reducing the noise of the extracted keywords. Furthermore, by determining the similarity between the first keyword and the target-oriented texts with the same analytical direction as the target object, target keywords with the same analytical direction as the target object can be selected from multiple first keywords based on similarity. This improves the relevance between the target keywords and the target object, reduces the noise of the extracted keywords, and improves the accuracy of subsequent analysis results.

[0044] Figure 1 An exemplary scenario architecture diagram of a text processing method is shown. Figure 1 This includes server 101. Figure 1 The server shown can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system. Additionally, the server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0045] Server 101 retrieves m initial keywords to be processed. These m initial keywords are derived from multiple raw texts, each associated with a target object, indicating the target application or event. m is an integer greater than or equal to 2. As an example, suppose the target object is game A, and the m initial keywords are derived from multiple raw texts associated with game A. These initial keywords might be phrases like "game over," "queue," "under maintenance," "can't log in," "unhappy," "guide," "goodbye," etc.

[0046] Server 101 matches m initial keywords with the target corpus to obtain n first keywords. The target corpus is constructed based on the target text set, where the target text is the commonly used text of the target object. The first keywords are the initial keywords that successfully match the target corpus. n is a positive integer less than or equal to m. Based on the previous example, assuming the target text is "game over", "queue", "under maintenance", "cannot log in", "bye", then the target corpus will include "game over", "queue", "under maintenance", "cannot log in", "bye", and matching the initial keywords with the target corpus will yield the first keywords: "game over", "queue", "under maintenance", "cannot log in", "bye".

[0047] Server 101 determines the similarity between each primary keyword and its corresponding target-oriented text, obtaining n similarity scores. The target-oriented text indicates text with the same analysis direction as the target object. Based on the previous example, assuming the analysis direction of the target object is "online direction", then the target-oriented text with the same "online direction" can be: "cannot log in", "under maintenance", "just opened", "waiting in line", "server shut down", etc. Then the similarity between the primary keyword and its corresponding target-oriented text can be determined. For example, the similarity between "cannot log in" and "cannot log in" is 90%.

[0048] Server 101 sorts the n primary keywords according to their similarity scores from highest to lowest, and selects the top q primary keywords as target keywords; q is an integer greater than or equal to 1. Based on the previous example, assuming the similarity scores of the primary keywords are "game over", 10%, "queue", 70%, "under maintenance", 80%, "cannot log in", 90%, and "bye", 1%, then the sorting results from highest to lowest similarity are: "cannot log in", "under maintenance", "queue", "game over", and "bye". Selecting the top 3 primary keywords as target keywords, the target keywords are "cannot log in", "under maintenance", and "queue".

[0049] Thus, this embodiment of the application can match the initial keywords with a target corpus of commonly used texts based on the target object, thereby removing some keywords that are irrelevant to the target object (such as "unhappy" and "strategy" in the example above) and reducing the noise of the extracted keywords. Furthermore, by determining the similarity between the first keyword and the target-oriented text with the same analysis direction as the target object, the target keywords with the same analysis direction as the target object can be selected from multiple first keywords based on the similarity. This improves the relevance between the target keywords and the target object, reduces the noise of the extracted keywords, and improves the accuracy of subsequent analysis results.

[0050] It should be noted that in some possible implementations, the aforementioned server can be replaced by a terminal device to perform the content delivery control method. The terminal device is a device with a display screen, including but not limited to mobile phones, tablets, computers, smart voice interaction devices, and in-vehicle terminals, but not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions.

[0051] In practice, the collection and processing of relevant data in this application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the subject of personal information, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0052] See Figure 2 The figure is a flowchart illustrating a text processing method provided in an embodiment of this application.

[0053] Combination Figure 2 As shown, the text processing method provided in this application includes:

[0054] S201: Obtain the m initial keywords to be processed.

[0055] Where m is an integer greater than or equal to 2.

[0056] Initial keywords refer to keywords derived from the original text. The *m* initial keywords are derived from multiple original texts. For example, initial keywords could be words like "game over," "queue," "under maintenance," "can't log in," "unhappy," "strategy guide," "goodbye," etc. Original text refers to text associated with the target object, such as "Under maintenance, 3 PM to 6 PM!" or "Okay, and XXX too!" or "#XXX mobile game#XXX mobile game under maintenance." The target object indicates the target application or event. The target application can be a game, audio software, video software, recording software, sharing software, or social software; the specific type of software is not limited here. The target event can be an event themed around a specific event, item, application, etc., such as a game review of game A, how to eat fruit deliciously, or clothing matching tips; these are not specifically limited here.

[0057] In one possible implementation, step S201 may include:

[0058] A1: Retrieves multiple raw texts.

[0059] The original text refers to the text obtained after processing original text data. In one possible implementation, the original text can be obtained by acquiring multiple original text data, preprocessing each original text data, and obtaining multiple original texts. The original text data has a relationship with the target object.

[0060] Raw text data refers to data obtained from different platforms that is related to the target object. Different platforms include, but are not limited to, recording platforms, social platforms, sharing platforms, audio and video platforms, etc. Raw text data includes, but is not limited to, content titles, comments, ASR (Automatic Speech Recognition), topics, descriptions, OCR (Optical Character Recognition), etc., without specific limitations.

[0061] As an example, suppose the set of original text data is T = {t1, t2, ..., t}. N}, t N Let T' represent the Nth original text data. Since the original text data contains a large amount of noise, it needs to be preprocessed. Preprocessing mainly involves removing non-informative data such as @xx, emoticons, accounts, and website addresses. The set of preprocessed original text can be represented as T' = f. c (T)={t1′,t2′,…,t M ′},f c This is a text cleaning function used to remove non-informative data from raw text data.

[0062] It should be understood that by preprocessing the original text data to obtain the original text, data without information can be removed, improving the data quality of the original text, which is beneficial for subsequent keyword extraction and improving the accuracy of the analysis results.

[0063] A2: Segment each original text into words to obtain multiple original keywords.

[0064] Original keywords refer to the keywords obtained by segmenting the original text. For example, if the original text A is "It will only recover the next day, did you fill up Game A today?", the original keywords corresponding to the original text A can be "will", "the next day", "only recover", "you", "today", "fill up", "Game A", "fill up", "did you?".

[0065] As an example, suppose the set of multiple original texts is T' = f c (T)={t1′,t2′,…,t M The set of multiple original keywords can be used to output w' = {w1, w2, ..., w}. M}

[0066] A3: Determine the left and right information entropy of each original keyword.

[0067] Left-right information entropy reflects the richness of left-right collocations of a keyword by calculating the information entropy of its left and right sides. Information entropy itself measures the uncertainty of a random variable; however, in the text processing method of this application, it can be used to measure the randomness of the left and right neighboring word sets of a keyword. When the left-right information entropy of a keyword is high, it indicates a high frequency of changes in its left-right collocations, meaning the original keyword has a rich and diverse contextual environment in the original text.

[0068] Here, the left and right information entropies are the sum of the left and right information entropies of the original keyword. The left information entropy refers to the information entropy of the set of words to the left of the original keyword, and the right information entropy refers to the information entropy of the set of words to the right of the original keyword.

[0069] In one possible implementation, step A3 may include:

[0070] B1: Determine the first frequency of the target original keyword among multiple original keywords.

[0071] The target original keyword is any one of multiple original keywords.

[0072] Term frequency refers to the number of times each word appears in a text or corpus. In this embodiment, the first term frequency refers to the number of times the target original keyword appears among multiple original keywords.

[0073] As an example, suppose the set of multiple original texts is T' = f c (T)={t1′,t2′,…,t M The set of multiple original keywords can be used to output w' = {w1, w2, ..., w}. M The set of multiple original keywords and their corresponding word frequencies can be W = {(w1,c1),(w2,c2),…,(w...}. K ,c K )},(w i ,c i () represents a specific original keyword and its corresponding word frequency.

[0074] B2: Determine the second word frequency of the left neighboring words of the target original keyword, and determine the third word frequency of the right neighboring words of the target original keyword.

[0075] The second term frequency refers to the number of times the left-neighbor words of the target original keyword appear in a set of multiple original keywords. The left-neighbor words refer to the words formed by the left-neighbor words and the target original keyword. For example, if the target original keyword is "cup", the left-neighbor words can be "take", "put", "smash", "wash", etc., then the formed left-neighbor words can be "take cup", "put cup", "smash cup", "wash cup", etc.

[0076] The third term frequency refers to the number of times the target original keyword appears in the combination of multiple original keywords, which are the right neighbor words. The right neighbor words refer to the words formed by the target original keyword and the right neighbor words. For example, if the target original keyword is "cup", the right neighbor words can be "inside", "outside", "up", "down", "of", etc., so the right neighbor words formed can be "inside the cup", "outside the cup", "upside the cup", "downside the cup", "of the cup", etc.

[0077] B3: Determine the left information entropy of the target original keyword based on the first and second word frequencies, and determine the right information entropy of the target original keyword based on the first and third word frequencies.

[0078] It should be understood that left information entropy reflects the richness and diversity of the left-neighboring words of the original target keyword, while right information entropy reflects the richness and diversity of the right-neighboring words of the original target keyword. The higher the left and / or right information entropy of the original target keyword, the greater the likelihood that the original target keyword is a single word.

[0079] As an example, suppose the set of multiple original keywords and their corresponding word frequencies can be W = {(w1,c1),(w2,c2),…,(w…}} K ,c K Then, the left information entropy of each original keyword can be determined by the following formula (1), and the right information entropy of each original keyword can be determined by the following formula (2):

[0080]

[0081]

[0082] Among them, Prew i For the original keyword w i The words that may exist to the left of (left neighbor words), w i Suf is the original keyword. i The word that may exist to the right of (right neighbor word).

[0083] Taking the left information entropy of the target original keyword as an example, the left information entropy of the target original keyword is calculated based on the first word frequency of the target original keyword, the multiple left neighbor words corresponding to the target original keyword, and the second word frequency of each left neighbor word. Similarly, following the calculation method of the left information entropy of the target original keyword, the right information entropy of the target original keyword is calculated based on the first word frequency of the target original keyword, the multiple right neighbor words corresponding to the target original keyword, and the third word frequency of each right neighbor word.

[0084] B4: The sum of the left and right information entropies is used as the left and right information entropies of the target original keyword.

[0085] After determining the left and right information entropy of the target original keyword, the left and right information entropy of the target original keyword can be determined based on the left and right information entropy, as shown in the following formula (3):

[0086]

[0087] It should be understood that, in the embodiments of this application, by calculating the left and right information entropy of each original keyword, more effective new words can be identified, thereby improving the accuracy of keyword recognition.

[0088] The left and right information entropies of each original keyword are obtained according to the method for determining the left and right information entropies of the target original keyword.

[0089] A4: Take the original keywords whose left and right information entropy is greater than the preset left and right information entropy as the initial keywords, and obtain m initial keywords.

[0090] Preset left and right information entropy refers to the left and right information entropy set according to needs, which is used as a condition for filtering initial keywords.

[0091] As an example, suppose the set of multiple original keywords and their corresponding word frequencies can be W = {(w1,c1),(w2,c2),…,(w…}} K ,c K )}, with the left and right information entropy preset to Tro b By filtering the left and right information entropies of each original keyword, we can obtain a set W′={(w1′,c1′),(w2′,c2′),…,(w K ′,c K ′)}.

[0092] It should be understood that left-right information entropy measures the richness of a word's left-right collocations. If a word can be collocated with multiple different words, then the left-right information entropy of that word will be high, meaning that the word is more important in the text and is likely a keyword. Therefore, by calculating the left-right information entropy of each original keyword, this embodiment of the application can more accurately identify the more important keywords in the original text among multiple original keywords.

[0093] Meanwhile, since new words may appear infrequently in text, traditional processing methods may not be able to effectively identify them. However, in this embodiment, by calculating the left and right information entropy of each original keyword, new words can be discovered by measuring the richness of left and right collocations of the original keywords, thus improving the accuracy of new word identification.

[0094] S202: Match the m initial keywords with the target corpus to obtain n first keywords.

[0095] Where n is a positive integer less than or equal to m.

[0096] The target corpus refers to a collection of texts associated with a target object. In one possible implementation, the target corpus is built based on a set of target texts, which are the commonly used texts of the target object. Commonly used texts refer to texts that are frequently used in the context of the target object. For example, if the target object is a game, the commonly used texts for the game could be "login", "download", "victory", "end", etc.

[0097] It should be understood that the initial keywords may also include common words related to the target object. These common words will be regarded as noise when analyzing the target object, which may affect the subsequent analysis results. Therefore, it is necessary to remove the common words of the target object to reduce the noise of the extracted keywords.

[0098] In one possible implementation, step S202 may include:

[0099] C1: Determine the fourth word frequency of the target initial keyword among the m initial keywords, and obtain the fifth word frequency of the target initial keyword in the target corpus.

[0100] Wherein, the target initial keyword is any one of the m initial keywords. It should be understood that, in this embodiment of the application, the target initial keyword is used as an example to illustrate the process of determining the matching between each initial keyword and the target corpus.

[0101] The fourth term frequency refers to the number of times the target initial keyword appears among m initial keywords. For example, for the target initial keyword w i Initial target keyword w i The fourth word frequency is tf i =c i / M all Among them, M all This represents the total number of initial keywords (i.e., n).

[0102] The fifth term, frequency, refers to the number of times the initial target keyword appears in the target corpus. For example, for the initial target keyword w... i Initial target keyword w i The fifth word frequency is Where, N all The total number of target texts in the target corpus. To include the target initial keyword w i The number of target texts.

[0103] C2: Use the product of the fourth and fifth word frequencies as the target word frequency of the initial target keyword.

[0104] It should be understood that, in the embodiments of this application, the TF-IDF algorithm can be used to process multiple initial keywords. TF-IDF (Term Frequency-Inverse Document Frequency) is used to evaluate the importance of an initial keyword to one of the target texts in the target corpus. The larger the TF-IDF value (target term frequency), the higher the importance of the target initial keyword among multiple initial keywords. If the target initial keyword is a common word of the target object, the TF-IDF value will be relatively low, and it will be removed.

[0105] The target term frequency refers to the product of the fourth term frequency (term frequency) and the fifth term frequency (inverse document frequency). As an example, the target term frequency can be TF-IDF. i =tf i ·idf i .

[0106] C3: Determine the target word frequency of each initial keyword according to the method of obtaining the target word frequency of the initial keywords, and obtain m target word frequencies.

[0107] It should be understood that, in this embodiment of the application, the process of determining the matching between each initial keyword and the target corpus is illustrated using the target initial keyword as an example. Therefore, the target word frequency of each initial keyword can be determined according to the method of obtaining the target word frequency of the target initial keyword, thereby obtaining m target word frequencies.

[0108] C4: Sort the m initial keywords in descending order of the frequency of the m target words, and take the first n initial keywords as the n first keywords.

[0109] It should be understood that after determining the target word frequency for each initial keyword, the m initial keywords can be arranged in descending order of target word frequency, and the first few keywords can be... c n initial keywords are used as the first keyword. Since the TF-IDF values ​​of commonly used words are relatively small, the method provided in the embodiments of this application can be used to remove commonly used words and reduce keyword noise.

[0110] S203: Determine the similarity between each primary keyword and its corresponding target text, and obtain n similarity scores.

[0111] Target-oriented text indicates text that follows the same analytical direction as the target object. The analytical direction refers to the analytical approach or trend when analyzing the target object. For example, if the target object is a game, the analytical direction could include, but is not limited to, criticism, praise, technical aspects, optimization, etc.

[0112] Similarity refers to the degree of similarity between the primary keyword and the target text. The higher the similarity, the more similar the primary keyword and the target text are, and the closer or the same the analysis direction they correspond to.

[0113] It should be understood that, in the embodiments of this application, by determining the similarity between the first keyword and the target targeted text, the first keyword with the same analysis direction as the target object can be determined, so that the target object can be analyzed using the first keyword with the same analysis direction, which can improve the accuracy of the analysis results and avoid interference from irrelevant content.

[0114] In one possible implementation, step S203 may include:

[0115] D1: Extract features from each primary keyword to obtain the primary features.

[0116] The first feature refers to the analytical direction of the first keyword. In one possible implementation, the sentiment classification sub-model in the trained Bert-Emo model can be used to analyze each first keyword t. i "Perform feature extraction to obtain the first feature corresponding to each first key feature."

[0117] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language representation model. It uses a large amount of unsupervised corpus for pre-training and can then be fine-tuned using a small amount of labeled corpus to perform specific tasks such as sentiment classification, question answering systems, and named entity recognition. BERT's network architecture is based on the Encoder part of a Transformer structure, giving it powerful language generation and understanding capabilities.

[0118] The BERT-Emo (BERT-Emotion, BRET-Sentiment) model refers to a sentiment classification model based on the BERT model. The BERT-Emo model combines BERT's powerful language understanding capabilities with sentiment classification tasks to achieve high-precision sentiment judgment. In this embodiment, sentiment refers to the analytical tendency.

[0119] The sentiment classification sub-model refers to a model used to extract sentiment features (or directional features) from the first keyword.

[0120] D2: Extract features from each target-oriented text to obtain the second feature.

[0121] The second feature refers to the directional characteristics of the target-oriented text analysis. In one possible implementation, the sentiment classification sub-model in a pre-trained Bert-Emo model can be used to analyze the target-oriented text set V = {v1, v2, ... v...}. Nv Each target-oriented text v in the} is subjected to feature extraction to obtain the second feature corresponding to the target-oriented text.

[0122] D3: Based on the first feature of each first keyword and the second feature of each target-oriented text, calculate the similarity between each first keyword and the corresponding target-oriented text feature to obtain n similarity scores.

[0123] It should be understood that, in the embodiments of this application, the similarity between each first keyword and its corresponding target-oriented text feature can be calculated, and then the similarity can be used to determine whether the first keyword conforms to the analysis direction of the target object. In one possible implementation, after determining the first feature f of each first keyword... t′i′ and the second feature F of each target-oriented text V Then, the feature cosine similarity s between the first feature and the second feature can be calculated. i =∑cos(f t′i′ ,f vi ), to measure the first keyword t i The degree of similarity between "and the target-oriented text v".

[0124] It should be understood that, in the embodiments of this application, by determining the similarity between the first keyword and the target targeted text, it can be determined whether the analysis direction of the first keyword and the target object is the same, and then the analysis can be performed using the first keyword that is the same as the analysis direction of the target object, thereby improving the accuracy of the word segmentation results.

[0125] S204: Sort the n primary keywords according to the n similarity scores from high to low, and take the top q primary keywords as target keywords.

[0126] Where q is an integer greater than or equal to 1.

[0127] It should be understood that, in the embodiments of this application, using the first q first keywords as target keywords can ensure that the analysis direction of the first keywords is consistent with the analysis direction of the target object, avoid the influence of low-relevance first keywords on the analysis results of the target object, and improve the accuracy of the analysis results of the target object.

[0128] In this embodiment, by matching the initial keywords with a target corpus of commonly used texts based on the target object, some keywords irrelevant to the target object can be removed, reducing the noise of the extracted keywords. Furthermore, by determining the similarity between the first keyword and target-oriented texts with the same analysis direction as the target object, target keywords with the same analysis direction as the target object can be selected from multiple first keywords based on similarity. This improves the relevance between the target keywords and the target object, reducing the noise of the extracted keywords. On the other hand, the target keywords extracted in this embodiment are more accurate than those extracted in related technologies, facilitating subsequent analysis by maintenance personnel and improving the accuracy of subsequent analysis results.

[0129] Based on the text processing method provided in the above embodiments, in one possible implementation, before executing step S204, it may further include: using a pre-trained keyword extraction model to extract i candidate keywords from n first keywords; determining the similarity corresponding to each of the i candidate keywords as candidate similarities, thereby obtaining i candidate similarities. Where i is a positive integer less than or equal to n.

[0130] A keyword extraction model refers to a model used to extract candidate keywords from multiple primary keywords. In one possible implementation, the keyword extraction model can be trained based on an LLM model. Large Language Model (LLM) is a deep learning model trained on massive amounts of text data. LLM models can not only generate natural language text, but also deeply understand the meaning of text and handle various natural language tasks, such as text summarization, question answering, and translation.

[0131] In one possible implementation, the keyword extraction model in this application embodiment can use a model based on Qwen14B, pre-trained and fine-tuned using a large dataset of target objects. This model possesses domain knowledge of the target objects and can better analyze text related to the target objects.

[0132] It should be understood that, in the embodiments of this application, a keyword extraction model can be used to further filter candidate keywords from multiple first keywords that are consistent with the analysis direction of the target object. As an example, let's take a task involving a keyword extraction model that extracts first keywords related to criticism or complaints:

[0133] Given content: {text}

[0134] Please specify if the above content contains any criticisms or negative values. The output format should be "xx,xx,xx,...". Please directly output these keywords and avoid other irrelevant content. If not, output "None:".

[0135] In addition, in this embodiment of the application, keywords related to the analysis direction of the target object can be selected from multiple initial keywords as candidate keywords to avoid missing words during keyword extraction.

[0136] As an example, suppose the list of candidate keywords is w i The similarity set is s i Therefore, a candidate keyword dictionary can be constructed. The term frequency kc used to store candidate keyword kw1 i Candidate similarity (ks) i Word frequency kc i The frequency of each candidate keyword in the initial keyword set T″ is given, while the candidate similarity is the normalized similarity s between each candidate keyword and the original text. i It comes from accumulation.

[0137] Correspondingly, step S204 includes: sorting the i candidate keywords according to their similarity ranking from highest to lowest. Based on the previous example, in constructing the candidate keyword dictionary KW... i Then, the candidate keywords in the candidate keyword dictionary are sorted according to their candidate similarity, and the top keywords are ranked... h The candidate keywords are used as the final target keywords.

[0138] It should be understood that, in the embodiments of this application, by further filtering the first keyword to obtain candidate keywords, the noise of the keywords can be further reduced.

[0139] Based on the text processing method provided in the above embodiments, in one possible implementation, before step S203, it may further include: selecting p first keywords from n first keywords according to a preset rule as p second keywords.

[0140] Preset rules refer to rules for selecting secondary keywords with a high degree of relevance to the target object from multiple primary keywords. For example, primary keywords with a relevance of more than 90% may be used as secondary keywords, or primary keywords with more information than a preset amount of information may be used as secondary keywords, etc. No specific limitations are made here.

[0141] In some possible implementations, embodiments of this application may also use a pre-trained LLM model to filter the first keyword. The pre-trained LLM model may be a model that has been pre-trained and fine-tuned based on a large dataset of target objects. The LLM model has domain knowledge of the target objects and can better filter out second keywords that are highly relevant to the target objects and of good quality from multiple first keywords.

[0142] Correspondingly, step S203 may include: determining the similarity between each second keyword and the corresponding target text, and obtaining p similarity scores.

[0143] Correspondingly, step S204 may include: sorting the p second keywords according to the p similarity ranking results from high to low. Here, p is a positive integer less than or equal to n.

[0144] It should be understood that after identifying n primary keywords, most irrelevant or low-relevance keywords can be filtered out. However, some low-information words or filler words, such as "have played" or "here it comes," may still exist, potentially affecting the accuracy of subsequent analysis results. In other words, the number (or scope) of primary keywords relative to the target audience's targeted text is still relatively large. Therefore, it is necessary to select high-relevance, high-quality secondary keywords from the n primary keywords to effectively remove low-information or irrelevant keywords, while also optimizing server computational efficiency.

[0145] Based on the text processing method described in any of the above embodiments, given the limitations of related technologies in terms of information density for keyword detection, which generally cannot directly map complex event or trend evolution processes, one possible implementation may further include: after obtaining q target keywords, constructing a full graph based on the q target keywords. The full graph includes q nodes, and there is a one-to-one correspondence between the q nodes and the q target keywords; the edges in the full graph indicate the edge information between two nodes.

[0146] A full target keyword graph refers to a complete graph constructed based on q target keywords. As an example, assume the set of target keywords obtained is... Each target keyword can then be treated as a node, and the relationships between words (i.e., edge information) can be constructed to form the entire graph. Edge information refers to the information about the edges connecting two nodes.

[0147] Specifically, for any two target keywords (kw) in the target keyword set i ,kw j ), can calculate their word distance in the original text. That is, kw i ,kw j The average distance between two target keywords that appear in the same original text. Represented as kw i ,kw j The amount of content appearing simultaneously in the same original text; T ij Represented as kwi ,kw j A collection that appears simultaneously in the same original text; D(kw i ,kw j () represents the distance between two words, that is, the number of characters between them.

[0148] In addition, the difference in similarity Δ can be calculated. ij =|ks i -ks j Then the edge between the two nodes can be defined as e. ij =-αlog(norm(Δ) ij ))-βlog(norm(d ij In this context, norm represents normalization, and α and β represent weighting parameters, which can be set according to different use cases. It should be understood that if... Therefore, there is no edge connecting the two nodes.

[0149] It should be understood that, in the embodiments of this application, by converting the target keywords into a full graph, the complex event or trend evolution process can be directly mapped, and the relationship between different nodes can be intuitively displayed, which is convenient for subsequent maintenance personnel to analyze the target object, increase the diversity of analysis results, and improve the accuracy of analysis results.

[0150] Based on the full spectrum provided in the above embodiments, in one possible implementation, it further includes: performing cluster analysis on the full spectrum to obtain clustering results.

[0151] It should be understood that after determining the full graph, cluster analysis is needed to facilitate the explanation of the relationships between different nodes and the connections between multiple nodes. In one possible implementation, a community detection algorithm can be used for cluster analysis of the full graph. In the field of graph computing, the community detection algorithm is used to divide the nodes in the full graph into different communities, such that nodes within a community have high internal connectivity (or high correlation), while the connectivity between communities is low. That is, through the community detection algorithm, the community structure in the full graph can be discovered, random walks can be used to simulate information flow, and communities can be identified by minimizing the amount of information required to describe this random walk process. The specific steps can be as follows:

[0152] E1: Initialize algorithm parameters (such as jump probabilities) and treat each node as an independent clustering result.

[0153] E2: Use a random walk to sample the nodes in the graph to obtain a sequence, and try to assign the node to the community of the neighbor node directly connected to the current node in order. Calculate the value of L(M), and take the community with the largest decrease in L(M) as the community of the current node. If L(M) does not decrease, the community of the current node is still itself.

[0154] E3: Repeat step E2 until L(M) remains unchanged, and obtain the final clustering result.

[0155] It should be noted that in some possible implementations, algorithms such as label propagation algorithms and hierarchical clustering algorithms can also be used to cluster the entire graph, and no specific limitations are made here.

[0156] It should be noted that in this embodiment, converting target keywords into a full graph and then performing cluster analysis based on the full graph facilitates the analysis of game ecosystem events. The game ecosystem refers to the unified whole formed by players and the game system within the space defined by the game. In this unified whole, players and the game system influence and constrain each other, maintaining a relatively stable dynamic equilibrium over a certain period. This equilibrium is achieved through player behavior, the design and adjustment of the game system, and the interaction between the two. That is, players share and comment on a specific event (i.e., the target object) in the game; maintenance personnel extract target keywords from these shares and comments, further determine the full graph based on the target keywords, and then perform cluster analysis on the full graph to obtain clustering results; based on the clustering results, maintenance personnel analyze the key points and needs discussed by players regarding a specific event, and optimize and adjust accordingly; the optimization and adjustment improve the player's gaming experience; thus, a game ecosystem is achieved: "Game → Player → Maintenance Personnel → Optimization and Adjustment → Game → Player".

[0157] It should be understood that, in the embodiments of this application, by performing cluster analysis on the full graph, scattered nodes can be transformed into meaningful communities, thereby providing clearer and more intuitive dynamic and trend predictions for subsequent maintenance personnel based on the analysis results of the target objects.

[0158] Based on the clustering results provided in the above embodiments, the clustering results can be visualized, such as... Figure 3 As shown, each node represents a target keyword. Two related nodes are connected by edges. Nodes from different communities are distinguished by different colors, indicating different analytical focuses on the target audience. Figure 3If the target of the display is "the first day data of a certain mobile game", it can be divided into communities such as "cannot log in", "disconnected", "under maintenance", "crashed", "quit". Each community includes corresponding target keywords, which makes it easier for developers to optimize "the certain mobile game" based on the clustering results.

[0159] Furthermore, after visualizing the clustering results, nodes can be selected, such as... Figure 4 As shown, assuming the selected node is "Maintenance", the nodes associated with the "Maintenance" node will be highlighted for easy viewing by developers.

[0160] Furthermore, after selecting a node, you can also see the degree of association between any two nodes, such as... Figure 5 As shown, assuming the selected nodes are "Cannot access" and "Uninstalled", you can view the degree of association between the two nodes, such as the degree of association being "Coor".

[0161] In addition, you can view the trends of more important events in the clustering results, such as Figure 6 As shown, Figure 6 In the text, 'a' and 'b' represent the target keywords corresponding to different important events (e.g., ...). Figure 6 The term 'a' in the table represents server crash, server overload, server shutdown, server failure, and quitting the game. Figure 6 The trend of (quitting, emulator, inability to log in, red envelopes, lag) shown in b in the figure can be used by developers to process and optimize the target object, and analyze whether the processing and optimization have a good effect based on the trend changes.

[0162] Furthermore, it's possible to trace back to the original content or comments based on target keywords, facilitating the maintenance of content or comments across different platforms. As shown in the image below, assuming the target is "first-day data for a mobile game," the relevant content could include:

[0163] 1. Maintenance is in progress, from 3 PM to 6 PM! Okay, and XXX is also under maintenance! #XXXMobileGame#XXXMobileGameMaintenance

[0164] 2. xxx mobile game server crash? One trick to fix it! [Link to video] There are also xxx redemption codes compiled for you at the end of the video! Use the code "123" today to get an extra speed-up card! #GameDaily #MobileGameRecommendation #VeteranPlayer…

[0165] Related comments can be:

[0166] 1. The server crashed and is undergoing maintenance.

[0167] 2. I didn't even log into the game. I worked every day, and after a few months I got tired of it and quit. What's the point?

[0168] 3. It won't recover until the next day. Did you fill up your xxx today? It won't recover until the next day. Did you fill up your xxx today? [Thumbs up][Thumbs up].

[0169] Based on the text processing method provided in the above embodiments, see [link to relevant documentation]. Figure 7 The figure is a schematic diagram of a text processing method provided in an embodiment of this application.

[0170] Combination Figure 7 As shown, the flow of the text processing method provided in this application embodiment can be as follows:

[0171] Step 1: Obtain the raw text data.

[0172] The raw text data includes, but is not limited to, comments, titles, ASRs, topics, descriptions, OCRs, etc.

[0173] Step 2: Preprocess the original text data to obtain the original text.

[0174] The preprocessing process can be found in the preprocessing process for the original text data in the above embodiments, and will not be repeated here.

[0175] Step 3: Segment the original text to obtain initial keywords.

[0176] The word segmentation process can utilize left and right information entropy for word segmentation. For related explanations, please refer to the above embodiments, which will not be repeated here.

[0177] Step 4: Use the target corpus to match the original keywords to obtain the first keyword.

[0178] Step 5: Extract keywords from the original text or the first keyword using preset rules to obtain the second keyword.

[0179] In this embodiment, keyword extraction can be performed not only on the first keyword but also on the original text. The preset rules refer to rules determined according to requirements. The process of extracting keywords from the original text is similar to that of extracting the first keyword and will not be repeated here.

[0180] Step 6: Determine the first feature of the second keyword and the second feature of the targeted text.

[0181] Step 7: Calculate the similarity based on the first and second features to obtain the similarity between the second keyword and the target text.

[0182] Step 8: Use the keyword extraction model to extract the second keyword and obtain candidate keywords.

[0183] Step 9: Match the candidate keywords with the calculated similarity to obtain the candidate similarity.

[0184] Step 10: Construct a candidate keyword dictionary based on candidate keywords and candidate similarity.

[0185] Step 11: Select the first q candidate keywords from the candidate keyword dictionary as target keywords.

[0186] Step 12: Use the detection list and stop word list to integrate the target keywords to obtain the target keyword set.

[0187] Step 13: Continuous detection of the target keyword set.

[0188] Step 14: Determine the correlation (edge ​​information) between every two target keywords in the target keyword set, and construct the full graph with the target keywords as nodes and the correlation as edges.

[0189] Step 15: Use the community detection algorithm to cluster the entire graph to obtain the clustering results.

[0190] Step 16: Perform post-processing and result verification based on the clustering results to obtain the detection results of the target object.

[0191] The detection results may include, but are not limited to, target keywords, similarity of target keywords, full graph, and trend changes of target keywords.

[0192] It should be noted that steps 1 to 16 provided in this application embodiment are similar to or the same as the steps in the above embodiments. For detailed explanations, please refer to the explanations of the above embodiments, which will not be repeated here. The beneficial effects of this application embodiment are the same as those of any of the above embodiments, which will not be repeated here.

[0193] Based on the text processing method provided in the foregoing embodiments, this application also provides a corresponding text processing apparatus. The following, in conjunction with... Figure 8 Please provide an explanation. Figure 8 This is a schematic diagram of the structure of a text processing device provided in an embodiment of this application. Figure 8 The text processing device shown includes:

[0194] The acquisition module 801 is used to acquire m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with the target object; the target object indicates the target application or target event; m is an integer greater than or equal to 2;

[0195] Matching module 802 is used to match m initial keywords with the target corpus to obtain n first keywords; the target corpus is constructed based on the target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords that have successfully matched with the target corpus; n is a positive integer less than or equal to m;

[0196] Module 803 is used to determine the similarity between each primary keyword and its corresponding target-oriented text, resulting in n similarity scores; the target-oriented text indicates text that is in the same direction of analysis as the target object.

[0197] The processing module 804 is used to sort the n primary keywords according to the sorting results of the n similarity scores from high to low, and to take the first q primary keywords as target keywords; q is an integer greater than or equal to 1.

[0198] In one possible implementation, the acquisition module 801 includes:

[0199] The raw text acquisition unit is used to acquire multiple raw texts.

[0200] The word segmentation unit is used to segment each original text into words to obtain multiple original keywords;

[0201] The left and right information entropy calculation unit is used to determine the left and right information entropy of each original keyword; the left and right information entropy is the sum of the left and right information entropy of the original keyword.

[0202] The acquisition unit is used to take the original keywords whose left and right information entropies are greater than the preset left and right information entropies as the initial keywords, and obtain m initial keywords.

[0203] In one possible implementation, the left and right information entropy calculation units are used for:

[0204] Determine the first frequency of the target original keyword among multiple original keywords; the target original keyword can be any one of the multiple original keywords.

[0205] Determine the second word frequency of the left neighboring words of the target original keyword, and determine the third word frequency of the right neighboring words of the target original keyword;

[0206] The left information entropy of the target original keyword is determined based on the first and second word frequencies, and the right information entropy of the target original keyword is determined based on the first and third word frequencies;

[0207] The sum of the left and right information entropies is taken as the left and right information entropies of the target original keyword; wherein, the left and right information entropies of each original keyword are obtained according to the method for determining the left and right information entropies of the target original keyword.

[0208] In one possible implementation, the raw text acquisition unit is used for:

[0209] Acquire multiple raw text data; the raw text data has a relationship with the target object;

[0210] Preprocessing is performed on each of the original text data to obtain multiple original texts.

[0211] In one possible implementation, the matching module 802 includes:

[0212] The determining unit is used to determine the fourth word frequency of the target initial keyword among m initial keywords, and to obtain the fifth word frequency of the target initial keyword in the target corpus; the target initial keyword is any one of the m initial keywords;

[0213] The target word frequency determination unit is used to take the product of the fourth word frequency and the fifth word frequency as the target word frequency of the initial target keyword; according to the method of obtaining the target word frequency of the initial target keyword, the target word frequency of each initial keyword is determined to obtain m target word frequencies;

[0214] The matching unit is used to sort the m initial keywords in descending order of the frequency of the m target words, and to take the first n initial keywords as the n first keywords.

[0215] In one possible implementation, before determining module 803, the module further includes: a filtering module, used to select p first keywords from n first keywords according to a preset rule, as p second keywords; where p is a positive integer less than or equal to n.

[0216] Module 803 is used to: determine the similarity between each second keyword and the corresponding target text, and obtain p similarity scores;

[0217] Processing module 804 is used to sort p secondary keywords according to the p similarity ranking results from high to low.

[0218] In one possible implementation, module 803 is defined for:

[0219] Feature extraction is performed on each primary keyword to obtain the primary features;

[0220] Feature extraction is performed on each target-oriented text to obtain the second feature;

[0221] Based on the first feature of each first keyword and the second feature of each target-oriented text, the similarity between each first keyword and the corresponding target-oriented text feature is calculated to obtain n similarity scores.

[0222] In one possible implementation, prior to processing module 804, an extraction module is also included, for:

[0223] Using a pre-trained keyword extraction model, extract i candidate keywords from n primary keywords; where i is a positive integer less than or equal to n.

[0224] Determine the similarity of each of the i candidate keywords as the candidate similarity, and obtain i candidate similarities;

[0225] The processing module 804 is used to sort the i candidate keywords according to the sorting results of the i candidate similarity from high to low.

[0226] In one possible implementation, after processing module 804, the system further includes: a construction module for constructing a full graph based on q target keywords; the full graph includes q nodes, and there is a one-to-one correspondence between the q nodes and the q target keywords; the edges in the full graph indicate the edge information between two nodes.

[0227] In one possible implementation, after the building module, a clustering analysis module is also included, which is used to perform clustering analysis on the full map to obtain clustering results.

[0228] The text processing apparatus provided in this application embodiment has the same beneficial effects as the text processing method provided in the above embodiments, and therefore will not be described again.

[0229] The following sections will describe the structures for server-based and terminal-based configurations.

[0230] Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the server 900.

[0231] Server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0232] CPU 922 is used to perform the following steps:

[0233] Obtain m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with the target object; the target object indicates the target application or target event; m is an integer greater than or equal to 2;

[0234] The m initial keywords are matched with the target corpus to obtain n first keywords; the target corpus is constructed based on the target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords that successfully match the target corpus; n is a positive integer less than or equal to m;

[0235] Determine the similarity between each primary keyword and its corresponding target-oriented text to obtain n similarity scores; the target-oriented text indicates text that is in the same direction of analysis as the target object.

[0236] Sort the n primary keywords according to their similarity scores from highest to lowest, and select the top q primary keywords as target keywords; q is an integer greater than or equal to 1.

[0237] This application also provides another embodiment, such as... Figure 10 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0238] Figure 10 This is a block diagram illustrating a portion of the structure of a mobile phone related to the terminal provided in the embodiments of this application. (Reference) Figure 10The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090, etc. Those skilled in the art will understand that... Figure 10 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0239] The following is combined Figure 10 A detailed introduction to each component of a mobile phone:

[0240] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0241] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0242] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0243] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1041. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 10 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0244] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0245] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0246] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 10 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0247] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby collecting overall data and information from the phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1080.

[0248] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0249] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0250] In this embodiment of the application, the processor 1080 included in the terminal also has the following functions:

[0251] Obtain m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with the target object; the target object indicates the target application or target event; m is an integer greater than or equal to 2;

[0252] The m initial keywords are matched with the target corpus to obtain n first keywords; the target corpus is constructed based on the target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords that successfully match the target corpus; n is a positive integer less than or equal to m;

[0253] Determine the similarity between each primary keyword and its corresponding target-oriented text to obtain n similarity scores; the target-oriented text indicates text that is in the same direction of analysis as the target object.

[0254] Sort the n primary keywords according to their similarity scores from highest to lowest, and select the top q primary keywords as target keywords; q is an integer greater than or equal to 1.

[0255] This application also provides a computer-readable storage medium for storing program code that executes any one of the text processing methods described in the foregoing embodiments.

[0256] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to execute any one of the text processing methods described in the foregoing embodiments.

[0257] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and equipment described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0258] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of the system is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple systems may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0259] The system described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0260] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0261] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0262] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A text processing method, characterized in that, include: Get the m initial keywords to be processed; The m initial keywords were obtained based on multiple original texts; The original text is the text associated with the target object; The target object indicates the target application or target event; m is an integer greater than or equal to 2; The m initial keywords are matched with the target corpus to obtain n first keywords; The target corpus is constructed based on the target text set, where the target text is the commonly used text of the target object; n is a positive integer less than or equal to m; Determine the similarity between each of the first keywords and the corresponding targeted text to obtain n similarity values; the targeted text indicates text that is in the same direction of analysis as the target object; Sort the n first keywords according to the n similarity ranking results from high to low, and take the first q first keywords as target keywords; The q is an integer greater than or equal to 1.

2. The method according to claim 1, characterized in that, The process of obtaining the m initial keywords to be processed includes: Retrieve multiple raw texts; The original texts are segmented to obtain multiple original keywords; Determine the left and right information entropies of each of the original keywords; the left and right information entropies are the sum of the left and right information entropies of the original keywords; The original keywords whose left and right information entropies are greater than the preset left and right information entropies are used as the initial keywords to obtain m initial keywords.

3. The method according to claim 2, characterized in that, Determining the left and right information entropy of each of the original keywords includes: Determine the first word frequency of the target original keyword among a plurality of original keywords; the target original keyword is any one of the plurality of original keywords. Determine the second word frequency of the left neighboring words of the target original keyword, and determine the third word frequency of the right neighboring words of the target original keyword; The left information entropy of the target original keyword is determined based on the first word frequency and the second word frequency, and the right information entropy of the target original keyword is determined based on the first word frequency and the third word frequency; The sum of the left information entropy and the right information entropy is taken as the left and right information entropy of the target original keyword; wherein, the left and right information entropy of each original keyword is obtained according to the method for determining the left and right information entropy of the target original keyword.

4. The method according to claim 2, characterized in that, The acquisition of multiple raw texts includes: Acquire multiple raw text data; the raw text data is associated with the target object; The original text data is preprocessed to obtain multiple original texts.

5. The method according to claim 1, characterized in that, The process of matching the m initial keywords with the target corpus to obtain n first keywords includes: Determine the fourth word frequency of the target initial keyword among the m initial keywords, and obtain the fifth word frequency of the target initial keyword in the target corpus; the target initial keyword is any one of the m initial keywords; The product of the fourth word frequency and the fifth word frequency is taken as the target word frequency of the target initial keyword; Based on the method for obtaining the target word frequency of the initial target keywords, the target word frequency of each initial keyword is determined, resulting in m target word frequencies; The m initial keywords are sorted in descending order of the frequency of the m target words, and the first n initial keywords are taken as the n first keywords.

6. The method according to claim 1, characterized in that, Before determining the similarity between each of the first keywords and the corresponding targeted text, the process also includes: According to a preset rule, p first keywords are selected from n first keywords to serve as p second keywords; where p is a positive integer less than or equal to the n keywords. Determining the similarity between each of the first keywords and the corresponding targeted text includes: Determine the similarity between each of the second keywords and the corresponding target text to obtain p similarity scores; The step of sorting the n first keywords according to the n similarity ranking results from high to low includes: Sort the p second keywords according to the p similarity ranking results from high to low.

7. The method according to claim 1, characterized in that, The similarity between each of the first keywords and the corresponding targeted text is determined, resulting in n similarity scores, including: Feature extraction is performed on each of the first keywords to obtain the first features; Feature extraction is performed on each of the target-oriented texts to obtain the second feature; Based on the first features of each first keyword and the second features of each target-oriented text, the similarity between each first keyword and the corresponding target-oriented text features is calculated to obtain n similarity scores.

8. The method according to claim 7, characterized in that, Before sorting the n first keywords according to the n similarity rankings from high to low, the method further includes: Using a pre-trained keyword extraction model, extract i candidate keywords from n first keywords; where i is a positive integer less than or equal to n. The similarity of each of the i candidate keywords is determined as the candidate similarity, thus obtaining i candidate similarities; The step of sorting the n first keywords according to the n similarity ranking results from high to low includes: The i candidate keywords are sorted from high to low according to the ranking of the similarity of the i candidate keywords.

9. The method according to any one of claims 1-8, characterized in that, After obtaining q target keywords, the method further includes: constructing a full graph based on the q target keywords; the full graph includes q nodes, and there is a one-to-one correspondence between the q nodes and the q target keywords; the edges in the full graph indicate the edge information between two nodes.

10. The method according to claim 9, characterized in that, After constructing a full graph based on q target keywords, the method further includes: performing cluster analysis on the full graph to obtain clustering results.

11. A text processing device, characterized in that, The device includes: The acquisition module is used to acquire m initial keywords to be processed; the m initial keywords are obtained based on multiple original texts; the original texts are texts associated with a target object; the target object indicates a target application or target event; and m is an integer greater than or equal to 2. A matching module is used to match m initial keywords with a target corpus to obtain n first keywords; the target corpus is constructed based on a target text set, and the target text is the commonly used text of the target object; the first keywords are the initial keywords whose matching result with the target corpus is successful; and n is a positive integer less than or equal to m. The determination module is used to determine the similarity between each of the first keywords and the corresponding target-oriented text, obtaining n similarity values; the target-oriented text indicates text that is in the same direction of analysis as the target object; The processing module is used to sort the n first keywords according to the sorting results of the n similarity in descending order, and to take the first q first keywords as target keywords; where q is an integer greater than or equal to 1.

12. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the steps of the text processing method according to any one of claims 1 to 10, based on instructions in the program code.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the steps of the text processing method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, Includes a computer program or instructions that, when executed, implement the steps of the text processing method according to any one of claims 1 to 10.

Images