Text processing method and device

Abstract

The present application discloses a text processing method and apparatus, wherein the method includes: performing word segmentation processing on an original search text to obtain M word segments, where M is a positive integer; performing synonym rewriting on the M word segments in sequence according to a synonym table to obtain an Mth candidate text set; the synonym table is used to indicate the correspondence between semantic words and synonyms of the semantic words; each candidate text in the Mth candidate text set has the same semantics as the original search text; selecting candidate texts from the Mth candidate text set whose rewriting probability exceeds a preset rewriting threshold to form a target text set, where the rewriting probability is determined by the number of plays of a playlist associated with the candidate text; determining a search input text based on the target text set, and using the search input text to search for the playlist. The present application can improve the accuracy of synonym rewriting of text.

Description

Technical Field

[0001] The present invention relates to the field of text processing technology, and in particular to a text processing method and device. Background Art

[0002] Usually, when users search for music, they use the singer's name, song name or semantic description as the search terms for matching on the music platform. Among them, users searching based on the singer's name or song name can accurately match the same type of songs. When users search based on semantic descriptions (such as semantic words such as "sleep" and "sleep aid"), different users often use rich and different search terms for the same type of songs. Therefore, the music platform can rewrite the search terms as synonyms to improve the recall ability of songs in the same category and enhance the diversity of search results. For example, when a user searches for "lullaby", "lullaby" can be rewritten as "lullaby", "lullaby", etc., so that the music platform can match songs based on "lullaby", "lullaby" and "sleep aid".

[0003] Currently, synonym rewriting is primarily based on synonym mining and replacement. This method requires first mining a synonym table from log information, then using this table to rewrite the search terms. This approach, however, lacks sufficient log information to meet the needs of synonym rewriting in the music field, suffers from missing or inaccurate data, and results in low synonym rewriting accuracy. Summary of the Invention

[0004] The embodiments of the present application provide a text processing method and apparatus, which can improve the accuracy of synonym rewriting of text.

[0005] In a first aspect, an embodiment of the present application provides a text processing method, the method comprising:

[0006] Perform word segmentation on the original search text to obtain M word segments, where M is a positive integer;

[0007] Rewriting the M word segments into synonyms in sequence according to a synonym table to obtain an Mth candidate text set; the synonym table is used to indicate the corresponding relationship between semantic words and synonyms of the semantic words; each candidate text in the Mth candidate text set has the same semantics as the original search text;

[0008] Selecting candidate texts with a rewriting probability exceeding a preset rewriting threshold from the Mth candidate text set to form a target text set, wherein the rewriting probability is determined by the number of plays of the playlist associated with the candidate text;

[0009] A search input text is determined based on the target text set, and the search input text is used to search for a playlist.

[0010] Based on the method described in the first aspect, this application can use a synonym table to sequentially rewrite the segmented words in the original search text into synonyms, and after rewriting, evaluate the rewriting probability (or rewriting quality) based on the associated playlist playback volume to screen the rewriting results. This method improves the accuracy of synonym rewriting.

[0011] In a possible implementation, the M word segments are sequentially rewritten into synonyms according to the synonym table to obtain the Mth candidate text set, including:

[0012] Set N to 1 and obtain the N-1th candidate text set, wherein the N-1th candidate text set includes at least one candidate text, each candidate text in the at least one candidate text has the same semantics as the original search text, and the first N-1 word segments of each candidate text are processed by synonym rewriting, where N is a positive integer and is less than or equal to M;

[0013] Searching for a synonym set corresponding to the Nth participle of each candidate text in a synonym table;

[0014] Rewriting the Nth word in each candidate text into a synonym based on the synonym set to obtain a candidate text set to be processed;

[0015] Determining a rewriting probability of each candidate text to be processed in the set of candidate texts to be processed;

[0016] Selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set;

[0017] If N is not equal to M, perform an addition operation on N, and return to the step of obtaining the N-1th candidate text set.

[0018] Based on this method, the present application can rewrite the M word segments into synonyms in sequence, and during the rewriting process, the rewriting probability is also used to screen the candidate texts to be processed corresponding to each word segment.

[0019] In one possible implementation, determining the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed includes:

[0020] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0021] Determining a second probability for each candidate text to be processed based on a replacement probability corresponding to an Nth word segment in each candidate text to be processed; the replacement probability corresponding to the Nth word segment is determined by the amount of playlists associated with a second processing segment in which the Nth word segment is located, the second processing segment being context-related to the Nth word segment;

[0022] The product of the first probability and the second probability is used as the rewriting probability.

[0023] Based on this method, the present application can evaluate the commonality of the recalled playlists of each candidate text to be processed through the correlation between the first processing segment and the second processing segment in each candidate text to be processed and the playlist playback volume, and then evaluate the rewriting quality of each candidate text to be processed (the higher the playback volume of the recalled playlist, the more common the playlist, and the better the rewriting quality of the candidate text to be processed).

[0024] In one possible implementation, the playback volume of the playlist associated with the first processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the first processing segment; the playback volume of the playlist associated with the second processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the second processing segment; or

[0025] The number of plays of the playlist related to the first processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the first processing segment; the number of plays of the playlist related to the second processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the second processing segment.

[0026] Based on this approach, the present application can evaluate the rewriting quality based on whether the first and second processed segments fully match or partially match the playlist title. Compared to a complete match approach, the partial match approach increases the number of matching situations and the diversity of matching situations.

[0027] In one possible implementation, determining the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed includes:

[0028] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0029] Obtain the historical click rate of the playlist related to each candidate text to be processed;

[0030] The product of the first probability and the historical click rate is used as the rewriting probability.

[0031] Based on this method, this application evaluates the rewriting quality of the candidate text to be processed based on the two dimensions of the playback volume and historical click rate of the relevant playlist, thereby improving the accuracy of the evaluation.

[0032] In one possible implementation, the selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form the Nth candidate text set includes:

[0033] Delete the first candidate text to be processed and the second candidate text to be processed from the candidate text set to be processed to obtain a pre-selected set; among the first N-1 participles of the first candidate text to be processed, there is a participle that is the same as the Nth participle or a synonym of the Nth participle; and the second candidate text to be processed is a candidate text to be processed that has the same participles but different combination orders and does not have the highest rewriting probability among the multiple candidate texts to be processed;

[0034] A preset number of candidate texts to be processed are selected from the preselected set based on the rewriting probability to form the Nth candidate text set.

[0035] Based on this method, this application can eliminate two situations: words that are the same as the previous N-1 word segmentations after rewriting (this situation reduces the diversity of the rewritten text), or different combination orders of multiple identical words (using the same words in different combination orders to search for playlists, the searched playlists are less different), so as to further improve the accuracy of synonym rewriting and improve the efficiency of synonym rewriting.

[0036] In one possible implementation, determining the search input text according to the target text set includes:

[0037] The target text with the highest rewriting probability in the target text set or each target text in the target text set is determined as the search input text.

[0038] Based on this approach, the present application can determine the search input text based on the target text set to search for a playlist.

[0039] In one possible implementation, the method further includes:

[0040] Obtain the playlist title set and the historical search set;

[0041] Performing word segmentation processing on the playlist titles in the playlist title set to obtain a first word segmentation set, and performing word segmentation processing on the historical searches in the historical search set to obtain a second word segmentation set;

[0042] Invoking an intent recognition model to perform semantic recognition on the first word segmentation set and the second word segmentation set, respectively, to obtain a first semantic word set and a second semantic word set;

[0043] Calling a vector embedding model to search for a first synonym set corresponding to the first semantic word set;

[0044] Searching for a second synonym set corresponding to the second semantic word set in a preset synonym mining vocabulary;

[0045] Two semantic words are randomly selected from the set including the first synonym set and the second synonym set for semantic determination. If the two semantic words are synonyms, the corresponding relationship between the two semantic words is recorded in the synonym table.

[0046] Based on this method, the present application can use a set of playlist titles and a set of historical searches that are strongly related to the music field as the corpus for constructing the synonym table, which better solves the problem of missing or inaccurate synonym data in the music field, thereby enabling the present application to have a more accurate synonym rewriting capability for the search text in the music platform, thereby improving the recall rate of high-quality playlists.

[0047] In a second aspect, an embodiment of the present application provides a text processing device, the device comprising:

[0048] The word segmentation unit is used to segment the original search text to obtain M words, where M is a positive integer;

[0049] a rewriting unit, configured to perform synonym rewriting on the M segmented words in sequence according to a synonym table to obtain an Mth candidate text set; the synonym table is configured to indicate a correspondence between a semantic word and its synonym; each candidate text in the Mth candidate text set has the same semantics as the original search text;

[0050] a selection unit, configured to select candidate texts having a rewriting probability exceeding a preset rewriting threshold from the Mth candidate text set to form a target text set, wherein the rewriting probability is determined by the number of plays of a playlist associated with the candidate text;

[0051] A determination unit is used to determine a search input text based on the target text set, and the search input text is used to search for a playlist.

[0052] In one possible implementation, the rewriting unit, when used to sequentially rewrite the M segmented words into synonyms according to the synonym table to obtain the Mth candidate text set, specifically includes:

[0053] Set N to 1 and obtain the N-1th candidate text set, wherein the N-1th candidate text set includes at least one candidate text, each candidate text in the at least one candidate text has the same semantics as the original search text, and the first N-1 word segments of each candidate text are processed by synonym rewriting, where N is a positive integer and is less than or equal to M;

[0054] Searching for a synonym set corresponding to the Nth participle of each candidate text in a synonym table;

[0055] Rewriting the Nth word in each candidate text into a synonym based on the synonym set to obtain a candidate text set to be processed;

[0056] Determining a rewriting probability of each candidate text to be processed in the set of candidate texts to be processed;

[0057] Selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set;

[0058] If N is not equal to M, perform an addition operation on N, and return to the step of obtaining the N-1th candidate text set.

[0059] In one possible implementation, the rewriting unit, when used to determine the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed, specifically includes:

[0060] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0061] Determining a second probability for each candidate text to be processed based on a replacement probability corresponding to an Nth word segment in each candidate text to be processed; the replacement probability corresponding to the Nth word segment is determined by the amount of playlists associated with a second processing segment in which the Nth word segment is located, the second processing segment being context-related to the Nth word segment;

[0062] The product of the first probability and the second probability is used as the rewriting probability.

[0063] In one possible implementation, the playback volume of the playlist associated with the first processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the first processing segment; the playback volume of the playlist associated with the second processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the second processing segment; or

[0064] The number of plays of the playlist related to the first processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the first processing segment; the number of plays of the playlist related to the second processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the second processing segment.

[0065] In one possible implementation, the rewriting unit, when used to determine the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed, specifically includes:

[0066] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0067] Obtain the historical click rate of the playlist related to each candidate text to be processed;

[0068] The product of the first probability and the historical click rate is used as the rewriting probability.

[0069] In one possible implementation, the selection unit, when configured to select a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form the Nth candidate text set, specifically includes:

[0070] Delete the first candidate text to be processed and the second candidate text to be processed from the candidate text set to be processed to obtain a pre-selected set; among the first N-1 participles of the first candidate text to be processed, there is a participle that is the same as the Nth participle or a synonym of the Nth participle; and the second candidate text to be processed is a candidate text to be processed that has the same participles but different combination orders and does not have the highest rewriting probability among the multiple candidate texts to be processed;

[0071] A preset number of candidate texts to be processed are selected from the preselected set based on the rewriting probability to form the Nth candidate text set.

[0072] In one possible implementation, the determining unit, when used to determine the search input text according to the target text set, specifically includes:

[0073] The target text with the highest rewriting probability in the target text set or each target text in the target text set is determined as the search input text.

[0074] In one possible implementation, the text processing apparatus further includes a construction unit, wherein the construction unit is configured to:

[0075] Obtain the playlist title set and the historical search set;

[0076] Performing word segmentation processing on the playlist titles in the playlist title set to obtain a first word segmentation set, and performing word segmentation processing on the historical searches in the historical search set to obtain a second word segmentation set;

[0077] Invoking an intent recognition model to perform semantic recognition on the first word segmentation set and the second word segmentation set, respectively, to obtain a first semantic word set and a second semantic word set;

[0078] Calling a vector embedding model to search for a first synonym set corresponding to the first semantic word set;

[0079] Searching for a second synonym set corresponding to the second semantic word set in a preset synonym mining vocabulary;

[0080] Two semantic words are randomly selected from the set including the first synonym set and the second synonym set for semantic determination. If the two semantic words are synonyms, the corresponding relationship between the two semantic words is recorded in the synonym table.

[0081] In a third aspect, an embodiment of the present application provides a terminal device comprising: a memory, a processor; the memory is used to store a computer program; the processor is used to call the computer program from the memory, so that the terminal device executes any one of the methods in the above-mentioned first aspect.

[0082] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed on the terminal device of the second aspect, the terminal device executes any one of the methods in the first aspect.

[0083] In a fifth aspect, an embodiment of the present application provides a computer program or computer program product, comprising code or instructions, which, when executed on a computer, enables the computer to execute any one of the methods in the first aspect above.

[0084] The beneficial effects of each possible implementation method in the second to fifth aspects can be found in the corresponding description in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments.

[0086] Figure 1 This is a schematic diagram of a text processing system architecture provided by an embodiment of the present application;

[0087] Figure 2 This is a flowchart of a text processing method provided by an embodiment of the present application;

[0088] Figure 3 This is a flow chart of a method for determining a candidate text set provided in an embodiment of the present application;

[0089] Figure 4 This is a flow chart of a method for constructing a synonym table provided in an embodiment of the present application;

[0090] Figure 5 is a structural diagram of a text processing device provided in an embodiment of the present application;

[0091] Figure 6 This is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0093] In the specification, claims, and drawings of this application, the terms "first," "second," and the like are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0094] In order to better understand the solution in this application, the following is an introduction to the technical fields involved in this application:

[0095] Artificial intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science. By understanding the essence of intelligence, it aims to produce new intelligent machines that can react in a manner similar to human intelligence, enabling them to possess diverse capabilities such as perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing several key areas, including computer vision (CV), speech processing, natural language processing, and machine learning (ML) or deep learning.

[0096] Natural language processing (NLP) studies the various theories and methods that enable effective communication between humans and computers using natural language. It is a discipline that integrates linguistics, computer science, and mathematics. Therefore, the study of this technology involves natural language, the language we use everyday, and is closely related to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0097] Based on the natural language processing technology in AI technology, this application can use a synonym table constructed based on a playlist title set and a historical search set to perform synonym rewriting on the segmented words in the original search text in turn to obtain the rewriting results. Then, the rewriting probability (or rewriting quality) is evaluated by the associated playlist playback volume to screen the rewriting results. This method better solves the problem of missing or inaccurate synonym data in the music field, and performs a quality assessment on the synonym rewriting results, thereby improving the accuracy of synonym rewriting.

[0098] The following describes the system architecture of the embodiment of the present application:

[0099] See also Figure 1 , is a diagram of a text processing system architecture provided by an embodiment of the present application. Figure 1As shown, the system includes a text processing device 101 and a search device 102. A search interface is displayed on the search device 102. When a user needs to search, the user can input search content in the search interface in the form of text or voice, and the search device 102 converts the search content into an original search text and sends it to the text processing device 101. After receiving the original search text, the text processing device 101 can rewrite the original search text and finally obtain a rewritten result of the original search text (or called a search input text, used to search for a playlist). In another implementation, the text processing device 101 is integrated in the search device 102, or the search device 102 has the ability to rewrite the search text introduced in the embodiment of the present application.

[0100] Below through Figure 1 The example in illustrates the rewriting process of the original search text, such as Figure 1 As shown: First, the text processing device 101 will perform word segmentation processing on the original search text "XXXXXX" to obtain the word segmentation results "w1, w2, w3", w1 is the first word segmentation result, w2 is the second word segmentation result, and w3 is the third word segmentation result; then, according to the position of the word segmentation result, w1, w2, and w3 are rewritten as synonyms in turn.

[0101] Specifically, when rewriting w1, first determine the synonym set corresponding to w1 (for example, the two words "A1, A2"). Replace w1 in "w1, w2, w3" with any word in the synonym set, or do not replace w1 in "w1, w2, w3". This results in three candidate texts to be processed: "w1, w2, w3", "A1, w2, w3", and "A2, w2, w3". The rewriting probabilities of the three candidate texts to be processed are calculated respectively. Based on the obtained rewriting probabilities, two texts, "w1, w2, w3" and "A1, w2, w3", are selected from the three candidate texts to be processed as the input for rewriting w2.

[0102] Next, w2 is rewritten. When rewriting w2, the synonym set corresponding to w2 is first determined (examples include the two words "B1, B2"), and w2 in "w1, w2, w3", "A1, w2, w3" is replaced with any word in the synonym set, or w2 in "w1, w2, w3", "A1, w2, w3" is not replaced, and six candidate texts to be processed are obtained: "w1, w2, w3", "w1, B1, w3", "w1, B2, w3", "A1, w2, w3", "A1, B1, w3", and "A1, B2, w3". The rewriting probabilities of these six candidate texts to be processed are calculated respectively, and based on the obtained rewriting probabilities, two texts, "w1, B2, w3" and "A1, w2, w3" are selected from these six candidate texts to be processed as the input for rewriting w3.

[0103] Finally, w3 is rewritten. When rewriting w3, the synonym set corresponding to w3 is first determined (examples include the two words "C1, C2"), and w3 in "w1, B2, w3", "A1, w2, w3" is replaced with any word in the synonym set, or w3 in "w1, B2, w3", "A1, w2, w3" is not replaced, and six candidate texts to be processed are obtained: "w1, B2, w3", "w1, B2, C1", "w1, B2, C2", "A1, w2, w3", "A1, w2, C1", and "A1, w2, C2". The rewriting probabilities of these six candidate texts to be processed are calculated respectively, and based on the obtained rewriting probabilities, "w1, B2, C2" and "A1, w2, w3" are screened out from these six candidate texts as the rewriting results of the original search text.

[0104] It should be noted that, in the rewriting process of each word segmentation mentioned above, in addition to screening multiple candidate texts to be processed based on the rewriting probability, the multiple candidate texts to be processed can also be further screened in combination with other screening strategies (for example, for multiple candidate texts to be processed that include the same words but different combination orders, only the candidate text to be processed with the highest rewriting probability is retained). This application does not impose any restrictions on this. In addition, after obtaining "w1, B2, C2" and "A1, w2, w3", the rewriting results can also be filtered in one step (such as using threshold filtering), and the filtered results are used as the final rewriting results.

[0105] The text processing device 101 in this application may be a terminal device or a server, etc., and the search device 101 may be a terminal device, etc. For example, the terminal device may be a device with a data processing function, including but not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a vehicle, a roadside device, an aircraft, a wearable device, etc., and this application does not impose any restrictions on this.

[0106] A server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0107] The above content introduces the system architecture of this application solution. Figures 2 to 6 The embodiments of the present invention provide a detailed introduction to the text processing method and device proposed in this application.

[0108] See also Figure 2 , is a flowchart of a text processing method provided in an embodiment of the present application. This method can be applied to the above-mentioned text processing device 101 and includes steps S201 to S204. Among them:

[0109] S201 : Perform word segmentation on the original search text to obtain M word segments, where M is a positive integer.

[0110] Among them, the original search text (or called original text, search text, etc.) is a semantic text that is not a song name or a singer name (when the user searches by song name or singer name, they often expect to accurately find a specific song or a singer's playlist. In this case, synonym rewriting is not applicable). The original search text is composed of one or more semantic words. These semantic words are used to describe the type of playlist, such as "classic old songs", "European and American songs", etc. The text processing device can receive various types of text input by the user from the search device (such as the client where the user is located), and determine whether these texts are semantic texts. If they are semantic texts, they will be used as original search texts for subsequent processing. It should be noted that the search device can receive text information typed by the user, or audio information input by voice, etc., convert this information into text and then send it to the text processing device. The search device here can be Figure 1 The search device 102 in.

[0111] Specifically, the text processing device can use a segmentation tool, such as SnowNLP, Thulac, HanLP, or Jieba, to segment the original search text. The segmentation tool generates M segmentations based on the semantic relationships between characters in the original search text. For example, if the original search text is "classic love songs in Cantonese," segmentation may yield three segmentations: "classic," "love songs," and "Cantonese."

[0112] Optionally, a series of text preprocessing tasks can be performed before word segmentation of the original search text, including removing punctuation marks, removing invalid characters, etc., to improve the processing efficiency of word segmentation of the original search text.

[0113] S202: rewrite the M word segments into synonyms in sequence according to the synonym table to obtain an M-th candidate text set.

[0114] Among them, the synonym table is constructed based on the playlist title set and the historical search set, and the synonym table is used to indicate the correspondence between semantic words and synonyms of semantic words; each candidate text in the Mth candidate text set has the same semantics as the original search text.

[0115] The following is a detailed introduction on how to obtain the Mth candidate text set.

[0116] When the text processing device rewrites the M words of the original search text into synonyms, it rewrites each word from the beginning to the end according to the position of the word in the original search text. For example, when the text processing device rewrites the text sequence W (W = {w1, w2, ..., w N ,...,w M},w N When rewriting synonyms for the Nth participle), w1, w2, ..., w N ,...,w M Rewrite, for example, after rewriting w1, we get the first candidate text set, after rewriting w2, we get the second candidate text set, after rewriting w3, we get the third candidate text set, ..., after rewriting w M After rewriting, the Mth candidate text set is obtained. Specifically, first, the text processing device obtains a second candidate text set based on the first candidate text set and the synonym table. Then, based on the second candidate text set and the synonym table, the device obtains a third candidate text set. Subsequent candidate text sets are obtained in this order until the Mth candidate text set is obtained.

[0117] The following is based on Figure 3A further introduction is given on how to obtain the Nth candidate text set based on the (N - 1)th candidate text set and the synonym table during the rewriting process. Here, N is a positive integer and N is less than or equal to M. For example, Figure 3 As shown, this process includes steps S1 to S5, where:

[0118] Step S1: Obtain the (N - 1)th candidate text set. The (N - 1)th candidate text set includes at least one candidate text. Each candidate text in the at least one candidate text has the same semantics as the original search text, and the first (N - 1) word segments of each candidate text have been processed by synonym rewriting.

[0119] That is to say, for each candidate text in the (N - 1)th candidate text set, w1 to w N-1 have been processed by synonym rewriting, while w N to w M have not been processed by synonym rewriting. It should be noted that when it is mentioned here that w1 to w N-1 have been processed by synonym rewriting, it means that each word segment in w1 to w N-1 may be replaced by any synonym of w1 to w N-1 or not replaced by a synonym of w1 to w N-1

[0120] Taking {inspiring, young, youth} as an example, when N takes the value of 2, "inspiring" in each candidate text in the 1st candidate text set has been processed by synonym rewriting (the rewriting process here also includes the case of not being rewritten as a synonym). If "inspiring" includes two synonyms, namely "encouraging" and "motivating", then the 1st candidate text set includes one or more of the three texts {inspiring, young, youth}, {encouraging, young, youth}, and {motivating, young, youth}.

[0121] It should be noted that when N takes the value of 1, the (N - 1)th candidate text set is the 0th candidate text set, and the 0th word segment in the 0th candidate text set has been processed by synonym rewriting, which means that all word segments of each candidate text in this set have not been processed by synonym rewriting. That is to say, the 0th candidate text set is the original search text.

[0122] S2: Search for the set of synonyms corresponding to the Nth word segment of each candidate text in the synonym table.

[0123] Among them, the synonym table is used to indicate the correspondence between semantic words and their synonyms. The synonym table is constructed based on the playlist title set and the historical search set. The specific construction method can be referred to the introduction of the corresponding embodiments in the following Figure 4 and will not be elaborated here.

[0124] ​Specifically, in each candidate text of the N-1 candidate text set, the Nth segment has not yet been rewritten as a synonym, so this step can first search for all synonyms of the Nth segment in the synonym table to form a synonym set S N , so that in the subsequent steps we can N Rewrite the Nth participle. Since the synonym table stores the correspondence between words with the same semantics, by searching the synonym table for the correspondence related to the Nth participle, the synonym of the Nth participle can be determined based on the correspondence. For example, see Table 1 below, which is a synonym expression table provided in an embodiment of the present application. If the Nth participle is "cute", the synonym set of "cute" can be determined as "like, lovely, well-behaved" according to Table 1.

[0125] Table 1

[0126] Serial number Synonyms 1 Cute, lovable, lovely, well-behaved 2 Romantic, tender, romantic ... ...

[0127] S3. Based on the synonym set, the Nth word in each candidate text is rewritten into a synonym to obtain a candidate text set to be processed.

[0128] For example, if the N-1th candidate text set includes I candidate texts, and the synonym set of the Nth participle includes J synonyms, then for any candidate text among the I candidate texts, there are J possibilities for rewriting the Nth participle in any candidate text into a synonym. Based on the synonym set, performing synonym rewriting on the Nth participle can obtain I*(J+1) combinations (the reason for adding 1 to J here is that: when performing synonym rewriting, in addition to rewriting the Nth participle in the I candidate texts into any of the J possible cases, it also includes the case where the Nth participle in the I candidate texts is not rewritten). These I*(J+1) combinations constitute the candidate text set to be processed.

[0129] S4. Determine the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed.

[0130] Among them, this application proposes two methods to determine the rewrite probability, which are introduced below:

[0131] Method 1:

[0132] In one possible implementation, the text processing device first determines a first probability and a second probability, respectively. The first probability is used to represent the semantic rationality of the candidate text to be processed, and the second probability is used to represent the probability that the Nth segment is replaced by any synonym in the synonym set when the contextual semantics of the Nth segment is considered. Finally, the product of the first probability and the second probability is used as the rewriting probability of the candidate text to be processed, that is, P 改写= P1*P2. The rewriting probability is used to select one or more candidate texts to be processed from the candidate text set to be processed including I*(J+1) combinations to form the Nth candidate text set.

[0133] The following describes how to determine the first probability and the second probability:

[0134] ①, First Probability

[0135] In an embodiment of the present application, the text processing device can determine the first probability based on the text processing principle of the N-gram language model. In the N-gram language model, the appearance of any word in the text is only related to the appearance of the previous N-1 words; for example, in the 2-gram language model, the appearance of any word in the text is only related to the appearance of the previous word and is not related to any other words in the text. In the 3-gram language model, the appearance of any word in the text is only related to the appearance of the previous two words and is not related to any other words in the text, and so on. And the probability that the entire text can be formed (that is, the semantic rationality of the entire text) is equal to the product of the probabilities of each word appearing. Therefore, the probability of text formation is equal to the product of the conditional probabilities of each word appearing in the text. It should be noted that the value of N in N-gram can be any integer greater than or equal to 2, and is independent of N in the Nth word segmentation and the Nth candidate text set in this application. This application does not limit the value of N in N-gram.

[0136] Based on the above-mentioned N-gram language model principle, the method of determining the first probability in this application specifically includes: determining the product of the conditional probabilities corresponding to each participle in each candidate text to be processed as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the playback volume of the playlist related to the first processing segment in which any participle is located, and the first processing segment includes at least two adjacent participles and any participle is the last participle of at least two adjacent participles.

[0137] For example, the result of word segmentation is W = {w1, w2, ..., w N ,...,w M}, with {v1,v2,...,v N ,...,v M} represents any candidate text set to be processed in the candidate text set to be processed, where “v N ,...,v M ” and “w B ,...,w M " are exactly the same (meaning that the Nth to Mth participles are not rewritten as synonyms), "v1, v2, ..., v N-1 ” is “w1,w2,...,wN-1 " is a combination of each participle or synonym of each participle (indicating that the first to N-1th participles have been rewritten as synonyms). Furthermore, the first probability can be calculated by multiplying the conditional probabilities corresponding to each participle in each candidate text to be processed. For example, taking the 3-gram language model with N being 3 as an example, the first probability can be obtained by referring to the following formula:

[0138] P1=p(B1,B2,v1,v2,...,v N ,...,v M ,E1,E2)

[0139] =p(v1|B1B2)*p(v2|B2v1)*...*p(v N |v N-2 v N-1 )*...*p(E2|v M E1)

[0140] Where M is the total number of segmented words, and "B1, B2" and "E1, E2" are additional head and tail markers used to mark the beginning and end of the candidate text to be processed. Adding "B1, B2" before the segmented result ensures that "v1" and "v2" have the first two associated words. The segmentation processing device can calculate the conditional probabilities corresponding to "v1" and "v2" based on "B1B2v1" and "B2v1v2", respectively.

[0141] "B1B2v1", "B2v1v2", "v N-2 v N-1 v N ", "v M E1E2" is the participle "v1", "v2", "v N ”, “The first processing segment of E2, p(v1|B1B2), p(v2|B2v1), p(v N |v N-2 v N-1 )、p(E2|v M E1) is the participle "v1", "v2", "v N ”, “E2” corresponding conditional probability. For example, “B1B2v1” is the first processing segment corresponding to the word “v1”, and p(v1|B1B2) is the conditional probability corresponding to the word “v1”. As can be seen from the formula, the first probability of the candidate text to be processed is the product of the conditional probabilities corresponding to each word.

[0142] Specifically, the conditional probability corresponding to each word segment is determined by the number of plays of the playlist related to the first processing segment in which the word segment is located. The following is an example of the conditional probability p(v1|B1B2) corresponding to "v1": For example, the conditional probability can be determined by the number of plays of the playlist related to "B1B2v1", as shown in the following formula:

[0143]

[0144] Among them, count(B1B2) is the number of plays of the playlist related to "B1B2", and count(B1B2v1) is the number of plays of the playlist related to "B1B2v1". p(v1|B1B2) is the ratio of the number of plays of the playlist related to "B1B2" to the number of plays of the playlist related to "B1B2v1". This application uses the ratio of the number of plays of the playlists to express the conditional probability of "B1B2v1" appearing under the premise of "B1B2". The method for determining count(B1B2) is the same as the method for determining count(B1B2v1).

[0145] In one possible implementation, the playback volume of the playlist associated with the first processing segment is the playback volume of the playlist in the playlist library whose playlist title is the same as the first processing segment. In other words, the playback volume of the playlist whose playlist title "completely matches" the first processing segment is the playback volume of the playlist associated with the first processing segment. For example, if the first processing segment is "B1B2v1", "complete match" means that the playlist title is exactly the same as "B1B2v1". In specific implementation, it is necessary to determine the playback volume of the playlist whose playlist title is exactly the same as "B1B2v1" from the playlist library, and the playback volume is the value of count(B1B2v1). The larger the value of count(B1B2v1), the more common the playlist with "B1B2v1" as the playlist title is, and the more it is listened to by users. count(B1B2) can be calculated in the same way.

[0146] In another possible implementation, the playback volume of the playlist related to the first processing segment is the playback volume of the playlist in the playlist library whose playlist title contains the first processing segment. In other words, the playback volume of the playlist whose playlist title "partially matches" the first processing segment is the playback volume of the playlist related to the first processing segment. In this application, "partial match" means that the playlist title contains the first processing segment, but is not exactly the same as the first processing segment. For example, when the first processing segment is "Classic", the playlist titles "Classic Old Songs", "Chinese Classics", "Singing Classic Old Songs", and "Classics" are all playlists that meet the "partial match" condition. Compared with the "complete match" method, this method increases the number of matching situations and increases the diversity of matching situations.

[0147] ② Second probability

[0148] In one possible implementation, the method for determining the second probability specifically includes: determining the second probability of each candidate text to be processed based on the replacement probability corresponding to the Nth word segment in each candidate text to be processed; the replacement probability corresponding to the Nth word segment is determined by the playback volume of the playlist related to the second processing segment where the Nth word segment is located, and the second processing segment is related to the context of the Nth word segment.

[0149] The second probability is related to the Nth word segment undergoing synonym rewriting. For example, the second probability of any candidate text to be processed can be obtained by the following formula:

[0150] P2=p(w N →s N |W L ,W R )

[0151] Among them, w N is the Nth participle, s N w N The result after synonym rewriting. In one case, s N Can be w N The corresponding synonym set S N In another case, s N Can be w N (This is the case where it has not been rewritten), W L Indicates w N The set of context-related segmentations in the original search text, W R Indicates w N The set of related word segments in the original search text. p(w N →s N |W L ,W R ) indicates that w is considered N Under the premise of the context, w N Replaced by s N probability.

[0152] To reduce the workload of P2 calculation, we can L and W R Select w N The nearest participle w L and w R Participate in p(w N →s N |W L ,W R ) calculation. Furthermore, this application can consider the above w N Replaced by s NThe probability of w N Replaced by s N The probability of w N Replaced by s N The probability of the three is the sum of the probabilities p(w N →s N |W L ,W R ). For example, P2 can be obtained by the following expression:

[0153] P2=0.5*rp(w L w N w R →w L s N w R )

[0154] +0.25*rp(w L w N →w L s N )

[0155] +0.25*rp(w N w R →s N w R )

[0156] Among them, rp(w L w N w R →w L s N w R )、rp(w L w N →w L s N )、rp(w N w R →s N w R ) respectively indicate the consideration of the preceding, following, and context w N Replaced by s N The probability of (that is, the replacement probability mentioned in this application). The sum of the products of these three replacement probabilities and the corresponding weights is the value of the second probability. For example, in the above expression, 0.5, 0.25 and 0.25 are the weights corresponding to the three replacement probabilities respectively. The weight distribution can also be other values, which is not limited in this application. rp(w L w N w R →w L s N w R )、rp(w L wN →w N s N ) and rp(w N w R →s N w R ) can be obtained by referring to the following expression:

[0157]

[0158]

[0159]

[0160] Among them, w L s N w R 、w L s N 、s N w R For the second processing segment, count(w L s N w R )、count(w L s N )、count(s N w R ) by w L s N w R 、w L s N 、s N w R The number of plays of the relevant playlist is determined. L w N w R →w L s N w R ) as an example, for rp(w L w N w R →w L s N w R ) is explained: Specifically, rp(w L w N w R →w L s N w R ) means that considering the context, "w L s N w R "The frequency of occurrence (count(w L s N w R)) accounts for "w L w N w R ” and all “w L iw R ”(i∈S N ) appears For example, if w N is B, the preceding conjunction and the following conjunction are A and C respectively, and w N The synonym set S N Including B1, B2, B3, N is B1, then rp(w L w N w R →w L s N w R ) can be expressed as the probability that the frequency of "AB1C" accounts for the sum of the frequencies of "ABC," "AB1C," "AB2C," and "AB3C." Based on this, this application determines the probability of selecting a word from all synonyms of the Nth participle (including the Nth participle itself) while considering the context of the Nth participle.

[0161] Among them, count(w L s N w R ) and “w L s N w R " is related to the number of plays of the relevant playlist. In one possible implementation, count(w L s N w R ) is equal to the value of the song list title determined in the song list library and "w L s N w R ” The number of plays of the same playlist. In another embodiment, count(w L s N w R ) is equal to the song list library to determine the song list title includes "w L s N w R ” playlist playback volume (for example, “w L s N w R XXX”). It should be noted that the above rp(w L w N w R →w L s N w R )、rp(w Lw N →w N s N ) and rp(w N w R →s N w R ) is calculated in the same way as count(w L s N w R ) are the same and will not be described here.

[0162] Method 2:

[0163] In another possible implementation, the method of determining the rewriting probability specifically includes: taking the product of the conditional probabilities corresponding to each participle in each candidate text to be processed as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the playback volume of the playlist related to the first processing segment in which any participle is located, and the first processing segment includes at least two adjacent participles and any participle is the last participle of at least two adjacent participles.

[0164] Among them, the method for determining the first probability is the same as the method for determining the first probability in the above-mentioned method 1, which will not be repeated here. The historical click-through rate is the click-through rate of the song list related to the candidate text to be processed by multiple users within a preset historical time period. The "song list related to the candidate text to be processed" here refers to the song list obtained by searching the candidate text to be processed. When the historical click-through rate is higher, it means that the song list related to the candidate text to be processed is more common and popular, and the quality of the candidate text to be processed is higher. In this case, the rewriting probability P 改写 =P1*P 历史点击率 .

[0165] S5. Select a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set.

[0166] According to step S4, the rewriting probability P of each candidate text to be processed in the candidate text set to be processed can be obtained. 改写 , according to the rewriting probability P of each candidate text to be processed 改写 As well as other joint screening methods, the candidate texts to be processed are screened, and the screened candidate texts to be processed are used as the Nth candidate text set. This process can be regarded as pruning the candidate text set to be processed, thereby limiting the size of the candidate text set to be processed to prevent the module from running too long due to too many candidate texts to be processed. The Nth candidate text set can be used to rewrite the N+1th word to obtain the N+1th candidate text set, and so on. The method for determining the N+1th candidate text set is the same as the method for determining the Nth candidate text set.

[0167] In one possible implementation, step S5 specifically includes: deleting the first candidate text to be processed and the second candidate text to be processed from the set of candidate texts to be processed to obtain a pre-selected set; among the first N-1 word segments of the first candidate text to be processed, there is a word that is the same as the Nth word segment or a synonym of the Nth word segment; the second candidate text to be processed is the candidate text to be processed whose rewriting probability is not the highest among multiple candidate texts to be processed with the same word segmentation and different combination orders; and selecting a preset number of candidate texts to be processed from the pre-selected set based on the rewriting probability to form the Nth candidate text set.

[0168] The following is an example to illustrate this possible implementation: Generally speaking, when a user uses a search text to search for a playlist, he or she may use multiple semantic words with similar semantics. For example, the original search text may be "sleep sleep". When the first word "sleep" is rewritten as "help sleep" to obtain "help sleep sleep" according to this solution, the second word "sleep" is also rewritten as "help sleep" to obtain "help sleep help sleep". The text semantic diversity of "help sleep help sleep" is not as good as "sleep sleep" or "help sleep sleep", so this application eliminates this situation (that is, the first candidate text to be processed mentioned above).

[0169] In addition, there may be multiple candidate texts in the candidate text set that have the same word segmentation but different combination orders. For example, "ABC," "ACB," and "BCA" have rewriting probabilities of "0.4," "0.5," and "0.6," respectively. In this case, this application only retains the candidate text with the highest rewriting probability (such as "BCA") and deletes the remaining candidate texts (i.e., the second candidate text) (such as "ABC" and "ACB").

[0170] After eliminating the above two situations and obtaining the preselected set, the present application may further select a preset number of candidate texts to be processed from the preselected set to form an Nth candidate text set. Optionally, if the preset number is K, the first K candidate texts to be processed with the highest rewriting probability may be selected from the second preselected set to form the Nth candidate text set, and the Nth candidate text set may be one or more of the I*(J+1) combinations included in the candidate text set to be processed.

[0171] S203 : Select candidate texts whose rewriting probabilities exceed a preset rewriting threshold from the Mth candidate text set to form a target text set.

[0172] Specifically, when the number of candidate texts in the Mth candidate text set determined according to the above steps is too large, a preset rewriting threshold can be used to further screen the candidate texts, and the target text set is composed of higher-quality candidate texts in the Mth candidate text set.

[0173] S204: Determine a search input text according to the target text set, and use the search input text to search for a playlist.

[0174] Specifically, after obtaining the target text set, the text processing device can send the target text with the highest probability of rewriting in the target text set to the search device, and the search device will search the playlist based on the target text with the highest probability of rewriting as the search input text; or, the text processing device can send each target text in the target text set to the search device, and the search device will first search for the playlist based on each target text, and then summarize the playlists of all target texts as the final search results. Optionally, if each target text in the target text set is different from the original search text, the search device can also use the original search text when searching the playlist to increase the number of playlists searched.

[0175] based on Figure 2 In the illustrated embodiment, the present application can sequentially rewrite synonyms for the segmented words in the original search text, and after rewriting, evaluate the rewriting probability (or rewriting quality) based on the associated playlist volume to further filter the rewriting results. This method evaluates the quality of synonym rewriting results and improves the accuracy of synonym rewriting.

[0176] Figure 2 and Figure 3 This paper introduces how to rewrite the synonyms of the original search text and the process of screening according to the rewriting probability. Figure 4 The illustrated embodiment introduces how to obtain the synonym table in the above step S2.

[0177] See also Figure 4 , is a flow chart of a synonym table construction method provided in an embodiment of the present application. This method can be applied to the above-mentioned text processing device, including steps S401 to S406.

[0178] S401. Obtain a playlist title set and a history search set.

[0179] The playlist title collection includes the playlist titles of multiple high-quality playlists (such as those with high click-through rates and high play counts), and the historical search collection includes search records entered by multiple users on the music platform within a preset historical time period. This application uses playlist titles and historical searches together as corpus for constructing a synonym table, which can improve the coverage of relevant terms in the music field and solve the problem of missing or inaccurate data in the music field.

[0180] S402. Perform word segmentation processing on the playlist titles in the playlist title set to obtain a first word segmentation set, and perform word segmentation processing on the historical searches in the historical search set to obtain a second word segmentation set.

[0181] Specifically, when step S402 is implemented, each playlist title in the playlist title set is segmented, and the segmentation results of all playlist titles constitute the first segmentation set. For example, the playlist title set includes 5 playlist titles, and each playlist title can be divided into 3 segmentations, then 5*3=15 segmentations are used as the first segmentation set. Similarly, each historical search in the historical search set is segmented, and the segmentation results of all historical searches constitute the second segmentation set. It should be noted that a segmentation tool can be used for segmentation. Optionally, a series of text preprocessing operations (such as removing punctuation marks, removing invalid characters, etc.) can be performed before segmentation to improve the efficiency of the segmentation process. This application does not impose any restrictions on this.

[0182] S403 : Call the intention recognition model to perform semantic recognition on the first word segmentation set and the second word segmentation set respectively to obtain a first semantic word set and a second semantic word set.

[0183] Since the word segments of playlist titles or historical searches may be semantic words or non-semantic words, this application needs to identify the semantic words through the intent recognition model so that the semantic words can be rewritten as synonyms later.

[0184] In specific implementation, the intent recognition model can adopt a deep learning model or other methods in machine learning to identify the semantics of the word segmentation of the playlist title or historical search, and filter the word segmentation with unclear semantics or no semantics (such as singer name) to obtain semantically obvious words to constitute the first semantic word set and the second semantic word set.

[0185] S404: Call the vector embedding model to search for a first synonym set corresponding to the first semantic word set.

[0186] Specifically, each word in the vector embedding model can be represented by a vector, and the semantics between any two words can be represented by the similarity between the corresponding two vectors, and the similarity between the two vectors can be calculated by the cosine value between the two vectors. When the cosine value is closer to 1, the higher the similarity between the two vectors (for example, when it is equal to 1, the two vectors completely overlap, that is, the two vectors are exactly the same), the more likely the two words are synonyms. Based on this, the first semantic word set can be input into the vector embedding model, and the synonyms corresponding to each word in the first semantic word set can be searched according to the cosine value of the vector to form a first synonym set. The vector embedding model can be a word2vec language model or other language models, and the vector embedding model can be trained using the above-mentioned first word segmentation set as training data.

[0187] S405: Search for a second synonymous word set corresponding to the second semantic word set in a preset synonym mining vocabulary.

[0188] The preset synonym mining vocabulary is an open source synonym mining vocabulary that has stored a large number of correspondences between synonyms. Therefore, the synonyms corresponding to each word in the second semantic word set can be directly matched in the preset synonym mining vocabulary to form the second synonym set.

[0189] S406 , arbitrarily selecting two semantic words from the set including the first synonym set and the second synonym set for semantic determination; if the two semantic words are synonyms, the corresponding relationship between the two semantic words is recorded in a synonym table.

[0190] In a specific implementation, a set comprising a first synonym set and a second synonym set can be input into a binary classification model, and the binary classification model randomly selects two semantic words from the set and judges the semantic similarity between the two semantic words. If the semantic similarity between the two semantic words is greater than a preset threshold, the corresponding relationship between the two semantic words is recorded in a synonym table. The binary classification model can adopt a deep learning model such as a bidirectional encoding model (bidirectional encoder representation from transformers, BERT) based on a machine translation model, and this application does not limit this. And the binary classification model needs to adopt a large amount of manually annotated synonym data pairs as training data during training.

[0191] based on Figure 4 In the embodiment shown, the present application can use a set of playlist titles and a set of historical searches that are strongly related to the music field as the corpus for constructing the synonym table, which better solves the problem of missing or inaccurate synonym data in the music field, thereby making the synonym table more suitable for use in music. Figure 2 The corresponding embodiment can provide a more accurate synonym rewriting capability for search texts in the music platform, thereby improving the recall rate of high-quality playlists.

[0192] See also Figure 5 , is a schematic diagram of the structure of a music review generation device provided in an embodiment of the present application, the device includes a word segmentation unit 501, a rewriting unit 502, a selection unit 503, a determination unit 504, and a construction unit 505. Among them:

[0193] The word segmentation unit 501 is used to perform word segmentation on the original search text to obtain M words, where M is a positive integer;

[0194] a rewriting unit 502 configured to sequentially rewrite the M segmented words into synonyms according to a synonym table to obtain an Mth candidate text set; the synonym table is configured to indicate a correspondence between a semantic word and its synonym; each candidate text in the Mth candidate text set has the same semantics as the original search text;

[0195] A selection unit 503 is configured to select candidate texts having a rewriting probability exceeding a preset rewriting threshold from the Mth candidate text set to form a target text set, wherein the rewriting probability is determined by the number of plays of the playlist associated with the candidate text;

[0196] The determination unit 504 is used to determine the search input text according to the target text set, and the search input text is used to search for a playlist.

[0197] In one possible implementation, the rewriting unit 502, when used to sequentially rewrite the M segmented words into synonyms according to the synonym table to obtain the Mth candidate text set, specifically includes:

[0198] Set N to 1 and obtain the N-1th candidate text set, wherein the N-1th candidate text set includes at least one candidate text, each candidate text in the at least one candidate text has the same semantics as the original search text, and the first N-1 word segments of each candidate text are processed by synonym rewriting, where N is a positive integer and is less than or equal to M;

[0199] Searching for a synonym set corresponding to the Nth participle of each candidate text in a synonym table;

[0200] Rewriting the Nth word in each candidate text into a synonym based on the synonym set to obtain a candidate text set to be processed;

[0201] Determining a rewriting probability of each candidate text to be processed in the set of candidate texts to be processed;

[0202] Selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set;

[0203] If N is not equal to M, perform an addition operation on N, and return to the step of obtaining the N-1th candidate text set.

[0204] In one possible implementation, the rewriting unit 502, when used to determine the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed, specifically includes:

[0205] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0206] Determining a second probability for each candidate text to be processed based on a replacement probability corresponding to an Nth word segment in each candidate text to be processed; the replacement probability corresponding to the Nth word segment is determined by the amount of playlists associated with a second processing segment in which the Nth word segment is located, the second processing segment being context-related to the Nth word segment;

[0207] The product of the first probability and the second probability is used as the rewriting probability.

[0208] In one possible implementation, the playback volume of the playlist associated with the first processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the first processing segment; the playback volume of the playlist associated with the second processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the second processing segment; or

[0209] The number of plays of the playlist related to the first processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the first processing segment; the number of plays of the playlist related to the second processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the second processing segment.

[0210] In one possible implementation, the rewriting unit 502, when used to determine the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed, specifically includes:

[0211] The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles;

[0212] Obtain the historical click rate of the playlist related to each candidate text to be processed;

[0213] The product of the first probability and the historical click rate is used as the rewriting probability.

[0214] In one possible implementation, the selection unit 503, when used to select a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form the Nth candidate text set, specifically includes:

[0215] The first candidate text to be processed and the second candidate text to be processed are deleted from the candidate text set to obtain a pre-selected set; the first N-1 word segments of the first candidate text to be processed contain a word that is the same as the Nth word segment or a synonym of the Nth word segment; the second candidate text to be processed is a candidate text to be processed that does not have the highest rewriting probability among multiple candidate texts to be processed that have the same word segmentation but different combination orders;

[0216] A preset number of candidate texts to be processed are selected from the preselected set based on the rewriting probability to form the Nth candidate text set.

[0217] In one possible implementation, the determining unit 504, when used to determine the search input text according to the target text set, specifically includes:

[0218] The target text with the highest rewriting probability in the target text set or each target text in the target text set is determined as the search input text.

[0219] In one possible implementation, the text processing apparatus further includes a construction unit 505, wherein the construction unit 505 is configured to:

[0220] Obtain the playlist title set and the historical search set;

[0221] Performing word segmentation processing on the playlist titles in the playlist title set to obtain a first word segmentation set, and performing word segmentation processing on the historical searches in the historical search set to obtain a second word segmentation set;

[0222] Invoking an intent recognition model to perform semantic recognition on the first word segmentation set and the second word segmentation set, respectively, to obtain a first semantic word set and a second semantic word set;

[0223] Calling a vector embedding model to search for a first synonym set corresponding to the first semantic word set;

[0224] Searching for a second synonym set corresponding to the second semantic word set in a preset synonym mining vocabulary;

[0225] Two semantic words are randomly selected from the set including the first synonym set and the second synonym set for semantic determination. If the two semantic words are synonyms, the corresponding relationship between the two semantic words is recorded in the synonym table.

[0226] It should be noted that the functions of each unit module of the text processing device in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment and will not be repeated here.

[0227] See also Figure 6 , is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. The terminal device may include: one or more processors 601, memory 602, and transceiver 603. The processor 601, memory 602, and transceiver 603 are connected via a bus 604. The memory 602 is used to store a computer program, which includes program instructions. The processor 601 and transceiver 603 are used to execute the program instructions stored in the memory 602 to perform the following operations:

[0228] Perform word segmentation on the original search text to obtain M word segments, where M is a positive integer;

[0229] Rewriting the M word segments into synonyms in sequence according to a synonym table to obtain an Mth candidate text set; the synonym table is used to indicate the corresponding relationship between semantic words and synonyms of the semantic words; each candidate text in the Mth candidate text set has the same semantics as the original search text;

[0230] Selecting candidate texts with a rewriting probability exceeding a preset rewriting threshold from the Mth candidate text set to form a target text set, wherein the rewriting probability is determined by the number of plays of the playlist associated with the candidate text;

[0231] A search input text is determined based on the target text set, and the search input text is used to search for a playlist.

[0232] It should be understood that in some feasible implementations, the processor 601 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The memory 602 may include a read-only memory and a random access memory, and provide instructions and data to the processor 601. A portion of the memory 602 may also include a non-volatile random access memory. For example, the memory 602 may also store device type information.

[0233] In a specific implementation, the terminal device can execute the above-mentioned functions through its built-in functional modules. Figure 2 、 Figure 3 or Figure 4For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0234] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions executed by the aforementioned text processing device, and the computer-readable instructions include program instructions. When the processor executes the program instructions, the above-mentioned Figure 2 、 Figure 3 or Figure 4 The method in the corresponding embodiment will therefore not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network. Multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain system.

[0235] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the above-mentioned Figure 2 、 Figure 3 or Figure 4 The method in the corresponding embodiment will therefore not be described again here.

[0236] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0237] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A text processing method, characterized in that: The method comprises: Perform word segmentation on the original search text to obtain M word segments, where M is a positive integer; Rewriting the M word segments into synonyms in sequence according to a synonym table to obtain an Mth candidate text set; the synonym table is used to indicate the corresponding relationship between semantic words and synonyms of the semantic words; each candidate text in the Mth candidate text set has the same semantics as the original search text; Selecting candidate texts with a rewriting probability exceeding a preset rewriting threshold from the Mth candidate text set to form a target text set, wherein the rewriting probability is determined by the number of plays of the playlist associated with the candidate text; Determine a search input text according to the target text set, wherein the search input text is used to search for a playlist; The step of rewriting the M word segments into synonyms in sequence according to the synonym table to obtain the Mth candidate text set includes: Set N to 1 and obtain an N-1th candidate text set, wherein the N-1th candidate text set includes at least one candidate text, each candidate text in the at least one candidate text has the same semantics as the original search text, and the first N-1 word segments of each candidate text are processed by synonym rewriting; Searching for a synonym set corresponding to the Nth participle of each candidate text in a synonym table; Rewriting the Nth word in each candidate text into a synonym based on the synonym set to obtain a candidate text set to be processed; Determining a rewriting probability of each candidate text to be processed in the set of candidate texts to be processed; Selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set; If N is not equal to M, perform an addition operation on N, and return to the step of obtaining the N-1th candidate text set.

2. The method according to claim 1, characterized in that Determining the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed includes: The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles; Determining a second probability for each candidate text to be processed based on a replacement probability corresponding to an Nth word segment in each candidate text to be processed; the replacement probability corresponding to the Nth word segment is determined by the amount of playlists associated with a second processing segment in which the Nth word segment is located, the second processing segment being context-related to the Nth word segment; The product of the first probability and the second probability is used as the rewriting probability.

3. The method according to claim 2, characterized in that The playback volume of the playlist related to the first processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the first processing segment; the playback volume of the playlist related to the second processing segment is the playback volume of the playlist in the playlist library with the same playlist title as the second processing segment; or The number of plays of the playlist related to the first processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the first processing segment; the number of plays of the playlist related to the second processing segment is the number of plays of the playlist in the playlist library whose playlist title includes the second processing segment.

4. The method according to claim 1, wherein Determining the rewriting probability of each candidate text to be processed in the set of candidate texts to be processed includes: The product of the conditional probabilities corresponding to the respective participles in each candidate text to be processed is used as the first probability of each candidate text to be processed; the conditional probability corresponding to any participle is determined by the number of plays of the playlist associated with the first processing segment in which the any participle is located, the first processing segment including at least two adjacent participles and the any participle being the last participle of the at least two adjacent participles; Obtain the historical click rate of the playlist related to each candidate text to be processed; The product of the first probability and the historical click rate is used as the rewriting probability.

5. The method according to claim 1, wherein The step of selecting a preset number of candidate texts to be processed from the set of candidate texts to be processed based on the rewriting probability of each candidate text to be processed to form an Nth candidate text set includes: Delete the first candidate text to be processed and the second candidate text to be processed from the candidate text set to be processed to obtain a pre-selected set; among the first N-1 participles of the first candidate text to be processed, there is a participle that is the same as the Nth participle or a synonym of the Nth participle; and the second candidate text to be processed is a candidate text to be processed that has the same participles but different combination orders and does not have the highest rewriting probability among the multiple candidate texts to be processed; A preset number of candidate texts to be processed are selected from the preselected set based on the rewriting probability to form the Nth candidate text set.

6. The method according to any one of claims 1 to 5, characterized in that The step of determining the search input text according to the target text set includes: The target text with the highest rewriting probability in the target text set or each target text in the target text set is determined as the search input text.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Obtain the playlist title set and history search set; Performing word segmentation processing on the playlist titles in the playlist title set to obtain a first word segmentation set, and performing word segmentation processing on the historical searches in the historical search set to obtain a second word segmentation set; Invoking an intent recognition model to perform semantic recognition on the first word segmentation set and the second word segmentation set, respectively, to obtain a first semantic word set and a second semantic word set; Calling a vector embedding model to search for a first synonym set corresponding to the first semantic word set; Searching for a second synonym set corresponding to the second semantic word set in a preset synonym mining vocabulary; Two semantic words are selected from the set including the first synonym set and the second synonym set for semantic determination. If the two semantic words are synonyms, the corresponding relationship between the two semantic words is recorded in the synonym table.

8. A terminal device, characterized in that: The terminal device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to call the computer program from the memory, so that the terminal device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on the terminal device, the terminal device executes the method according to any one of claims 1 to 7.

10. A computer program product comprising codes or instructions, characterized in that When the codes or instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 7.

Images