Key word library construction method and device, computer equipment, medium and product

Abstract

The application provides a keyword library construction method and device, computer equipment, medium and product. The method comprises the following steps: acquiring a keyword library of a source language, the keyword library of the source language comprising at least one first keyword; converting the first keyword into a backup keyword of a target language; acquiring a target language corpus related to the backup keyword of the target language, the target language corpus comprising M corpora of the target language, M being a positive integer; performing keyword labeling on the M corpora in the target language corpus according to the keyword library of the source language, to obtain M labeled corpora, each labeled corpus comprising a labeled second keyword; and constructing a keyword library of the target language according to the M labeled corpora, and outputting the keyword library of the target language. According to the application, a more accurate and rich keyword library of the target language can be constructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular to a keyword library construction method, a keyword library construction device, a computer device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the rapid development of Internet technology, people's social interaction scenarios in the Internet are widespread in all aspects of life. Therefore, maintaining the network security of the Internet is an important research topic in Internet application. Taking the online transaction between object A and object B as an example, the online transaction in the Internet is usually filtered based on a keyword library to ensure the security of the transaction between the two parties.

[0003] With the expansion of business scenarios, when the transaction parties come from different language regions, it is necessary to establish one or more keyword libraries in other languages. Therefore, how to construct a more accurate and rich keyword library in the target language is a technical problem to be solved at present. SUMMARY

[0004] The embodiments of the present application provide a keyword library construction method, device, computer device, medium and product, which can construct a more accurate and rich keyword library in the target language.

[0005] In one aspect, the present application provides a keyword library construction method, which comprises:

[0006] obtaining a keyword library in a source language, the keyword library in the source language comprising at least one first keyword, and converting the first keyword into a backup keyword in a target language;

[0007] obtaining a corpus in the target language related to the backup keyword in the target language, the corpus in the target language comprising M corpora in the target language, M being a positive integer;

[0008] performing keyword annotation on the M corpora in the corpus in the target language according to the keyword library in the source language, to obtain M annotated corpora, each annotated corpus comprising an annotated second keyword;

[0009] constructing a keyword library in the target language according to the M annotated corpora, and outputting the keyword library in the target language.

[0010] In one aspect, the present application provides a keyword library construction device, which comprises:

[0011] an obtaining unit configured to obtain a keyword library in a source language, the keyword library in the source language comprising at least one first keyword, and convert the first keyword into a backup keyword in a target language;

[0012] The acquisition unit is further configured to acquire a target language corpus related to the backup keyword in the target language, the target language corpus including M corpora in the target language, M being a positive integer;

[0013] The processing unit is configured to perform keyword labeling on the M corpora in the target language corpus according to the keyword library in the source language, to obtain M labeled corpora, and each labeled corpus includes a labeled second keyword.

[0014] The processing unit is further configured to construct a keyword library in the target language according to the M labeled corpora, and output the keyword library in the target language.

[0015] In a possible implementation, any one of the M corpora is denoted as a target corpus, and a labeled corpus corresponding to the target corpus is denoted as a target labeled corpus.

[0016] The processing unit performs keyword labeling on the target corpus in the target language corpus according to the keyword library in the source language to obtain the target labeled corpus, and is configured to perform the following operations:

[0017] performing word segmentation processing on the target corpus to obtain k second associated words in the target language, k being a positive integer;

[0018] translating each second associated word in the k second associated words into a first associated word in the source language, one second associated word in the target language corresponding to one first associated word in the source language;

[0019] performing keyword labeling on the target corpus in the target language corpus according to the keyword library in the source language and the k first associated words to obtain the target labeled corpus.

[0020] In a possible implementation, the processing unit performs keyword labeling on the target corpus in the target language corpus according to the keyword library in the source language and the k first associated words to obtain the target labeled corpus, and is configured to perform the following operations:

[0021] According to the k first associated words, n to-be-labeled keywords are hit from the keyword library in the source language, n being a positive integer and n≤k;

[0022] According to the n to-be-labeled keywords, the target corpus in the target language corpus is labeled by using a remote supervision algorithm to obtain the target labeled corpus.

[0023] In a possible implementation, the processing unit performs keyword labeling on the target corpus in the target language corpus according to the n to-be-labeled keywords by using a remote supervision algorithm to obtain the target labeled corpus, and is configured to perform the following operations:

[0024] The n second keywords having a mapping relationship with the n to-be-labeled keywords are selected from the k second associated keywords in the target corpus based on a distant supervision algorithm, one second keyword corresponding to one to-be-labeled keyword;

[0025] The n second keywords are labeled to obtain a target labeled corpus;

[0026] The label corresponding to any one of the n second keywords is a keyword label, and the label corresponding to any one of the k-n second associated keywords other than the n second keywords is a non-keyword label.

[0027] In a possible implementation, the processing unit constructs a keyword library of the target language according to the M labeled corpora, for performing the following operations:

[0028] The second keywords in each of the M labeled corpora are obtained;

[0029] The obtained second keywords are added to the keyword library of the target language.

[0030] In a possible implementation, any one of the M corpora is represented as a target corpus, and the labeled corpus corresponding to the target corpus is represented as a target labeled corpus;

[0031] The processing unit constructs a keyword library of the target language according to the M labeled corpora, for performing the following operations:

[0032] The keyword labeling model is trained according to the target labeled corpus;

[0033] The trained keyword labeling model is called to correct the target labeled corpus to obtain a corrected target labeled corpus;

[0034] The keyword library of the target language is constructed based on the corrected target labeled corpus.

[0035] In a possible implementation, the keyword labeling model includes a first network model and a second network model; the first network model is used to obtain a vector representation of a word, and the second network model is used for sequence labeling; the target labeled corpus includes k second associated keywords;

[0036] The processing unit trains the keyword labeling model according to the target labeled corpus, for performing the following operations:

[0037] The first network model is called to identify the k second associated keywords in the target labeled corpus to obtain a word semantic feature of each of the k second associated keywords;

[0038] The processing unit calls the second network model to identify the word semantic feature of each of the k second associated words, and obtains a prediction result of each of the k second associated words.

[0039] The processing unit trains the keyword labeling model according to the target labeled corpus and the prediction result of each of the second associated words.

[0040] In a possible implementation, the processing unit calls the first network model to identify the target labeled corpus, and obtains the word semantic feature of each of the k second associated words, for performing the following operations:

[0041] The processing unit obtains a word vector, a text vector and a position vector corresponding to the i th second associated word in the target labeled corpus, where i is an integer greater than 1 and i≤k;

[0042] The processing unit combines the word vector, the text vector and the position vector corresponding to the i th second associated word to obtain a combined vector of the i th second associated word;

[0043] The processing unit calls the first network model to identify the combined vector of the i th second associated word, and obtains the word semantic feature of the i th second associated word.

[0044] In a possible implementation, the processing unit calls the second network model to identify the word semantic feature of each of the k second associated words, and obtains a prediction result of each of the k second associated words, for performing the following operations:

[0045] The processing unit obtains the prediction result of the i-1 th second associated word;

[0046] The processing unit calls the second network model to identify the word semantic feature of the i th second associated word and the prediction result of the i-1 th second associated word, and obtains the prediction result of the i th second associated word.

[0047] In a possible implementation, the target labeled corpus includes a label of each of the k second associated words, and the label includes a keyword label and a non-keyword label.

[0048] The processing unit trains the keyword labeling model according to the target labeled corpus and the prediction result of each of the second associated words, for performing the following operations:

[0049] The processing unit adjusts a model parameter of the keyword labeling model according to the label of each of the k second associated words and a difference between the prediction result of each of the k second associated words.

[0050] If the adjusted keyword labeling model meets a model convergence condition, the processing unit stops training the keyword labeling model.

[0051] In one possible implementation, the processing unit acquires a target language corpus related to alternative keywords for the target language, and performs the following operations:

[0052] Web crawling is used to search for alternative keywords in the target language to obtain corpus to be parsed related to the alternative keywords in the target language;

[0053] The corpus to be parsed is parsed to obtain the corpus of the target language, and a corpus of the target language is constructed based on the corpus of the target language.

[0054] In one possible implementation, the keyword library for the target language includes multiple keywords; the processing unit is also used to perform the following operations:

[0055] Obtain a transaction request, which carries a text sequence to be identified, including text words in multiple target languages;

[0056] Based on multiple keywords in the keyword library of the target language, each text word in the text sequence to be identified is matched sequentially to obtain the matching result;

[0057] Based on the matching results, perform business processing operations on the transaction request.

[0058] On one hand, embodiments of this application provide a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the aforementioned method for constructing a keyword library.

[0059] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when read and executed by a processor of a computer device, causes the computer device to perform the aforementioned keyword library construction method.

[0060] On one hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned method for constructing a keyword library.

[0061] In this embodiment, a keyword library for the source language is first obtained. This keyword library includes at least one first keyword, which can be converted into alternative keywords for the target language. Then, a target language corpus related to the alternative keywords is obtained. This target language corpus includes M corpora in the target language, where M is a positive integer. Next, based on the keyword library for the source language, the M corpora in the target language corpus are annotated with keywords, resulting in M ​​annotated corpora, each containing an annotated second keyword. Finally, a target language keyword library is constructed based on the M annotated corpora and output. Therefore, the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the alternative keywords, richer text related to the alternative keywords can be incorporated into the target language corpus, resulting in a richer target language keyword library constructed subsequently based on the target language corpus. Furthermore, compared to directly constructing a keyword database for the target language based on the keyword database of the source language, the keyword database for the target language constructed in this application is more accurate. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a schematic diagram illustrating the principle of a keyword library construction scheme provided in an embodiment of this application;

[0064] Figure 2 This is a schematic diagram of the architecture of a keyword library construction system provided in an embodiment of this application;

[0065] Figure 3 This is a flowchart illustrating a method for constructing a keyword library according to an embodiment of this application;

[0066] Figure 4 This is a schematic diagram of a web crawler search interface provided in an embodiment of this application;

[0067] Figure 5 This is a flowchart illustrating a translation target corpus provided in an embodiment of this application;

[0068] Figure 6 This is a flowchart illustrating another method for constructing a keyword library provided in an embodiment of this application;

[0069] Figure 7aThis is a schematic diagram of a process for training a first network model provided in an embodiment of this application;

[0070] Figure 7b This is a schematic diagram of the structure of a first network model provided in an embodiment of this application;

[0071] Figure 7c This is a schematic diagram of a process for obtaining semantic features of words provided in an embodiment of this application;

[0072] Figure 8 This is a flowchart illustrating the processing of a keyword annotation model provided in an embodiment of this application;

[0073] Figure 9 This is a schematic diagram illustrating an application scenario of a keyword library provided in an embodiment of this application;

[0074] Figure 10 This is a schematic diagram of the structure of a keyword library construction device provided in an embodiment of this application;

[0075] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0077] This application proposes a keyword database construction scheme that can be applied to business scenarios such as black market interception. Next, in conjunction with... Figure 1 The principles behind the keyword database construction scheme provided in this application are explained. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram illustrating the principle of a keyword library construction scheme provided in an embodiment of this application. For example... Figure 1 As shown, the general principle of this scheme (including steps 1-6) is as follows:

[0078] Step 1: First, you can obtain the keyword library of the source language. The keyword library of the source language can include at least one primary keyword (where the language corresponding to any primary keyword is the source language). The primary keywords can be converted (for example, by machine translation) into backup keywords for the target language (the keywords of the target language can be added to the backup keyword library of the target language).

[0079] Step 2: Obtain a target language corpus related to the alternative keywords of the target language (e.g., through web crawling). This target language corpus can include M corpora of the target language, where M is a positive integer.

[0080] Step 3: For any corpus in the target language corpus, the corpus can be translated into the source language using machine translation. One possible implementation is to call a machine translation model to perform machine translation on the corpus in the target language corpus.

[0081] Step 4: Based on the keyword database of the source language, the machine-translated source language translation can be annotated using a remote supervision algorithm.

[0082] Step 5: Further, based on the annotated source language, the M corpora in the target language corpus are annotated with keywords to obtain M annotated corpora. Each annotated corpus includes an annotated second keyword (where the language corresponding to the second keyword is the target language).

[0083] Step 6: Finally, a keyword library for the target language can be constructed based on the M labeled corpora and output. This keyword library can then be applied to business scenarios such as black market interception. One possible implementation is as follows: Figure 1 As shown, a keyword library for the target language can be constructed based on M labeled corpora using sequence labeling (e.g., by calling a neural network model with sequence labeling capabilities).

[0084] Therefore, it is evident that the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the alternative keywords for the target language, richer texts related to the alternative keywords can be incorporated into the target language corpus, resulting in a richer target language keyword database subsequently constructed based on the target language corpus. Furthermore, compared to directly constructing a target language keyword database based on the source language keyword database, the target language keyword database constructed in this application is more accurate.

[0085] The keyword database construction scheme proposed in this application can be combined with artificial intelligence technology, cloud technology, and blockchain technology. The following section introduces the keyword database construction scheme provided in this application using relevant technical terminology:

[0086] I. Artificial Intelligence:

[0087] Artificial Intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0088] In one possible implementation, the keyword library construction scheme provided in this application embodiment can be combined with natural language processing (NLP) technology, a branch of artificial intelligence. Specifically, machine translation technology within NLP can be used to convert the first keyword into alternative keywords in the target language. Natural Language Processing (NLP) is an important direction in computer science and artificial intelligence. It studies various theories and methods that enable effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language, i.e., the language people use daily, and thus it is closely related to linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0089] In another possible implementation, the keyword library construction scheme provided in this application embodiment can also be combined with deep learning technology under the field of artificial intelligence. Specifically, a keyword library for the target language can be constructed based on M labeled corpora using deep learning technology. Deep learning (DL) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to enable computers to possess intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0090] II. Cloud Technology:

[0091] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0092] In one possible implementation, when executing the keyword library construction scheme of this application, keywords are labeled on M corpora in the target language corpus according to the keyword library of the source language to obtain M labeled corpora, and the keyword library of the target language is constructed based on the M labeled corpora. These two processes involve large-scale computing and require a large amount of computing power and storage space. Therefore, computer equipment can obtain sufficient computing power and storage space through cloud computing technology to execute the construction of the keyword library of the target language involved in this application.

[0093] III. Blockchain:

[0094] Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product and service layer, and an application service layer.

[0095] The data processing in this application can be combined with blockchain technology. For example, the keyword database of the source language, the keyword database of the target language, and the target language corpus can be uploaded to the blockchain for storage, which can ensure that the data on the blockchain is not easily tampered with.

[0096] It should be noted that in the subsequent specific embodiments of this application, when data related to object information (such as the object's identity) is applied to specific products or technologies, permission or consent from the object is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0097] Please refer to Figure 2 , Figure 2This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application. The system architecture includes a server 240 and a cluster of terminal devices, wherein the cluster of terminal devices may include terminal devices 210, 220, 230, etc. The cluster of terminal devices and the server 240 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0098] Figure 2 The server 240 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0099] Figure 2 The terminal devices 210, 220, and 230 shown can be mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), vehicles, in-vehicle devices, roadside devices, aircraft, wearable devices such as smartwatches, smart bracelets, pedometers, and other smart devices with data processing capabilities.

[0100] In one possible implementation, taking the joint execution of the aforementioned keyword database construction scheme by terminal device 210 and server 240 as an example, the specific operations performed by terminal device 210 and server 240 are explained below. Terminal device 210 can obtain a keyword database for the source language, which includes at least one first keyword. Then, terminal device 210 can send the keyword database to server 240. Server 240 can convert the first keywords into alternative keywords for the target language. Next, server 240 can obtain a target language corpus related to the alternative keywords, which includes M corpora in the target language, where M is a positive integer. Then, server 240 can annotate the M corpora in the target language corpus with keywords based on the keyword database for the source language, obtaining M annotated corpora, each of which includes an annotated second keyword. Finally, server 240 can construct a keyword database for the target language based on the M annotated corpora. Server 240 can send a keyword database of the target language to terminal device 210, and then terminal device 210 can use the keyword database for business applications, such as black market interception.

[0101] In another possible implementation, the above is merely an illustrative description of the specific operations performed by terminal device 210 and server 240. For example, converting the first keyword into alternative keywords in the target language does not necessarily have to be performed by server 240; it can also be performed by terminal device 210 (or any other terminal device in the terminal device cluster). Similarly, converting the first keyword into alternative keywords in the target language, and obtaining a target language corpus related to the alternative keywords in the target language, also does not necessarily have to be performed by server 240; it can also be performed by terminal device 210 (or any other terminal device in the terminal device cluster).

[0102] In another possible implementation, the keyword database construction scheme mentioned above can be executed independently by terminal device 210 (or any other terminal device in the terminal device cluster) or server 240. Specifically, taking server 240 as an example, server 240 obtains a keyword database for the source language, which includes at least one first keyword, and converts the first keyword into a backup keyword for the target language. Then, server 240 obtains a target language corpus related to the backup keywords for the target language, which includes M corpora in the target language, where M is a positive integer. Next, server 240 annotates the M corpora in the target language corpus with keywords based on the keyword database for the source language, obtaining M annotated corpora, each of which includes an annotated second keyword. Finally, server 240 can construct a keyword database for the target language based on the M annotated corpora. Subsequently, server 240 can send the keyword database for the target language to terminal device 210, which can then save the keyword database for the target language. When terminal device 210 receives a transaction request, the request carries a text sequence to be identified, which includes multiple text words in the target language. Then, terminal device 210 can sequentially match each text word in the text sequence to be identified based on multiple keywords from a keyword library in the target language, obtaining a matching result. Finally, terminal device 210 can perform business processing operations on the transaction request based on the matching result.

[0103] In one possible implementation, the data processing system provided in this application embodiment can be deployed on a blockchain node. For example, each terminal device included in the server 240 and the terminal device cluster can be treated as a blockchain node device, jointly forming a blockchain network. Therefore, the data processing flow in this application can be executed on the blockchain, which can ensure the fairness and impartiality of the data processing flow, while also making the data processing flow traceable and improving its security.

[0104] It is understood that the system architecture diagrams described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0105] Based on the above analysis, the following will combine... Figure 3 The method for constructing the keyword library in this application is described in detail. Please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a method for constructing a keyword database according to an embodiment of this application. This method for constructing the keyword database can be executed by a computer device, which can be a terminal device or a server. Figure 3 As shown, the method for constructing this keyword database may include steps S301 to S304. Wherein:

[0106] S301: Obtain the keyword library of the source language, which includes at least one primary keyword, and convert the primary keyword into alternative keywords for the target language.

[0107] In this application, the source language and the target language refer to two different languages. The source language or the target language can be, but is not limited to, any of the following languages: Chinese, English, Korean, Japanese, German, French, Russian, etc. For ease of explanation, subsequent embodiments of this application will use Chinese as the source language and English as the target language as an example. Furthermore, the first keyword mentioned in this application refers to the keyword in the source language; that is, the language corresponding to the first keyword is the source language.

[0108] In one possible implementation, a computer device can invoke a machine translation model to convert each first keyword in the keyword library of the source language into a backup keyword in the target language. The machine translation model can include, but is not limited to, Transformer models, sequence-to-sequence (seq2seq) models, etc. This application embodiment does not specifically limit the model structure of the machine translation model. Each first keyword corresponds to one backup keyword in the target language. For example, if the first keyword is "mahjong," then the backup keyword in the target language obtained by translating that first keyword could be "mahjong."

[0109] In another possible implementation, the computer device may also invoke a machine translation model to convert a portion of the first keywords in the keyword library of the source language into alternative keywords in the target language. In yet another possible implementation, the computer device may also invoke a machine translation model to convert any one of the first keywords in the keyword library of the source language into an alternative keyword in the target language. For ease of explanation, subsequent embodiments of this application will use the example of a computer device converting any one of the first keywords into an alternative keyword in the corresponding target language.

[0110] S302: Obtain a target language corpus related to the alternative keywords for the target language. The target language corpus includes M corpora of the target language, where M is a positive integer.

[0111] In one possible implementation, the process by which a computer device acquires a target language corpus related to alternative keywords in the target language may include: first, performing a web crawler search for the alternative keywords in the target language to obtain corpus to be parsed related to the alternative keywords in the target language; then, parsing the corpus to be parsed to obtain the target language corpus, and constructing a target language corpus based on the target language corpus.

[0112] Specifically, web crawler search tools may include, but are not limited to, any one or more of the following: HttpClient (a client that supports the HTTP protocol), Jsoup (a Java HTML parser), HtmlUnit (an open-source Java page analysis tool), Watij (a web application testing tool developed in Java), Selenium (a tool for testing web applications), Webspec (an open-source Java browser with a user interface), and Google (Google Chrome).

[0113] For example, to demonstrate web scraping using Google Chrome, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of a web crawler search interface provided in an embodiment of this application. For example... Figure 4As shown, if you enter the alternative keyword "Mahjong" in the target language into the web crawler search interface, a Google web crawler search will retrieve corpora related to the alternative keyword. These corpora may or may not include the alternative keyword. Specifically, the corpora can be academic papers, literature, journal articles, etc., related to the alternative keyword; alternatively, they can be explanations or interpretations of the alternative keyword. For example, the corpora could be: "Gambling is one of mankind's oldest activities, evidenced by writings and equipment found in tombs and other places." Another example is: "Mahjong (originated in China) is the most popular tile puzzle game around." Yet another example is: "Mahjong is an entertainment game."

[0114] Furthermore, the searched corpus can be parsed (for example, parsing corpus related to the alternative keywords from papers and journals to obtain corpus for the target language), resulting in corpus for the target language, and a corpus for the target language can be constructed based on this corpus. For example, the corpus for the target language could be: "Mahjong (originated in China) is the most popular tile puzzle game around".

[0115] In another possible implementation, a target language corpus can be obtained by consulting books, papers, and other literature related to alternative keywords for the target language. This application does not limit the method of obtaining the target language corpus. Through the above methods, a target language corpus related to alternative keywords for the target language can be obtained, thus resulting in a rich target language corpus.

[0116] S303: Based on the keyword database of the source language, the M corpora in the target language corpus are labeled with keywords to obtain M labeled corpora, and each labeled corpus includes the labeled second keyword.

[0117] In the embodiments of the present application, the so-called labeled corpus means that each word in the corpus is labeled with a corresponding tag, and the tags include keyword tags and non-keyword tags. Among them, the words labeled with keyword tags in the corpus can also be called second keywords; the words labeled with non-keyword tags can also be called non-keywords in the target language. In addition, any one of the M corpora is denoted as the target corpus, and the labeled corpus corresponding to the target corpus is denoted as the target labeled corpus. Moreover, the second keywords involved in the present application refer to the keywords in the target language, that is, the language corresponding to the second keywords is the target language.

[0118] For the convenience of description, in the subsequent embodiments of the present application, the target corpus is taken as an example for corresponding description, and the specific implementation process for other corpora can refer to the processing process for the target corpus.

[0119] In a possible implementation manner, the process for a computer device to perform keyword labeling on the target corpus in the target language corpus according to the keyword library in the source language to obtain the target labeled corpus may include: First, perform word segmentation on the target corpus to obtain k second associated words in the target language, where k is a positive integer. Then, translate each second associated word in the target language among the k second associated words into a first associated word in the source language, and one second associated word in the target language corresponds to one first associated word in the source language. Finally, perform keyword labeling on the target corpus in the target language corpus according to the keyword library in the source language and the k first associated words to obtain the target labeled corpus.

[0120] Specifically, a machine translation model can be used to translate each second associated word in the target language among the k second associated words into a first associated word in the source language. It should be noted that when performing translation, the translation is performed word by word, that is, factors such as word order in the target corpus are not considered. For example, please refer to Figure 5 , Figure 5 which is a schematic flow chart of a process for translating the target corpus provided by the embodiments of the present application. As Figure 5 shown, assume the target corpus is: Trading is the wagering something of value. Then, first perform word segmentation on the target corpus to obtain 6 second associated words: "Trading", "is", "the wagering", "something", "of", "value". Then, translate the above 6 second associated words word by word into Chinese (first associated words) as: "Trading" (交易), "is" (是), "the wagering" (下注), "something" (东西), "of" (有), "value" (价值).

[0121] For example, suppose the target corpus is: "Trading is the wagering something of value". The mapping relationship between the corresponding second and first conjunctions is shown in Table 1.1 below:

[0122] Table 1.1

[0123]

[0124] For example, suppose the target corpus is: "Mahjong (originated in China) is the most popular tile puzzle game around". The mapping relationship between the corresponding second and first conjunctions is shown in Table 1.2 below:

[0125] Table 1.2

[0126]

[0127] In one possible implementation, the process of a computer device tagging target corpus in a target language corpus with keywords based on a keyword library of the source language and k first related words, to obtain the target tagged corpus, may include: First, based on the k first related words, n keywords to be tagged are obtained from the keyword library of the source language, where n is a positive integer and n≤k. Then, based on the n keywords to be tagged, a remote supervision algorithm is used to tag the target corpus in the target language corpus, resulting in the target tagged corpus. The principle of the remote supervision algorithm includes aligning the text with entities in a large-scale knowledge graph and using the existing inter-entity relationships in the knowledge graph to tag the text. In this embodiment, the mapping relationship between the first and second related words can be obtained using the remote supervision algorithm, and subsequently, the second keyword (a word that has a mapping relationship with the first related words) can be determined based on the keywords to be tagged (part or all of the first related words).

[0128] For example, suppose we have k primary keywords: "transaction," "betting," and "valuable." Each primary keyword can be compared with the primary keyword in the source language's keyword library. If they match, the primary keyword has been found in the source language's keyword library, and this keyword is identified as a target keyword. For instance, we can compare "transaction" with the primary keyword in the source language's keyword library. If the source language's keyword library includes "transaction," then "transaction" is identified as a target keyword. Similarly, we can compare "is" with the primary keyword in the source language's keyword library. If the source language's keyword library does not include "is," then this primary keyword ("is") has not been found. This process continues until we have identified n target keywords, let's say "transaction" and "betting."

[0129] In one possible implementation, the computer device uses a remote supervision algorithm to annotate the target language corpus with n keywords to be annotated, obtaining the target annotated corpus. The specific process may include: First, based on the remote supervision algorithm, selecting n second keywords from k second related words in the target corpus that have a mapping relationship with the n keywords to be annotated, with one second keyword corresponding to one keyword to be annotated. Then, annotating the n second keywords yields the target annotated corpus. Here, the tag corresponding to any second keyword is a keyword tag, and the tag corresponding to any of the kn second related words (excluding the n second keywords) is a non-keyword tag.

[0130] For example, suppose the identified n keywords to be tagged are "trading" and "betting". Then, based on a remote supervision algorithm, from the k second-related words in the target corpus, n second keywords that have a mapping relationship with the n keywords to be tagged are selected as "Trading" and "wagering". Then, "Trading" and "wagering" in the target corpus can be labeled as keywords, meaning the tags carried by "Trading" and "wagering" are keyword tags. Further, any one of the kn second-related words in the target corpus, excluding the n second keywords, carries a non-keyword tag. Through this method, each word (k second-related words) in the target corpus carries a corresponding tag, thus generating the labeled target corpus. In one possible implementation, the language of the tags mentioned above can be the target language; for example, the keyword tag could be "kw", and the non-keyword tag could be "f".

[0131] Following the above process, keyword annotation can be performed on each of the M corpora in the target language corpus, resulting in M ​​annotated corpora. The keyword annotation process for any corpus can refer to the specific process described above for the target corpus. In this embodiment, remote supervision is used to annotate the target language corpora based on the source language's keyword database, thereby improving the efficiency and accuracy of the annotation process.

[0132] S304: Construct a keyword database for the target language based on M labeled corpora and output the keyword database for the target language.

[0133] In one possible approach, a computer device constructs a keyword library for a target language based on M labeled corpora. This can include: first, obtaining the second keyword from each of the M labeled corpora; then, adding the obtained second keywords to the keyword library for the target language. For example, assuming the second keywords in labeled corpus 1 are "Trading" and "wagering," then "Trading" and "wagering" can be added to the keyword library for the target language. Similarly, assuming the second keywords in labeled corpus 2 are "Mahjong" and "game," then "Mahjong" and "game" can be added to the keyword library for the target language. And so on, adding the second keyword from each labeled corpus to the keyword library for the target language, thereby constructing the keyword library for the target language. In this way, based on the backup keywords of the target language, a richer corpus can be obtained. Then, based on remote supervision and the keyword library of the source language, the keywords in the target language corpus are back-tracked to construct the keyword library of the target language. Compared with directly translating the first keyword to construct the keyword library of the target language, this method can introduce richer and more comprehensive keywords, thus constructing a richer keyword library.

[0134] In another possible approach, any one of the M corpora is represented as the target corpus, and the corresponding labeled corpus is represented as the target labeled corpus. The computer device constructs a keyword database for the target language based on the M labeled corpora, which may further include: first, training a keyword annotation model based on the target labeled corpus; then, using the trained keyword annotation model to correct the target labeled corpus, obtaining the corrected target labeled corpus; and finally, constructing the keyword database for the target language based on the corrected target labeled corpus. This method, by using labeled corpora to train a keyword annotation model and then using the trained keyword annotation model to correct the target labeled corpus again, can improve the accuracy of the target language keyword database. Furthermore, the trained keyword annotation model can annotate keywords on more target language corpora, thus constructing a more comprehensive and richer target language keyword database.

[0135] In this embodiment, a keyword library for the source language is first obtained. This keyword library includes at least one first keyword, and each first keyword can be converted into a backup keyword for the target language. Then, a target language corpus related to the backup keywords is obtained. This target language corpus includes M corpora in the target language, where M is a positive integer. Next, based on the keyword library for the source language, the M corpora in the target language corpus are annotated with keywords, resulting in M ​​annotated corpora, each containing an annotated second keyword. Finally, a target language keyword library is constructed based on the M annotated corpora and output. Therefore, the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the backup keywords, richer text related to the backup keywords can be incorporated into the target language corpus, resulting in a richer target language keyword library constructed subsequently based on the target language corpus. Furthermore, compared to directly constructing a keyword database for the target language based on the keyword database of the source language, the keyword database for the target language constructed in this application is more accurate.

[0136] Please see Figure 6 , Figure 6 This is a flowchart illustrating another method for constructing a keyword library according to an embodiment of this application. This method for constructing the keyword library can be executed by a computer device, which can be a terminal device or a server. Figure 6 As shown, the method for constructing this keyword database may include steps S601 to S606. Wherein:

[0137] S601: Obtain the keyword library of the source language, which includes at least one primary keyword, and convert the primary keyword into alternative keywords for the target language.

[0138] S602: Obtain a target language corpus related to the alternative keywords for the target language. The target language corpus includes M corpora of the target language, where M is a positive integer.

[0139] S603: Based on the keyword database of the source language, the M corpora in the target language corpus are labeled with keywords to obtain M labeled corpora, and each labeled corpus includes a labeled second keyword.

[0140] It should be noted that the specific procedures executed by the computer device in steps S601-S603 of this embodiment can be found in [reference needed]. Figure 3 The specific processes executed by the computer device in steps S301-S303 of the embodiments will not be repeated here.

[0141] S604: Train a keyword annotation model based on the target labeled corpus.

[0142] In this embodiment of the application, any one of the M corpora is represented as the target corpus, and the labeled corpus corresponding to the target corpus is represented as the target labeled corpus.

[0143] In one possible implementation, the keyword annotation model can include a first network model and a second network model. The first network model is used to obtain the vector representation of words, and the second network model is used for sequence annotation. As mentioned above, the target annotated corpus includes k second related words, where each second keyword carries a corresponding label (keyword label or non-keyword label). The second related word carrying the keyword label can also be called a second keyword.

[0144] Specifically, keyword tagging models can be sequence tagging models, where the input is a sequence and the output is also a sequence. Sequence tagging includes part-of-speech tagging, named entity tagging, semantic role tagging, and so on. In essence, "sequence tagging" means assigning a label from a set of tags to each text in an input sequence (e.g., a text sequence), and these tags are interconnected. For example, part-of-speech tagging assigns a part-of-speech tag to each word; named entity recognition assigns an entity category to each word in the text. In this embodiment, each word in the input corpus can be tagged with either a non-keyword label or a keyword label.

[0145] Specifically, the keyword labeling model can be a sequence labeling model, which can include a first network model and a second network model. The first network model can be an encoding model, and the second network model can be a decoding model. Encoding models can include, but are not limited to: RNN (Recurrent Neural Network) models, LSTM (Long Short-Term Memory) models, GRU (Gated Recurrent Neural Network) models, and BERT (Bidirectional Encoder Representation from Transformers) models. Decoding models can include, but are not limited to: CRF (Conditional Random Field) models.

[0146] Next, the specific process of training a keyword tagging model based on the target labeled corpus will be explained in detail. Specifically, the process of training a keyword tagging model using computer equipment based on the target labeled corpus can include:

[0147] (1) Call the first network model to identify k second related words in the target labeled corpus and obtain the word semantic features of each second related word in the k second related words.

[0148] In one possible implementation, the process by which a computer device calls a first network model to identify the target labeled corpus and obtain the semantic features of each of the k second related words can include: First, obtaining the word vector, text vector, and position vector corresponding to the i-th second related word in the target labeled corpus, where i is an integer greater than 1 and i ≤ k. Then, combining the word vector, text vector, and position vector corresponding to the i-th second related word to obtain the combined vector of the i-th second related word. Finally, calling the first network model to identify the combined vector of the i-th second related word to obtain the semantic features of the i-th second related word.

[0149] For ease of explanation, this application embodiment uses the BERT model as an example where the first network model (encoding model) is BERT, and combines it with... Figures 7a-7c A detailed description of the BERT model processing procedure is provided below:

[0150] Please see Figure 7a , Figure 7a This is a schematic diagram illustrating a process for training a first network model according to an embodiment of this application. Figure 7aAs shown, the BERT model is a pre-trained model, and its model structure is equivalent to the encoder of the Transformer. The word representations (word semantic features) learned by the BERT model will vary depending on the context, that is, the word semantic features corresponding to each word integrate the semantic information of other words in the whole text.

[0151] Specifically, the BERT model needs to jointly complete two tasks during pre-training: ① cloze test, i.e., predicting missing words in the text; ② determining whether the preceding and following sentences form context. For example... Figure 7a As shown, the input of the BERT model consists of three parts: token embedding (the semantic vector of a word, also known as a word vector), segment embedding (used to mark which sentence it belongs to, also known as a text vector), and position embedding (used to numerically label the relative positions of words, also known as position vectors). It can be understood that the input text sequence can contain one or more sentences. For example, suppose the input target language text sequence is the two sentences: "my dog ​​is cute" and "he likes playing". First, the target language text sequence can be segmented into sentences by inserting a [SEP] marker between sentence A (my dog ​​is cute) and sentence B (he likes playing). This [SEP] marker separates the two input sentences; for example, if the input is sentences A and B, the [SEP] marker should be added after sentences A and B. Furthermore, a [CLS] marker can be added at the beginning of the first sentence. Figure 7a As shown, the text sequence obtained after sentence segmentation of the target language text sequence is: [CLS]my dog ​​is cute[SEP]he likes playing[SEP]. Subsequently, the text sequence obtained after sentence segmentation is further processed by word segmentation, and the token embedding corresponding to each word is obtained (e.g., E). my ), segment embedding (e.g., E A ), and position embedding (e.g., E1).

[0152] Next, please see Figure 7b , Figure 7b This is a schematic diagram of the structure of a first network model provided in an embodiment of this application. The main structure of the BERT model consists of a multi-head self-attention mechanism. An attention mechanism means that for each word, it uses other words in the sentence to represent that word. The weights of the other words in representing that word are different. "Multi-head" refers to multiple attention mechanisms, which can capture relationships at different abstract levels from different perspectives. For example...Figure 7b As shown, for example, T1 is determined based on E1, E2, ..., En. Specifically, it can be based on a multi-head self-attention mechanism to determine the expression weight of each word in E2, ..., En to E1, and finally output T1. The above process can be represented by the following formulas (1) and (2):

[0153] MultiHead(Q,K,V)=Concat(Head1;Head2;…;Head n W 0 (1)

[0154]

[0155] In formulas (1) and (2) above, Q, K, and V are all high-dimensional representations of the output of the previous layer. In the self-attention mechanism, Q and K are used to learn the importance representation of other words in the sentence to this word. Then, by multiplying them with V, the representation of other words in the sentence to this word can be obtained.

[0156] Finally, please see Figure 7c , Figure 7c This is a flowchart illustrating a process for obtaining semantic features of words, provided in an embodiment of this application. For ease of description, this application uses a text sequence in the source language as an example. It can be understood that the processing procedures for the text sequence in the target language are similar to those for the text sequence in the source language. Figure 7c As shown, the input text sequence is assumed to be: "I love China". This text sequence is then segmented into words, yielding the words "I", "love", "China", and "country". The word vectors for the word "I" are then extracted. In one possible implementation, a word embedding model can be used to obtain the word vector for each word. This word embedding model can include, but is not limited to, the word2vec model, which specifically includes the FastText model and the CBOW model. Further, the text vector (segment embedding) and position embedding for the word "I" are extracted. Finally, the word vector, text vector, and position embedding for each word are combined into a combined vector. This combined vector is then input into the BERT model for recognition, outputting the semantic features of each word.

[0157] In this way, the lower layers of the BERT model can learn lexical-level representations, and the higher layers can learn semantic-level representations. The word semantic features output by the pre-trained BERT model can overcome complex situations such as polysemy, that is, the semantic features of the same word are different in different contexts, thus improving the accuracy of the acquired word semantic features.

[0158] (2) Call the second network model to identify the semantic features of each of the k second associated words and obtain the prediction result of each second associated word.

[0159] In one possible implementation, the specific process by which the computer device calls the second network model to identify the semantic features of each of the k second related words and obtain the prediction result of each second related word may include: first, obtaining the prediction result of the (i-1)th second related word; then, calling the second network model to identify the semantic features of the ith second related word and the prediction result of the (i-1)th second related word and obtain the prediction result of the ith second related word.

[0160] For ease of explanation, this application embodiment uses a CRF model as an example to describe the processing procedure of the CRF model in detail:

[0161] CRF models can be used for sequence labeling, which includes part-of-speech tagging, named entity tagging, and semantic role tagging. In named entity tagging, experience shows that the output of each term influences the output of the next. For example, in part-of-speech tagging, if the previous output is a verb, the next output is highly unlikely to also be a verb. CRF models can use label transition constraints to ensure that the output of each term depends on its input and the output of the previous term; that is, each output can be derived from f(x,i,y). i ,y i-1 We obtain the result, where x is the input, i is the current time, and y is the input. i y represents the output of the previous layer at the current moment. i-1 This is the output of the previous layer at the previous time step. It can be expressed by the following formula (3):

[0162] Z t =Z t-1 G⊙H(y t |x) (3)

[0163] Among them, Z t Let H(y) be the output at time t, G be the transition matrix, and H(y) be the output at time t t |x) represents the output of the previous layer at time t.

[0164] CRF models can learn the optimal path using the Viterbi algorithm. The Viterbi algorithm is essentially an optimal choice problem in a multi-step, multi-choice model. At each step, it stores the minimum total cost (or maximum value) from all previous steps to the current choice, as well as the choices of previous steps given the current cost. After calculating all steps sequentially, the optimal path is found through backtracking.

[0165] In summary, the keyword tagging model combining the BERT model and the CRF model in this embodiment can effectively perform sequence tagging, thereby improving the accuracy of the prediction results for each second related word in the target tagged corpus. Please refer to... Figure 8 , Figure 8 This is a flowchart illustrating the processing of a keyword tagging model provided in an embodiment of this application. For example... Figure 8 As shown, this keyword tagging model can include the BERT model and the CRF model. Assuming the labeled corpus input to this keyword tagging model is "Mahjong is an entertainment game", then through the keyword model composed of the BERT and CRF models, the prediction result for each of the five second-related words included in the labeled corpus can be obtained. The prediction result indicates whether the second-related word is a keyword.

[0166] like Figure 8 As shown in Table 2, the prediction results for each word in "Mahjong is an entertainmentgame" obtained by calling the keyword model can be seen below:

[0167] Table 2. Prediction results for each second related term

[0168] Second association word Prediction result Mahjong kw is f an f entertainment kw game kw

[0169] As shown in Table 2 above, the prediction result "kw" indicates that the second related word is a keyword, and the prediction result "f" indicates that the second related word is not a keyword.

[0170] (3) Train the keyword labeling model based on the target labeled corpus and the prediction results of each second related word.

[0171] In this embodiment of the application, the labeled corpus includes k second related words, each of which carries a corresponding tag, which includes keyword tags or non-keyword tags.

[0172] Specifically, the process of training the keyword labeling model using computer equipment based on the target labeled corpus and the prediction results of each second related word can include: First, adjusting the model parameters of the keyword labeling model based on the labels of each of the k second related words and the differences between the prediction results of each of the k second related words. If the adjusted keyword labeling model meets the model convergence condition, training of the keyword labeling model is stopped.

[0173] For example, the labels for each of the k second-related words can be shown in Table 3 below:

[0174] Table 3. Tags for each second related term

[0175] Second association word Label Mahjong kw is f an f entertainment f game kw

[0176] Based on Tables 2 and 3, the model parameters of the keyword tagging model are adjusted according to the difference between the label of each second related word and the prediction result of each second related word.

[0177] In one possible implementation, the convergence condition of the keyword tagging model may include, but is not limited to: when the number of training iterations of the keyword tagging model reaches a preset training threshold, such as 100, the keyword tagging model can be considered to have met the convergence condition. Alternatively, when the difference between the predicted result of the second associated word and the label of the second associated word is less than an error threshold, the keyword tagging model can be considered to have met the convergence condition. Or, when the change between the predicted results of the second associated word obtained from two consecutive training iterations of the keyword tagging model is less than a change threshold, the keyword tagging model can be considered to have met the convergence condition.

[0178] S605: Call the trained keyword annotation model to correct the target annotated corpus and obtain the corrected target annotated corpus.

[0179] It is understandable that the trained keyword tagging model can be obtained through the model training process in step S604 above. Then, the computer device can call the trained keyword tagging model to correct the target labeled corpus, and obtain the corrected target labeled corpus.

[0180] Specifically, a trained keyword tagging model can include a word semantic feature model and a sequence labeling model. The process by which a computer device can call the trained keyword tagging model to correct the target labeled corpus can include: First, calling the word semantic feature model to extract features from the target labeled corpus, obtaining the word semantic features of each of the k second-related words. Then, calling the sequence labeling model to identify the word semantic features of each of the k second-related words, obtaining the identification result for each second-related word. Finally, based on the identification results of each of the k second-related words, correcting the label of each of the k second-related words.

[0181] It should be noted that the detailed process of the computer device performing "calling the word semantic feature model to perform feature extraction processing on the target labeled corpus, and obtaining the word semantic features of each of the k second related words" can be referred to in the relevant process of the computer device performing "calling the first network model to identify the k second related words in the target labeled corpus, and obtaining the word semantic features of each of the k second related words," which will not be repeated here in this embodiment. Similarly, the detailed process of the computer device performing "calling the sequence labeling model to identify the word semantic features of each of the k second related words, and obtaining the identification result of each second related word" can be referred to in the relevant process of the computer device performing "calling the second network model to identify the word semantic features of each of the k second related words, and obtaining the prediction result of each second related word," which will also not be repeated here in this embodiment.

[0182] For example, if the corpus is labeled as "Mahjong is an entertainment game", and the labels for each second related word in the corpus are shown in Table 3 above, then after calling the trained keyword labeling model to re-identify "Mahjong is an entertainment game", the recognition results for each second related word in the labeled corpus can be shown in Table 4 below:

[0183] Table 4. Identification results for each second related word

[0184] Figure 9 Figure 9 Figure 9 Figure 10 Figure 10 f Figure 11 f Figure 11 f ​ ​

[0185] Therefore, combining Tables 3 and 4, each second related word in the target labeled corpus is corrected. For example, for "Mahjong," its corresponding label is "kw," and its corresponding recognition result is also "kw," so no correction is required. Similarly, for "is," its corresponding label is "f," and its corresponding recognition result is also "f," so no correction is required. Furthermore, for "entertainment," its corresponding label is "kw," and its corresponding recognition result is "f," so this word needs correction, and its label is changed to the recognition result "f." This process continues for each second related word. It's understood that if the label and recognition result are the same, no correction is required; if the label and recognition result are different, the label is changed to match the recognition result.

[0186] In this way, on the one hand, keywords labeled "kw" have similar meanings and therefore similar word vectors, which can be quickly filtered out by the keyword labeling model. On the other hand, during the labeling of corpora in the target language using the remote supervision algorithm, some noise is often introduced. For example, a word that should be a secondary keyword may not be in the keyword database of the source language after translation and therefore is not labeled as "kw"; or a word that should not be a keyword may be labeled as "kw". When the target language corpus is large enough, these situations can be corrected by the keyword labeling model, resulting in more accurate keyword recognition results. The reason is as follows: words that should be secondary keywords often have similar word vectors to a large number of other keywords labeled "kw", so they can be filtered out during the keyword labeling model processing; while words that should not be keywords but are labeled as "kw" are discarded by the keyword labeling model because their word vectors differ significantly from those of other keywords labeled "kw".

[0187] S606: Construct a keyword library for the target language based on the corrected target-annotated corpus.

[0188] Specifically, the process of a computer device constructing a keyword library for a target language based on the corrected target labeled corpus may include: obtaining second keywords from the corrected target labeled corpus; and adding the obtained second keywords to the keyword library for the target language.

[0189] It is understood that, in the embodiments of this application, the processing procedure for any of the M labeled corpora can refer to the processing procedure of the computer device on the target labeled corpus in steps S603-S606 above, and will not be repeated here.

[0190] In one possible implementation, the computer device constructing a keyword library for the target language based on M labeled corpora may include: obtaining a second keyword from each of the M labeled corpora (wherein, the number of second keywords in each labeled corpus may be the same or different; in addition, the number of second keywords in the labeled corpus may be one, multiple, or zero), and adding the obtained multiple second keywords to the keyword library for the target language, thereby constructing the keyword library for the target language.

[0191] In another possible implementation, the computer device can also invoke a trained keyword tagging model to process M corpora in the target language corpus, obtaining the recognition result for each of the M corpora (wherein the recognition result is used to indicate whether the expected word is a keyword). Then, based on the recognition result of each of the M corpora, a keyword database for the target language is constructed.

[0192] In another possible implementation, the computer device can also invoke the trained keyword tagging model to identify other corpora of the target language. These other corpora refer to corpora not included in the target language corpus constructed in the embodiments of this application. These other corpora can be obtained through online real-time collection, web crawling, or other methods. In this way, the trained keyword tagging model can also identify other corpora of the target language, thereby adding the identified keywords to the target language keyword library, ensuring the diversity of the target language keyword library.

[0193] In the above method, by training a keyword annotation model using multiple labeled corpora, the annotation results can be corrected based on the labeled corpora, making the keyword annotations in the labeled corpora more accurate, and the resulting keyword database for the target language more accurate. Furthermore, the trained keyword annotation model can also identify other unlabeled corpora, thereby obtaining keywords from the unlabeled corpora and constructing a keyword database, making the keyword database for the target language more comprehensive and richer.

[0194] Finally, the constructed keyword library can be applied to keyword recognition scenarios, such as black market interception. In one possible implementation, the target language keyword library includes multiple keywords; the computer device is also used to perform the following operations: First, it acquires a transaction request carrying a text sequence to be recognized, which includes multiple text words in the target language. Then, based on the multiple keywords in the target language keyword library, each text word in the text sequence to be recognized is matched sequentially to obtain a matching result. Finally, based on the matching result, business processing operations are performed on the transaction request.

[0195] Next, taking the black market interception scenario as an example, the application scenarios of the keyword library for the target language in this application embodiment (wherein, the keywords contained in the keyword library for the target language in this scenario can be sensitive words that need to be intercepted in the business scenario) will be explained. Please refer to ​ , ​ This is a schematic diagram illustrating an application scenario of a keyword library provided in an embodiment of this application. For example... ​ As shown, suppose object A initiates a transaction request, which carries a text sequence to be identified. This text sequence includes multiple words in the target language. Object A can send the transaction request to a server. After receiving the request, the server can use a keyword database in the target language to verify it. Specifically, the server can sequentially match each word in the text sequence with multiple keywords from the database to obtain a matching result. If a word in the text sequence matches a keyword in the database, the match is successful; otherwise, the match fails.

[0196] Furthermore, if the matching result is successful, it means that the transaction request contains sensitive words that need to be blocked in the current transaction scenario. The server can then intercept the transaction request, preventing object B from receiving the transaction request from object A, and thus preventing object A and object B from completing the transaction. If the matching result is unsuccessful, it means that the transaction request does not contain sensitive words that need to be blocked in the current transaction scenario. The server can then respond to the transaction request and send it to object B. Object B can then complete the transaction with object A based on the received transaction request. This method ensures the security of transactions between objects and maintains and fosters a healthy network environment.

[0197] In summary, this application's embodiments, through keyword annotation of a large volume of corpus, can quickly and accurately filter out keywords applicable to the target language context. Guided by similar word vectors, it avoids introducing words meaningless to the target language and improves the recall rate of secondary keywords. Furthermore, this solution employs machine translation and remote supervision for sequence annotation, resulting in low reliance on human intervention. Even those completely unfamiliar with the target language can quickly build a keyword library using this method. Subsequently, the constructed keyword library can be applied to various business scenarios (such as black market interception scenarios).

[0198] In this embodiment, a keyword library for the source language is first obtained. This keyword library includes at least one first keyword, and each first keyword can be converted into a backup keyword for the target language. Then, a target language corpus related to the backup keywords is obtained. This target language corpus includes M corpora in the target language, where M is a positive integer. Next, based on the keyword library for the source language, the M corpora in the target language corpus are annotated with keywords, resulting in M ​​annotated corpora, each containing an annotated second keyword. Finally, a target language keyword library is constructed based on the M annotated corpora and output. Therefore, the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the backup keywords, richer text related to the backup keywords can be incorporated into the target language corpus, resulting in a richer target language keyword library constructed subsequently based on the target language corpus. Furthermore, compared to directly constructing a keyword database for the target language based on the keyword database of the source language, the keyword database for the target language constructed in this application is more accurate.

[0199] Please see ​ , ​ This is a schematic diagram of a keyword database construction apparatus provided in an embodiment of this application. The keyword database construction apparatus 1000 can be applied to the computer device described in the foregoing embodiments. The keyword database construction apparatus 1000 can be a computer program (including program code) running on the computer device; for example, the keyword database construction apparatus 1000 is an application software. The network configuration management device can be used to execute corresponding steps in the data processing method provided in the embodiments of this application. The keyword database construction apparatus 1000 may include:

[0200] The acquisition unit 1001 is used to acquire a keyword library of the source language, which includes at least one primary keyword, and convert the primary keyword into a backup keyword of the target language.

[0201] The acquisition unit 1001 is also used to acquire a target language corpus related to the alternative keywords of the target language. The target language corpus includes M corpora of the target language, where M is a positive integer.

[0202] The processing unit 1002 is used to annotate M corpora in the target language corpus with keywords according to the keyword library of the source language, so as to obtain M annotated corpora, and each annotated corpus includes an annotated second keyword;

[0203] The processing unit 1002 is also used to construct a keyword library for the target language based on M labeled corpora and output the keyword library for the target language.

[0204] In one possible implementation, any one of the M corpora is represented as the target corpus, and the labeled corpus corresponding to the target corpus is represented as the target labeled corpus;

[0205] Processing unit 1002 performs keyword annotation on the target corpus in the target language corpus based on the keyword library of the source language, obtaining the target annotated corpus, which is used to perform the following operations:

[0206] The target corpus is segmented to obtain k second related words in the target language, where k is a positive integer;

[0207] Translate each of the k second-link words in the target language into a first-link word in the source language, with one second-link word in the target language corresponding to one first-link word in the source language.

[0208] Based on the keyword database of the source language and k first related words, the target corpus in the target language corpus is labeled with keywords to obtain the labeled target corpus.

[0209] In one possible implementation, the processing unit 1002 performs keyword annotation on the target corpus in the target language corpus based on the keyword library of the source language and k first related words, to obtain the target annotated corpus, which is used to perform the following operations:

[0210] Based on the k first related words, n keywords to be back-targeted are obtained from the keyword database of the source language, where n is a positive integer and n≤k;

[0211] Based on n keywords to be labeled, the target language corpus in the target language corpus is labeled with keywords using a remote supervision algorithm to obtain the labeled target corpus.

[0212] In one possible implementation, the processing unit 1002 uses a remote supervision algorithm to annotate the target corpus in the target language corpus based on n keywords to be annotated, obtaining the annotated target corpus, which is then used to perform the following operations:

[0213] Based on the remote supervision algorithm, n second keywords that have a mapping relationship with n keywords to be re-indexed are selected from k second related words in the target corpus, and one second keyword corresponds to one keyword to be re-indexed;

[0214] Tag n secondary keywords to obtain the target tagged corpus;

[0215] Among them, the tag corresponding to any second keyword is a keyword tag, and the tag corresponding to any of the kn second related words other than the n second keywords is a non-keyword tag.

[0216] In one possible implementation, the processing unit 1002 constructs a keyword library for the target language based on M labeled corpora, for performing the following operations:

[0217] Obtain the second keyword from each of the M labeled corpora;

[0218] Add the obtained second keyword to the keyword library for the target language.

[0219] In one possible implementation, any one of the M corpora is represented as the target corpus, and the labeled corpus corresponding to the target corpus is represented as the target labeled corpus;

[0220] Processing unit 1002 constructs a keyword database for the target language based on M labeled corpora, and performs the following operations:

[0221] Train a keyword annotation model based on the target labeled corpus;

[0222] The trained keyword annotation model is called to correct the target annotated corpus, resulting in the corrected target annotated corpus;

[0223] Based on the corrected target-annotated corpus, a keyword database for the target language is constructed.

[0224] In one possible implementation, the keyword annotation model includes a first network model and a second network model; the first network model is used to obtain the vector representation of words, and the second network model is used for sequence annotation; the target annotated corpus includes k second related words;

[0225] Processing unit 1002 trains a keyword annotation model based on the target annotated corpus, and performs the following operations:

[0226] The first network model is invoked to identify k second-related words in the target labeled corpus, and the semantic features of each second-related word in the k second-related words are obtained;

[0227] The second network model is invoked to identify the semantic features of each of the k second related words, and the prediction result of each second related word is obtained.

[0228] A keyword labeling model is trained based on the target labeled corpus and the prediction results of each second related word.

[0229] In one possible implementation, the processing unit 1002 calls the first network model to identify the target labeled corpus, obtains the word semantic features of each of the k second related words, and performs the following operations:

[0230] Obtain the word vector, text vector, and position vector corresponding to the i-th second related word in the target annotated corpus, where i is an integer greater than 1 and i≤k;

[0231] Combine the word vector, text vector, and position vector corresponding to the i-th second related word to obtain the combined vector of the i-th second related word;

[0232] The first network model is invoked to identify the combined vector of the i-th second associated word, thereby obtaining the semantic features of the i-th second associated word.

[0233] In one possible implementation, the processing unit 1002 calls the second network model to identify the semantic features of each of the k second related words, and obtains the prediction result of each second related word, which is used to perform the following operations:

[0234] Obtain the prediction result of the (i-1)th second related word;

[0235] The second network model is invoked to identify the semantic features of the i-th second associated word and the prediction result of the (i-1)-th second associated word, thereby obtaining the prediction result of the i-th second associated word.

[0236] In one possible implementation, the target labeled corpus includes labels for each of the k second related words, and the labels include keyword labels and non-keyword labels;

[0237] Processing unit 1002 trains a keyword annotation model based on the target annotated corpus and the prediction results of each second related word, for the following operations:

[0238] Based on the labels of each of the k second related words and the differences between the prediction results of each of the k second related words, adjust the model parameters of the keyword labeling model;

[0239] If the adjusted keyword tagging model meets the convergence condition, stop training the keyword tagging model.

[0240] In one possible implementation, the processing unit 1002 acquires a target language corpus related to alternative keywords for the target language, and performs the following operations:

[0241] Web crawling is used to search for alternative keywords in the target language to obtain corpus to be parsed related to the alternative keywords in the target language;

[0242] The corpus to be parsed is parsed to obtain the corpus of the target language, and a corpus of the target language is constructed based on the corpus of the target language.

[0243] In one possible implementation, the keyword library for the target language includes multiple keywords; the processing unit 1002 is also used to perform the following operations:

[0244] Obtain a transaction request, which carries a text sequence to be identified, including text words in multiple target languages;

[0245] Based on multiple keywords in the keyword library of the target language, each text word in the text sequence to be identified is matched sequentially to obtain the matching result;

[0246] Based on the matching results, perform business processing operations on the transaction request.

[0247] In this embodiment, a keyword library for the source language is first obtained. This keyword library includes at least one first keyword, and each first keyword can be converted into a backup keyword for the target language. Then, a target language corpus related to the backup keywords is obtained. This target language corpus includes M corpora in the target language, where M is a positive integer. Next, based on the keyword library for the source language, the M corpora in the target language corpus are annotated with keywords, resulting in M ​​annotated corpora, each containing an annotated second keyword. Finally, a target language keyword library is constructed based on the M annotated corpora and output. Therefore, the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the backup keywords, richer text related to the backup keywords can be incorporated into the target language corpus, resulting in a richer target language keyword library constructed subsequently based on the target language corpus. Furthermore, compared to directly constructing a keyword database for the target language based on the keyword database of the source language, the keyword database for the target language constructed in this application is more accurate.

[0248] Please see ​ , ​ This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device 1100 is used to execute the steps performed by the computer device in the aforementioned method embodiment. The computer device 1100 includes: one or more processors 1110; one or more input devices 1120; one or more output devices 1130; and a memory 1140. The processors 1110, input devices 1120, output devices 1130, and memory 1140 are connected via a bus 1150. The memory 1140 is used to store a computer program, which includes program instructions. The processor 1110 is used to call the program instructions stored in the memory 1140 to perform the following operations:

[0249] Obtain the keyword database of the source language, which includes at least one primary keyword, and convert the primary keyword into alternative keywords for the target language;

[0250] Obtain a target language corpus related to the alternative keywords for the target language. The target language corpus includes M corpora of the target language, where M is a positive integer.

[0251] Based on the keyword database of the source language, keywords are labeled on M corpora in the target language corpus to obtain M labeled corpora. Each labeled corpus includes a labeled second keyword.

[0252] Construct a keyword database for the target language based on M labeled corpora, and output the keyword database for the target language.

[0253] In one possible implementation, any one of the M corpora is represented as the target corpus, and the labeled corpus corresponding to the target corpus is represented as the target labeled corpus;

[0254] The processor 1110 uses the keyword library of the source language to annotate the target corpus in the target language corpus with keywords, and obtains the target annotated corpus, which is used to perform the following operations:

[0255] The target corpus is segmented to obtain k second related words in the target language, where k is a positive integer;

[0256] Translate each of the k second-link words in the target language into a first-link word in the source language, with one second-link word in the target language corresponding to one first-link word in the source language.

[0257] Based on the keyword database of the source language and k first related words, the target corpus in the target language corpus is labeled with keywords to obtain the labeled target corpus.

[0258] In one possible implementation, processor 1110 performs keyword annotation on the target corpus in the target language corpus based on the keyword library of the source language and k first related words, obtaining the target annotated corpus, which is used to perform the following operations:

[0259] Based on the k first related words, n keywords to be back-targeted are obtained from the keyword database of the source language, where n is a positive integer and n≤k;

[0260] Based on n keywords to be labeled, the target language corpus in the target language corpus is labeled with keywords using a remote supervision algorithm to obtain the labeled target corpus.

[0261] In one possible implementation, the processor 1110 uses a remote supervision algorithm to annotate the target corpus in the target language corpus based on n keywords to be annotated, obtaining the target annotated corpus, which is then used to perform the following operations:

[0262] Based on the remote supervision algorithm, n second keywords that have a mapping relationship with n keywords to be re-indexed are selected from k second related words in the target corpus, and one second keyword corresponds to one keyword to be re-indexed;

[0263] Tag n secondary keywords to obtain the target tagged corpus;

[0264] Among them, the tag corresponding to any second keyword is a keyword tag, and the tag corresponding to any of the kn second related words other than the n second keywords is a non-keyword tag.

[0265] In one possible implementation, processor 1110 constructs a keyword library for the target language based on M annotated corpora, for performing the following operations:

[0266] Obtain the second keyword from each of the M labeled corpora;

[0267] Add the obtained second keyword to the keyword library for the target language.

[0268] In one possible implementation, any one of the M corpora is represented as the target corpus, and the labeled corpus corresponding to the target corpus is represented as the target labeled corpus;

[0269] Processor 1110 constructs a keyword library for the target language based on M labeled corpora, and performs the following operations:

[0270] Train a keyword annotation model based on the target labeled corpus;

[0271] The trained keyword annotation model is called to correct the target annotated corpus, resulting in the corrected target annotated corpus;

[0272] Based on the corrected target-annotated corpus, a keyword database for the target language is constructed.

[0273] In one possible implementation, the keyword annotation model includes a first network model and a second network model; the first network model is used to obtain the vector representation of words, and the second network model is used for sequence annotation; the target annotated corpus includes k second related words;

[0274] Processor 1110 trains a keyword annotation model based on the target annotated corpus, which is used to perform the following operations:

[0275] The first network model is invoked to identify k second-related words in the target labeled corpus, and the semantic features of each second-related word in the k second-related words are obtained;

[0276] The second network model is invoked to identify the semantic features of each of the k second related words, and the prediction result of each second related word is obtained.

[0277] A keyword labeling model is trained based on the target labeled corpus and the prediction results of each second related word.

[0278] In one possible implementation, processor 1110 calls a first network model to identify the target labeled corpus, obtains the word semantic features of each of the k second related words, and performs the following operations:

[0279] Obtain the word vector, text vector, and position vector corresponding to the i-th second related word in the target annotated corpus, where i is an integer greater than 1 and i≤k;

[0280] Combine the word vector, text vector, and position vector corresponding to the i-th second related word to obtain the combined vector of the i-th second related word;

[0281] The first network model is invoked to identify the combined vector of the i-th second associated word, thereby obtaining the semantic features of the i-th second associated word.

[0282] In one possible implementation, the processor 1110 calls a second network model to identify the semantic features of each of the k second related words, obtaining a prediction result for each second related word, which is then used to perform the following operations:

[0283] Obtain the prediction result of the (i-1)th second related word;

[0284] The second network model is invoked to identify the semantic features of the i-th second associated word and the prediction result of the (i-1)-th second associated word, thereby obtaining the prediction result of the i-th second associated word.

[0285] In one possible implementation, the target labeled corpus includes labels for each of the k second related words, and the labels include keyword labels and non-keyword labels;

[0286] Processor 1110 trains a keyword annotation model based on the target annotated corpus and the prediction results of each second related word, for the following operations:

[0287] Based on the labels of each of the k second related words and the differences between the prediction results of each of the k second related words, adjust the model parameters of the keyword labeling model;

[0288] If the adjusted keyword tagging model meets the convergence condition, stop training the keyword tagging model.

[0289] In one possible implementation, processor 1110 acquires a target language corpus related to alternative keywords for the target language, and performs the following operations:

[0290] Web crawling is used to search for alternative keywords in the target language to obtain corpus to be parsed related to the alternative keywords in the target language;

[0291] The corpus to be parsed is parsed to obtain the corpus of the target language, and a corpus of the target language is constructed based on the corpus of the target language.

[0292] In one possible implementation, the keyword library for the target language includes multiple keywords; the processor 1110 is also used to perform the following operations:

[0293] Obtain a transaction request, which carries a text sequence to be identified, including text words in multiple target languages;

[0294] Based on multiple keywords in the keyword library of the target language, each text word in the text sequence to be identified is matched sequentially to obtain the matching result;

[0295] Based on the matching results, perform business processing operations on the transaction request.

[0296] In this embodiment, a keyword library for the source language is first obtained. This keyword library includes at least one first keyword, and each first keyword can be converted into a backup keyword for the target language. Then, a target language corpus related to the backup keywords is obtained. This target language corpus includes M corpora in the target language, where M is a positive integer. Next, based on the keyword library for the source language, the M corpora in the target language corpus are annotated with keywords, resulting in M ​​annotated corpora, each containing an annotated second keyword. Finally, a target language keyword library is constructed based on the M annotated corpora and output. Therefore, the target language corpus can be obtained based on the first keyword. Since the target language corpus is related to the backup keywords, richer text related to the backup keywords can be incorporated into the target language corpus, resulting in a richer target language keyword library constructed subsequently based on the target language corpus. Furthermore, compared to directly constructing a keyword database for the target language based on the keyword database of the source language, the keyword database for the target language constructed in this application is more accurate.

[0297] Furthermore, it should be noted that this application embodiment also provides a computer storage medium, which stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the methods in the corresponding embodiments described above. Therefore, it will not be described again here. This computer-readable storage medium is a memory device in the terminal, used to store programs and data. It is understood that the computer-readable storage medium here can include the built-in storage medium in the terminal, or it can include the extended storage medium supported by the terminal. The computer-readable storage medium provides storage space, which stores the processing system of the terminal. In addition, the storage space also stores one or more instructions suitable for being loaded and executed by the processor 1001. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located far away from the aforementioned processor. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed on a single computer device, or executed on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0298] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the preceding embodiments; therefore, further details will not be provided here.

[0299] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0300] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0301] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0302] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. A method for constructing a keyword database, characterized in that, include: Obtain a keyword library for the source language, which includes at least one primary keyword, and convert the primary keyword into alternative keywords for the target language; Obtain a target language corpus related to the alternative keywords of the target language, wherein the target language corpus includes M corpora of the target language, where M is a positive integer; The target corpus in the M corpora is segmented to obtain k second related words in the target language, where k is a positive integer; Each of the k second related words in the target language is translated into a first related word in the source language, with one second related word in the target language corresponding to one first related word in the source language. Based on the k first related words, n keywords to be back-targeted are obtained from the keyword library of the source language, where n is a positive integer and n≤k; Based on the n keywords to be labeled, the target corpus in the target language corpus is labeled with keywords using a remote supervision algorithm to obtain the labeled target corpus. The target labeled corpus includes labeled second keywords; the target labeled corpus means that each word in the corpus is labeled with a corresponding tag, the tags include keyword tags and non-keyword tags; wherein, the words in the corpus labeled with the keyword tags are called second keywords; Based on the target labeled corpus, a keyword labeling model is trained; wherein, the keyword labeling model includes a sequence labeling model, the sequence labeling means that the input is a sequence and the output is also a sequence, the sequence labeling includes at least one of part-of-speech tagging, named entity tagging, and semantic role tagging; The trained keyword annotation model is invoked to correct the target-annotated corpus, resulting in the corrected target-annotated corpus. Based on the revised target labeled corpus, a keyword library for the target language is constructed; Output the keyword database for the target language.

2. The method as described in claim 1, characterized in that, The step involves using a remote supervision algorithm to annotate the target language corpus with the n keywords to be annotated, resulting in the annotated target corpus, including: Based on the remote supervision algorithm, n second keywords that have a mapping relationship with the n keywords to be re-indexed are selected from the k second related words in the target corpus, and one second keyword corresponds to one keyword to be re-indexed; The n second keywords are labeled to obtain the target labeled corpus; Among them, the tag corresponding to any second keyword is a keyword tag, and the tag corresponding to any of the kn second related words other than the n second keywords is a non-keyword tag.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: Obtain the second keyword from each of the M labeled corpora corresponding to the M corpora; Add the obtained second keyword to the keyword library for the target language.

4. The method as described in claim 1, characterized in that, The keyword annotation model includes a first network model and a second network model; the first network model is used to obtain the vector representation of words, and the second network model is used for sequence annotation; the target annotated corpus includes k second related words; The step of training a keyword annotation model based on the target-annotated corpus includes: The first network model is invoked to identify k second related words in the target labeled corpus, and the semantic features of each second related word in the k second related words are obtained; The second network model is invoked to identify the semantic features of each of the k second associated words, and the prediction result of each second associated word is obtained. The keyword annotation model is trained based on the target-annotated corpus and the prediction results of each second related word.

5. The method as described in claim 4, characterized in that, The step of calling the first network model to identify k second related words in the target labeled corpus, and obtaining the word semantic features of each of the k second related words, includes: Obtain the word vector, text vector, and position vector corresponding to the i-th second related word in the target annotated corpus, where i is an integer greater than 1 and i≤k; The word vector, text vector, and position vector corresponding to the i-th second associated word are combined to obtain the combined vector of the i-th second associated word; The first network model is invoked to identify the combined vector of the i-th second associated word, thereby obtaining the semantic features of the i-th second associated word.

6. The method as described in claim 5, characterized in that, The step of calling the second network model to identify the semantic features of each of the k second related words, and obtaining the prediction result for each second related word, includes: Obtain the prediction result of the (i-1)th second related word; The second network model is invoked to identify the semantic features of the i-th second associated word and the prediction result of the (i-1)-th second associated word, thereby obtaining the prediction result of the i-th second associated word.

7. The method as described in claim 4, characterized in that, The target labeled corpus includes a label for each of the k second related words, and the label includes keyword labels and non-keyword labels; The step of training the keyword annotation model based on the target annotated corpus and the prediction results of each second related word includes: The model parameters of the keyword labeling model are adjusted based on the label of each of the k second associated words and the difference between the prediction results of each of the k second associated words. If the adjusted keyword tagging model meets the model convergence condition, training of the keyword tagging model shall be stopped.

8. The method as described in claim 1, characterized in that, The acquisition of a target language corpus related to the alternative keywords for the target language includes: A web crawler is used to search for alternative keywords in the target language to obtain corpus to be parsed related to the alternative keywords in the target language. The corpus to be parsed is parsed to obtain the corpus of the target language, and a corpus of the target language is constructed based on the corpus of the target language.

9. The method as described in claim 1, characterized in that, The keyword database for the target language includes multiple keywords; the method further includes: Obtain a transaction request, wherein the transaction request carries a text sequence to be identified, and the text sequence to be identified includes multiple text words in the target language; Based on multiple keywords in the keyword library of the target language, each text word in the text sequence to be identified is matched sequentially to obtain the matching result; Based on the matching result, perform business processing operations on the transaction request.

10. A device for constructing a keyword library, characterized in that, include: The acquisition unit is used to acquire a keyword library of the source language, wherein the keyword library of the source language includes at least one first keyword, and convert the first keyword into a backup keyword of the target language; The acquisition unit is further configured to acquire a target language corpus related to the backup keywords of the target language, wherein the target language corpus includes M corpora of the target language, where M is a positive integer; The processing unit is used to perform word segmentation on the target corpus in the M corpora to obtain k second related words in the target language, where k is a positive integer; The processing unit is further configured to translate each second associated word in the target language into a first associated word in the source language from the k second associated words, wherein a second associated word in the target language corresponds to a first associated word in the source language; The processing unit is further configured to obtain n keywords to be back-labeled from the keyword library of the source language based on the k first related words, where n is a positive integer and n≤k; The processing unit is further configured to perform keyword annotation on the target corpus in the target language corpus based on the n keywords to be annotated, using a remote supervision algorithm, to obtain the target annotated corpus; the target annotated corpus includes annotated second keywords; the target annotated corpus refers to a corpus in which each word is annotated with a corresponding tag, the tags including keyword tags and non-keyword tags; wherein, the words in the corpus annotated with the keyword tags are called second keywords; The processing unit is further configured to train a keyword annotation model based on the target annotated corpus; wherein the keyword annotation model includes a sequence annotation model, the sequence annotation refers to a sequence as input and a sequence as output, and the sequence annotation includes at least one of part-of-speech tagging, named entity tagging, and semantic role tagging; The processing unit is also used to call the trained keyword annotation model to correct the target annotated corpus and obtain the corrected target annotated corpus. The processing unit is also used to construct a keyword library for the target language based on the corrected target labeled corpus; The processing unit is also used to output a keyword library for the target language.

11. A computer device, characterized in that, include: Storage devices and processors; A memory, wherein one or more computer programs are stored; A processor for loading one or more computer programs to implement the method for constructing the keyword library as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-9 to construct the keyword library.

13. A computer program product, characterized in that, The computational program product includes a computer program adapted to be loaded by a processor and execute the method for constructing a keyword library as described in any one of claims 1-9.

Images