Method and equipment for processing text data

Text features are extracted through the branch model of the lightweight text processing model, combined with the information of the complex text processing model, the problem of low-quality text recognition in massive Internet content is solved, efficient and accurate text classification is achieved, and content quality and model efficiency are improved.

CN118966211APending Publication Date: 2024-11-15TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410668487.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2020-07-09
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

It is difficult for the existing technology to effectively identify and classify the exaggerated, false, pornographic, vulgar and other inferior text data in massive Internet content, resulting in a decline in content quality and poor user experience. It mainly relies on manual review and user reporting, with a low recognition rate and high cost.

Method used

Using a lightweight text processing model, the text data to be classified is obtained and converted into numerical vectors. The order information, association relationship and keyword information are extracted using three branch models (Bi-LSTM, TextCNN, FastText), and the sentence vector is fused into sentence vectors. The identification and classification of text data are trained in combination with the information of complex text processing models to reduce the complexity of the model.

Benefits of technology

It realizes the rapid and accurate identification and classification of text data such as titles, user names, introductions, etc., improves the content quality of the content aggregation platform, reduces the cost of identification and classification, improves the training and inference speed of the model, and is suitable for insufficient computing power. equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966211A_ABST
    Figure CN118966211A_ABST
Patent Text Reader

Abstract

The invention discloses a method and equipment for processing text data, a method and equipment for simplifying a complex text processing model into a lightweight text processing model, and a computer readable storage medium. The method for processing the text data comprises the steps of obtaining to-be-classified text data; converting the to-be-classified text data into a numerical vector; converting the numerical vector into a sentence vector by utilizing a lightweight text processing model; and determining a category label of the text data based on the sentence vector. According to the method, a lightweight text processing model of three branch models is utilized, and text data can be quickly and accurately identified and classified.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the invention patent application with the application date of July 9, 2020, application number 202010655433.6, and invention name “Method and device for processing text data”. Technical Field

[0002] The present disclosure relates to the field of artificial intelligence services, and more specifically to a method, device, and computer-readable storage medium based on text processing. The present disclosure also relates to a method and device for simplifying a complex text processing model into a lightweight text processing model. Background Art

[0003] There is a huge amount of information on the current Internet. Many mobile applications have built-in content aggregators. These content aggregators aggregate the information that has been published on these applications. The content aggregation server corresponding to the content aggregator can push corresponding articles, pictures, long videos, short videos, music and other data sources to users based on user subscription information, user interests, etc.

[0004] At present, in order to attract readers or viewers, some data source publishers (such as public account bloggers, video account bloggers, music creators, etc.) will add some exaggerated, misleading, false, pornographic, vulgar, and policy and regulatory violation titles to the data sources they publish. Some data source publishers even set up some false, deceptive, and misleading user names (nicknames) and profiles to attract readers or viewers.

[0005] If the above content appears in large quantities, it will reduce the content quality and the user experience of the application, and bring negative impacts to content aggregation products. Currently, the recognition and classification of text information such as titles, user names, and profiles are mainly carried out through manual review and user reports, with low recognition rate and high cost. Summary of the invention

[0006] Embodiments of the present disclosure provide a method and device for processing text data, a method and device for simplifying a complex text processing model into a lightweight text processing model, and a computer-readable storage medium.

[0007] An embodiment of the present disclosure provides a method for processing text data, which also includes: obtaining text data to be classified; converting the text data to be classified into a numerical vector; using a lightweight text processing model to convert the numerical vector into a sentence vector; and determining the category label of the text data based on the sentence vector; wherein, using the lightweight text processing model to convert the numerical vector into a sentence vector includes: using the first branch model of the lightweight text processing model to obtain a first clause vector representing the sequential information of the text data from the numerical vector; using the second branch model of the lightweight text processing model to obtain a second clause vector representing the association relationship between each word in the text data from the numerical vector; using the third branch model of the lightweight text processing model to obtain a third clause vector representing the keyword information in the text data from the numerical vector; and fusing the first clause vector, the second clause vector and the third clause vector into a sentence vector.

[0008] An embodiment of the present disclosure provides a method for simplifying a complex text processing model into a lightweight text processing model, comprising: obtaining a complex text processing model trained based on a first training text library, wherein each sample in the first training text library includes text data of the sample; obtaining a second training text library, wherein each sample in the second training text library includes a category label of the sample and a word segmentation sequence of the sample, and the sample size in the second training text library is smaller than the sample size in the first training text library; using the complex text processing model, converting the category label and word segmentation sequence of the sample in the second training text library into a first sample sentence vector; and training the lightweight text processing model based on the category label, word segmentation sequence and first sample sentence vector of each sample in the second training text library, wherein the complexity of the lightweight text processing model is lower than that of the complex text processing model.

[0009] An embodiment of the present disclosure provides a device for processing text data, comprising: a processor; and a memory, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the above method is performed.

[0010] An embodiment of the present disclosure provides a device for simplifying a complex text processing model into a lightweight text processing model, including: a processor; a memory, the memory storing computer instructions, and the computer instructions implement the above method when executed by the processor.

[0011] An embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the above method is implemented.

[0012] According to another aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various aspects or various optional implementations of the above-mentioned various aspects.

[0013] The embodiments of the present disclosure provide a method for processing text data. The method utilizes a lightweight text processing model of three branch models, which can quickly and accurately identify and classify text data such as titles, user names, and profiles, thereby helping content aggregators avoid recommending data sources that are exaggerated, misleading, false, pornographic, vulgar, or violate policies and regulations to users, thereby improving the quality of content provided by the content aggregation platform.

[0014] The method for processing text data provided by the embodiments of the present disclosure also integrates the information in the complex text processing model into the lightweight text processing model, so that the lightweight text processing model can still quickly and accurately identify and classify text data on the basis of low complexity, thereby improving the training speed and reasoning speed of the lightweight text processing model.

[0015] The embodiments of the present disclosure provide a method for simplifying a complex text processing model into a lightweight text processing model, thereby improving the processing efficiency of text data. In industrial applications, the simplified lightweight text processing model is used to process text data, which can achieve accuracy and recall similar to that of processing text data using a complex text processing model, while also greatly improving the efficiency of reasoning and training, and can be more widely used in devices with insufficient computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the following briefly introduces the drawings required for describing the embodiment. The drawings described below are only exemplary embodiments of the present disclosure.

[0017] Figure 1 The diagram is an example diagram showing a scenario of recommending a data source to a user by analyzing text data related to the data source according to an embodiment of the present disclosure.

[0018] Figure 2A is a flowchart illustrating a method for processing text data according to an embodiment of the present disclosure.

[0019] Figure 2B is a schematic diagram illustrating a method for processing text data according to an embodiment of the present disclosure.

[0020] Figure 2C and Figure 2D is a schematic diagram showing a lightweight text processing model according to an embodiment of the present disclosure.

[0021] Figure 3A is a flowchart illustrating a process of training a lightweight text processing model according to an embodiment of the present disclosure.

[0022] Figure 3B is a schematic diagram showing the training of a lightweight text processing model according to an embodiment of the present disclosure.

[0023] Figure 3C is a schematic diagram showing a complex text processing model according to an embodiment of the present disclosure.

[0024] Figure 3D is a schematic diagram illustrating calculation of processing loss according to an embodiment of the present disclosure.

[0025] Figure 4A The invention is a flowchart showing a method for simplifying a complex text processing model into a lightweight text processing model according to an embodiment of the present invention.

[0026] Figure 4B It is a schematic diagram showing a method for simplifying a complex text processing model into a lightweight text processing model according to an embodiment of the present disclosure.

[0027] Figure 5 is a structural diagram showing a device for processing text data according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0029] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance or ranking.

[0030] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0031] The above-mentioned content aggregator may be based on artificial intelligence (AI). Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. For example, for content aggregators based on artificial intelligence, it can classify data sources in a way similar to how humans read text data related to data sources. By studying the design principles and implementation methods of various intelligent machines, artificial intelligence enables content aggregators to have the functions of perceiving text data, reasoning about text data and making decisions.

[0032] The title of a data source, the user name (nickname) and introduction of the publisher, etc. can all be referred to as text data associated with the data source. Specifically, each data source also has different text data. For example, for image and article data sources, the text data associated with the data source may include comments, titles, summaries, authors, author nicknames, etc. For music data sources, the associated text data may include singers, lyricists, composers, album names, music reviews, lyrics, etc. For video data sources, the associated text data may include actors, directors, screenwriters, lines, movie names, scripts, etc.

[0033] The content aggregator that processes the above text data uses natural language processing (NLP) technology. Natural language processing technology is an important direction in the fields of computer science and artificial intelligence. It can realize various theories and methods for effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Natural language processing technology generally includes text processing, semantic understanding, machine translation, robot question answering, knowledge graphs and other technologies. Therefore, based on natural language processing technology, content aggregators can analyze text data associated with data sources, classify text data, and identify low-quality text data (for example, exaggerated, misleading, false, pornographic, vulgar, and text data that violates policies and regulations) to facilitate the operators of content communities to proceed to the next step.

[0034] Natural language processing technology can also be based on machine learning (ML) and deep learning. Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Natural language processing technology uses machine learning to study how computers simulate or implement human language learning behavior, acquire new knowledge or skills by analyzing existing and classified text data, and reorganize existing knowledge structures to continuously improve their performance. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0035] Optionally, each model that can be used for the content aggregator below can be an artificial intelligence model, especially an artificial intelligence-based neural network model. Typically, an artificial intelligence-based neural network model is implemented as an acyclic graph in which neurons are arranged in different layers. Typically, a neural network model includes an input layer and an output layer, which are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating an output in the output layer. The network nodes are fully connected to the nodes in the adjacent layers via edges, and there are no edges between the nodes in each layer. The data received at the nodes of the input layer of the neural network is propagated to the nodes of the output layer via any one of the hidden layers, activation layers, pooling layers, convolutional layers, etc. The input and output of the neural network model can take various forms, and the present disclosure is not limited to this.

[0036] The solutions provided by the embodiments of the present disclosure involve technologies such as artificial intelligence, natural language processing and machine learning, which are specifically explained through the following embodiments.

[0037] Figure 1 1 is an exemplary schematic diagram showing a scenario 100 of recommending a data source to a user by analyzing text data related to the data source according to an embodiment of the present disclosure.

[0038] At present, there are already multiple content aggregation and sharing platforms. The data source publisher can upload the data source to the server of the content aggregation and sharing platform through the network, so as to publish the video on the content aggregation and sharing platform. The network can be an Internet of Things (Internet of Things) based on the Internet and / or telecommunications network, which can be a wired network or a wireless network, for example, it can be a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a cellular data communication network, and other electronic networks that can realize information exchange functions.

[0039] like Figure 1As shown, the server of the content aggregation and sharing platform can receive data sources published by multiple data source publishers. Assume that two data source publishers (the publisher of data source A and the publisher of data source B) upload data source A and data source B to the server respectively. Among them, the text data associated with data source A is: "Professionals teach you how to place orders, with a winning rate of up to 90%!". The text data associated with data source B is: "The provincial meteorological station issued a yellow warning for heavy rain, and most cities and states in the province are within the warning range today!".

[0040] The server can use the method disclosed in the embodiment of the present disclosure to determine that the content of data source A is related to gambling and lottery, which violates policies and regulations. Therefore, the server can classify data source A as lottery, and may restrict the account of the publisher of data source A, and not publish data source A on the content aggregation and sharing platform.

[0041] The server can use the method disclosed in the embodiment of the present disclosure to determine that data source B is classified as a weather forecast, and the server may also identify the publisher of data source B as an institution number. Therefore, if the data source receiver subscribes to weather-related information, the server can push data source B to the data source receiver.

[0042] The method for processing text data provided by the embodiments of the present disclosure also integrates the information in the complex text processing model into the lightweight text processing model, so that the lightweight text processing model can still quickly and accurately identify and classify text data on the basis of low complexity, thereby improving the training speed and reasoning speed of the lightweight text processing model.

[0043] The embodiments of the present disclosure improve the processing efficiency of text data by providing a method for simplifying a complex text processing model into a lightweight text processing model. In industrial applications, the simplified lightweight text processing model is used to process text data, which can achieve accuracy and recall similar to that of processing text data using a complex text processing model, while also greatly improving the efficiency of reasoning and training, and can be more widely used in devices with insufficient computing power.

[0044] Figure 2A is a flowchart illustrating a method 200 for processing text data according to an embodiment of the present disclosure. Figure 2B is a schematic diagram illustrating a method 200 for processing text data according to an embodiment of the present disclosure. Figure 2C and Figure 2D is a schematic diagram showing a lightweight text processing model according to an embodiment of the present disclosure.

[0045] The method 200 for processing text data according to an embodiment of the present disclosure can be applied to any electronic device. It is understood that the electronic device can be different types of hardware devices, such as a personal digital assistant (PDA), an audio / video device, a mobile phone, an MP3 player, a personal computer, a laptop computer, a server, etc. For example, the electronic device can be Figure 1 The server in the data source A, the application terminal of the publisher of the data source B, the application terminal of the data source receiver, etc. In the following, the present disclosure is described by taking the server as an example, and those skilled in the art should understand that the present disclosure is not limited thereto.

[0046] First, in step S201, the server obtains text data to be classified.

[0047] Optionally, the text data to be classified is associated with at least one data source, and the text data to be classified represents the data source in the form of text. The title of the data source, the user name (nickname) and introduction of the publisher, etc. can all be called text data associated with the data source. Specifically, each data source also has different text data. For example, for data sources of pictures and articles, the text data associated with the data source may include comments, titles, abstracts, authors, author nicknames, etc. For music data sources, the associated text data may include singers, lyricists, composers, album names, music reviews, lyrics, etc. For video data sources, the associated text data may include actors, directors, screenwriters, lines, movie names, scripts, etc.

[0048] Next, in step S202, the server converts the text data to be classified into a numerical vector.

[0049] refer to Figure 2B , the server can convert the text data to be classified into a numerical vector through an embedding operation. For example, the server can divide the text data to be classified into multiple words, and then convert these words into word vectors through word embedding, and finally concatenate these word vectors as a numerical vector.

[0050] Optionally, the server may first divide the text data to be classified into multiple segmented words, and the multiple segmented words form a segmented word sequence. For example, assuming that the text data to be classified is "Professionals teach you how to place an order", the segmented word sequence may be {professionals, teach, you, how, to place an order}.

[0051] The server then encodes each word in the word sequence into a numerical value.

[0052] Optionally, the server can use a preset dictionary to convert each word segment into a numerical value. The preset dictionary can be a set in which each element includes a word segment and its corresponding numerical value. The elements in the dictionary are represented in the form of <word segment, numerical value>. Assume that the dictionary is {<professional, 5>, <teaching, 7>, <you, 4>, <how, 1>, <order, 2>}. Each word segment in the word segment sequence can be converted into: 5, 7, 4, 1, 2 in sequence.

[0053] Optionally, the server can also dynamically encode each segmentation in the segmentation sequence. For example, the server can dynamically build a dictionary, and as long as a segmentation is not included in the dictionary, the segmentation is encoded as the number of elements in the dictionary plus one, and added to the dictionary. For example, for the above segmentation sequence: {professional, teach, you, how, place an order}, the server can dynamically build a dictionary {<professional, 1>, <teach, 2>, <you, 3>, <how, 4>, <order, 5>}. Thus, each segmentation in the segmentation sequence can be converted to: 1, 2, 3, 4, 5 in sequence. After dynamically building the dictionary, if a new segmentation sequence {professional, teach, you, how, dress} is obtained, then correspondingly, the dictionary will become {<professional, 1>, <teach, 2>, <you, 3>, <how, 4>, <order, 5>, <dress, 6>}. According to the dictionary, each segmentation in the new segmentation sequence can be converted to: 1, 2, 3, 4, 6 in sequence.

[0054] Next, the server combines the numerical values ​​corresponding to each word in the word segmentation sequence to convert the word segmentation sequence into a numerical vector.

[0055] Next, in step S203, the server uses a lightweight text processing model to convert the numerical vector into a sentence vector.

[0056] Lightweight text processing models can be Figure 2C and Figure 2D See Figure 2C and Figure 2D , the lightweight text processing model can include three branch models: a first branch model, a second branch model, and a third branch model.

[0057] Step S203 may include: using the first branch model of the lightweight text processing model to obtain a first clause vector representing the sequential information of the text data from the numerical vector; using the second branch model of the lightweight text processing model to obtain a second clause vector representing the association relationship between each word in the text data from the numerical vector; using the third branch model of the lightweight text processing model to obtain a third clause vector representing keyword information in the text data from the numerical vector; and fusing the first clause vector, the second clause vector and the third clause vector into a sentence vector.

[0058] Optionally, the first branch model can be a Bi-LSTM (Bi-directional Long Short-Term Memory) model. The Bi-LSTM model is a time recursive neural network that is suitable for processing and predicting sequential information in an ordered sequence. It can effectively solve the problem of long path dependency in traditional recurrent neural networks. The Bi-LSTM model implements bidirectional memory based on the usual long short-term memory model (LSTM only has forward memory, while Bi-LSTM can achieve forward and reverse memory). See Figure 2D ,The Bi-LSTM model fully considers the sequential relationship between context words and ,makes full use of the bidirectional information, so that it can ,fully extract the sequential information of text data and construct the ,first clause vector containing the sequential information of text data.

[0059] Optionally, the second branch model can be a TextCNN (text convolutional neural network model). The TextCNN model is a text processing model based on the convolutional neural network model (CNN). A typical TextCNN model may include a convolution layer, a pooling layer, and a fully connected layer (FC), which can effectively capture pseudo-random features. Therefore, the TextCNN model can effectively extract information representing the association relationship between the various words in the text data from the numerical vector (such as Figure 2D As shown, this information is also called co-occurrence information), and then a second clause vector containing the co-occurrence information of the text data is constructed.

[0060] Optionally, the third branch model is a FastText model. The FastText model is a fast text classification algorithm that speeds up training and testing while maintaining high accuracy. The FastText model takes into account the similarity between words, so that it is conducive to quickly training word vectors. The FastText model usually has a 1-layer neural network, so it has the advantage of fast learning and prediction speed.

[0061] Alternatively, if Figure 2DAs shown, the third branch model can also be an improved FastText model. For example, the third branch model can also be a model that is a mixture of the FastText model and the TextBank model. For example, the third branch model can be a model obtained by connecting the FastText model and the TextBank model in series or in parallel. The TextBank model is a neural network model that can quickly extract the weights of keywords in a sentence, and it uses the similarity between sentences to extract the weights of keywords. The third branch model can also be a mixture of the FastText model and the TextBank model (for example, by hashing, matching the weights of keywords with keywords), which can more accurately capture the keyword information in the text data, and it performs weighted averaging of the word vector obtained by the FastText model and the keyword weight obtained by the TextBank model to construct a third clause vector containing the keyword information of the text data.

[0062] The present disclosure does not limit the method of extracting the weights of keywords. For example, the third branch model can also be a mixture of the FastText model and the N-gram model (bag of words model). The N-gram model extracts the weights of keywords by the order of words. The third branch model performs a weighted average of the word vectors obtained through the FastText model and the keyword weights obtained through the TextBank model to construct a third clause vector containing the keyword information of the text data. For example, the third branch model can also extract weights through TF-IDF (term frequency-inverse document frequency). TF-IDF assigns higher weights to words that appear more frequently in the text data and less frequently in the entire language environment. The third branch model can even combine the weight values ​​obtained by the above three methods to obtain a more accurate third clause vector.

[0063] The three branch models mentioned above may be in a parallel relationship. The numerical vectors are respectively input into the three branch models to obtain three clause vectors, and then the three clause vectors are fused to obtain the sentence vector.

[0064] For example, Figure 2C As shown, each branch model can obtain a weighted clause vector through a fully connected layer. The three clause vectors are combined and then passed through a pooling layer (i.e., the three clause vectors are averaged) to fuse into the final sentence vector. Of course, other methods can also be used to fuse the clause vectors output by the three branch models, and this application does not limit this.

[0065] Finally, in step S204, the server determines the category label of the text data based on the sentence vector.

[0066] For example, the server can use a fully connected layer / classification model based on the sentence vector to calculate the probability that the text data belongs to each category label. For example, the server can calculate the sentence vector of the text data associated with data source A based on the text data associated with data source A, "Professionals teach you how to place orders, with a winning rate of up to 90%!" Then, based on the values ​​of each element in the sentence vector, the probability that its category label is gambling is much greater than other categories, and then the category label of the text data is determined to be gambling. Similarly, if the text data associated with data source B is: "The provincial meteorological observatory issued a yellow warning for heavy rain, and most cities and states in the province are within the warning range today!", the server can also use a similar method to determine that the category label of the text data is the agency number.

[0067] Optionally, method 200 further includes generating recommendation information of the data source based on the category label of the text data to be classified. The server automatically generates the category label of the text data to be classified using method 200, and recommends the data source that complies with policies and regulations and has benign content to users who are interested in the data source of this category. Thus, while increasing the exposure of benign content, it avoids recommending low-quality content to users, thereby improving the quality of content provided by the content aggregation platform.

[0068] The method for processing text data according to an embodiment of the present disclosure utilizes a lightweight text processing model of three branch models, which can quickly and accurately identify and classify text data such as titles, user names, and profiles, thereby helping content aggregators to avoid recommending data sources that are exaggerated, misleading, false, pornographic, vulgar, or violate policies and regulations to users, thereby improving the quality of content provided by the content aggregation platform.

[0069] Figure 3A is a flowchart illustrating a process 300 of training a lightweight text processing model according to an embodiment of the present disclosure. Figure 3B is a schematic diagram showing the training of a lightweight text processing model according to an embodiment of the present disclosure. Figure 3C is a schematic diagram showing a complex text processing model according to an embodiment of the present disclosure. Figure 3D is a schematic diagram illustrating calculation of processing loss according to an embodiment of the present disclosure.

[0070] The above-mentioned lightweight text processing model is trained based on a complex text processing model, wherein the complexity of the lightweight text processing model is lower than that of the complex text processing model.

[0071] See also Figure 3A ,Training a lightweight text processing model may include the following steps.

[0072] In step S301, a complex text processing model trained based on a first training text library is obtained, where each sample in the first training text library includes text data of the sample.

[0073] The complex text processing model may include multiple transformers. For example, the complex text processing model may be Figure 3C The BERT (Bidirectional Encoder Representations from Transformers) model shown in the figure. The BERT model is a semantic encoding model. After training, the model can obtain corresponding semantic information (i.e., sentence vectors) by inputting characters, words, or sentences. The BERT model uses bidirectional conversion for language models. Traditional language models are all unidirectional language models. Through its bidirectional conversion structure, the BERT model can obtain a deeper understanding than a unidirectional language model, that is, it captures more contextual information between characters in a word and between words in a sentence. Compared with traditional language models, the BERT model has a stronger learning ability and better prediction effect.

[0074] The BERT model can be a BERT-BASE model, which includes 12 operation layers, i.e., 12 converters, each of which can perform feature extraction on text data based on the attention mechanism, as well as encode and decode text data. The BERT-BASE model can also include 768 hidden units and 12 attention heads. The number of parameters of the BERT-BASE model is approximately 1.1 billion. In addition, the BERT model can also be a BERT-LARGE model, which includes 24 converters, 1024 hidden units, and 16 attention heads. The number of parameters of the BERT-LARGE model is approximately 3.4 billion. The present disclosure does not limit which BERT model is used.

[0075] If the BERT-BASE model is used, its output will be 12 sub-vectors with a dimension of 768. Then, a pooling layer (e.g., a mean pooling layer) can be used to merge these 12 sub-vectors into a 768-dimensional vector as the sentence vector output by the complex text processing model. If the BERT-LARGE model is used, its output will be 24 sub-vectors with a dimension of 1024. Then, a pooling layer (e.g., a mean pooling layer) can be used to merge these 24 sub-vectors into a 1024-dimensional vector as the sentence vector output by the complex text processing model. The present disclosure does not limit the dimension of the sentence vector finally output.

[0076] The above-mentioned complex text processing model can be pre-trained using the first training text library. Each sample in the first training text library includes the text data of the sample. For example, the first training text library can be corpus information publicly available on the Internet. For example, in the case where the complex text processing model is a BERT model, the first training text library can be Wikipedia, which includes approximately 2.5 billion words and the corresponding explanations of these words. The first training sample library can also include text data obtained from various public / private databases, such as encyclopedias, dictionaries, news, Q&A, etc. The sample size of the first training text library is usually large. Therefore, the complex text processing model trained by the first training text library contains a general understanding of prior knowledge (for example, most words, terms, etc.). Therefore, by using a complex text processing model to process text data (especially short text data), a more accurate sentence vector containing sufficient information can be obtained.

[0077] Since the number of parameters of complex text processing models is large, in industry, if complex text processing models are used directly, the training and reasoning efficiency will be low. In order to improve the processing efficiency of text processing models in industry, it is necessary to compress and simplify the complex text processing model to obtain a lightweight text processing model that can be used in industry. The lightweight text processing model obtained by compression and simplification will have a smaller number of parameters while also incorporating the complex text processing model's understanding of the first training text library, thereby improving the efficiency of training and reasoning. At the same time, the lightweight text processing model and the complex text processing model can obtain approximate sentence vectors when facing the same input, thereby ensuring the accuracy of text data processing.

[0078] In step S302, the information about the first training text library in the complex text processing model is integrated into the lightweight text processing model.

[0079] Optionally, step S302 may also include step S3021, step S3022 and step S3023.

[0080] In step S3021, a second training text library is obtained, each sample in the second training text library includes a category label of the sample and a word segmentation sequence of the sample, and the sample size in the second training text library is smaller than the sample size in the first training text library.

[0081] Several examples of samples in the second training text library are given below. The following examples show labels of some samples in the second training text library and word segmentation sequences of the samples. In the following examples, slashes are used to separate each word in the word segmentation sequence.

[0082]

[0083]

[0084] The samples in the second training text library can be collected from a content aggregation platform. For example, the text data of the data source can be marked by manual review and user reporting. Then, the text data with category labels is segmented and stored in the second training text library. The present disclosure does not limit the expression form of the segmentation sequence and the method of obtaining the samples in the second training text library.

[0085] In step S3022, the complex text processing model is used to convert the word segmentation sequence of the sample in the second training text library into a first sample sentence vector.

[0086] Optionally, before the samples in the second training text library are input into the complex text processing model, the word segmentation sequence can be preprocessed. For example, the word segmentation sequence can be encoded using bert tokenizer (a built-in word segmentation tool in the BERT model). For example, suppose the word segmentation sequence includes three word segmentations {fund, combination, information}. Assign a code to each of these three word segmentations, for example, "fund" can have code 0, "combination" can have code 1, and "information" can have code 2. Thus, the "word segmentation-coding" dictionary corresponding to the word segmentation sequence can be {fund-0, combination-1, information-2}. Those skilled in the art should understand that the dictionary may be the same as the dictionary in step S202, or it may be different, and the present disclosure does not limit this.

[0087] Each word in the word segmentation sequence is replaced based on the above dictionary, and a word segmentation is converted into a numerical value to obtain a numerical sequence corresponding to the word segmentation sequence. The numerical sequence is input into the complex text processing model. In the case where the complex text processing model is a BERT-BASE model, the 12 converters in the BERT-BASE model perform encoding and decoding operations on the numerical sequence to extract the text features of the sample, and annotate it with a category label to form a first sample sentence vector including the text features of the sample.

[0088] In step S3023, a lightweight text processing model is trained based on the category label, word segmentation sequence and first sample sentence vector of each sample in the second training text library.

[0089] See also Figure 3B , training the lightweight text processing model includes: using the lightweight text processing model to convert the word segmentation sequence of each sample in the second training text library into a second sample sentence vector of the sample, and determining the processing loss of the lightweight text processing model based on the first sample sentence vector and the second sample sentence vector.

[0090] The word segmentation sequence corresponding to the sample in the second training text library can be encoded using the method described in step S202 to obtain a numerical vector, and then the numerical vector is converted into a second sample sentence vector through a lightweight text processing model.

[0091] Then, the processing loss of the lightweight text processing model relative to the complex text processing model can be determined by comparing the difference between the first sample sentence vector and the second sample sentence vector.

[0092] For example, a loss function may be used to determine the processing loss of the lightweight text processing model.

[0093] The processing loss can be an L2 loss (square loss function) in the Euclidean space. For example, suppose the i elements in the first sample sentence vector are recorded as T(x i ), the i-th element of the second sample sentence vector is recorded as S(x i ).

[0094] The first sample sentence vector and the second sample sentence vector have the same dimension N. i is less than or equal to N and greater than 0.

[0095] The loss function corresponding to L2 loss (denoted as L2) can be recorded as:

[0096]

[0097] Wherein, j is less than or equal to N and approximately 0.

[0098] The processing loss can also be a kernel loss on the RKHS space (reproducing kernel Hilbert space). The loss function is a kernel function based on the reproducing kernel Hilbert space, and the processing loss is a kernel loss. The calculation of the above L2 loss involves a large number of inner product terms, and the amount of calculation is huge. Therefore, the L2 loss on the Euclidean space can be converted to the kernel loss on the RKHS space for calculation.

[0099] In the process of calculating the kernel loss on the RKHS space, the kernel function can be used to replace the inner product term to simplify the calculation. At the same time, the RKHS space is a high-dimensional space, which can often capture more correlation information between vectors.

[0100] Assume that the kernel function K(m, n) is used to calculate the kernel loss, where m and n represent different parameters. Then the kernel loss (denoted as L kernel ) can be written as:

[0101]

[0102] Since the kernel function K(m, n) is reproducible, positively definite and symmetric, using the kernel technique, formula (2) can be written as:

[0103]

[0104] Among them, φ(T(x i )) represents the conversion of T(x i ) is mapped to the RKHS space, φ(S(x i )) represents the conversion of S(x i ) is mapped to the RKHS space.

[0105] For example, assuming that the kernel function K(m, n) is a Gaussian kernel function, the Gaussian kernel function can be expressed as follows:

[0106]

[0107] Then the above formula (2) can be written as:

[0108]

[0109] It can be seen that the amount of calculation of formula (5) is greatly reduced compared with that of formula (1).

[0110] In addition, the processing loss may also be a cosine loss. The present disclosure does not further limit the processing loss.

[0111] Optionally, the loss function may also respectively calculate the first sample sentence vector and the processing loss components between the three clause vectors obtained by the three branch models of the lightweight text processing model, and add the three processing loss components to obtain the final processing loss.

[0112] That is, the following formula (6) can be used to calculate the total processing loss L of the model: 处理损失 :

[0113] L 处理损失 =L 第一支路模型 +L 第二支路模型 +L 第三支路模型 (6)

[0114] Among them, L 第一支路模型 represents the processing loss between the first sample sentence vector and the first clause vector, L 第二支路模型 represents the processing loss between the first sample sentence vector and the second sentence vector, L 第三支路模型 Represents the processing loss between the first sample sentence vector and the third sentence vector.

[0115] Then, the processing loss can be minimized by updating the parameters in the lightweight text processing model. For example, the parameters in the lightweight text processing model can be updated iteratively, and the processing loss is reduced as much as possible in each iteration. When the above processing loss converges, it can be said that the training of the lightweight text processing model using the second training text library is completed.

[0116] In addition, in practical industrial applications, lightweight text processing models (i.e. Figure 3B For example, more text data and their corresponding category labels are continuously collected from the content aggregation platform, and then the text data is converted into a numerical vector and input into the lightweight text processing model. The predicted category label of the text data is calculated by the sentence vector output by the lightweight text processing model. The predicted category label is compared with the text data and its corresponding category label to adjust the parameters in the lightweight text processing model.

[0117] Therefore, the method for processing text data provided by the embodiments of the present disclosure also integrates the information in the complex text processing model into the lightweight text processing model, so that the lightweight text processing model can still quickly and accurately identify and classify text data on the basis of low complexity, thereby improving the training speed and reasoning speed of the lightweight text processing model.

[0118] Figure 4A 4 is a flowchart showing a method 400 for simplifying a complex text processing model into a lightweight text processing model according to an embodiment of the present disclosure. Figure 4B 4 is a schematic diagram showing a method 400 for simplifying a complex text processing model into a lightweight text processing model according to an embodiment of the present disclosure.

[0119] See also Figure 4A , simplifying a complex text processing model into a lightweight text processing model can include the following steps.

[0120] In step S401, a complex text processing model trained based on a first training text library is obtained, where each sample in the first training text library includes text data of the sample.

[0121] The complex text processing model may include multiple transformers. For example, the complex text processing model may be a BERT (Bidirectional Encoder Representations from Transformers) model. The BERT model is a semantic encoding model. After training, the model can obtain corresponding semantic information (i.e., sentence vectors) by inputting characters, words, or sentences. The BERT model uses bidirectional conversion for language models. Traditional language models are all unidirectional language models. The BERT model can obtain a deeper understanding than a unidirectional language model through its bidirectional conversion structure, that is, it captures more contextual information between characters in words and between words in sentences. Compared with traditional language models, the BERT model has a stronger learning ability and better prediction effect.

[0122] The BERT model can be a BERT-BASE model, which includes 12 operation layers, i.e., 12 converters, each of which can perform feature extraction on text data based on the attention mechanism, as well as encode and decode text data. The BERT-BASE model can also include 768 hidden units and 12 attention heads. The number of parameters of the BERT-BASE model is approximately 1.1 billion. In addition, the BERT model can also be a BERT-LARGE model, which includes 24 converters, 1024 hidden units, and 16 attention heads. The number of parameters of the BERT-LARGE model is approximately 3.4 billion. The present disclosure does not limit which BERT model is used.

[0123] The above-mentioned complex text processing model can be pre-trained using the first training text library. Each sample in the first training text library includes the text data of the sample. For example, the first training text library can be corpus information publicly available on the Internet. For example, in the case where the complex text processing model is a BERT model, the first training text library can be Wikipedia, which includes approximately 2.5 billion words and the corresponding explanations of these words. The first training sample library can also include text data obtained from various public / private databases, such as encyclopedias, dictionaries, news, Q&A, etc. The sample size of the first training text library is usually large. Therefore, the complex text processing model trained by the first training text library contains a general understanding of prior knowledge (for example, most words, terms, etc.). Therefore, by using a complex text processing model to process text data (especially short text data), a more accurate sentence vector containing sufficient information can be obtained.

[0124] Since the number of parameters of complex text processing models is large, in industry, if complex text processing models are used directly, the training and reasoning efficiency will be low. In order to improve the processing efficiency of text processing models in industry, it is necessary to compress and simplify the complex text processing model to obtain a lightweight text processing model that can be used in industry. The lightweight text processing model obtained by compression and simplification will have a smaller number of parameters while also incorporating the complex text processing model's understanding of the first training text library, thereby improving the efficiency of training and reasoning. At the same time, the lightweight text processing model and the complex text processing model can obtain approximate sentence vectors when facing the same input, thereby ensuring the accuracy of text data processing.

[0125] In step S402, a second training text library is obtained, each sample in the second training text library includes a category label of the sample and a word segmentation sequence of the sample, and the sample size in the second training text library is smaller than the sample size in the first training text library.

[0126] The samples in the second training text library can be collected from a content aggregation platform. For example, the text data of the data source can be marked by manual review and user reporting. Then, the text data with category labels is segmented and stored in the second training text library. The present disclosure does not limit the expression form of the segmentation sequence and the method of obtaining the samples in the second training text library.

[0127] In step S403, the complex text processing model is used to convert the word segmentation sequence of the sample in the second training text library into a first sample sentence vector.

[0128] Optionally, before the samples in the second training text library are input into the complex text processing model, the word segmentation sequence can be preprocessed. For example, the word segmentation sequence can be encoded using bert tokenizer (a built-in word segmentation tool in the BERT model). For example, suppose the word segmentation sequence includes three word segmentations {fund, combination, information}. Assign a code to each of these three word segmentations, for example, "fund" can have code 0, "combination" can have code 1, and "information" can have code 2. Thus, the "word segmentation-coding" dictionary corresponding to the word segmentation sequence can be {fund-0, combination-1, information-2}. Those skilled in the art should understand that the dictionary may be the same as the dictionary in step S202, or it may be different, and the present disclosure does not limit this.

[0129] Each word in the word segmentation sequence is replaced based on the above dictionary, and a word segmentation is converted into a numerical value to obtain a numerical sequence corresponding to the word segmentation sequence. The numerical sequence is input into the complex text processing model. In the case where the complex text processing model is a BERT-BASE model, the 12 converters in the BERT-BASE model perform encoding and decoding operations on the numerical sequence to extract the text features of the sample, and annotate it with a category label to form a first sample sentence vector including the text features of the sample.

[0130] In step S404, a lightweight text processing model is trained based on the category label, word segmentation sequence and first sample sentence vector of each sample in the second training text library.

[0131] The lightweight text processing model includes a first branch model for extracting sequential information of a word segmentation sequence, a second branch model for extracting association information between words in the word segmentation sequence, and a third branch model for extracting keyword information in the word segmentation sequence.

[0132] Training the lightweight text processing model includes: using the lightweight text processing model to convert the category label and word segmentation sequence of each sample in the second training text library into a second sample sentence vector of the sample, and determining the processing loss of the lightweight text processing model based on the first sample sentence vector and the second sample sentence vector.

[0133] The word segmentation sequence corresponding to the sample in the second training text library can be encoded using the method described in step S202 to obtain a numerical vector, and then the numerical vector is converted into a second sample sentence vector through a lightweight text processing model.

[0134] For example, using the lightweight text processing model to convert the word segmentation sequence of each sample in the second training text library into the second sample sentence vector of the sample also includes: using the first branch model of the lightweight text processing model to obtain a first sample clause vector representing the sequential information of the text data from the word segmentation sequence; using the second branch model of the lightweight text processing model to obtain a second sample clause vector representing the association relationship between each word in the text data from the word segmentation sequence; using the third branch model of the lightweight text processing model to obtain a third sample clause vector representing keyword information in the text data from the word segmentation sequence; and fusing the first sample clause vector, the second sample clause vector and the third sample clause vector into a second sample sentence vector.

[0135] Then, the processing loss of the lightweight text processing model relative to the complex text processing model can be determined by comparing the difference between the first sample sentence vector and the second sample sentence vector.

[0136] For example, a loss function may be used to determine the processing loss of the lightweight text processing model.

[0137] The processing loss may be an L2 loss (square loss function) on a Euclidean space, a kernel loss on a RKHS space (reproducing kernel Hilbert space), or a cosine loss. The present disclosure does not further limit the processing loss.

[0138] Optionally, the loss function may also respectively calculate the first sample sentence vector and the processing loss components between the three clause vectors obtained by the three branch models of the lightweight text processing model, and add the three processing loss components to obtain the final processing loss.

[0139] Then, the processing loss can be minimized by updating the parameters in the lightweight text processing model. When the processing loss converges, it can be said that the training of the lightweight text processing model is completed using the second training text library.

[0140] In addition, in practical industrial applications, lightweight text processing models can also be trained dynamically. For example, by continuously collecting more text data and its corresponding category labels from the content aggregation platform, the text data with category labels is converted into a numerical vector and input into the lightweight text processing model to obtain a sentence vector. The sentence vector is then normalized to calculate the predicted category label of the text data. The predicted category label is compared with the category label of the text data to adjust the parameters in the lightweight text processing model.

[0141] Therefore, the method for simplifying a complex text processing model into a lightweight text processing model provided by the embodiments of the present disclosure integrates the information in the complex text processing model into the lightweight text processing model, thereby enabling the lightweight text processing model to quickly and accurately identify and classify text data on the basis of low complexity, thereby improving the training speed and reasoning speed of the lightweight text processing model.

[0142] Figure 5 is a structural diagram showing a device 500 for processing text data according to an embodiment of the present disclosure.

[0143] See also Figure 5 , the device 500 may include a processor 501 and a memory 502. The processor 501 and the memory 502 may be connected via a bus 503.

[0144] The processor 501 can perform various actions and processes according to the program stored in the memory 502. Specifically, the processor 501 can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X87 architecture or an ARM architecture.

[0145] The memory 502 has computer instructions stored thereon, which implement the method 200 when executed by the microprocessor. The memory 502 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronously linked dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the method described in the present disclosure is intended to include, but is not limited to, these and any other suitable types of memory.

[0146] The device provided by the embodiment of the present disclosure for simplifying a complex text processing model into a lightweight text processing model also has a structure that is the same as or similar to the device 500 , and therefore, it will not be described in detail in this disclosure.

[0147] An embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the above method is implemented.

[0148] According to another aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various aspects or various optional implementations of the above-mentioned various aspects.

[0149] The embodiments of the present disclosure provide a method for processing text data. The method utilizes a lightweight text processing model of three branch models, which can quickly and accurately identify and classify text data such as titles, user names, and profiles, thereby helping content aggregators avoid recommending data sources that are exaggerated, misleading, false, pornographic, vulgar, or violate policies and regulations to users, thereby improving the quality of content provided by the content aggregation platform.

[0150] The method for processing text data provided by the embodiments of the present disclosure also integrates the information in the complex text processing model into the lightweight text processing model, so that the lightweight text processing model can still quickly and accurately identify and classify text data on the basis of low complexity, thereby improving the training speed and reasoning speed of the lightweight text processing model.

[0151] The embodiments of the present disclosure provide a method for simplifying a complex text processing model into a lightweight text processing model, thereby improving the processing efficiency of text data. In industrial applications, the simplified lightweight text processing model is used to process text data, which can achieve accuracy and recall similar to that of processing text data using a complex text processing model, while also greatly improving the efficiency of reasoning and training, and can be more widely used in devices with insufficient computing power.

[0152] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0154] The exemplary embodiments of the present invention described in detail above are merely illustrative, rather than restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present invention, and such modifications should fall within the scope of the present invention.

Claims

1. A method for simplifying a complex text processing model into a lightweight text processing model, comprising: Acquire a complex text processing model trained based on a first training text library, wherein each sample in the first training text library includes text data of the sample; Acquire a second training text library, wherein each sample in the second training text library includes a category label of the sample and a word segmentation sequence of the sample, and the sample size in the second training text library is smaller than the sample size in the first training text library; Using the complex text processing model, converting the word segmentation sequence of the sample in the second training text library into the first sample sentence vector; and Based on the category label, word segmentation sequence and first sample sentence vector of each sample in the second training text library, a lightweight text processing model is trained, wherein the complexity of the lightweight text processing model is lower than that of the complex text processing model.

2. The method of claim 1, wherein: The lightweight text processing model includes a first branch model for extracting sequential information of a word segmentation sequence, a second branch model for extracting association information between words in the word segmentation sequence, and a third branch model for extracting keyword information in the word segmentation sequence.

3. The method of claim 2, wherein: The training lightweight text processing model includes: The word segmentation sequence of each sample in the second training text library is converted into a second sample sentence vector of the sample by using a lightweight text processing model. Determining a processing loss of the lightweight text processing model based on the first sample sentence vector and the second sample sentence vector; Update parameters in the lightweight text processing model to minimize the processing loss.

4. The method of claim 3, wherein: The lightweight text processing model is used to convert the word segmentation sequence of each sample in the second training text library into a second sample sentence vector of the sample: Using the first branch model of the lightweight text processing model, obtaining a first sample clause vector representing sequential information of the text data from the word segmentation sequence; Using the second branch model of the lightweight text processing model, obtaining a second sample clause vector representing the association relationship between each word in the text data from the word segmentation sequence; Using the third branch model of the lightweight text processing model, obtaining a third sample clause vector representing keyword information in the text data from the word segmentation sequence; The first sample clause vector, the second sample clause vector and the third sample clause vector are fused into a second sample clause vector.

5. A device for simplifying a complex text processing model into a lightweight text processing model, comprising: processor; as well as A memory, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the method according to any one of claims 1 to 4 is executed.

6. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 4.