Text processing method, computer device and medium

Abstract

The application provides a text processing method, computer equipment and medium, the method comprises: identifying the theme of input text and the keyword in the input text through a model; determining the risk level of the input text based on the theme and the keyword; and adjusting the probability distribution of the output word element of the model based on the risk level, and determining the output text based on the probability distribution. Through the above text processing method, the output text can be adjusted based on the risk level of the text without affecting the quality of the text, so that the output text contains marks that are not easy to remove.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer processing, and in particular to a text processing method, a computer device and a medium. BACKGROUND

[0002] With the wide application of large language model (LLM) generated content, the traceability and copyright protection of the content have become increasingly prominent. In the prior art, a large model content watermarking technology is used as a solution. The core of the technology is to embed imperceptible hidden marks in the generated text during the generation of the text by the LLM. However, the embedded marks are prone to be removed or affect the quality of the generated text. SUMMARY

[0003] Embodiments of the present application disclose a text processing method, a computer device and a medium, which can solve the problem that the added marks for the text are prone to be removed or affect the quality of the text.

[0004] In a first aspect, embodiments of the present application provide a text processing method, which includes: receiving an input text through a model; identifying a topic of the input text and a keyword in the input text; determining a risk level of the input text based on the topic and the keyword; and adjusting a probability distribution of an output token of the model based on the risk level, and determining an output text based on the probability distribution.

[0005] In some possible implementation manners of the present application, the adjusting the probability distribution of the output token of the model based on the risk level, and determining the output text based on the probability distribution includes: generating a candidate list of the output token through the model, wherein each first token in the candidate list corresponds to a probability value; performing a hash operation on each first token based on a preset key to obtain a hash value corresponding to each first token; determining a coefficient based on the risk level; determining a second token in the candidate list based on the first coefficient and the hash value; adjusting a probability value corresponding to the second token based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold value, wherein the probability threshold value is equal to a maximum value in the probability values corresponding to the first tokens in the candidate list.

[0006] In some possible implementation manners of the present application, the determining the second token in the candidate list based on the first coefficient and the hash value includes: comparing the size of the hash value corresponding to each first token with the first coefficient; and determining the first token corresponding to the hash value as the second token in response to the hash value being less than or equal to the first coefficient.

[0007] In some possible implementation of the present application, the determining the first coefficient based on the risk level comprises: determining the first coefficient corresponding to the risk level by querying a first correspondence table between risk levels and first coefficients.

[0008] In some possible implementation of the present application, the determining the risk level of the input text based on the topic and the keywords comprises: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient and the third risk coefficient.

[0009] In some possible implementation of the present application, the determining the first risk coefficient of the input text based on the topic comprises: obtaining a fourth risk coefficient corresponding to the topic according to a second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; in response to the existence of the preset word related to the topic in the input text, summing the preset value corresponding to the preset word and the fourth risk coefficient to obtain the first risk coefficient.

[0010] In some possible implementation of the present application, the determining the second risk coefficient of the input text based on the sensitive words in the keywords comprises: determining the number of occurrences and the weight corresponding to each sensitive word in the input text in a preset time period; determining a maximum number of occurrences based on the number of occurrences; and obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

[0011] In some possible implementation of the present application, the determining the third risk coefficient of the input text based on the named entities in the keywords comprises: determining the total length of the input text and the total number of named entities contained in the input text; determining the probability corresponding to each named entity based on the number of occurrences of the named entity in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities and the probability.

[0012] In the second aspect, the embodiments of the present application further provide a computer device, which comprises a processor and a memory, and the processor implements the text processing method as described above when executing a computer program stored in the memory.

[0013] In the third aspect, the embodiments of the present application further provide a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor to implement the text processing method as described above.

[0014] The text processing method provided by the embodiments of the present application receives an input text through a model; identifies a topic of the input text and a keyword in the input text; determines a risk level of the input text based on the topic and the keyword; and adjusts a probability distribution of output tokens of the model based on the risk level to obtain an output text. Through the above text processing method, the output text of the model can be modified based on the risk level of the text without affecting the quality of the text, thereby realizing the function of adding marks to the text and increasing the anti-tampering capability of the text. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 FIG. 1 is a schematic diagram of an application environment of the text processing method provided by the embodiments of the present application.

[0016] Figure 2 FIG. 2 is a flowchart of the text processing method provided by the embodiments of the present application.

[0017] Figure 3 FIG. 3 is a structural diagram of the text processing device in the embodiments of the present application.

[0018] Figure 4 FIG. 4 is a structural diagram of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0019] For the convenience of understanding, exemplary descriptions of some concepts related to the embodiments of the present application are given for reference.

[0020] It should be noted that "at least one" in the present application means one or more, and "multiple" means two or more than two. "And / or" describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0021] Large model content watermarking is a technology that embeds hidden markers in generated content. The core idea of large model content watermarking is to insert specific characters or phrases into the text that do not affect the readability of the text, but can identify the source of the generated content. For example, invisible characters (such as \u200b) or specific symbols can be inserted at specific positions. Large model content watermarking can help determine the source of the generated content, making it easier to track and manage, especially when the source of the content is unclear. Large model content watermarking can also help protect copyright. For example, after embedding a watermark in the generated content, it can prevent the generated content from being misused or misused by others, protecting the rights of the original creator. Large model content watermarking can help with content review and supervision, especially in sensitive language tracking to ensure that the generated content meets relevant regulations and ethical standards. The presence of the watermark can provide additional information sources for content review. And let users identify which content is generated by AI, increase the transparency and credibility of information. This helps users understand and judge the source of the content. However, the embedded markers are prone to removal or affect the quality of the generated text.

[0022] Based on the above technical problems, the text processing method provided by the present application can adjust the probability distribution of the output word units of the model based on the risk level of the text without affecting the quality of the text to obtain the output text. The text processing method provided by the present application can add a watermark to the text based on the risk level of the text without affecting the quality of the text to adjust the output text, increase the anti-tampering ability of the text, and make the output text contain markers that are not easy to remove.

[0023] In order to better understand the text processing method and related products provided by the embodiments of the present application, first, the application scenario of the text processing method of the present application will be described.

[0024] Figure 1 is a schematic diagram of the application environment of the text processing method provided by the embodiments of the present application. As Figure 1As shown, in some embodiments of the present application, the text processing method can be applied in a computer device 1 (for example, a server) which can be in communication connection with a terminal device 2. The terminal device 2 can receive the input text of the user and send the input text to the model in the computer device 1 for processing, and output the corresponding output text through the model. For example, when the user inputs the input text of consulting travel through the terminal device 2 as "Please give a travel plan for summer vacation in Shenzhen". The terminal device 2 sends the input text to the model in the computer device 1 for processing. After the model processes the input text, it will output an output text containing a travel plan. In order to identify the generation source of the travel plan output by the model, the output text can be processed based on the risk level of the input text when the model generates the output text. For example, according to the risk level of the input text, the probability distribution of the output word units of the model is adjusted to obtain an output text containing a travel plan, which not only ensures the semantic invariance of the output text, but also ensures the readability of the output text. The method for processing the text is specifically referred to Figure 2 The flowchart shown.

[0025] The terminal device 2 can be a computer, a server, a smart screen, a mobile phone, a tablet computer, a smart wearable device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, a netbook, etc. The specific type of electronic device is not limited in the embodiments of the present application.

[0026] The technical solutions of the present application will be described in detail below with specific embodiments. The specific embodiments described below can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments.

[0027] Figure 2 The flowchart of the text processing method provided in the embodiments of the present application, the method of the present embodiment can be executed by a computer device, and a model (for example, a large language model) can be deployed in the computer device. As Figure 2 As shown, the method of the present embodiment can include:

[0028] Step S201, receiving an input text through a model.

[0029] In some embodiments of the present application, when the user needs to generate text by using the model (for example, LLM), the input text can be sent to the computer device through the terminal device, and the corresponding output text is output after the input text is processed by the model in the computer device. In order to not affect the readability of the output text, the content of the output text can be adjusted according to the risk level of the input text when the model outputs the corresponding output text according to the input text.

[0030] In some embodiments of the present application, the input text can also be received directly by the computer device. For example, the input text input by the user is received through the input / output interface of the computer device.

[0031] In step S202, the topic of the input text and the keywords in the input text are identified.

[0032] In some embodiments of the present application, in order to perform risk assessment on the input text, the topic of the input text and the keywords in the input text are identified, and the input text is risk assessed by the topic and the keywords. The keywords at least include sensitive words and named entities.

[0033] In some embodiments of the present application, the topic of the input text can include finance, life, learning, etc. The input text can include at least one sentence. For example, the input text can be a sentence, or a short article containing multiple sentences. For example, when the input text “Please give me some suggestions for weekend travel” is received by the model, the topic of the input text is identified as life. For example, when the input text “What is the latest loan interest rate of XX Bank?” is received by the model, the topic of the input text is identified as finance. For example, when the input text “How to quickly learn a foreign language?” is received by the model, the topic of the input text is identified as learning.

[0034] In some embodiments of the present application, identifying the topic of the input text includes: extracting a context vector representation of the input text by a pre-trained language model; calculating a Term Frequency-Inverse Document Frequency (TF-IDF) and a topic saliency score according to the context vector representation; constructing a text semantic dependency graph based on the TF-IDF and the topic saliency score, and identifying the topic of the input text by the semantic dependency graph. Specifically, a pre-trained language model (such as BERT, RoBERTa) can be used to extract a context vector representation to obtain a deep semantic representation of the text. Based on the context vector, a TF-IDF and a topic saliency score are calculated. The TF-IDF is used to measure the importance of word frequency, and the topic saliency score is used to quantify the topic relevance of the word. Based on the TF-IDF and the topic saliency score, a semantic dependency graph is constructed. The topic saliency score is used to initialize or weight the nodes of the semantic dependency graph, so that the semantic dependency graph focuses more on the topic significant elements. The construction of the semantic dependency graph can depend on the parsing ability of the pre-trained language model (such as using its output to generate dependency relationships). The semantic dependency graph is analyzed (for example, PageRank, community detection or clustering algorithms are applied), and the topic of the text is output.

[0035] In some embodiments of the present application, a sensitive word library can be constructed in advance, different sensitive word libraries can be constructed according to different application scenarios, and if there are words in the input text that match the sensitive word library, the sensitive words in the input text can be identified.

[0036] In some embodiments of the present application, in the embodiments of the present application, a named entity (Named Entity, or simply entity) refers to an entity in the text that has a specific meaning or strong reference, usually including names, place names, organization names, dates and times, proper nouns, etc. More extensive entities also include numbers, currencies, addresses, etc. Because it is difficult to effectively identify non-fixed format information to be encrypted (for example, names and addresses, etc.) through pattern matching, in order to improve the identification efficiency of the information to be encrypted in the information to be processed, the named entity recognition model can be used to identify the named entity in each sentence.

[0037] In embodiments of the present application, the named entity recognition model can also be referred to as a named entity recognition system. The model can extract the above-mentioned named entities from the input text, and can identify more categories of named entities according to business needs. The named entity recognition model can use models in existing related technologies, such as methods based on feature templates, methods based on neural networks, etc. In an embodiment, the named entity recognition model can use a model of Word Embedding (word embedding) + LSTM (Long Short Term Memory, long short-term memory network) / BiLSTM (Bidirectional Long Short Term Memory, bidirectional long short-term memory network) + CRF (Conditional Random Field, conditional random field).

[0038] The named entity in the input text can be identified by a named entity recognition (Named Entity Recognition, NER) model. Specifically, the input text is divided into multiple division fields; each division field is assigned a part-of-speech tag; each division field after assigning the part-of-speech tag is subjected to dependency analysis; the boundaries of the input text are detected to obtain multiple sentences; and the named entities in each sentence are identified based on the results of the dependency analysis through the named entity model.

[0039] In embodiments of the present application, the division field refers to the basic unit of sentence division. For example, a single Chinese character can be used as the most basic division field, or a word after performing word segmentation processing can be used as the most basic division field. The word segmentation processing can use a word segmentation tool in existing related technologies, such as the Jieba word segmentation tool, etc. The goal of word segmentation processing is to divide the sentence into multiple words.

[0040] In embodiments of the present application, after obtaining multiple division fields, in order to analyze the context semantics of the input text, each division field can be assigned a part-of-speech tag to label the part-of-speech of each division field. For example, verbs, nouns, adverbs, conjunctions, etc.

[0041] In the embodiments of the present application, after the part-of-speech tags are assigned, a dependency analysis can be performed on each divided field to determine the relationship between the divided fields. The relationship between the divided fields includes a subject-predicate relationship (SBV), a head of the sentence relationship (HED), an attribute-modifier relationship (ATT), an adverbial structure (ADV), a coordinate relationship (COO), a right adjunct relationship (RAD), and a verb-object structure (VOB). The subject-predicate relationship SBV represents the structural relationship between the subject and the predicate in a sentence. In Chinese syntactic analysis, SBV is used to describe the relationship between the subject (Subject) and its corresponding predicate verb (Predicate). The core relationship HED refers to the core component of a sentence, which is usually the center of the meaning of the entire sentence. In dependency syntactic analysis, HED is used to mark the core words or phrases that dominate other components of the sentence. The attribute-modifier relationship ATT represents the relationship between the modifier and the modified. In Chinese syntactic analysis, ATT is used to describe the case where an adjective (Attribute) modifies a noun (Modifier). The adverbial structure ADV refers to the role and position of adverbs or adverbials in a sentence. In dependency syntactic analysis, ADV is used to mark the words or phrases used as adverbials, which usually modify verbs, adjectives, or other adverbs. The coordinate relationship COO represents the parallel relationship between two or more same grammatical components. In Chinese syntactic analysis, COO is used to describe the relationship between parallel words, phrases, or clauses. The right adjunct relationship RAD refers to the relationship between the component attached on the right side and the center word. In dependency syntactic analysis, RAD is used to mark the components that are located on the right side of the center word and play a supplementary explanatory role. The verb-object structure VOB represents the structural relationship between a verb and its direct object. In Chinese syntactic analysis, VOB is used to describe the relationship between a verb (Verb) and its direct object (Object).

[0042] In some embodiments, morphological reduction can also be performed on each divided field. By reducing the word form of each divided field to its basic form, the semantic ambiguity caused by the inflection of the divided field due to tense, person, number, etc. can be eliminated, which helps to improve the accuracy of identifying the information to be encrypted.

[0043] In the embodiments of the present application, the boundaries of each sentence in the input text are identified according to punctuation marks, and the text information is divided according to the identified boundaries of each sentence to obtain a plurality of complete sentences. The sentences can be segmented by searching for punctuation marks such as commas and periods in the text information. In some embodiments of the present application, the text can be preprocessed before identifying the topic of the text. Specifically, the preprocessing of the text includes tokenization, part-of-speech tagging, word frequency feature extraction, and stop word filtering of the original text.

[0044] In step S203, the risk level of the input text is determined based on the topic and the keywords.

[0045] In the embodiments of the present application, the input text needs to be classified into a risk level, and the corresponding output text is output based on the risk level of the input text, so that the text can be marked without embedding specific characters or phrases. Specifically, determining the risk level of the input text based on the topic and the keywords includes: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient, and the third risk coefficient.

[0046] In the embodiments of the present application, determining the first risk coefficient of the input text based on the topic includes: obtaining a fourth risk coefficient corresponding to the topic according to a second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; in response to the presence of the preset word related to the topic in the input text, summing the preset value corresponding to the preset word and the fourth risk coefficient to obtain the first risk coefficient.

[0047] Specifically, after identifying the topic of the input text, a correspondence table is constructed. The second correspondence table can be a topic and risk coefficient correspondence table, which describes the correspondence between the topic and the risk coefficient. For example, when the topic is finance, the corresponding risk coefficient is large; when the topic is life, the corresponding risk coefficient is small. After determining the fourth risk coefficient corresponding to the topic according to the topic and risk coefficient correspondence table, a certain coefficient can be added to the fourth risk coefficient according to the presence of the preset word related to the topic in the input text, indicating that the probability of adjusting the output text obtained by the model according to the input text is large. The preset word can be a word involved when the text under the topic is complained.

[0048] For example, when the theme is finance, the corresponding fourth risk coefficient is 0.7, and if the input text further includes preset words "exchange rate" and "stock market" related to the theme, the fourth risk coefficient is increased by a preset value 0.12, and the first risk coefficient 0.82 is obtained based on the sum of the fourth risk coefficient 0.7 and the preset value 0.12. When the theme is people's livelihood, the corresponding fourth risk coefficient is 0.5, and if the input text further includes preset words "medical treatment" and "education" related to the theme, the fourth risk coefficient is increased by a preset value 0.1, and the first risk coefficient 0.7 is obtained based on the sum of the fourth risk coefficient 0.5 and the preset value 0.2. When the theme is technology, the corresponding fourth risk coefficient is 0.3, and if the input text further includes preset words "patent" and "artificial intelligence" related to the theme, the fourth risk coefficient is increased by a preset value 0.1, and the first risk coefficient 0.4 is obtained based on the sum of the fourth risk coefficient 0.3 and the preset value 0.1.

[0049] In the embodiment of the present application, the second risk coefficient of the input text is determined based on the sensitive words in the keywords, including: determining the corresponding number of occurrences of each sensitive word in the input text in a preset time period and the corresponding weight; determining the maximum number of occurrences based on the number of occurrences; and obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

[0050] In the embodiment of the present application, the number of occurrences of each sensitive word in the input text in a preset time period (for example, 30 days) can be calculated. For example, the number of occurrences of the sensitive word is determined by the received dialogue content of the complaint case. The ratio between the number of occurrences of each sensitive word in the preset time period and the number of occurrences of all sensitive words in the preset time period is calculated to obtain the corresponding weight of each sensitive word in the preset time period.

[0051] In the embodiment of the present application, the second risk coefficient is calculated based on the weight and the maximum number of occurrences by the following formula

[0052] ,

[0053] wherein, is the weight of each sensitive word in the preset time period, is the number of occurrences of each sensitive word in the input text in the preset time period, is the maximum number of occurrences, is the total number of all sensitive words in the input text.

[0054] In the embodiment of the present application, the maximum number of occurrences is the maximum value of the number of occurrences of all sensitive words in the preset time period.

[0055] ​In the embodiments of the present application, determining the third risk coefficient of the input text based on the named entities comprises: determining the total length of the input text and the total number of all named entities; determining the probability corresponding to each named entity based on the number of times each named entity appears in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities and the probability.

[0056] Specifically, the total length of the input text can be determined according to the total number of characters of the input text. For example, if the input text is a sentence including ten characters, the total length of the input text is ten. The probability corresponding to each named entity is determined based on the ratio between the number of times each named entity appears in the input text and the total number of named entities, and the third risk coefficient of the input text is calculated based on the total length of the input text, the total number of named entities and the probability by the following formula:

[0057] ,

[0058] wherein, is the probability corresponding to each named entity, is the total number of named entities, is the total length of the input text.

[0059] In the embodiments of the present application, the risk level of the input text can be obtained by accumulating the sum of the first risk coefficient, the second risk coefficient and the third risk coefficient, or the risk level of the input text can be obtained by calculating the first risk coefficient, the second risk coefficient and the third risk coefficient through weighted average calculation. The present application does not make any limitation in this regard.

[0060] In step S204, the probability distribution of the output tokens of the risk level adjustment model is adjusted based on the probability distribution, and the output text is determined based on the probability distribution.

[0061] In some embodiments, after the model receives the input text, the model can generate the output text based on one output token at a time. For example, the model generates a first output token, a first output token and a third output token based on the input text, and then concatenates the first output token, the second output token and the third output token to obtain the output text. Specifically, in order to generate a coherent output text, the model predicts the next most likely output token. Specifically, before the model outputs the first output token, a candidate list of the first output token can be generated based on the input text, where each first token in the candidate list corresponds to a probability value. The first token with the maximum probability value in the candidate list can be selected as the first output token. The next most likely output token is predicted based on the input text and the first output token, and the model generates a candidate list of the second output token based on the input text and the first output token. The first token with the maximum probability value in the candidate list of the second output token is selected as the second output token. The next most likely output token is predicted based on the input text, the first output token and the second output token, and the model generates a candidate list of the third output token based on the input text, the first output token and the second output token. The first token with the maximum probability value in the candidate list of the third output token is selected as the third output token. The output text is obtained according to the first output token, the second output token and the third output token.

[0062] The above process of generating the input text by the model does not consider the risk level of the input text, cannot identify the output text, and cannot determine the source of the output text. The text processing method provided in the embodiments of the present application can adjust the probability distribution of the output token of the model according to the risk level of the input text to obtain the output text. Specifically, a candidate list of the output token is generated by the model, where each first token in the candidate list corresponds to a probability value; a corresponding hash value is obtained by performing a hash operation on the first token based on a preset key; a first coefficient is determined based on the risk level; a second token in the candidate list is determined based on the first coefficient and the hash value; the probability value corresponding to the second token is adjusted based on the risk level, and in response to the adjusted probability value being greater than or equal to a probability threshold, the output text is output, wherein the probability threshold is equal to the maximum value of the probability values corresponding to the first tokens in the candidate list.

[0063] In the embodiments of the present application, determining the second token in the candidate list based on the first coefficient and the hash value includes: comparing the size of the hash value corresponding to each first token with the first coefficient; and in response to the hash value being less than or equal to the first coefficient, determining the first token corresponding to the hash value as the second token.

[0064] In the embodiments of the present application, the determining the first coefficient based on the risk level comprises: determining the first coefficient corresponding to the risk level by querying a first correspondence table (such as Table 1) between the risk level and the first coefficient. For example, when the risk level of the text is determined to be low risk, the corresponding first coefficient is a first value; when the risk level of the text is determined to be medium risk, the corresponding first coefficient is a second value; and when the risk level of the text is determined to be high risk, the corresponding first coefficient is a third value. The first value is less than the second value, and the second value is less than the third value.

[0065] Table 1

[0066]

[0067] For example, before the model outputs a first output token, a candidate list of the first output token can be generated based on the input text, wherein the first token in the candidate list corresponds to a probability value. The probability values corresponding to the first tokens in the candidate list are sorted in descending order, and the first tokens ranked in the top five can be selected to form the candidate list. Five hash values are obtained by performing a hash operation on the preset key and the five first tokens, a first coefficient is determined based on the risk level of the input text, and the five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first token corresponding to the hash value is selected as a second token. The probability value corresponding to the second token is adjusted based on the risk level, and the first token is output when the adjusted probability value is greater than or equal to a probability threshold.

[0068] The next most likely output token is predicted based on the input text and the second token. The model generates a candidate list of the second output token based on the input text and the second token, and selects the first tokens ranked in the top five to form the candidate list. Five hash values are obtained by performing a hash operation on the preset key and the five first tokens, a first coefficient is determined based on the risk level of the input text, and the five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first token corresponding to the hash value is selected as a third token. The probability value corresponding to the third token is adjusted based on the risk level, and the third token is output when the adjusted probability value is greater than or equal to a probability threshold.

[0069] The model generates a candidate list of the third output word units based on the input text, the second word unit and the third word unit, and selects the first word units ranked in the top five to form the candidate list. Five hash values are obtained by performing a hash operation based on a preset key and the five first word units. A first coefficient is determined based on a risk level of the input text. The five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first word unit corresponding to the hash value is selected as a fourth word unit. The probability value corresponding to the fourth word unit is adjusted based on the risk level. If the adjusted probability value is greater than or equal to a probability threshold, the fourth word unit is output. The output text is obtained according to the second word unit, the third word unit and the fourth word unit.

[0070] In some embodiments of the present application, if there are multiple second word units whose adjusted probability values are greater than or equal to the probability threshold, the word unit corresponding to the largest probability value of the second word units greater than or equal to the probability threshold is selected as the second word unit.

[0071] In some embodiments of the present application, the output word unit can include a part of a character, a word or a phrase.

[0072] In some embodiments of the present application, the preset key is a unique key corresponding to the owner of the model, used to identify the generation source of the output text. For example, company A generates a first output text through the model, and company B generates a second output text through the model. In order to distinguish that the generation source of the first output text is the model owned by company A, and the generation source of the second output text is the model owned by company B, a hash value can be obtained by performing a hash operation on the preset key A and the first word unit when company A generates the first output text through the model; a hash value can be obtained by performing a hash operation on the preset key B and the first word unit when company B generates the second output text through the model.

[0073] In an embodiment of the present application, adjusting the probability value corresponding to the second word unit based on the risk level includes increasing a preset probability for the probability value corresponding to the second word unit based on the risk level. The risk level is negatively correlated with the preset probability. For example, if the risk level is low, the corresponding increased preset probability is a first probability; if the risk level is medium, the corresponding increased preset probability is a second probability; if the risk level is high, the corresponding increased preset probability is a third probability. The first probability is greater than the second probability, and the second probability is greater than the third probability. For example, if the risk level is low, the corresponding increased preset probability is 0.03%; if the risk level is medium, the corresponding increased preset probability is 0.02%; if the risk level is high, the corresponding increased preset probability is 0.01%.

[0074] For example, when the input text received by the model is "Let the financial AI large model be like a planner", the topic of the input text is identified as finance, and the risk level of the input text is determined to be high risk based on the topic and keywords of the input text. In the process of generating the output text by the model, a candidate list of output word units is first generated, which may include 100 words, and the top five words with the highest probability values are selected to obtain the corresponding candidate list: come to see (probability 42%), think (probability 38%), know (probability 12%), latest (probability 5%), and do (probability 3%). Among them, "come to see" is the first word unit, the corresponding probability value is 42%; "think" is the first word unit, the corresponding probability value is 38%; "know" is the first word unit, the corresponding probability value is 12%; "latest" is the first word unit, the corresponding probability value is 5%; and "do" is the first word unit, the corresponding probability value is 3%. Based on the preset key (for example, 12345) and the above five first word units, a hash value is obtained by hash operation, and a first coefficient is determined to be a third value 40 based on the risk level. Determine that "think (hash value 37)" and "latest (hash value 18)" in the candidate list are the second word units, and increase the preset probability by 1% based on the probability value 42% corresponding to the second word unit "think". Increase the preset probability by 1% based on the probability value 5% corresponding to the second word unit "latest". Then the adjusted probability value of the second word unit "think" is 43%, which is greater than the maximum value 42% of the corresponding probability value in the candidate list. Then the adjusted probability value of the second word unit "latest" is 6%, which is less than the maximum value 42% of the corresponding probability value in the candidate list. Thus, the first output word unit determined by the model is "think".

[0075] In the embodiments of the present application, if in some application scenarios, after the computer device outputs the text information to the user of the terminal device through the model, in order to avoid the user using the text information by abnormal means and then thinking that the text information has a problem and causing trouble to the owner of the model. The output text can be verified by the mark in the text. Specifically, verifying the output text includes: counting the frequency of the adjusted probability distribution of the output word unit appearing in the output text, confirming the size of the frequency and the frequency threshold; if the frequency is greater than or equal to the frequency threshold, it is determined that the output text contains the mark; if the frequency is less than the frequency threshold, it is determined that the output text does not contain the mark. In the embodiments of the present application, the frequency of the mark appearing in the output text is counted by calculating the frequency of the adjusted probability distribution of the output word unit appearing in the output text.

[0076] In the embodiments of the present application, the output text is verified by calculating the frequency of the output word element after the adjustment of the probability distribution appearing in the output text. For example, if the model output text is "a certain country announced that it would implement a new trade policy in 2025, which would significantly affect the global energy market", the frequency of the output word elements "policy" and "significantly" after the adjustment of the probability distribution in the above output text is counted. If the frequency is greater than or equal to the frequency threshold (for example, 0.95), it is determined that the output text contains the label; if the frequency is less than the frequency threshold (for example, 0.95), it is determined that the output text does not contain the label. For another example, if the model output text is "weekend recommendation to go to the park for a walk, suitable for relaxing!", the frequency of the exclamation mark after the adjustment of the probability distribution in the above output text is counted. If the frequency is greater than or equal to the frequency threshold (for example, 0.7), it is determined that the output text contains the label; if the frequency is less than the frequency threshold (for example, 0.7), it is determined that the output text does not contain the label.

[0077] It should be noted that the frequency threshold is related to the risk level of the text and the first coefficient. When it is determined that the risk level of the text is low risk, the corresponding first coefficient is a first value, and the corresponding frequency threshold is a first threshold value; when it is determined that the risk level of the text is medium risk, the corresponding first coefficient is a second value, and the corresponding frequency threshold is a second threshold value; when it is determined that the risk level of the text is high risk, the corresponding first coefficient is a third value, and the corresponding frequency threshold is a third threshold value. The first value is less than the second value, and the second value is less than the third value; the first threshold value is less than the second threshold value, and the second threshold value is less than the third threshold value. This is the key to detecting and verifying the labels in the output text of the model.

[0078] Figure 3 is a structural diagram of a text processing apparatus provided in the embodiments of the present application. The text processing apparatus 300 can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the text processing apparatus 300 can be stored in the memory of the computer device and executed by at least one processor to perform the text processing function (see Figure 2 for details).

[0079] In the embodiments, the text processing apparatus 300 can be divided into a plurality of functional modules according to the functions it performs. The functional modules can include a receiving module 301, an identifying module 302, a determining module 303, and a processing module 304. The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory. In the embodiments, the text processing apparatus 300 can be used to implement the text processing method as shown in Figure 2 . As shown in Figure 3 , the text processing apparatus 300 is applied to a computer device (such as a server or a terminal). Figure 1The text processing apparatus 300 comprises:

[0080] The receiving module 301 is configured to receive an input text through a model.

[0081] The identifying module 302 is configured to identify a topic of the input text and keywords in the input text.

[0082] The processing module 304 is configured to adjust a probability distribution of output tokens of the model based on the risk level to obtain an output text.

[0083] The processing module 304 adjusts the probability distribution of the output tokens of the model based on the risk level, and determines the output text based on the probability distribution, including: generating a candidate list of the output tokens through the model, wherein each first token in the candidate list corresponds to a probability value; performing a hash operation on each first token based on a preset key to obtain a hash value corresponding to each first token; determining a coefficient based on the risk level; determining a second token in the candidate list based on the first coefficient and the hash value; adjusting a probability value corresponding to the second token based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold, wherein the probability threshold is equal to a maximum value in the probability values corresponding to the first tokens in the candidate list.

[0084] The processing module 304 determines the second token in the candidate list based on the first coefficient and the hash value, including: comparing the size of the hash value corresponding to each first token with the first coefficient; and determining the first token corresponding to the hash value as the second token in response to the hash value being less than or equal to the first coefficient.

[0085] The processing module 304 determines the first coefficient based on the risk level, including: determining the first coefficient corresponding to the risk level by querying a first correspondence table between risk levels and first coefficients.

[0086] The determining module 303 determines the risk level of the input text based on the topic and the keywords, including: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient, and the third risk coefficient.

[0087] The determination module 303 determines the first risk coefficient of the input text based on the topic by: obtaining the fourth risk coefficient corresponding to the topic according to the second correspondence table, and confirming the preset words related to the topic and the preset values ​​corresponding to the preset words; in response to the existence of preset words related to the topic in the input text, summing the preset values ​​corresponding to the preset words and the fourth risk coefficient to obtain the first risk coefficient.

[0088] The determination module 303 determines the second risk coefficient of the input text based on the sensitive words in the keywords, including: determining the number of times each sensitive word in the input text appears within a preset time period and its corresponding weight; determining the maximum number of times it appears based on the number of times it appears; and obtaining the second risk coefficient based on the weight and the maximum number of times it appears.

[0089] The determination module 303 determines the third risk coefficient of the input text based on the named entities in the keywords, including: determining the total length of the input text and the total number of named entities contained in the input text; determining the probability corresponding to each named entity based on the number of times each named entity appears in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities, and the probability.

[0090] Figure 4 This is a schematic diagram of the structure of the computer device provided in an embodiment of this application. Figure 4 As shown, computer device 1 can be a server, for example, a central server, an edge server, or a local server in a local data center. Each computer device 1 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication interface 101, the memory 102, and the I / O interface 104 via the bus 105.

[0091] The communication module 101 can include a wired communication module and / or a wireless communication module. The wired communication module can provide one or more of the following wired communication solutions: universal serial bus (USB), Controller Area Network (CAN) bus, etc. The wireless communication module can provide one or more of the following wireless communication solutions: wireless fidelity (Wi-Fi), bluetooth (BT), mobile communication network, frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0092] The memory 102 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The random access memory can be directly readable and writable by the processor 103, and can be used to store executable programs (e.g., machine instructions) of an operating system or other programs that are running, and can also be used to store data of users and applications, etc. The random access memory can include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0093] The non-volatile memory can also store executable programs and store data of users and applications, etc., and can be loaded in advance into the random access memory for direct reading and writing by the processor 103. The non-volatile memory can include a magnetic disk storage device, a flash memory.

[0094] The memory 102 is configured to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions, which when executed by the processor 103, can implement the text processing method executed on the computer device 1.

[0095] In other embodiments, the computer device 1 further comprises an external memory interface for connecting an external memory, so as to extend the storage capability of the computer device 1.

[0096] The processor 103 can include one or more processing units, for example: the processor 103 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0097] The processor 103 provides computing and control capabilities, for example, the processor 103 is used to execute a computer program stored in the memory 102.

[0098] The I / O interface 104 is used to provide a channel for user input or output, for example, the I / O interface 104 can be used to connect various input and output devices, such as a mouse, a keyboard, a touch device, a display screen, etc., so that the user can enter information, or make the information visualized.

[0099] The bus 105 is used to provide a communication channel between the communication module 101, the memory 102, the processor 103, and the I / O interface 104 in the computer device 1.

[0100] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the computer device 1. In other embodiments of the present application, the computer device 1 can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0101] The embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, and the method implemented when the program instructions are executed can refer to the method in each of the above embodiments of the present application.

[0102] The computer readable storage medium can be an internal storage of the electronic device, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0103] In some embodiments, the computer readable storage medium can include a program storage area and a data storage area. The program storage area can store an operating system, an application required by at least one function, and the like. The data storage area can store data created according to use of the electronic device, and the like.

[0104] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0105] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0106] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented by other ways. For example, the apparatus / terminal device embodiments described above are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0107] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A text processing method characterized by, The method comprises: receiving an input text through a model; identifying a topic of the input text and keywords in the input text; determining a risk level of the input text based on the topic and the keywords; adjusting a probability distribution of output word elements of the model based on the risk level, and determining an output text based on the probability distribution, comprising: generating a candidate list of the output word elements through the model, wherein each first word element in the candidate list corresponds to a probability value; performing a hash operation on each first word element based on a preset key to obtain a hash value corresponding to each first word element; determining a first coefficient based on the risk level, wherein the higher the risk level, the larger the first coefficient; determining a second word element in the candidate list based on the first coefficient and the hash value; adjusting a probability value corresponding to the second word element based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold, wherein the determining the second word element in the candidate list based on the first coefficient and the hash value comprises: comparing the size of the hash value corresponding to each first word element with the first coefficient; and determining the first word element corresponding to the hash value as the second word element in response to the hash value being less than or equal to the first coefficient.

2. The text processing method of claim 1, wherein, The probability threshold is equal to the maximum value of the probability values corresponding to the first word elements in the candidate list.

3. The text processing method of claim 2, wherein, The determining the first coefficient based on the risk level comprises: determining the first coefficient corresponding to the risk level by querying a first correspondence table between risk levels and first coefficients.

4. The text processing method of claim 1, wherein, The determining the risk level of the input text based on the topic and the keywords comprises: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; determining the risk level of the input text based on the first risk coefficient, the second risk coefficient, and the third risk coefficient.

5. The text processing method of claim 4, wherein, The determining the first risk coefficient of the input text based on the topic comprises: obtaining a fourth risk coefficient corresponding to the topic according to a second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; in response to the presence of the preset word related to the topic in the input text, summing the preset value corresponding to the preset word and the fourth risk coefficient to obtain the first risk coefficient.

6. The text processing method of claim 4, wherein, The determining the second risk coefficient of the input text based on the sensitive words in the keywords comprises: determining the number of occurrences and the corresponding weight of each sensitive word in the input text within a preset time period; determining a maximum number of occurrences based on the number of occurrences; obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

7. The text processing method of claim 4, wherein, The determining the third risk coefficient of the input text based on the named entities in the keywords comprises: determining the total length of the input text and the total number of named entities contained in the input text; determining a probability corresponding to each of the named entities based on a number of times each of the named entities appears in the input text and a total number of the named entities; calculating a third risk coefficient of the input text based on a total length of the input text, the total number of the named entities and the probability.

8. A computer device, comprising: The computer device comprises a processor and a memory, and the processor implements the text processing method as claimed in any one of claims 1 to 7 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the text processing method as claimed in any one of claims 1 to 7.

Images