Text processing method, computer equipment and medium

Abstract

The invention provides a text processing method, computer equipment and a medium. The method comprises the following steps: recognizing a theme of an input text and keywords in the input text through a model; determining a risk level of the input text based on the theme and the keyword; and adjusting probability distribution of output lexical elements of the model based on the risk level, and determining an output text based on the probability distribution. Through the text processing method, the output text can be adjusted based on the risk level of the text under the condition that the text quality is not influenced, so that the output text contains marks which are not easy to remove.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer processing, and in particular to a text processing method, a computer device and a medium. BACKGROUND

[0002] With the wide application of large language model (LLM) generated content, the traceability and copyright protection of the content have become increasingly prominent. In the prior art, a large model content watermarking technology is used as a solution. The core of the technology is to embed imperceptible hidden marks in the generated text during the generation of the text by the LLM. However, the embedded marks are prone to be removed or affect the quality of the generated text. SUMMARY

[0003] Embodiments of the present application disclose a text processing method, a computer device and a medium, which can solve the problem that the added marks for the text are prone to be removed or affect the quality of the text.

[0004] In a first aspect, embodiments of the present application provide a text processing method, which includes: receiving an input text through a model; identifying a topic of the input text and a keyword in the input text; determining a risk level of the input text based on the topic and the keyword; and adjusting a probability distribution of an output token of the model based on the risk level, and determining an output text based on the probability distribution.

[0005] In some possible implementation manners of the present application, the adjusting the probability distribution of the output token of the model based on the risk level, and determining the output text based on the probability distribution includes: generating a candidate list of the output token through the model, wherein each first token in the candidate list corresponds to a probability value; performing a hash operation on each first token based on a preset key to obtain a hash value corresponding to each first token; determining a coefficient based on the risk level; determining a second token in the candidate list based on the first coefficient and the hash value; adjusting a probability value corresponding to the second token based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold value, wherein the probability threshold value is equal to a maximum value in the probability values corresponding to the first tokens in the candidate list.

[0006] In some possible implementation manners of the present application, the determining the second token in the candidate list based on the first coefficient and the hash value includes: comparing the size of the hash value corresponding to each first token with the first coefficient; and determining the first token corresponding to the hash value as the second token in response to the hash value being less than or equal to the first coefficient.

[0007] In some possible implementation of the present application, the determining the first coefficient based on the risk level comprises: determining the first coefficient corresponding to the risk level by querying a first correspondence table between risk levels and first coefficients.

[0008] In some possible implementation of the present application, the determining the risk level of the input text based on the topic and the keywords comprises: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient and the third risk coefficient.

[0009] In some possible implementation of the present application, the determining the first risk coefficient of the input text based on the topic comprises: obtaining a fourth risk coefficient corresponding to the topic according to a second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; in response to the existence of the preset word related to the topic in the input text, summing the preset value corresponding to the preset word and the fourth risk coefficient to obtain the first risk coefficient.

[0010] In some possible implementation of the present application, the determining the second risk coefficient of the input text based on the sensitive words in the keywords comprises: determining the number of occurrences and the weight corresponding to each sensitive word in the input text in a preset time period; determining a maximum number of occurrences based on the number of occurrences; and obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

[0011] In some possible implementation of the present application, the determining the third risk coefficient of the input text based on the named entities in the keywords comprises: determining the total length of the input text and the total number of named entities contained in the input text; determining the probability corresponding to each named entity based on the number of occurrences of the named entity in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities and the probability.

[0012] In the second aspect, the embodiments of the present application further provide a computer device, which comprises a processor and a memory, and the processor implements the text processing method as described above when executing a computer program stored in the memory.

[0013] In the third aspect, the embodiments of the present application further provide a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor to implement the text processing method as described above.

[0014] The text processing method provided by the embodiments of the present application receives an input text through a model; identifies a topic of the input text and a keyword in the input text; determines a risk level of the input text based on the topic and the keyword; and adjusts a probability distribution of output tokens of the model based on the risk level to obtain an output text. Through the above text processing method, the output text of the model can be modified based on the risk level of the text without affecting the quality of the text, thereby realizing the function of adding marks to the text and increasing the anti-tampering capability of the text. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 FIG. 1 is a schematic diagram of an application environment of the text processing method provided by the embodiments of the present application.

[0016] Figure 2 FIG. 2 is a flowchart of the text processing method provided by the embodiments of the present application.

[0017] Figure 3 FIG. 3 is a structural diagram of the text processing device in the embodiments of the present application.

[0018] Figure 4 FIG. 4 is a structural diagram of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0019] For the convenience of understanding, exemplary descriptions of some concepts related to the embodiments of the present application are given for reference.

[0020] It should be noted that “at least one” in the present application means one or more, and “multiple” means two or more than two. “And / or” describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms “first”, “second”, “third”, “fourth” and the like (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0021] Large model content watermarking is a technology that embeds hidden markers in generated content. The core idea of large model content watermarking is to insert specific characters or phrases into the text that do not affect the readability of the text, but can identify the source of the generated content. For example, invisible characters (such as \u200b) or specific symbols can be inserted at specific positions. Large model content watermarking can help determine the source of the generated content, making it easier to track and manage, especially when the source of the content is unclear. Large model content watermarking can also help protect copyright. For example, after embedding a watermark in the generated content, it can prevent the generated content from being misused or misused by others, protecting the rights of the original creator. Large model content watermarking can help with content review and supervision, especially in sensitive language tracking to ensure that the generated content meets relevant regulations and ethical standards. The presence of the watermark can provide additional information sources for content review. And let users identify which content is generated by AI, increase the transparency and credibility of information. This helps users understand and judge the source of the content. However, the embedded markers are prone to removal or affect the quality of the generated text.

[0022] Based on the above technical problems, the text processing method provided by the present application can adjust the probability distribution of the output word units of the model based on the risk level of the text without affecting the quality of the text to obtain the output text. The text processing method provided by the present application can add a watermark to the text based on the risk level of the text without affecting the quality of the text to adjust the output text, increase the anti-tampering ability of the text, and make the output text contain markers that are not easy to remove.

[0023] In order to better understand the text processing method and related products provided by the embodiments of the present application, first, the application scenario of the text processing method of the present application will be described.

[0024] Figure 1 is a schematic diagram of the application environment of the text processing method provided by the embodiments of the present application. As Figure 1As shown, in some embodiments of the present application, the text processing method can be applied in a computer device 1 (for example, a server) which can be in communication connection with a terminal device 2. The terminal device 2 can receive the input text of the user and send the input text to the model in the computer device 1 for processing, and output the corresponding output text through the model. For example, when the user inputs the input text of consulting travel through the terminal device 2 as "Please give a travel plan for summer vacation in Shenzhen". The terminal device 2 sends the input text to the model in the computer device 1 for processing. After the model processes the input text, it will output an output text containing a travel plan. In order to facilitate the identification of the generation source of the travel plan output by the model, the output text can be processed based on the risk level of the input text when the model generates the output text. For example, according to the risk level of the input text, the probability distribution of the output word units of the model is adjusted to obtain an output text containing a travel plan, which not only ensures that the semantics of the output text is unchanged, but also ensures the readability of the output text. The method for processing the text is specifically referred to Figure 2 the flowchart.

[0025] The terminal device 2 can be a computer, a server, a smart screen, a mobile phone, a tablet computer, a smart wearable device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, a netbook, etc. The specific type of electronic device is not limited in the embodiments of the present application.

[0026] The technical solutions of the present application will be described in detail below with specific embodiments. The specific embodiments described below can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments.

[0027] Figure 2 The flowchart of the text processing method provided in the embodiments of the present application can be executed by a computer device, and a model (for example, a large language model) can be deployed in the computer device. As shown in the figure, Figure 2 The method of the present embodiment can include: Step S201, receiving an input text through a model.

[0028] In some embodiments of the present application, when the user needs to generate text by using the model (for example, LLM), the input text can be sent to the computer device through the terminal device, and the corresponding output text is output after the input text is processed by the model in the computer device. In order to not affect the readability of the output text, the content of the output text can be adjusted according to the risk level of the input text when the model outputs the corresponding output text according to the input text.

[0029] In some embodiments of the present application, the input text can also be received directly by the computer device. For example, the input text input by the user is received through the input / output interface of the computer device.

[0030] In step S202, the topic of the input text and the keywords in the input text are identified.

[0031] In some embodiments of the present application, in order to perform risk assessment on the input text, the topic of the input text and the keywords in the input text are identified, and the input text is risk assessed by the topic and the keywords. The keywords at least include sensitive words and named entities.

[0032] In some embodiments of the present application, the topic of the input text can include finance, life, learning, etc. The input text can include at least one sentence. For example, the input text can be a sentence, or a short article containing multiple sentences. For example, when the input text “Please give me some suggestions for weekend travel” is received by the model, the topic of the input text is identified as life. For example, when the input text “What is the latest loan interest rate of XX Bank?” is received by the model, the topic of the input text is identified as finance. For example, when the input text “How to quickly learn a foreign language?” is received by the model, the topic of the input text is identified as learning.

[0033] In some embodiments of the present application, identifying the topic of the input text includes: extracting a context vector representation of the input text by a pre-trained language model; calculating a Term Frequency-Inverse Document Frequency (TF-IDF) and a topic saliency score according to the context vector representation; constructing a text semantic dependency graph based on the TF-IDF and the topic saliency score, and identifying the topic of the input text by the semantic dependency graph. Specifically, a pre-trained language model (such as BERT, RoBERTa) can be used to extract a context vector representation to obtain a deep semantic representation of the text. Based on the context vector, a TF-IDF and a topic saliency score are calculated. The TF-IDF is used to measure the importance of word frequency, and the topic saliency score is used to quantify the topic relevance of the word. Based on the TF-IDF and the topic saliency score, a semantic dependency graph is constructed. The topic saliency score is used to initialize or weight the nodes of the semantic dependency graph, so that the semantic dependency graph focuses more on the topic significant elements. The construction of the semantic dependency graph can depend on the parsing ability of the pre-trained language model (such as using its output to generate dependency relationships). The semantic dependency graph is analyzed (for example, PageRank, community detection or clustering algorithms are applied), and the topic of the text is output.

[0034] In some embodiments of the present application, a sensitive word library can be constructed in advance, different sensitive word libraries can be constructed according to different application scenarios, and if there are words in the input text that match the sensitive word library, the sensitive words in the input text can be identified.

[0035] In some embodiments of the present application, in the embodiments of the present application, a named entity (Named Entity, or simply entity) refers to an entity in a text that has a specific meaning or strong reference, usually including a person's name, a place name, an organization name, a date and time, a proper noun, etc. More extensive entities also include numbers, currencies, addresses, etc. Because it is difficult to effectively identify non-fixed format information to be encrypted (for example, names and addresses, etc.) through pattern matching, in order to improve the identification efficiency of the information to be encrypted in the information to be processed, the named entity recognition model can be used to identify the named entity in each sentence.

[0036] In embodiments of the present application, the named entity recognition model can also be referred to as a named entity recognition system. The model can extract the above-mentioned named entities from the input text, and can identify more categories of named entities according to business needs. The named entity recognition model can use models in existing related technologies, such as methods based on feature templates, neural network-based methods, etc. In an embodiment, the named entity recognition model can use a model of Word Embedding (word embedding) + LSTM (Long Short Term Memory, long short-term memory network) / BiLSTM (Bidirectional Long Short Term Memory, bidirectional long short-term memory network) + CRF (Conditional Random Field, conditional random field).

[0037] The named entity in the input text can be identified by a named entity recognition (Named Entity Recognition, NER) model. Specifically, the input text is divided into multiple division fields; each division field is assigned a part-of-speech tag; each division field after assigning the part-of-speech tag is subjected to dependency analysis; the boundaries of the input text are detected to obtain multiple sentences; and the named entities in each sentence are identified based on the results of the dependency analysis through the named entity model.

[0038] In embodiments of the present application, the division field refers to the basic unit of sentence division. For example, a single Chinese character can be used as the most basic division field, or a word after performing word segmentation processing can be used as the most basic division field. The word segmentation processing can use a word segmentation tool in existing related technologies, such as the Jieba word segmentation tool, etc. The goal of word segmentation processing is to divide the sentence into multiple words. In embodiments of the present application, after obtaining multiple division fields, in order to analyze the context semantics of the input text, each division field can be assigned a part-of-speech tag to label the part-of-speech of each division field. For example, verbs, nouns, adverbs, conjunctions, etc.

[0039] In the embodiments of the present application, after the part-of-speech tags are assigned, dependency analysis can be performed on each divided field to determine the relationship between the divided fields. The relationship between the divided fields includes a subject-predicate relationship (SBV), a head of the sentence (HED), an attribute-modifier relationship (ATT), an adverbial structure (ADV), a coordinate relationship (COO), a right adjunct relationship (RAD), and a verb-object structure (VOB). The subject-predicate relationship SBV represents the structural relationship between the subject and the predicate in a sentence. In Chinese syntactic analysis, SBV is used to describe the relationship between the subject (Subject) and its corresponding predicate verb (Predicate). The core relationship HED refers to the core component of a sentence, which is usually the center of the meaning of the entire sentence. In dependency syntactic analysis, HED is used to mark the core words or phrases that dominate other components of the sentence. The attribute-modifier relationship ATT represents the relationship between the modifier and the modified. In Chinese syntactic analysis, ATT is used to describe the case where an adjective (Attribute) modifies a noun (Modifier). The adverbial structure ADV refers to the role and position of adverbs or adverbials in a sentence. In dependency syntactic analysis, ADV is used to mark the words or phrases used as adverbials, which usually modify verbs, adjectives, or other adverbs. The coordinate relationship COO represents the parallel relationship between two or more same grammatical components. In Chinese syntactic analysis, COO is used to describe the relationship between parallel words, phrases, or clauses. The right adjunct relationship RAD refers to the relationship between the component attached on the right side and the center word. In dependency syntactic analysis, RAD is used to mark the components that are located on the right side of the center word and play a supplementary explanatory role. The verb-object relationship VOB represents the structural relationship between a verb and its direct object. In Chinese syntactic analysis, VOB is used to describe the relationship between a verb (Verb) and its direct object (Object).

[0040] In some embodiments, morphological reduction can also be performed on each divided field. By reducing the word form of each divided field to its basic form, semantic ambiguity caused by inflection of tense, person, number, etc. of the divided field can be eliminated, which helps to improve the accuracy of identifying information to be encrypted.

[0041] In the embodiments of the present application, the boundaries of each sentence in the input text are identified according to punctuation marks, and the text information is divided according to the identified boundaries of each sentence to obtain a plurality of complete sentences. The sentences can be segmented by searching for punctuation marks such as commas and periods in the text information. In some embodiments of the present application, the text can be preprocessed before identifying the topic of the text. Specifically, the preprocessing of the text includes tokenization, part-of-speech tagging, word frequency feature extraction, and stop word filtering of the original text.

[0042] In step S203, the risk level of the input text is determined based on the topic and the keywords.

[0043] In the embodiments of the present application, the input text needs to be classified into a risk level, and the corresponding output text is output based on the risk level of the input text, so that the text can be marked without embedding specific characters or phrases. Specifically, the risk level of the input text is determined based on the topic and the keywords, including: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient, and the third risk coefficient.

[0044] In the embodiments of the present application, the first risk coefficient of the input text is determined based on the topic, including: obtaining a fourth risk coefficient corresponding to the topic according to a second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; in response to the presence of the preset word related to the topic in the input text, summing the preset value corresponding to the preset word and the fourth risk coefficient to obtain the first risk coefficient.

[0045] Specifically, after identifying the topic of the input text, a correspondence table is constructed. The second correspondence table can be a topic and risk coefficient correspondence table, which describes the correspondence between the topic and the risk coefficient. For example, when the topic is finance, the corresponding risk coefficient is larger; when the topic is life, the corresponding risk coefficient is smaller. After determining the fourth risk coefficient corresponding to the topic according to the topic and risk coefficient correspondence table, a certain coefficient can be added to the fourth risk coefficient according to the presence of the preset word related to the topic in the input text, indicating that the probability of adjusting the output text obtained by the model according to the input text is larger. The preset word can be a word involved when the text under the topic is complained.

[0046] For example, when the theme is finance, the corresponding fourth risk coefficient is 0.7, and if the input text further includes preset words "exchange rate" and "stock market" related to the theme, the fourth risk coefficient is increased by a preset value 0.12, and the first risk coefficient 0.82 is obtained based on the sum of the fourth risk coefficient 0.7 and the preset value 0.12. When the theme is people's livelihood, the corresponding fourth risk coefficient is 0.5, and if the input text further includes preset words "medical treatment" and "education" related to the theme, the fourth risk coefficient is increased by a preset value 0.1, and the first risk coefficient 0.7 is obtained based on the sum of the fourth risk coefficient 0.5 and the preset value 0.2. When the theme is technology, the corresponding fourth risk coefficient is 0.3, and if the input text further includes preset words "patent" and "artificial intelligence" related to the theme, the fourth risk coefficient is increased by a preset value 0.1, and the first risk coefficient 0.4 is obtained based on the sum of the fourth risk coefficient 0.3 and the preset value 0.1.

[0047] In the embodiment of the present application, determining the second risk coefficient of the input text based on the sensitive words in the keywords comprises: determining the corresponding number of occurrences of each sensitive word in the input text in a preset time period and the corresponding weight; determining the maximum number of occurrences based on the number of occurrences; and obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

[0048] In the embodiment of the present application, the number of occurrences of each sensitive word in the input text in a preset time period (for example, 30 days) can be calculated. For example, the number of occurrences of the sensitive word is determined by the received conversation content of the complaint case. The ratio between the number of occurrences of each sensitive word in the preset time period and the number of occurrences of all sensitive words in the preset time period is calculated to obtain the weight corresponding to each sensitive word in the preset time period.

[0049] In the embodiment of the present application, the second risk coefficient is calculated based on the weight and the maximum number of occurrences by the following formula wherein, is the weight corresponding to each sensitive word in the preset time period, is the number of occurrences of each sensitive word in the input text in the preset time period, is the maximum number of occurrences, is the total number of all sensitive words in the input text.

[0050] In the embodiment of the present application, the maximum number of occurrences is the maximum value among the numbers of occurrences of all sensitive words in the preset time period.

[0051] ​​In an embodiment of the present application, determining the third risk coefficient of the input text based on named entities includes: determining the total length of the input text and the total number of all named entities; determining the probability corresponding to each named entity based on the number of times each named entity appears in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities, and the probability.

[0052] Specifically, the total length of the input text can be determined based on the total number of characters in the input text. For example, if the input text is a sentence consisting of ten characters, then the total length of the input text is ten. The probability corresponding to each named entity is determined based on the ratio between the number of times each named entity appears in the input text and the total number of named entities. The third risk coefficient is calculated based on the total length of the input text, the total number of named entities, and the probability using the following formula: , in, For each named entity, the probability is the total number of named entities, The total length of the input text.

[0053] In the embodiment of the present application, the risk level of the input text can be obtained by accumulating the sum of the first risk coefficient, the second risk coefficient, and the third risk coefficient; or the risk level of the input text can be obtained by calculating the first risk coefficient, the second risk coefficient, and the third risk coefficient using a weighted average calculation method. This application is not limited to this.

[0054] Step S204 : adjusting the probability distribution of the output word-units of the model based on the risk level, and determining the output text based on the probability distribution.

[0055] In some embodiments, after the model receives the input text, the model can generate the output text based on one output token at a time. For example, the model generates a first output token, a first output token and a third output token based on the input text, and then concatenates the first output token, the second output token and the third output token to obtain the output text. Specifically, in order to generate a coherent output text, the model predicts the next most likely output token. Specifically, before the model outputs the first output token, a candidate list of the first output token can be generated based on the input text, where each first token in the candidate list corresponds to a probability value. The first token with the maximum probability value in the candidate list can be selected as the first output token. The next most likely output token is predicted based on the input text and the first output token, and the model generates a candidate list of the second output token based on the input text and the first output token. The first token with the maximum probability value in the candidate list of the second output token is selected as the second output token. The next most likely output token is predicted based on the input text, the first output token and the second output token, and the model generates a candidate list of the third output token based on the input text, the first output token and the second output token. The first token with the maximum probability value in the candidate list of the third output token is selected as the third output token. The output text is obtained according to the first output token, the second output token and the third output token.

[0056] The above process of generating the input text by the model does not consider the risk level of the input text, cannot identify the output text, and cannot determine the source of the output text. The text processing method provided in the embodiments of the present application can adjust the probability distribution of the output token of the model according to the risk level of the input text to obtain the output text. Specifically, a candidate list of the output token is generated by the model, where each first token in the candidate list corresponds to a probability value; a corresponding hash value is obtained by performing a hash operation on the first token based on a preset key; a first coefficient is determined based on the risk level; a second token in the candidate list is determined based on the first coefficient and the hash value; the probability value corresponding to the second token is adjusted based on the risk level, and in response to the adjusted probability value being greater than or equal to a probability threshold, the output text is output, wherein the probability threshold is equal to the maximum value of the probability values corresponding to the first tokens in the candidate list.

[0057] In the embodiments of the present application, determining the second token in the candidate list based on the first coefficient and the hash value includes: comparing the size of the hash value corresponding to each first token with the first coefficient; and in response to the hash value being less than or equal to the first coefficient, determining the first token corresponding to the hash value as the second token.

[0058] In the embodiments of the present application, the determining the first coefficient based on the risk level comprises: determining the first coefficient corresponding to the risk level by querying a first correspondence table (such as Table 1) between the risk level and the first coefficient. For example, when the risk level of the text is determined to be low risk, the corresponding first coefficient is a first value; when the risk level of the text is determined to be medium risk, the corresponding first coefficient is a second value; and when the risk level of the text is determined to be high risk, the corresponding first coefficient is a third value. The first value is less than the second value, and the second value is less than the third value.

[0059] Table 1 For example, before the model outputs a first output token, a candidate list of the first output token can be generated based on the input text, wherein the first token in the candidate list corresponds to a probability value. The probability values corresponding to the first tokens in the candidate list are sorted in descending order, and the first tokens ranked in the top five can be selected to form the candidate list. Five hash values are obtained by performing a hash operation on the preset key and the five first tokens, a first coefficient is determined based on the risk level of the input text, and the five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first token corresponding to the hash value is selected as a second token. The probability value corresponding to the second token is adjusted based on the risk level, and the first token is output when the adjusted probability value is greater than or equal to a probability threshold.

[0060] The next most likely output token is predicted based on the input text and the second token. The model generates a candidate list of the second output token based on the input text and the second token, and selects the first tokens ranked in the top five to form the candidate list. Five hash values are obtained by performing a hash operation on the preset key and the five first tokens, a first coefficient is determined based on the risk level of the input text, and the five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first token corresponding to the hash value is selected as a third token. The probability value corresponding to the third token is adjusted based on the risk level, and the third token is output when the adjusted probability value is greater than or equal to a probability threshold.

[0061] The model generates a candidate list of the third output word units based on the input text, the second word unit and the third word unit, and selects the first word units ranked in the top five to form the candidate list. Five hash values are obtained by performing a hash operation based on a preset key and the five first word units. A first coefficient is determined based on a risk level of the input text. The five hash values are compared with the first coefficient respectively. If there is a hash value in the five hash values that is less than or equal to the first coefficient, the first word unit corresponding to the hash value is selected as a fourth word unit. The probability value corresponding to the fourth word unit is adjusted based on the risk level. If the adjusted probability value is greater than or equal to a probability threshold, the fourth word unit is output. The output text is obtained according to the second word unit, the third word unit and the fourth word unit.

[0062] In some embodiments of the present application, if there are multiple second word units whose adjusted probability values are greater than or equal to the probability threshold, the word unit corresponding to the largest probability value of the second word units greater than or equal to the probability threshold is selected as the second word unit.

[0063] In some embodiments of the present application, the output word unit can include a part of a character, a word or a phrase.

[0064] In some embodiments of the present application, the preset key is a unique key corresponding to the owner of the model, used to identify the generation source of the output text. For example, company A generates a first output text through the model, and company B generates a second output text through the model. In order to distinguish that the generation source of the first output text is the model owned by company A, and the generation source of the second output text is the model owned by company B, a hash value can be obtained by performing a hash operation on the preset key A and the first word unit when company A generates the first output text through the model; a hash value can be obtained by performing a hash operation on the preset key B and the first word unit when company B generates the second output text through the model.

[0065] In an embodiment of the present application, adjusting the probability value corresponding to the second word unit based on the risk level includes increasing a preset probability for the probability value corresponding to the second word unit based on the risk level. The risk level is negatively correlated with the preset probability. For example, if the risk level is low, the corresponding increased preset probability is a first probability; if the risk level is medium, the corresponding increased preset probability is a second probability; if the risk level is high, the corresponding increased preset probability is a third probability. The first probability is greater than the second probability, and the second probability is greater than the third probability. For example, if the risk level is low, the corresponding increased preset probability is 0.03%; if the risk level is medium, the corresponding increased preset probability is 0.02%; if the risk level is high, the corresponding increased preset probability is 0.01%.

[0066] For example, when the model receives the input text "Let the financial AI model be like a planner," it identifies the subject of the input text as finance and, based on the subject and keywords, determines the risk level of the input text as high. When the model generates the output text, it first generates a candidate list of output tokens, which may include 100 words. The five words with the highest probability values ​​are selected to form the following candidate list: "come and see" (42% probability), "think" (38% probability), "accordingly" (12% probability), "latest" (5% probability), and "conduct" (3% probability). "come and see" is the first token, with a probability of 42%; "think" is the first token, with a probability of 38%; "accordingly" is the first token, with a probability of 12%; "latest" is the first token, with a probability of 5%; and "conduct" is the first token, with a probability of 3%. A hash operation is performed on the five first tokens using a preset key (e.g., 12345) to obtain the corresponding hash value. Based on the risk level, the first coefficient is determined to be a third value of 40. The candidate list identifies "think (hash value 37)" and "latest" (hash value 18) as the second word. The probability value of the second word "think" is increased by 1% based on the 42% probability value, and the probability value of the second word "latest" is increased by 1% based on the 5% probability value. The adjusted probability value of the second word "think" is 43%, which is greater than the maximum probability value of 42% in the candidate list. The adjusted probability value of the second word "latest" is 6%, which is less than the maximum probability value of 42% in the candidate list. The first output word determined by this model is "think."

[0067] In an embodiment of the present application, if in some application scenarios, after a computer device outputs text information to a user of a terminal device through a model, in order to prevent the user from using the text information by abnormal means, but then believing that there is a problem with the text information and embarrassing the model owner, the output text can be verified by the marks in the text. Specifically, verifying the output text includes: counting the frequency of occurrence of output word units after adjusting the probability distribution in the output text, and confirming the size of the frequency and the frequency threshold; if the frequency is greater than or equal to the frequency threshold, determining that the output text contains the mark; if the frequency is less than the frequency threshold, determining that the output text does not contain the mark. In an embodiment of the present application, the frequency of occurrence of the mark in the output text is counted by calculating the frequency of occurrence of the output word units after adjusting the probability distribution in the output text.

[0068] In the embodiments of the present application, the output text is verified by calculating the frequency of the output word elements after the adjustment of the probability distribution appearing in the output text. For example, if the model output text is "a certain country announced that it would implement a new trade policy in 2025, which would significantly affect the global energy market", the frequency of the output word elements "policy" and "significantly" after the adjustment of the probability distribution in the above output text is counted. If the frequency is greater than or equal to the frequency threshold (for example, 0.95), it is determined that the output text contains the label; if the frequency is less than the frequency threshold (for example, 0.95), it is determined that the output text does not contain the label. For another example, if the model output text is "it is recommended to go to the park for a walk on the weekend, which is suitable for relaxing", the frequency of the exclamation mark after the adjustment of the probability distribution in the above output text is counted. If the frequency is greater than or equal to the frequency threshold (for example, 0.7), it is determined that the output text contains the label; if the frequency is less than the frequency threshold (for example, 0.7), it is determined that the output text does not contain the label.

[0069] It should be noted that the frequency threshold is related to the risk level of the text and the first coefficient. When it is determined that the risk level of the text is low risk, the corresponding first coefficient is a first value, and the corresponding frequency threshold is a first threshold value; when it is determined that the risk level of the text is medium risk, the corresponding first coefficient is a second value, and the corresponding frequency threshold is a second threshold value; when it is determined that the risk level of the text is high risk, the corresponding first coefficient is a third value, and the corresponding frequency threshold is a third threshold value. The first value is less than the second value, and the second value is less than the third value; the first threshold value is less than the second threshold value, and the second threshold value is less than the third threshold value. This is the key to detecting and verifying the labels in the output text of the model.

[0070] Figure 3 is a structural diagram of a text processing apparatus provided in the embodiments of the present application. The text processing apparatus 300 can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the text processing apparatus 300 can be stored in the memory of the computer device and executed by at least one processor to perform the text processing function (see Figure 2 for details).

[0071] In the embodiments, the text processing apparatus 300 can be divided into a plurality of functional modules according to the functions it performs. The functional modules can include a receiving module 301, an identifying module 302, a determining module 303, and a processing module 304. The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory. In the embodiments, the text processing apparatus 300 can be used to implement the text processing method as shown in Figure 2 . As shown in Figure 3 , the text processing apparatus 300 is applied to a computer device (such as a server or a terminal). Figure 1The text processing apparatus 300 comprises: a receiving module 301 configured to receive an input text via a model; an identifying module 302 configured to identify a topic of the input text and keywords in the input text; a processing module 304 configured to adjust a probability distribution of output tokens of the model based on the risk level to obtain an output text.

[0072] The processing module 304 adjusts the probability distribution of the output tokens of the model based on the risk level and determines the output text based on the probability distribution, including: generating a candidate list of the output tokens via the model, wherein each first token in the candidate list corresponds to a probability value; performing a hash operation on each first token based on a preset key to obtain a hash value corresponding to each first token; determining a coefficient based on the risk level; determining a second token in the candidate list based on the first coefficient and the hash value; adjusting a probability value corresponding to the second token based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold, wherein the probability threshold is equal to a maximum value among the probability values corresponding to the first tokens in the candidate list.

[0073] The processing module 304 determines the second token in the candidate list based on the first coefficient and the hash value, including: comparing the size of the hash value corresponding to each first token with the first coefficient; and determining the first token corresponding to the hash value as the second token in response to the hash value being less than or equal to the first coefficient.

[0074] The processing module 304 determines the first coefficient based on the risk level, including: determining the first coefficient corresponding to the risk level by querying a first correspondence table between risk levels and first coefficients.

[0075] The determining module 303 determines the risk level of the input text based on the topic and the keywords, including: determining a first risk coefficient of the input text based on the topic; determining a second risk coefficient of the input text based on sensitive words in the keywords; determining a third risk coefficient of the input text based on named entities in the keywords; and determining the risk level of the input text based on the first risk coefficient, the second risk coefficient, and the third risk coefficient.

[0076] The determination module 303 determines the first risk coefficient of the input text based on the topic, including: obtaining the fourth risk coefficient corresponding to the topic according to the second correspondence table, and confirming the preset words related to the topic and the preset values ​​corresponding to the preset words; in response to the presence of preset words related to the topic in the input text, summing the preset values ​​corresponding to the preset words and the fourth risk coefficient to obtain the first risk coefficient.

[0077] The determination module 303 determines the second risk coefficient of the input text based on the sensitive words in the keywords, including: determining the number of occurrences and the corresponding weight of each sensitive word in the input text within a preset time period; determining the maximum number of occurrences based on the number of occurrences; and obtaining the second risk coefficient based on the weight and the maximum number of occurrences.

[0078] The determination module 303 determines the third risk coefficient of the input text based on the named entities in the keywords, including: determining the total length of the input text and the total number of named entities contained in the input text; determining the probability corresponding to each named entity based on the number of times each named entity appears in the input text and the total number of named entities; and calculating the third risk coefficient of the input text based on the total length of the input text, the total number of named entities and the probability.

[0079] Figure 4 This is a schematic diagram of the structure of the computer device provided in the embodiment of the present application. Figure 4 As shown, the computer device 1 can be a server, for example, a central server, an edge server, or a local server in a local data center. Each computer device 1 can include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication interface 101, the memory 102, and the I / O interface 104 via the bus 105.

[0080] The communication module 101 can include a wired communication module and / or a wireless communication module. The wired communication module can provide one or more of the following wired communication solutions: universal serial bus (USB), Controller Area Network (CAN) bus, etc. The wireless communication module can provide one or more of the following wireless communication solutions: wireless fidelity (Wi-Fi), bluetooth (BT), mobile communication network, frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0081] The memory 102 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The random access memory can be directly readable and writable by the processor 103, and can be used to store executable programs (e.g., machine instructions) of an operating system or other programs that are running, and can also be used to store data of users and applications, etc. The random access memory can include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0082] The non-volatile memory can also store executable programs and store data of users and applications, etc., and can be loaded in advance into the random access memory for direct reading and writing by the processor 103. The non-volatile memory can include a magnetic disk storage device, a flash memory.

[0083] The memory 102 is configured to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions, which when executed by the processor 103, can implement the text processing method executed on the computer device 1.

[0084] In other embodiments, the computer device 1 further comprises an external memory interface for connecting an external memory, so as to extend the storage capability of the computer device 1.

[0085] The processor 103 can include one or more processing units, for example: the processor 103 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0086] The processor 103 provides computing and control capabilities, for example, the processor 103 is used to execute a computer program stored in the memory 102.

[0087] The I / O interface 104 is used to provide a channel for user input or output, for example, the I / O interface 104 can be used to connect various input and output devices, such as a mouse, a keyboard, a touch device, a display screen, etc., so that the user can enter information, or make the information visualized.

[0088] The bus 105 is used to provide a communication channel between the communication module 101, the memory 102, the processor 103, and the I / O interface 104 in the computer device 1.

[0089] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the computer device 1. In other embodiments of the present application, the computer device 1 can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0090] The embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, and the method implemented when the program instructions are executed can refer to the method in each of the above embodiments of the present application.

[0091] The computer-readable storage medium may be an internal memory of the electronic device described in the above embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device.

[0092] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, applications required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0093] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0094] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0095] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0097] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A text processing method, characterized in that: The method comprises: Receive input text through the model; identifying the subject of the input text and keywords in the input text; determining a risk level of the input text based on the subject and the keywords; Adjusting the probability distribution of the output word of the model based on the risk level, and determining the output text based on the probability distribution, including: generating a candidate list of the output word through the model, wherein each first word in the candidate list corresponds to a probability value; performing a hash operation on each of the first word based on a preset key to obtain a hash value corresponding to each first word; determining a first coefficient based on the risk level; determining a second word in the candidate list based on the first coefficient and the hash value; adjusting the probability value corresponding to the second word based on the risk level, and outputting the output text in response to the adjusted probability value being greater than or equal to a probability threshold.

2. The text processing method according to claim 1, wherein: The probability threshold is equal to the maximum value of the probability values ​​corresponding to the first word in the candidate list.

3. The text processing method according to claim 2, wherein: Determining the second word-gram in the candidate list based on the first coefficient and the hash value includes: comparing the hash value corresponding to each of the first word elements with the first coefficient; In response to the hash value being less than or equal to the first coefficient, a first word-gram corresponding to the hash value is determined to be the second word-gram.

4. The text processing method according to claim 2, wherein: Determining the first coefficient based on the risk level includes: The first coefficient corresponding to the risk level is determined by querying a first correspondence table between risk levels and first coefficients.

5. The text processing method according to claim 1, wherein: Determining the risk level of the input text based on the subject and the keywords includes: determining a first risk factor for the input text based on the subject matter; determining a second risk factor of the input text based on sensitive words in the keywords; determining a third risk factor of the input text based on the named entities in the keywords; A risk level of the input text is determined based on the first risk factor, the second risk factor, and the third risk factor.

6. The text processing method according to claim 5, wherein: Determining a first risk factor of the input text based on the topic includes: Obtaining a fourth risk coefficient corresponding to the topic according to the second correspondence table, and confirming a preset word related to the topic and a preset value corresponding to the preset word; In response to the presence of a preset word related to the topic in the input text, a preset value corresponding to the preset word is summed with the fourth risk coefficient to obtain the first risk coefficient.

7. The text processing method according to claim 5, wherein: Determining the second risk factor of the input text based on the sensitive words in the keywords includes: Determine the number of occurrences and the corresponding weight of each sensitive word in the input text within a preset time period; determining a maximum number of occurrences based on the number of occurrences; The second risk coefficient is obtained based on the weight and the maximum number of occurrences.

8. The text processing method according to claim 5, wherein: Determining a third risk factor of the input text based on the named entities in the keywords includes: Determining the total length of the input text and the total number of named entities contained in the input text; Determining a probability corresponding to each named entity based on the number of times each named entity appears in the input text and the total number of the named entities; A third risk coefficient of the input text is calculated based on the total length of the input text, the total number of the named entities, and the probability.

9. A computer device, characterized in that: The computer device includes a processor and a memory, and when the processor executes the computer program stored in the memory, the text processing method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the text processing method according to any one of claims 1 to 8 is implemented.

Images