Text auditing method and device, storage medium and program product

Abstract

The invention discloses a text auditing method and device, a storage medium and a program product, and relates to the technical field of computers. The method comprises the steps that an input text and a knowledge base are obtained, the knowledge base comprises text information corresponding to at least two sample texts, and the at least two sample texts comprise at least one violation text and at least one compliance text; performing fine-grained splitting on the input text to obtain at least one suspicious text fragment in the input text; according to the at least one suspicious text fragment and a knowledge base, obtaining a target violation text and a target compliance text corresponding to the at least one suspicious text fragment; and generating an audit result of the input text through a large language model according to the input text, the at least one suspicious text fragment, and the target violation text and the target compliance text corresponding to the at least one suspicious text fragment. According to the method, the text auditing of the input text is converted into the text auditing of the suspicious text fragment, so that the accuracy of the text auditing is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a text auditing method and device, a storage medium and a program product. BACKGROUND

[0002] When a user makes a public comment on an application, the text input by the user will be subjected to text auditing, the text that fails to pass the auditing will be intercepted, and the text that passes the auditing will be displayed on the interface of the application.

[0003] In the related art, a large language model internally contains a large amount of knowledge, and based on a prompt word, a to-be-audited text is subjected to deep understanding and reasoning, the semantic relationship of each word in the to-be-audited text is recognized, and it is determined whether each word in the to-be-audited text is in violation, so as to determine whether the to-be-audited text is in violation, and obtain an auditing result of the to-be-audited text.

[0004] However, the above method is prone to cover up the violation content in the to-be-audited text, affecting the accuracy of the auditing result. SUMMARY

[0005] Embodiments of the present application provide a text auditing method, device, storage medium and program product. The technical solutions provided by the embodiments of the present application are as follows: According to an aspect of the embodiments of the present application, a text auditing method is provided, and the method comprises: obtaining an input text and a knowledge base, the knowledge base comprising text information corresponding to at least two sample texts respectively, the at least two sample texts comprising at least one violation text and at least one compliance text; performing fine-grained splitting on the input text to obtain at least one suspicious text segment in the input text, the suspicious text segment being a text segment in the input text that is suspected to contain violation content; obtaining target violation texts and target compliance texts corresponding to the at least one suspicious text segment according to the at least one suspicious text segment and the knowledge base, the target violation text corresponding to the suspicious text segment being a violation text in the at least one violation text that has the highest similarity with the suspicious text segment, and the target compliance text corresponding to the suspicious text segment being a compliance text in the at least one compliance text that has the highest similarity with the suspicious text segment; generating an auditing result of the input text by a large language model according to the input text, the at least one suspicious text segment, the target violation texts and the target compliance texts corresponding to the at least one suspicious text segment, the auditing result of the input text being used to indicate that the input text is a violation text or the input text is a compliance text.

[0006] According to an aspect of some embodiments of the present application, a text auditing device is provided, the device comprising: a text acquisition module configured to acquire an input text and a knowledge base, the knowledge base comprising text information corresponding to at least two sample texts, the at least two sample texts comprising at least one rule violation text and at least one rule compliance text; a text splitting module configured to perform fine-grained splitting on the input text to obtain at least one suspicious text segment in the input text, the suspicious text segment being a text segment in the input text suspected of containing rule violation content; a knowledge base extraction module configured to obtain, according to the at least one suspicious text segment and the knowledge base, target rule violation texts and target rule compliance texts corresponding to the at least one suspicious text segment, the target rule violation text corresponding to the suspicious text segment being a rule violation text in the at least one rule violation text having the highest similarity with the suspicious text segment, and the target rule compliance text corresponding to the suspicious text segment being a rule compliance text in the at least one rule compliance text having the highest similarity with the suspicious text segment; a result generation module configured to generate, by a large language model, an auditing result of the input text according to the input text, the at least one suspicious text segment, the target rule violation texts and the target rule compliance texts corresponding to the at least one suspicious text segment, the auditing result of the input text being configured to indicate that the input text is a rule violation text or the input text is a rule compliance text.

[0007] According to an aspect of some embodiments of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned text auditing method.

[0008] According to an aspect of some embodiments of the present application, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being loaded and executed by a processor to implement the above-mentioned text auditing method.

[0009] According to an aspect of some embodiments of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being loaded and executed by a processor to implement the above-mentioned text auditing method.

[0010] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects: By performing fine-grained splitting on the input text, the input text is split into multiple text segments, so that suspicious text segments suspected of containing illegal content can be screened from the multiple text segments, thereby converting the text review of the input text into the text review of the suspicious text segments, fully reducing the review difficulty of the text review, helping to improve the review efficiency of the text review, and by obtaining the target illegal text and the target compliant text corresponding to the suspicious text segment from the knowledge base, and performing text review on the suspicious text segment based on the target illegal text and the target compliant text, the accuracy of the text review can be improved, and the reliability of the review result is enhanced. Compared with the method of directly performing text review according to the entire input text in the related art, the technical method provided in the present application effectively identifies potential illegal information in the input text through the suspicious text segment, avoids false positives or missed judgments on the input text in the case where the input text includes illegal text segments and compliant text segments, effectively ensures that the illegal content in the input text will not be concealed, and improves the review accuracy and review effect of the text review. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a schematic diagram of a computer system provided by an embodiment of the present application; Figure 2 is a flowchart of a text review method provided by an embodiment of the present application; Figure 3 is a process schematic diagram of generating a review result of an input text based on a prompt text provided by an embodiment of the present application; Figure 4 is a schematic diagram of a review process of a text review provided by an embodiment of the present application; Figure 5 is a schematic diagram of a background interface of a text review platform provided by an embodiment of the present application; Figure 6 is a schematic diagram of a display interface of an application provided by an embodiment of the present application; Figure 7 is a process schematic diagram of fine-grained combination of text segments after word segmentation of an input text provided by an embodiment of the present application; Figure 8 is a process schematic diagram of screening suspicious text segments from at least one text segment provided by an embodiment of the present application; Figure 9 is a schematic diagram of a retrieval process of a target text provided by an embodiment of the present application; Figure 10 is a schematic diagram of distributed cluster processing of review records provided by an embodiment of the present application; Figure 11 is a schematic diagram of an analysis process of illegal types of illegal texts provided by an embodiment of the present application; Figure 12 is a schematic diagram of the overall process of text auditing provided by an embodiment of the present application; Figure 13 is a block diagram of a text auditing apparatus provided by an embodiment of the present application; Figure 14 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0013] Reference is made to Figure 1 which shows a schematic diagram of a computer system provided by an embodiment of the present application. The computer system can include a terminal device 10 and a server 20.

[0014] The number of terminal devices 10 can be one or more. The terminal device 10 can be an electronic device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a game console, an e-book reader, a multimedia playback device, a wearable device, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc.

[0015] A client of a target application can be installed in the terminal device 10, the target application having a function of text auditing. A user can input a text in the target application, and the target application outputs an auditing result of the text, the auditing result being used to indicate whether the input text is in violation. The type of the target application is not limited in the present application. Optionally, the target application can be an application that needs to be downloaded and installed, or an application that can be used immediately after being clicked, which is not limited in the present application.

[0016] The server 20 is used to provide background services for the client of the target application installed and running in the terminal device 10. For example, the server 20 can be a background server of the target application described above. The server 20 can be a physical server, or a server cluster composed of multiple servers, or a cloud computing service center. Optionally, the server 20 provides background services for target applications in multiple terminal devices 10 at the same time. The terminal device 10 and the server 20 can communicate with each other through a network.

[0017] In the embodiments of the present application, input text and a knowledge base are obtained, the knowledge base includes text information corresponding to at least two sample texts respectively, and the at least two sample texts include at least one rule violation text and at least one rule compliance text. The present application is used to determine whether the input text is rule violation according to the sample texts in the knowledge base, and obtain an audit result of the input text. First, fine-grained splitting is performed on the input text to obtain at least one suspicious text segment in the input text, the suspicious text segment being a text segment in the input text that is suspected to contain rule violation content. Then, according to the at least one suspicious text segment and the knowledge base, target rule violation texts and target rule compliance texts corresponding to the at least one suspicious text segment are obtained, the target rule violation text corresponding to the suspicious text segment being the rule violation text with the highest similarity to the suspicious text segment among the at least one rule violation text, and the target rule compliance text corresponding to the suspicious text segment being the rule compliance text with the highest similarity to the suspicious text segment among the at least one rule compliance text. Finally, whether the at least one suspicious text segment is rule violation is determined according to the target rule violation texts and the target rule compliance texts corresponding to the at least one suspicious text segment respectively, and the audit result of the input text is obtained. The audit result of the input text is generated by a large language model according to the input text, the at least one suspicious text segment, the target rule violation texts and the target rule compliance texts corresponding to the at least one suspicious text segment respectively, and is used to indicate that the input text is a rule violation text or a rule compliance text.

[0018] Reference is made to Figure 2 which shows a flowchart of a text audit method provided by an embodiment of the present application. The execution subject of each step of the method can be a computer device. The method can include at least one of the following steps 210-240: Step 210, obtaining input text and a knowledge base, the knowledge base including text information corresponding to at least two sample texts respectively, and the at least two sample texts including at least one rule violation text and at least one rule compliance text.

[0019] The input text can be any text, and the present application does not limit the text length of the input text or the text content of the input text.

[0020] In some embodiments, the text information corresponding to the sample text includes the sample text and label information of the sample text, and the label information of the rule violation text is used to indicate the rule violation type of the rule violation text.

[0021] The text information corresponding to the at least two sample texts respectively includes text information corresponding to at least one compliant text respectively and text information corresponding to at least one non-compliant text respectively. The text information corresponding to the compliant text includes the compliant text and label information of the compliant text, the label information of the compliant text is a compliant label, the compliant label is used to indicate that the compliant text does not contain non-compliant content, and the compliant text is a text that passes the text review. The text information corresponding to the non-compliant text includes the non-compliant text and label information of the non-compliant text, the label information of the non-compliant text is a non-compliant label and a non-compliant type, the non-compliant label is used to indicate that the non-compliant text contains non-compliant content, and the non-compliant text is a text that fails the text review. The non-compliant label can be used to indicate the specific non-compliant type of the non-compliant text.

[0022] In some embodiments, the knowledge base further includes an inverted index list and a vector index library.

[0023] The inverted index list includes inverted lists corresponding to at least two sample vocabularies respectively and position information corresponding to the at least two sample vocabularies respectively, the at least two sample vocabularies are extracted from at least two sample texts, the inverted list corresponding to each sample vocabulary includes at least one sample text associated with the sample vocabulary, and the position information corresponding to each sample vocabulary includes position information of the sample vocabulary in the inverted index list and position information of the sample vocabulary in the at least one sample text associated with the sample vocabulary.

[0024] Optionally, the at least two sample vocabularies can include all vocabularies extracted from the at least two sample texts, or can include vocabularies present in a preset vocabulary library among all vocabularies extracted from the at least two sample texts.

[0025] The inverted lists corresponding to the at least two sample vocabularies respectively are arranged in sequence in the inverted index list according to a certain sorting rule, for example, the at least two sample vocabularies can be sorted in alphabetical order. The position information of the sample vocabulary in the inverted index list is used to indicate the sorting position of the sample vocabulary in the inverted index list, for example, if the sample vocabulary is sorted first in the inverted index list, the position information of the sample vocabulary in the inverted index list can be 01, and if the sample vocabulary is sorted eleventh in the inverted index list, the position information of the sample vocabulary in the inverted index list can be 11.

[0026] The inverted list corresponding to each sample vocabulary includes at least one sample text associated with the sample vocabulary, and the at least one sample text associated with each sample vocabulary refers to at least one sample text in which each sample vocabulary is located, that is, the at least one sample text associated with each sample vocabulary refers to at least one sample text containing the sample vocabulary. The position information of the sample vocabulary in the sample text associated with the sample vocabulary includes the sequential position and the offset of the sample vocabulary in the sample text associated with the sample vocabulary. The sequential position of the sample vocabulary in the sample text is used to indicate the position of the sample vocabulary in the sample text associated with the sample vocabulary, for example, if the sample text is "Elasticsearch is powerful", the sequential position of the sample vocabulary "Elasticsearch" in the sample text is 0, the sequential position of the sample vocabulary "is" in the sample text is 1, and the sequential position of the sample vocabulary "powerful" in the sample text is 2. The offset of the sample vocabulary in the sample text associated with the sample vocabulary is used to indicate the start and end positions of the characters of the sample vocabulary in the sample text associated with the sample vocabulary, for example, if the sample text is "Elasticsearch is powerful", the start position of the sample vocabulary "Elasticsearch" in the sample text is 0, and the end position is 12, the start position of the sample vocabulary "is" in the sample text is 13, and the end position is 14, the start position of the sample vocabulary "powerful" in the sample text is 15, and the end position is 22.

[0027] The position information corresponding to each sample vocabulary can further include the term frequency of the sample vocabulary in the at least one sample text associated with the sample vocabulary, and the term frequency of the sample vocabulary in the sample text associated with the sample vocabulary is used to indicate the number of occurrences of the sample vocabulary in the sample text associated with the sample vocabulary, for example, if the sample text is "Elasticsearch is powerful", the term frequency of the sample vocabulary "Elasticsearch", "is" and "powerful" in the sample text is 1.

[0028] The vector index library includes the feature vectors corresponding to the at least two sample texts respectively. Optionally, each sample text can be converted into a 312-dimensional feature vector by using an ALBERT model for storage.

[0029] The specific construction process of the knowledge base can refer to the following embodiments, which will not be described in detail here.

[0030] In step 220, fine-grained splitting is performed on the input text to obtain at least one suspicious text segment in the input text, and the suspicious text segment is a text segment in which the input text is suspected to contain illegal content.

[0031] The fine-grained splitting refers to splitting the input text into multiple words, and then determining at least one suspicious text segment in the input text according to the characteristics of each word. Each suspicious text segment can be a text segment containing one word in the input text, or a text segment containing multiple consecutive words in the input text.

[0032] The suspicious text segment is a text segment in the input text that is preliminarily judged to possibly contain illegal content, that is, the suspicious text segment can also not be a text segment in the input text that contains illegal content, and therefore, it is necessary to further determine whether the suspicious text segment is illegal in the subsequent step.

[0033] The specific fine-grained splitting process performed on the input text can refer to the following embodiments, which will not be described in detail here.

[0034] In step 230, at least one suspicious text segment and a knowledge base are used to obtain target illegal text and target compliant text corresponding to each suspicious text segment, respectively. The target illegal text corresponding to the suspicious text segment is the illegal text with the highest similarity to the suspicious text segment in the at least one illegal text. The target compliant text corresponding to the suspicious text segment is the compliant text with the highest similarity to the suspicious text segment in the at least one compliant text.

[0035] For each suspicious text segment in the at least one suspicious text segment, the illegal text with the highest similarity to the suspicious text segment is filtered from the at least one illegal text in the knowledge base to obtain the target illegal text corresponding to the suspicious text segment. And the compliant text with the highest similarity to the suspicious text segment is filtered from the at least one compliant text in the knowledge base to obtain the target compliant text corresponding to the suspicious text segment.

[0036] The specific process of filtering the target illegal text and the target compliant text can refer to the following embodiments, which will not be introduced here.

[0037] In step 240, a large language model is used to generate an audit result of the input text according to the input text, the at least one suspicious text segment, the target illegal text and the target compliant text corresponding to each suspicious text segment, respectively. The audit result of the input text is used to indicate whether the input text is illegal or compliant.

[0038] The large language model here can be any publicly available large language model, such as a natural language model based on a transformer structure trained on a large amount of data, and the amount of data can reach a sample level of hundreds of millions or more. The present application does not limit this.

[0039] In some embodiments, step 240 includes at least one of sub-steps 241-242.

[0040] In sub-step 241, a prompt text is generated according to the input text, the at least one auditing rule, the at least one suspicious text segment, the target violation text and the target compliance text corresponding to the at least one suspicious text segment respectively, and the violation type of the target violation text corresponding to the at least one suspicious text segment respectively, and the auditing rule is used to indicate the text violation type.

[0041] Optionally, the prompt text is generated by the prompt text generator according to the input text, the at least one auditing rule, the at least one suspicious text segment, the target violation text and the target compliance text corresponding to the at least one suspicious text segment respectively, and the violation type of the target violation text corresponding to the at least one suspicious text segment respectively, and the violation type of the target violation text corresponding to the at least one suspicious text segment is obtained from the knowledge base after the target violation text is determined.

[0042] Optionally, the at least one auditing rule can be a template content in the prompt text configuration template, and then the input text, the at least one suspicious text segment, the target violation text and the target compliance text corresponding to the at least one suspicious text segment respectively, and the violation type of the target violation text corresponding to the at least one suspicious text segment are input to the prompt text generator, and the prompt text generator outputs the prompt text. Optionally, the at least one auditing rule can also be a custom input auditing rule, and then the input text, the at least one auditing rule, the at least one suspicious text segment, the target violation text and the target compliance text corresponding to the at least one suspicious text segment respectively, and the violation type of the target violation text corresponding to the at least one suspicious text segment are input to the prompt text generator, and the prompt text generator outputs the prompt text.

[0043] Figure 3 The left part of FIG. 1 shows a process diagram of generating an auditing result of an input text based on a prompt text, Figure 3 The left part of FIG. 1 shows a process diagram of generating an auditing result of an input text based on a prompt text,

[0044] Optionally, the optimal prompt text can also be generated by the prompt text optimization iteration framework.

[0045] In sub-step 242, the auditing result of the input text is generated by the large language model according to the prompt text.

[0046] The prompt text is input to the large language model, and the large language model outputs the auditing result of the input text. For example, Figure 3As shown, the prompt text defines one of the two words "violation" or "normal", and the large language model outputs "violation" after reasoning according to the prompt text, indicating that the input text in the prompt text is a violation text.

[0047] Alternatively, the large language model can also output the audit analysis text of the input text and the audit result of the input text. The audit analysis text of the input text is the reasoning analysis text of the audit result of the input text, which is used to analyze whether at least one suspicious text segment in the input text hits at least one audit rule. As shown Figure 3 As shown, the large language model can also output the reasoning analysis process and the audit result to enhance the credibility of the audit result. If the audit result of the input text is used to indicate that the input text is a violation text, the audit analysis text of the input text is used to describe the reasoning analysis process of at least one suspicious text segment in the input text hitting the audit rule, and the type of violation of the input text can be determined according to the audit rule hit by at least one suspicious text segment in the input text. If the audit result of the input text is used to indicate that the input text is a compliant text, the audit analysis text of the input text is used to describe the reasoning analysis process of at least one suspicious text segment in the input text not hitting the audit rule.

[0048] By generating a prompt text first and inputting the prompt text into the large language model, the large language model can help the large language model understand the prompt text according to the text format of the prompt text, better understand the user's audit requirements, and help improve the efficiency of text audit.

[0049] The technical scheme provided by the embodiments of the present application can reduce the difficulty of text audit by performing fine-grained splitting on the input text to split the input text into multiple text segments, so that suspicious text segments suspected of containing violation content can be screened from the multiple text segments, thereby converting the text audit of the input text into the text audit of the suspicious text segments. The accuracy of the text audit can be improved, and the reliability of the audit result can be enhanced. Compared with the method of directly performing text audit on the entire input text in the related art, the technical method provided by the present application effectively identifies the potential violation information in the input text through the suspicious text segments, avoids false positives or missed judgments of the input text in the case where the input text includes violation text segments and compliant text segments, effectively ensures that the violation content in the input text will not be concealed, and improves the audit precision and audit effect of the text audit.

[0050] Figure 4A schematic diagram of an audit process of text audit is shown, the audit process of text audit includes the following audit nodes: an audit node of receiving input text, an audit node of obtaining word after text segmentation, an audit node of obtaining text segment after fine-grained combined word, an audit node of extracting suspicious text segment, an audit node of obtaining target text through hybrid retrieval, an audit node of constructing prompt text, and an audit node of long language model reasoning output audit result. For a long time, algorithm developers and business personnel can only communicate through model output. If the adjustment of audit rules is involved, the business personnel can only submit orders, and then the algorithm developers can implement the orders. This undoubtedly causes the rise of communication cost. Therefore, by building a business workflow platform, the business personnel can directly understand the existing algorithm process and strategy configuration in a visual manner, which significantly improves the development efficiency. Through each audit node, the output of each audit step can be observed, so that for the examples of false positives or false negatives, the problem node can be traced back, facilitating the next optimization and iteration. The logic reasoning and multi-level audit are realized, and flexibility is provided for subsequent manual supervision and adjustment, thereby greatly improving the overall audit efficiency and reducing the labor burden.

[0051] Figure 5 A schematic diagram of a background interface of a text audit platform is shown. In the background interface 500 of the text audit platform, a text audit bar 501, an audit state bar 502, and an audit result bar 503 are displayed. The text content of each text is displayed in the text audit bar 501. Whether each text has been audited is displayed in the audit state bar 502. The audit result of the audited text is displayed in the audit result bar 503. If the text has not been audited, the audit result is not displayed in the corresponding audit result bar 503.

[0052] Figure 6 A schematic diagram of a display interface of an application is shown. The application is a music application. The display interface 600 of the application is a music comment interface. In the display interface 600 of the application, a user can input text (music comment) in a text input box 602. After receiving the input text, the client of the application performs text audit on the input text. If the input text passes the text audit, the input text is displayed in the display interface 600 of the application, as shown in text 601. If the input text does not pass the text audit, the input text is not displayed in the display interface 600 of the application.

[0053] In some embodiments, step 220 includes at least one of sub-steps 221-223.

[0054] Sub-step 221, performing segment extraction on the input text to obtain at least one text segment in the input text.

[0055] The segment extraction is used to extract at least one text segment in the input text, each text segment comprising at least one character in the input text, the character in each text segment comprising but not limited to a literal character, a numeric character, a punctuation character, a symbol character, and the like. For example, there can be a text segment comprising only one numeric character, and there can be a text segment comprising only one punctuation character. There can be repeated characters between different text segments, or there can be no repeated characters.

[0056] In some embodiments, a word segmentation processing is performed on the input text to obtain at least one word contained in the input text, each word comprising at least one character. At least one word is sequentially obtained in a left-to-right order to obtain at least one text segment, each text segment comprising a number of characters less than or equal to a preset maximum threshold.

[0057] Optionally, a word segmentation tool is used to perform a word segmentation processing on the input text to obtain at least one word contained in the input text, each word comprising at least one character. Generally, one or more literal characters are segmented into one word, one or more numeric characters are segmented into one word, one label character is segmented into one word, and one symbol character is segmented into one character.

[0058] At least one word is sequentially obtained in a left-to-right order to obtain a text segment comprising a first word, a text segment comprising the first and second words, a text segment comprising the first and second and third words, and so on, until a number of characters in a text segment comprising the first and second and third and fourth and fifth and sixth and seventh and eighth and ninth and tenth words is greater than the preset maximum threshold, and the obtaining of the text segment with the first word as the starting word is stopped, to obtain n text segments with the first word as the starting word, n being a positive integer. Then a text segment is obtained with the second word as the starting word to obtain m text segments with the second word as the starting word, m being a positive integer. Text segments with each word as the starting word are sequentially obtained to obtain at least one text segment corresponding to the input text. Each text segment corresponding to the input text comprises at least one word.

[0059] The specific value of the preset maximum threshold is set by an algorithm developer according to the text review requirements, and the present application does not make any limitation. Exemplarily, the preset maximum threshold can be 10, and each text segment comprises a number of characters less than or equal to 10. If a text segment is obtained with the first word as the starting word, a number of characters in a text segment comprising the first and second and third and fourth and fifth and sixth and seventh and eighth and ninth and tenth words is greater than 10, and a number of characters in a text segment comprising the first and second and third and fourth and fifth and sixth and seventh and eighth and ninth words is less than or equal to 10, then the obtaining of the text segment with the first word as the starting word is stopped after the text segment comprising the first and second and third and fourth and fifth and sixth and seventh and eighth and ninth words is obtained.

[0060] Figure 7 The process diagram of input text segmentation and fine-grained combination into text segments is shown. The input text is "vibrant archer village, bustling and prosperous abandoned city, quiet and deep magic forest, desert and barren warrior tribe, these places have left our footprints, I will always be here waiting for your return, group: 5000? Ba long ⑦ ⑦ 4". The input text is segmented to obtain a plurality of words contained in the input text: "vibrant", "of", "archer village", "of", "bustling", "prosperous", "of", "abandoned", "city", "of", "quiet", "deep", "of", "magic", "forest", "of", "desert", "barren", "of", "warrior", "tribe", "of", "these", "places", "all", "once", "left", "you and me", "of", "footprints", "of", "I", "will", "always", "here", "wait", "for", "your", "return", "of", "group", ":", "5000", "?", "Ba", "long", "⑦", "⑦", "4".

[0061] From the above words, the first word "vibrant" is taken as the starting word in the order from left to right, and the text segments containing "vibrant", "vibrant" and "of", "vibrant", "of", "archer village", "vibrant", "of", "archer village", "of", "bustling" are obtained. Since the number of characters in the text segment containing "vibrant", "of", "archer village", "of", "bustling" and "prosperous" is greater than 10, the text segment is stopped. Then the second word is taken as the starting word to obtain the text segment, and so on, until the last word is taken as the starting word to obtain the text segment, and at least one text segment corresponding to the input text is obtained.

[0062] By segmenting the input text and forming at least one text segment corresponding to the input text in a fine-grained manner, the text information in the input text is fully mined, providing multiple possible ways of text segments, enhancing the understanding of the input text, and facilitating the subsequent identification of potential violation information in the input text based on the text segment.

[0063] In sub-step 222, at least one suspicious score corresponding to at least one text segment is obtained by a text scoring model according to at least one text segment. The suspicious score corresponding to the text segment is used to measure the probability of containing violation content in the text segment.

[0064] Exemplarily, the text scoring model can be a trained DPCNN (Deep Pyramid Convolutional Neural Networks for Text Categorization) model. The text scoring model is configured to score the likelihood of the text segment containing the rule violation content, and obtain a suspicious score corresponding to the text segment. The higher the suspicious score corresponding to the text segment, the higher the likelihood of the text segment containing the rule violation content, i.e., the higher the possibility of the text segment violating the rules.

[0065] In some embodiments, if the text segment includes a sensitive word in the pre-set sensitive word library, the suspicious score corresponding to the text segment is 1.

[0066] In sub-step 223, at least one suspicious text segment is obtained according to the suspicious scores respectively corresponding to the at least one text segment.

[0067] At least one suspicious text segment is filtered from the at least one text segment according to the suspicious scores respectively corresponding to the at least one text segment. The suspicious text segment is a text segment whose suspicious score meets the score requirement.

[0068] By combining the word library to extract the text segments from the input text and using the machine learning scoring mechanism to score the text segments, the probability of each text segment containing the rule violation content can be intuitively observed, so that the potential rule violation text segment in the input text can be identified according to the suspicious score, the rule violation content in the input text can be avoided to be covered by the compliance content, and the accuracy of the text review can be improved.

[0069] In some embodiments, in the process of obtaining the suspicious scores respectively corresponding to the at least one text segment, the text segments whose suspicious scores are greater than or equal to a first threshold value are obtained, and the filtered at least one text segment is obtained. According to the suspicious scores respectively corresponding to the filtered at least one text segment from large to small, the text segments corresponding to at least one target suspicious score in the front are obtained, and the at least one suspicious text segment is obtained. Among them, there is no repeated word between the text segments corresponding to the at least one target suspicious score, and there is no repeated word between the at least one text segment before each target suspicious score and the text segment corresponding to the target suspicious score.

[0070] The specific value of the first threshold value is set by the algorithm developer according to the text review requirement, and the present application does not make any limitation. Exemplarily, the first threshold value is 0.5, and the text segments whose suspicious scores are greater than or equal to 0.5 are obtained from the suspicious scores respectively corresponding to the at least one text segment, and the filtered at least one text segment is obtained.

[0071] Among the at least one text segment after screening, there may be repeated words between different text segments, or there may be no repeated words between different text segments. Then, it is necessary to remove the text segments with relatively low suspicious scores that have repeated words in the at least one text segment after screening, and retain the text segment with the highest suspicious score that has repeated words, so as to obtain at least one suspicious text segment. The suspicious text segment is the text segment corresponding to the target suspicious score.

[0072] Arrange the at least one text segment after screening in descending order according to the suspicious scores corresponding to the at least one text segment after screening, and obtain a text list. Determine the first text segment in the text list as the suspicious text segment, and remove the text segments in the text list that have repeated words with the first text segment, so as to obtain the text list after the first filtering. Determine the second text segment in the text list after the first filtering as the suspicious text segment, and remove the text segments in the text list after the first filtering that have repeated words with the second text segment, so as to obtain the text list after the second filtering. Continue to perform the operations of extracting suspicious text segments and filtering the text segments that have repeated words with the already determined suspicious text segments until no suspicious text segment can be extracted. Determine the already determined suspicious text segments as the at least one suspicious text segment. There are no repeated words between the at least one suspicious text segment, and there are no text segments with repeated words with the suspicious text segment before the suspicious text segment in the text list.

[0073] Through the above idea of simulating manual review, the text segment with the largest suspicious score is preferentially retained, and the remaining text segments cannot have repeated words with the already retained text segments, so that the finally obtained suspicious text segment is the text segment with the highest probability of containing illegal content, which can improve the accuracy and precision of text review. And it avoids retaining the text segments with repeated words with the suspicious text segments, thus avoiding increasing the computational burden of the model and improving the review efficiency of text review.

[0074] Figure 8 The process diagram of screening suspicious text segments from at least one text segment is shown. The suspicious scores corresponding to the at least one text segment are obtained through a text scoring model, as shown in Figure 8 the second figure in. Retain the text segments with suspicious scores greater than or equal to 0.5 in the at least one text segment, as shown in Figure 8 the third figure in. Arrange in descending order of suspicious scores. First, determine the text segment corresponding to the maximum value of the suspicious score as the suspicious text segment, that is, determine "Group: Five 0o0o? Bar Nine" as the suspicious text segment, and then remove the text segments in the retained text segments that have repeated words with the suspicious text segment. Figure 8All text fragments except "Group: Five 0o0o? Ba Jiu" will be removed, resulting in a suspicious text fragment: "Group: Five 0o0o? Ba Jiu".

[0075] In some embodiments, step 230 includes at least one of sub-steps 231 to 233.

[0076] Sub-step 231, for each suspicious text fragment in at least one suspicious text fragment, obtain at least one suspicious word in the suspicious text fragment.

[0077] Based on the aforementioned word segmentation record and fine-grained combination record of the input text, at least one suspicious word in the suspicious text fragment is obtained. A suspicious word is a word included in the suspicious text fragment. Exemplarily, for the suspicious text fragment "Group: Five 0o0o? Ba Jiu", at least one suspicious word includes "Group", ":", "Five 0o0o", "?", "Ba", "Jiu".

[0078] Sub-step 232, in the case where any suspicious word in at least one suspicious word exists in the inverted index list, perform an inverted retrieval on the suspicious text fragment to obtain the target violation text and the target compliance text corresponding to the suspicious text fragment.

[0079] Optionally, if at least two sample words in the inverted index list include all the words extracted from at least two sample texts, then there must be at least one suspicious word in the inverted index list. Then, directly perform an inverted retrieval on the suspicious text fragment to obtain the target violation text corresponding to the suspicious text fragment and the target compliance text corresponding to the suspicious text fragment.

[0080] Optionally, if at least two sample words in the inverted index list include the words existing in the preset vocabulary among all the words extracted from at least two sample texts, then there may or may not be a suspicious word in the inverted list. Therefore, different retrieval steps need to be taken for the two possible situations respectively. In the case where any suspicious word in at least one suspicious word exists in the inverted index list, perform sub-step 232 to perform an inverted retrieval on the suspicious text fragment to obtain the target violation text and the target compliance text corresponding to the suspicious text fragment. In the case where no suspicious word in at least one suspicious word exists in the inverted index list, perform sub-step 233 to perform a vector retrieval on the suspicious text fragment to obtain the target violation text and the target compliance text corresponding to the suspicious text fragment.

[0081] The specific process of the inverted retrieval can be referred to the following embodiments and will not be introduced here.

[0082] Sub-step 233, in the case that there is no at least one suspicious vocabulary in the inverted index list, performing vector retrieval on the suspicious text segment to obtain the target violation text and the target compliance text corresponding to the suspicious text segment.

[0083] The specific process of vector retrieval can refer to the following embodiments, which will not be introduced here.

[0084] By giving priority to inverted retrieval, the advantages of inverted retrieval can be fully utilized in the case of meeting the conditions, and the target violation text and the target compliance text with the highest similarity to the suspicious text segment can be extracted as much as possible, thereby improving the accuracy of text review. And in the case where the condition is not met, the vector retrieval mode is used to retrieve the target violation text and the target compliance text with the highest similarity to the suspicious text segment through the feature vector, and the vector retrieval is used as a bottom-up measure to ensure the smooth execution of the text review scheme and avoid the interruption of the scheme execution due to the condition not being met. And it provides a sample reference for large language models, so that the inference analysis process of the large language model is more in line with business standards, reduces the deviation caused by pre-training knowledge, enhances the reliability of the review result, and reduces the illusion problem of the large language model.

[0085] In some embodiments, sub-step 232 includes at least one of sub-steps 2321-2324.

[0086] Sub-step 2321, from the inverted index list, obtain at least one suspicious sample vocabulary existing in the inverted index list corresponding to the at least one suspicious vocabulary respectively. The inverted list corresponding to the suspicious sample vocabulary includes at least one similar sample text associated with the suspicious sample vocabulary, and the similar sample text is a sample text associated with the suspicious sample vocabulary.

[0087] First, at least one suspicious sample vocabulary existing in the inverted index list is obtained, and then the inverted list corresponding to at least one suspicious sample vocabulary is obtained from the inverted index list, that is, at least one sample text associated with at least one suspicious sample vocabulary is obtained. The sample text associated with the suspicious sample vocabulary can be referred to as a similar sample text.

[0088] If at least two sample vocabularies in the inverted index list include all the vocabularies extracted from at least two sample texts, then the at least one suspicious sample vocabulary is the at least one suspicious vocabulary. If at least two sample vocabularies in the inverted index list include all the vocabularies extracted from at least two sample texts, and the at least one suspicious sample vocabulary is a vocabulary existing in the preset vocabulary library.

[0089] In sub-step 2322, for each similar sample text in the inverted list corresponding to the at least one suspicious sample vocabulary respectively, a similarity between the suspicious text segment and the similar sample text is calculated to obtain a first similarity score corresponding to the similar sample text.

[0090] The similar sample text is a sample text containing one or more suspicious vocabularies in the suspicious text segment. Therefore, the similar sample text is a sample text associated with the suspicious text segment. The similarity between each suspicious text segment and the similar sample text needs to be calculated, so that the target violation text and the target compliance text corresponding to the suspicious text segment can be determined according to the first similarity score corresponding to each similar sample text in the inverted list corresponding to the at least one suspicious sample vocabulary respectively.

[0091] Exemplarily, the at least one suspicious sample vocabulary in the suspicious text segment includes a suspicious sample vocabulary a and a suspicious sample vocabulary b. The inverted list corresponding to the suspicious sample vocabulary a includes a similar sample text 1 and a similar sample text 2 associated with the suspicious sample vocabulary a. The inverted list corresponding to the suspicious sample vocabulary b includes a similar sample text 3 and a similar sample text 4 associated with the suspicious sample vocabulary b. Then, the similarity between the suspicious text segment and the similar sample text 1, the similar sample text 2, the similar sample text 3 and the similar sample text 4 respectively needs to be calculated.

[0092] In some embodiments, the first similarity score corresponding to the similar sample text is obtained according to the term frequency of the at least one suspicious sample vocabulary in the similar sample text, the length of the similar sample text, the average length of the at least two sample texts, the number of texts in the inverted list corresponding to the at least one suspicious sample vocabulary respectively, and the number of texts of the at least two sample texts.

[0093] The term frequency of the suspicious sample vocabulary in the similar sample text refers to the number of occurrences of the suspicious sample vocabulary in the similar sample text. If the similar sample text is not the sample text associated with the suspicious sample vocabulary, the term frequency of the suspicious sample vocabulary in the similar sample text is 0. The length of the similar sample text refers to the number of vocabularies contained in the similar sample text. The average length of the at least two sample texts refers to the average number of vocabularies contained in the at least two sample texts. The number of texts in the inverted list corresponding to the suspicious sample vocabulary refers to the number of similar sample texts associated with the suspicious sample vocabulary. The number of texts of the at least two sample texts can also be understood as the total number of sample texts contained in the knowledge base.

[0094] Exemplarily, the BM25 algorithm can be used to calculate the similarity between the suspicious text segment and the similar sample text. The first similarity score corresponding to the similar sample text can be represented as:

[0095]

[0096] wherein, represents the similarity between the suspicious text segment and the similar sample text (the first similarity score corresponding to the similar sample text), d represents the similar sample text, q represents the suspicious text segment, t represents the suspicious sample vocabulary in the suspicious text segment, represents the word frequency of the suspicious sample vocabulary t in the similar sample text d, represents the length of the similar sample text d, represents the average length of the at least two sample texts, represents the number of texts in the at least two sample texts, represents the number of texts in the inverted list corresponding to the suspicious sample vocabulary t. and is an adjustable parameter, and exemplarily, is 1.2, is 0.75.

[0097] The BM25 algorithm adjusts the values of and so that the first similarity score is smoother, avoiding the excessive influence of some high-frequency words on the scoring of the similar sample text.

[0098] By using the above method to calculate the similarity between the suspicious text segment and the similar sample text, the common words in the suspicious text segment can be suppressed, and the rare words in the suspicious text segment can be highlighted, so that the violation content in the suspicious text segment can be found, thereby the relevance between each similar sample text and the suspicious text segment can be more accurately evaluated, and the accuracy of text retrieval can be improved.

[0099] In sub-step 2323, in the case that there are violations in the inverted list corresponding to at least one suspicious sample vocabulary, the similar violation text corresponding to the maximum value in the first similarity score of at least one similar violation text is determined as the target violation text corresponding to the suspicious text segment, and the similar violation text is the violation text in the inverted list corresponding to at least one suspicious sample vocabulary.

[0100] Exemplarily, if the similar sample text 1 and the similar sample text 2 are violation samples, the similar sample text 1 and the similar sample text 2 belong to suspicious violation samples, and the similar violation text corresponding to the maximum value in the first similarity score corresponding to the similar sample text 1 and the first similarity score corresponding to the similar sample text 2 is determined as the target violation text corresponding to the suspicious text segment. For example, if the first similarity score corresponding to the similar sample text 1 is greater than the first similarity score corresponding to the similar sample text 2, the similar sample text 1 is determined as the target violation text.

[0101] Sub-step 2324, in the case that there is a compliance sample in the inverted list corresponding to each of the at least one suspicious sample vocabulary, the similar compliance text corresponding to the maximum value in the first similarity scores of the similar compliance texts corresponding to each of the at least one suspicious sample vocabulary is determined as the target compliance text corresponding to the suspicious text segment.

[0102] For example, if similar sample text 3 and similar sample text 4 are compliance samples, similar sample text 3 and similar sample text 4 belong to suspicious compliance samples, and the similar compliance text corresponding to the maximum value in the first similarity scores of similar sample text 3 and similar sample text 4 is determined as the target compliance text corresponding to the suspicious text segment. For example, if the first similarity score of similar sample text 3 is greater than the first similarity score of similar sample text 4, similar sample text 3 is determined as the target compliance text.

[0103] By determining at least one similar sample text associated with the suspicious text segment according to the suspicious sample vocabulary, and then determining the target violation text and the target compliance text from the at least one similar sample text, the search range of the target violation text and the target compliance text is narrowed, the data volume required for similarity calculation is reduced, and the efficiency of text review can be improved.

[0104] In some embodiments, sub-step 232 further includes at least one of sub-steps 2325-2326.

[0105] Sub-step 2325, in the case that there is no violation text in the inverted list corresponding to each of the at least one suspicious sample vocabulary, vector retrieval is performed on the suspicious text segment to obtain the target violation text corresponding to the suspicious text segment.

[0106] For example, if similar sample text 1, similar sample text 2, similar sample text 3, and similar sample text 4 are all compliance samples, the target violation text cannot be retrieved from the inverted list corresponding to each of the at least one suspicious sample vocabulary, and therefore, vector retrieval needs to be performed on the suspicious text segment to obtain the target violation text corresponding to the suspicious text segment.

[0107] Sub-step 2326, in the case that there is no compliance text in the inverted list corresponding to each of the at least one suspicious sample vocabulary, vector retrieval is performed on the suspicious text segment to obtain the target compliance text corresponding to the suspicious text segment.

[0108] Exemplarily, if the similar sample text 1, the similar sample text 2, the similar sample text 3 and the similar sample text 4 are all violation samples, the target compliance text cannot be retrieved from the inverted list corresponding to the at least one suspicious sample vocabulary, and therefore, the vector retrieval needs to be performed on the suspicious text segment to obtain the target compliance text corresponding to the suspicious text segment.

[0109] By sampling the vector retrieval when the condition is not met, the target violation text and the target compliance text with the highest similarity to the suspicious text segment are retrieved through the feature vector, and the vector retrieval serves as a backup measure to ensure the smooth execution of the text auditing scheme and avoid interruption of the scheme execution due to the condition not being met.

[0110] In some embodiments, the sub-step 233 includes at least one of the sub-steps 2331-2334.

[0111] The sub-step 2331 performs feature extraction on the input text to obtain a feature vector of the input text.

[0112] Optionally, the ALBERT model can be used to perform feature extraction on the input text to convert the input text into a 312-dimensional feature vector.

[0113] The sub-step 2332 calculates the similarity between the feature vector of the input text and the feature vectors of the at least two sample texts respectively to obtain the second similarity scores of the at least two sample texts respectively.

[0114] The second similarity score of the sample text refers to the similarity between the feature vector of the input text and the feature vector of the sample text. Exemplarily, the cosine similarity can be used to calculate the similarity between the feature vector of the input text and the feature vector of the sample text, and the second similarity score of the sample text can be represented as:

[0115]

[0116]

[0117] wherein, the second similarity score of the sample text, the feature vector of the input text, the feature vector of the sample text, the dot product of the feature vector of the input text and the feature vector of the sample text. the L2 norm of the feature vector of the input text, the L2 norm of the feature vector of the sample text. The parameter of the $i$-th dimension in the feature vector of the input text The parameter of the $i$-th dimension in the feature vector of the sample text, where $n$ represents the number of dimensions of the feature vector.

[0118] The value range of the cosine similarity is usually between $[-1, 1]$. In specific applications, the vectors can be normalized before use, so that the similarity score will be between 0 and 1. The closer the value is to 1, the more similar the two vectors are in direction, which also means that the corresponding data semantics are closer. This method can effectively capture the semantic correlation between data without relying on keyword matching on the surface of the data.

[0119] Sub-step 2333: Determine the target violation text corresponding to the suspicious text segment as the violation text corresponding to the maximum value among the second similarity scores corresponding to at least one violation text.

[0120] Sub-step 2334: Determine the target compliant text corresponding to the suspicious text segment as the compliant text corresponding to the maximum value among the second similarity scores corresponding to at least one compliant text.

[0121] By obtaining the target violation text and target compliant text corresponding to the suspicious text segment through the above method, the large language model can perform reasoning and analysis on the suspicious text segment based on sample cases, guiding the large language model to think specifically about the suspicious text segment, and avoiding the problems of false killing or missed judgment caused by reasoning and analyzing the entire input text.

[0122] Figure 9 Fig. shows a schematic diagram of the retrieval process of the target text. The suspicious text segment is "Group: Five 0o0o? Bar nine". According to the suspicious text segment, a hybrid retrieval is performed. Index retrieval is preferred first, and vector retrieval is used when the target text cannot be obtained by index retrieval, obtaining the target compliant text "The bar friends in the group all love" and the target violation text "Group number: xxxxxx". Thus, based on the input text, the suspicious text segment, the target violation text and target compliant text corresponding to the suspicious text segment, a prompt text as shown in Figure 9 can be generated.

[0123] In some embodiments, the construction process of the knowledge base includes at least one of the following steps A1 to A6.

[0124] Step A1: According to at least one review record corresponding to at least two sample texts, determine at least one compliant text among the at least two sample texts, and determine the violation type corresponding to at least one violation text, so as to obtain the label information corresponding to the at least two sample texts.

[0125] The at least one audit record corresponding to each of the at least two sample texts is obtained by the distributed engine, which can solve the problem of insufficient memory and hard disk when cleaning massive data on a single machine, and has been deployed to a timing scheduling platform to continuously and uninterruptedly obtain the latest data for statistics.

[0126] The determination result of the at least one audit record corresponding to each sample text is counted, and the case of conflict before and after is eliminated to ensure that the final result state of the sample is determined through a voting mechanism.

[0127] Figure 10 A schematic diagram of distributed cluster processing audit records is shown. The sample texts include "this song is really good" and "this song XXXX". In the audit record of "this song is really good", the number of audit passes is 100 and the number of audit violations is 1, so the final result of "this song is really good" is a compliant text. In the audit record of "this song XXXX", the number of audit passes is 1 and the number of audit violations is 100, so the final result of "this song XXXX" is a violation text. Therefore, it can be determined that the label information of "this song is really good" is a compliance label, and the label information of "this song XXXX" is a violation label.

[0128] The violation type of the violation text is analyzed by the deep learning model, so that the large language model can better understand the violation point in the subsequent process, help the large language model understand the text audit business, and make correct reasoning and judgment.

[0129] Figure 11 A schematic diagram of the analysis process of the violation type of the violation text is shown. For example, the violation texts include "do you want XXX" and "this song XXXX". After analyzing the violation type by the deep learning model, the violation types of "do you want XXX" and "this song XXXX" are obtained.

[0130] The label information corresponding to each of the at least two sample texts is obtained. The label information of the compliant text is a compliance label, and the label information of the violation text is a violation label and a violation type.

[0131] Step A2, feature extraction is performed on the at least two sample texts respectively to obtain feature vectors corresponding to the at least two sample texts respectively.

[0132] Optionally, each sample text can be converted into a 312-dimensional feature vector by an ALBERT model.

[0133] Step A3, at least two sample vocabularies are extracted from the at least two sample texts.

[0134] Optionally, the at least two sample vocabularies can include all vocabularies extracted from the at least two sample texts, or can include vocabularies existing in a preset vocabulary library from all vocabularies extracted from the at least two sample texts.

[0135] Step A4, performing aggregation processing on the sample texts in which the at least two sample vocabularies are respectively located, to obtain at least two inverted lists corresponding to the at least two sample vocabularies respectively.

[0136] For each sample vocabulary in the at least two sample vocabularies, the sample text in which the sample vocabulary is located is determined as the sample text associated with the sample vocabulary, to obtain at least one sample text associated with the sample vocabulary. According to the at least one sample text associated with the sample vocabulary, an inverted list corresponding to the sample vocabulary is constructed.

[0137] Step A5, generating an inverted index list according to the inverted lists corresponding to the at least two sample vocabularies respectively and the position information corresponding to the at least two sample vocabularies respectively.

[0138] Step A6, obtaining a knowledge base according to the label information corresponding to the at least two sample texts respectively, the feature vectors corresponding to the at least two sample texts respectively, and the inverted index list.

[0139] Figure 12 The schematic diagram of the overall process of text review is shown. First, the distributed cluster processing mode is used to process massive data, including obtaining historical compliance samples and historical violation samples from historical review records. In the knowledge base, a deep learning model is used to add violation types to the violation samples. Finally, a pre-trained vector model and an inverted index technology are used to construct an index retrieval library for compliance samples and violation samples. The input text is finely divided to solve the case of adversarial samples, and a deep learning model is used to score the text segments obtained by division to obtain suspicious text segments in advance. Then, hybrid retrieval is used on the suspicious text segments to obtain target violation texts and target compliance texts corresponding to the suspicious text segments. Using these dynamic information, the final prompt text of the large language model is constructed. Finally, the obtained prompt text is input into the large language model to obtain the final review result.

[0140] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0141] Please refer to Figure 13 which shows a block diagram of a text review apparatus provided by an embodiment of the present application. The apparatus has the functions of implementing the above-mentioned text review method. The functions can be implemented by hardware, or by executing corresponding software by hardware. The apparatus can be the computer device introduced above, or can be arranged in the computer device. For example, Figure 13As shown, the apparatus 1300 can include a text acquisition module 1310, a text splitting module 1320, a knowledge base extraction module 1330, and a result generation module 1340.

[0142] The text acquisition module 1310 is configured to acquire input text and a knowledge base, wherein the knowledge base includes text information corresponding to at least two sample texts, and the at least two sample texts include at least one non-compliant text and at least one compliant text.

[0143] The text splitting module 1320 is configured to perform fine-grained splitting on the input text to obtain at least one suspicious text segment in the input text, wherein the suspicious text segment is a text segment in the input text that is suspected to contain non-compliant content.

[0144] The knowledge base extraction module 1330 is configured to obtain, according to the at least one suspicious text segment and the knowledge base, target non-compliant text and target compliant text corresponding to the at least one suspicious text segment, wherein the target non-compliant text corresponding to the suspicious text segment is a non-compliant text in the at least one non-compliant text that has the highest similarity with the suspicious text segment, and the target compliant text corresponding to the suspicious text segment is a compliant text in the at least one compliant text that has the highest similarity with the suspicious text segment.

[0145] The result generation module 1340 is configured to generate, by a large language model, an audit result of the input text according to the input text, the at least one suspicious text segment, the target non-compliant text and the target compliant text corresponding to the at least one suspicious text segment, wherein the audit result of the input text is used to indicate that the input text is a non-compliant text or the input text is a compliant text.

[0146] In some embodiments, the text splitting module 1320 is configured to: perform segment extraction on the input text to obtain at least one text segment in the input text; obtain, by a text scoring model, suspicious scores corresponding to the at least one text segment, wherein the suspicious score corresponding to the text segment is used to measure the probability of the text segment containing non-compliant content; obtain the at least one suspicious text segment according to the suspicious scores corresponding to the at least one text segment.

[0147] In some embodiments, the text splitting module 1320 is configured to: perform word segmentation processing on the input text to obtain at least one word included in the input text, wherein each word includes at least one character; According to the at least one vocabulary, at least one continuous distributed vocabulary is sequentially obtained in a left-to-right order, and at least one text segment is obtained, wherein the number of characters contained in each text segment is less than or equal to a preset maximum threshold value.

[0148] In some embodiments, the text splitting module 1320 is configured to: Among the suspicious scores corresponding to the at least one text segment, a text segment with a suspicious score greater than or equal to a first threshold value is obtained, and at least one filtered text segment is obtained. According to the suspicious scores corresponding to the at least one filtered text segment from large to small, a text segment corresponding to at least one target suspicious score in a front order is obtained, and at least one suspicious text segment is obtained. Among the at least one target suspicious score, there is no repeated vocabulary between the text segments corresponding to the at least one target suspicious score, and there is no repeated vocabulary between at least one text segment before each target suspicious score and the text segment corresponding to the target suspicious score.

[0149] In some embodiments, the knowledge base further includes an inverted index list, the inverted index list includes at least two sample vocabularies corresponding to an inverted list and position information of the at least two sample vocabularies, the at least two sample vocabularies are extracted from the at least two sample texts, the inverted list corresponding to each sample vocabulary includes at least one sample text associated with the sample vocabulary, and the position information corresponding to each sample vocabulary includes position information of the sample vocabulary in the inverted index list and position information of the sample vocabulary in the at least one sample text associated with the sample vocabulary; the knowledge base extraction module 1330 is configured to: For each suspicious text segment in the at least one suspicious text segment, at least one suspicious vocabulary in the suspicious text segment is obtained. In the case that any suspicious vocabulary in the at least one suspicious vocabulary exists in the inverted index list, inverted retrieval is performed on the suspicious text segment, and target violation text and target compliance text corresponding to the suspicious text segment are obtained. In the case that the at least one suspicious vocabulary does not exist in the inverted index list, vector retrieval is performed on the suspicious text segment, and target violation text and target compliance text corresponding to the suspicious text segment are obtained.

[0150] In some embodiments, the knowledge base extraction module 1330 is configured to: From the inverted index list, obtain an inverted list corresponding to each suspicious sample vocabulary in the at least one suspicious vocabulary present in the inverted index list, the inverted list corresponding to the suspicious sample vocabulary including at least one similar sample text, the similar sample text being a sample text associated with the suspicious sample vocabulary; For each similar sample text in the inverted list corresponding to each suspicious sample vocabulary, calculate a similarity between the suspicious text segment and the similar sample text to obtain a first similarity score corresponding to the similar sample text; In a case where there is a violation sample in the inverted list corresponding to each suspicious sample vocabulary, determine, as a target violation text corresponding to the suspicious text segment, a similar violation text corresponding to a maximum value in the first similarity scores of at least one similar violation text, the similar violation text being a violation text in the inverted list corresponding to each suspicious sample vocabulary; In a case where there is a compliance sample in the inverted list corresponding to each suspicious sample vocabulary, determine, as a target compliance text corresponding to the suspicious text segment, a similar compliance text corresponding to a maximum value in the first similarity scores of at least one similar compliance text, the similar compliance text being a compliance text in the inverted list corresponding to each suspicious sample vocabulary.

[0151] In some embodiments, the knowledge base extraction module 1330 is configured to: obtain the first similarity score corresponding to the similar sample text according to a term frequency of the at least one suspicious sample vocabulary in the similar sample text, a text length of the similar sample text, an average text length of the at least two sample texts, a number of texts in the inverted list corresponding to each suspicious sample vocabulary, and a number of texts of the at least two sample texts.

[0152] In some embodiments, the knowledge base extraction module 1330 is configured to: In a case where there is no violation text in the inverted list corresponding to each suspicious sample vocabulary, perform vector retrieval on the suspicious text segment to obtain a target violation text corresponding to the suspicious text segment; In a case where there is no compliance text in the inverted list corresponding to each suspicious sample vocabulary, perform vector retrieval on the suspicious text segment to obtain a target compliance text corresponding to the suspicious text segment.

[0153] In some embodiments, the knowledge base further includes a vector index library, the vector index library including feature vectors corresponding to the at least two sample texts respectively; and the knowledge base extraction module 1330 is configured to: perform feature extraction on the input text to obtain a feature vector of the input text; calculate similarities between the feature vector of the input text and feature vectors respectively corresponding to the at least two sample texts, to obtain second similarity scores respectively corresponding to the at least two sample texts; determine, as a target non-compliant text corresponding to the suspicious text segment, a non-compliant text corresponding to a maximum value in the second similarity scores respectively corresponding to the at least one non-compliant text; determine, as a target compliant text corresponding to the suspicious text segment, a compliant text corresponding to a maximum value in the second similarity scores respectively corresponding to the at least one compliant text.

[0154] In some embodiments, the text information corresponding to the sample text further includes label information of the sample text, and the label information of the non-compliant text is used to indicate a non-compliance type of the non-compliant text; and the result generation module 1340 is configured to: generate a prompt text according to the input text, at least one audit rule, the at least one suspicious text segment, a target non-compliant text and a target compliant text respectively corresponding to the at least one suspicious text segment, and a non-compliance type of the target non-compliant text, the audit rule being used to indicate a text non-compliance type; generate, by the large language model, an audit result of the input text according to the prompt text.

[0155] In some embodiments, the apparatus 1300 further includes a knowledge base construction module, which is configured to: determine the at least one compliant text in the at least two sample texts and determine a non-compliance type of the at least one non-compliant text according to at least one audit record respectively corresponding to the at least two sample texts, to obtain label information respectively corresponding to the at least two sample texts; perform feature extraction on the at least two sample texts respectively to obtain feature vectors respectively corresponding to the at least two sample texts; extract at least two sample vocabularies from the at least two sample texts; perform aggregation processing on sample texts in which the at least two sample vocabularies respectively exist, to obtain inverted lists respectively corresponding to the at least two sample vocabularies; generate an inverted index list according to the inverted lists respectively corresponding to the at least two sample vocabularies and position information respectively corresponding to the at least two sample vocabularies; According to the label information corresponding to the at least two sample texts respectively, the feature vectors corresponding to the at least two sample texts respectively, and the inverted index list, the knowledge base is obtained.

[0156] It should be noted that the apparatus provided in the above embodiments, in realizing its functions, only takes the above-mentioned division of each functional module as an example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the content structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0157] Please refer to Figure 14 which shows a structural block diagram of a computer device 1400 provided in an embodiment of the present application. The computer device 1400 can be any electronic device with data computing, processing and storage functions. The computer device 1400 can be used to implement the text auditing method provided in the above embodiments.

[0158] Generally, the computer device 1400 includes a processor 1401 and a memory 1402.

[0159] The processor 1401 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1401 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), a PLA (Programmable Logic Array). The processor 1401 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1401 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1401 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0160] The memory 1402 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 1402 can also include high-speed random access memory and can include nonvolatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other nonvolatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium of the memory 1402 is configured to store computer programs that are configured to be executed by one or more processors to implement the text review method described above.

[0161] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the computer device 1400, and the computer device 1400 can include more or fewer components than those shown, or combine certain components, or use different arrangements of components. Figure 14 Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the computer device 1400, and the computer device 1400 can include more or fewer components than those shown, or combine certain components, or use different arrangements of components.

[0162] In an exemplary embodiment, a computer-readable storage medium is also provided, and the storage medium stores a computer program. When the computer program is executed by a processor of a computer device, the computer program implements the text review method described above. Optionally, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0163] In an exemplary embodiment, a computer program product is also provided, and the computer program product includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the text review method described above.

[0164] It should be noted that before collecting the relevant data of the user and in the process of collecting the relevant data of the user, the prompt interface, the pop-up window or the output voice prompt information can be displayed, the prompt interface, the pop-up window or the voice prompt information is used to prompt that the relevant data of the user is currently being collected, so that the application only starts to perform the related steps of obtaining the relevant data of the user after obtaining the confirmation operation of the user to the prompt interface or the pop-up window, otherwise (that is, without obtaining the confirmation operation of the user to the prompt interface or the pop-up window), ending the related steps of obtaining the relevant data of the user, that is, not obtaining the relevant data of the user. In other words, all the user data collected by the application is strictly processed according to the requirements of relevant national laws and regulations, and the informed consent or separate consent of the personal information subject is obtained under the condition that the user agrees and authorizes, and the subsequent data use and processing behavior is carried out within the authorization range of the laws and regulations and the personal information subject, and the collection, use and processing of the relevant user data need to comply with the relevant laws and regulations and standards of the relevant countries and regions.

[0165] It should be understood that "multiple" referred to herein means two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as simultaneously executing two steps with different numbers or executing two steps with different numbers in an order opposite to the illustration, and the embodiments of the application are not limited to this.

[0166] The above only illustrates the exemplary embodiments of the application, and does not limit the application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A text auditing method, characterized in that, The method comprises: obtaining input text and a knowledge base, wherein the knowledge base comprises text information corresponding to at least two sample texts respectively, the at least two sample texts comprise at least one rule violation text and at least one rule compliance text; performing fine-grained splitting on the input text to obtain at least one suspicious text segment in the input text, wherein the suspicious text segment is a text segment suspected of containing rule violation content in the input text; obtaining target rule violation text and target rule compliance text corresponding to the at least one suspicious text segment respectively according to the at least one suspicious text segment and the knowledge base, wherein the target rule violation text corresponding to the suspicious text segment is a rule violation text having the highest similarity to the suspicious text segment among the at least one rule violation text, and the target rule compliance text corresponding to the suspicious text segment is a rule compliance text having the highest similarity to the suspicious text segment among the at least one rule compliance text; generating an audit result of the input text by a large language model according to the input text, the at least one suspicious text segment, the target rule violation text and the target rule compliance text corresponding to the at least one suspicious text segment respectively, wherein the audit result of the input text is used to indicate that the input text is a rule violation text or the input text is a rule compliance text.

2. The method of claim 1, wherein, The method further comprises: performing segment extraction on the input text to obtain at least one text segment in the input text; obtaining a suspicious score corresponding to each text segment by a text scoring model according to the at least one text segment, wherein the suspicious score corresponding to the text segment is used to measure a probability of containing rule violation content in the text segment; and obtaining the at least one suspicious text segment according to the suspicious score corresponding to each text segment.

3. The method of claim 2, wherein, The method further comprises: performing word segmentation processing on the input text to obtain at least one word contained in the input text, wherein each word comprises at least one character; obtaining at least one word in a continuous distribution in a left-to-right order from the at least one word to obtain the at least one text segment, wherein the number of characters contained in each text segment is less than or equal to a preset maximum threshold value.

4. The method according to claim 2 or 3, characterized in that, The method further comprises: obtaining a text segment whose suspicious score is greater than or equal to a first threshold value from the suspicious score corresponding to each text segment of the at least one text segment to obtain at least one filtered text segment; obtaining a text segment corresponding to at least one target suspicious score in a front order according to the suspicious score corresponding to each text segment of the at least one filtered text segment from large to small to obtain the at least one suspicious text segment; and obtaining a text segment corresponding to at least one target suspicious score in a front order according to the suspicious score corresponding to each text segment of the at least one filtered text segment from large to small to obtain the at least one suspicious text segment. The at least one target suspicious score respectively corresponds to a text segment, and there is no repeated vocabulary between the text segments corresponding to the at least one target suspicious score.

5. The method according to any one of claims 1 to 4, characterized in that, The knowledge base further includes an inverted index list, the inverted index list including inverted lists corresponding to at least two sample vocabularies and position information corresponding to the at least two sample vocabularies, the at least two sample vocabularies being extracted from the at least two sample texts, each sample vocabulary corresponding to an inverted list including at least one sample text associated with the sample vocabulary, and each sample vocabulary corresponding to position information including a position of the sample vocabulary in the inverted index list and a position of the sample vocabulary in the at least one sample text associated with the sample vocabulary; The obtaining of the target violation text and the target compliance text corresponding to the at least one suspicious text segment according to the at least one suspicious text segment and the knowledge base includes: For each suspicious text segment in the at least one suspicious text segment, at least one suspicious vocabulary in the suspicious text segment is obtained; In a case where any suspicious vocabulary in the at least one suspicious vocabulary exists in the inverted index list, inverted retrieval is performed on the suspicious text segment to obtain the target violation text and the target compliance text corresponding to the suspicious text segment; In a case where the at least one suspicious vocabulary does not exist in the inverted index list, vector retrieval is performed on the suspicious text segment to obtain the target violation text and the target compliance text corresponding to the suspicious text segment.

6. The method of claim 5, wherein, The inverted retrieval performed on the suspicious text segment to obtain the target violation text and the target compliance text corresponding to the suspicious text segment includes: From the inverted index list, an inverted list corresponding to at least one suspicious sample vocabulary in the at least one suspicious vocabulary existing in the inverted index list is obtained, the inverted list corresponding to the suspicious sample vocabulary including at least one similar sample text, the similar sample text being a sample text associated with the suspicious sample vocabulary; For each similar sample text in the inverted list corresponding to the at least one suspicious sample vocabulary, a similarity between the suspicious text segment and the similar sample text is calculated to obtain a first similarity score corresponding to the similar sample text; In a case where a violation sample exists in the inverted list corresponding to the at least one suspicious sample vocabulary, a similar violation text corresponding to a maximum value in first similarity scores corresponding to at least one similar violation text is determined as the target violation text corresponding to the suspicious text segment, the similar violation text being a violation text in the inverted list corresponding to the at least one suspicious sample vocabulary; and In a case where a violation sample exists in the inverted list corresponding to the at least one suspicious sample vocabulary, a similar violation text corresponding to a maximum value in first similarity scores corresponding to at least one similar violation text is determined as the target violation text corresponding to the suspicious text segment, the similar violation text being a violation text in the inverted list corresponding to the at least one suspicious sample vocabulary. In the case that there is a compliance sample in the inverted list corresponding to each of the at least one suspicious sample vocabulary, a similar compliance text corresponding to a maximum value in the first similarity scores of similar compliance texts corresponding to the at least one suspicious sample vocabulary respectively is determined as the target compliance text corresponding to the suspicious text segment, the similar compliance text being the compliance text in the inverted list corresponding to each of the at least one suspicious sample vocabulary.

7. The method of claim 6, wherein, The calculating of the similarity between the suspicious text segment and the similar sample text to obtain the first similarity score corresponding to the similar sample text comprises: According to the word frequency of the at least one suspicious sample vocabulary in the similar sample text, the length of the similar sample text, the average length of the at least two sample texts, the number of texts in the inverted list corresponding to each of the at least one suspicious sample vocabulary, and the number of texts of the at least two sample texts, the first similarity score corresponding to the similar sample text is obtained.

8. The method according to claim 6 or 7, characterized in that, The method further comprises: In the case that there is no non-compliance text in the inverted list corresponding to each of the at least one suspicious sample vocabulary, vector retrieval is performed on the suspicious text segment to obtain the target non-compliance text corresponding to the suspicious text segment; In the case that there is no compliance text in the inverted list corresponding to each of the at least one suspicious sample vocabulary, vector retrieval is performed on the suspicious text segment to obtain the target compliance text corresponding to the suspicious text segment.

9. The method according to any one of claims 5 to 8, characterized in that, The knowledge base further comprises a vector index library, and the vector index library comprises feature vectors corresponding to the at least two sample texts respectively; The vector retrieval performed on the suspicious text segment to obtain the target non-compliance text and the target compliance text corresponding to the suspicious text segment comprises: Feature extraction is performed on the input text to obtain a feature vector of the input text; Similarities between the feature vector of the input text and the feature vectors corresponding to the at least two sample texts respectively are calculated to obtain second similarity scores corresponding to the at least two sample texts respectively; A maximum value in the second similarity scores corresponding to the at least one non-compliance text respectively corresponds to a non-compliance text, which is determined as the target non-compliance text corresponding to the suspicious text segment; A maximum value in the second similarity scores corresponding to the at least one compliance text respectively corresponds to a compliance text, which is determined as the target compliance text corresponding to the suspicious text segment.

10. The method according to any one of claims 1 to 9, characterized in that, The text information corresponding to the sample text further comprises label information of the sample text, and the label information of the non-compliance text is used to indicate a non-compliance type of the non-compliance text; The generation of the audit result of the input text by the large language model according to the input text, the at least one suspicious text segment, the target non-compliance text and the target compliance text corresponding to the at least one suspicious text segment comprises: According to the input text, at least one audit rule, the at least one suspicious text segment, the target non-compliant text and the target compliant text corresponding to the at least one suspicious text segment respectively, and the non-compliant type of the target non-compliant text corresponding to the at least one suspicious text segment, a prompt text is generated, and the audit rule is used to indicate a text non-compliant type; According to the prompt text, an audit result of the input text is generated by the large language model.

11. The method according to any one of claims 1 to 10, characterized in that, The method further comprises: According to the at least one audit record corresponding to the at least two sample texts respectively, the at least one compliant text in the at least two sample texts is determined, and the non-compliant type corresponding to the at least one non-compliant text is determined, to obtain label information corresponding to the at least two sample texts respectively; Feature extraction is performed on the at least two sample texts respectively, to obtain feature vectors corresponding to the at least two sample texts respectively; At least two sample vocabularies are extracted from the at least two sample texts; Aggregation processing is performed on the sample texts in which the at least two sample vocabularies are located respectively, to obtain inverted lists corresponding to the at least two sample vocabularies respectively; According to the inverted lists corresponding to the at least two sample vocabularies respectively and the position information corresponding to the at least two sample vocabularies respectively, an inverted index list is generated; According to the label information corresponding to the at least two sample texts respectively, the feature vectors corresponding to the at least two sample texts respectively, and the inverted index list, the knowledge base is obtained.

12. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores a computer program, which is loaded and executed by the processor to implement the text audit method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is loaded and executed by the processor to implement the text audit method according to any one of claims 1 to 11.

14. A computer program product, characterised in that, The computer program product comprises a computer program, which is loaded and executed by the processor to implement the text audit method according to any one of claims 1 to 11.

Images