Large model cue word processing method, large model content auditing method and device and electronic equipment
By identifying and updating audit examples of audit error text content in large language models, the problem of low accuracy of large model audit is solved, and the accuracy and efficiency of text content audit is improved.
Patent Information
- Application Number
- CN202411047843.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-12
AI Technical Summary
The large language model has the problem of low audit accuracy in text content review. It is mainly because the accuracy of the prompt words is not sufficient, which leads to inconsistent with the reference results, and it is impossible to effectively update the audit prompt words of the model.
When the review results of the text content are inconsistent with the reference results, the review error text content under the first audit rule is determined, and the audit prompt words of the large language model are updated based on the content to improve the audit accuracy. The specific steps include: reviewing the text content based on the large language model and audit rules, identifying the review error text content, determining its corresponding audit example, and updating the audit prompt word to reduce the number of audit errors.
It improves the accuracy of the large model in text content review, reduces the number of audit errors, and enhances the reliability of audit results.
Smart Images

Figure CN118940754A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of natural language understanding and text content review, and can be applied to scenarios involving review, such as generative search, intelligent document editing, intelligent assistants, virtual assistants, and intelligent e-commerce. The present disclosure specifically relates to a large model prompt word processing method, a large model content review method, a device, an electronic device, and a storage medium. Background Art
[0002] A popular text review method is to use a large language model to review text content, such as advertising and marketing content, social platform information, etc., and only allow it to be made public or published after it passes the review. Among them, the accuracy of the large language model and the accuracy of the prompt words provided to the large language model will have a certain impact on the accuracy of the model review text. Summary of the invention
[0003] The present disclosure provides a large model prompt word processing method, a large model content review method, a device, an electronic device and a storage medium.
[0004] According to one aspect of the present disclosure, a large model prompt word processing method is provided, comprising:
[0005] Based on the large language model and the audit prompt words under each audit rule, the text content is audited to obtain the audit result of the text content;
[0006] In the case where the audit result of the text content is inconsistent with the reference audit result, based on the text content, determining the audit error text content under the first audit rule, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result;
[0007] Determine a review example under the first review rule based on the review error text content under the first review rule;
[0008] Based on the review example under the first review rule, the review prompt words of the large language model under the first review rule are updated.
[0009] According to another aspect of the present disclosure, a large model content audit method is provided, comprising:
[0010] A prompt word determination module is used to determine the corresponding review prompt word based on the text content;
[0011] A segmentation module, used to segment the text content to obtain multiple text paragraphs;
[0012] A text splicing module, used for splicing the audit prompt word with each of the text paragraphs to obtain the target prompt word corresponding to each of the text paragraphs;
[0013] A text review module, used to input each of the target prompt words into a large language model respectively, and obtain a review result corresponding to each of the text paragraphs output by the large language model;
[0014] The audit result determination module is used to determine the audit result of the text content based on the audit results corresponding to each of the text paragraphs.
[0015] According to another aspect of the present disclosure, a large model prompt word processing device is provided, comprising:
[0016] A text content review module is used to review the text content based on the large language model and the review prompt words under various review rules to obtain the review result of the text content;
[0017] A sample screening module, used for determining, based on the text content, the audit error text content under a first audit rule when the audit result of the text content is inconsistent with the reference audit result, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result;
[0018] An audit example determination module, configured to determine an audit example under the first audit rule based on the audit error text content under the first audit rule;
[0019] A prompt word determination module is used to update the audit prompt words of the large language model under the first audit rule based on the audit examples under the first audit rule.
[0020] According to another aspect of the present disclosure, a large model content review device is provided, comprising:
[0021] A prompt word determination module is used to determine the corresponding review prompt word based on the text content;
[0022] A segmentation module, used to segment the text content to obtain multiple text paragraphs;
[0023] A text splicing module, used for splicing the audit prompt word with each of the text paragraphs to obtain the target prompt word corresponding to each of the text paragraphs;
[0024] A text review module, used to input each of the target prompt words into a large language model respectively, and obtain a review result corresponding to each of the text paragraphs output by the large language model;
[0025] The audit result determination module is used to determine the audit result of the text content based on the audit results corresponding to each of the text paragraphs.
[0026] According to another aspect of the present disclosure, there is provided an electronic device, comprising:
[0027] at least one central processing unit; and
[0028] A memory in communication with the at least one central processing unit; wherein,
[0029] The memory stores instructions that can be executed by the at least one central processing unit, and the instructions are executed by the at least one central processing unit so that the at least one central processing unit can execute any large model prompt word processing method or large model content review method in the embodiments of the present disclosure.
[0030] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any large model prompt word processing method or large model content review method in the embodiments of the present disclosure.
[0031] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements any large model prompt word processing method or large model content review method according to the embodiments of the present disclosure.
[0032] According to the technology disclosed in the present invention, based on the large language model and the audit prompt words under each audit rule, the text content is audited to obtain the audit result of the text content; when the audit result of the text content is inconsistent with the reference audit result, based on the text content, the audit error text content under the first audit rule is determined, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result; based on the audit error text content under the first audit rule, the audit example under the first audit rule is determined; based on the audit example under the first audit rule, the audit prompt words of the large language model under the first audit rule are updated. In this way, when the large model audits the text content, these updated audit prompt words are used for auditing, which can improve the audit accuracy and reduce the number of audit errors.
[0033] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0035] Figure 1 is a flow chart of a large model prompt word processing method according to an embodiment of the present disclosure;
[0036] Figure 2 is a flow chart of a large model prompt word processing method according to another embodiment of the present disclosure;
[0037] Figure 3 is a flow chart of a large model content review method according to an embodiment of the present disclosure;
[0038] Figure 4 It is a structural block diagram of a large model content review system according to an embodiment of the present disclosure;
[0039] Figure 5 It is a structural block diagram of a large model prompt word processing device according to an embodiment of the present disclosure;
[0040] Figure 6 is a structural block diagram of a large model prompt word processing device according to another embodiment of the present disclosure;
[0041] Figure 7 It is a structural block diagram of a large model content review device according to an embodiment of the present disclosure;
[0042] Figure 8 is a structural block diagram of a large model content review device according to another embodiment of the present disclosure;
[0043] Fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0044] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0045] In order to facilitate understanding of the large model prompt word processing method provided by the embodiment of the present disclosure, the relevant technologies of the embodiment of the present disclosure are explained below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiment of the present disclosure as optional solutions, and they all belong to the protection scope of the embodiment of the present disclosure.
[0046] Figure 1: is a flowchart of a large model prompt word processing method of an embodiment of the present disclosure. The method can be applied to an electronic device. The electronic device is, for example, a terminal, a server or other processing device, wherein the terminal can be a user equipment (UE) such as a desktop computer, a mobile device, a PDA (Personal Digital Assistant), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some implementations, the electronic device can implement the large model prompt word processing method of the embodiment of the present disclosure by calling a computer-readable instruction stored in a memory by a central processing unit.
[0047] like Figure 1 As shown, the method may include the following steps:
[0048] S110, based on the large language model and the review prompt words under each review rule, review the text content to obtain the review result of the text content;
[0049] S120, when the audit result of the text content is inconsistent with the reference audit result, determine the audit error text content under the first audit rule based on the text content, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result;
[0050] S130, determining a review example under the first review rule based on the review error text content under the first review rule;
[0051] S140: Based on the review example under the first review rule, update the review prompt words of the large language model under the first review rule.
[0052] Exemplarily, in the above step S110, the above step S110 is performed one by one for multiple text contents to obtain the review results of the multiple text contents.
[0053] Exemplarily, the above step S120 may be performed for each text content respectively, so that a plurality of audit error texts may be obtained, and added to the audit error text set under the corresponding rule according to the first audit rule corresponding to each audit error text.
[0054] Exemplarily, for the audit error set under each rule, the following steps can be performed: when the number of texts in the audit error text set exceeds a preset number, the audit examples under the corresponding rule can be determined based on the audit error texts in the audit error text set. In this way, the audit examples under each rule can be obtained, and the audit prompt words under each rule of the large language model can be updated using the audit examples under each rule.
[0055] Alternatively, according to the meaning of the above steps, each time an audit error text is determined, a corresponding audit example is determined, and the audit prompt words of the large language model are updated.
[0056] Exemplarily, in the above step S110, the audit prompt words under each audit rule are concatenated with the text content to obtain the concatenated text content, and the concatenated text content is input into the large language model to obtain the audit result output by the large language model.
[0057] Exemplarily, in the above step S110, the text content is divided into multiple text paragraphs, and each text paragraph is spliced with the review prompt words under each review rule to obtain the spliced text content corresponding to each text paragraph, and the spliced text content corresponding to each text paragraph is input into the large language model to obtain the review results corresponding to each text paragraph output by the large language model, and the review results of the text content are determined based on the review results corresponding to each text paragraph.
[0058] It is understandable that there may be one or more audit rules. Each audit rule has a corresponding audit prompt word. The audit prompt word may include the rule content and the audit example under the rule.
[0059] It can be understood that the audit prompts of the large language model may include, in addition to the audit prompts under the audit rules, the audit task description, the audit role played by the large language model, the audit process, the audit blacklist and whitelist, the audit intensity, the output format and output examples of the audit results, etc.
[0060] Exemplarily, the audit result may include audit passed, or audit failed and the audit rules violated by the text content.
[0061] Exemplarily, the reference audit result may include audit passed, or the reference audit result may include audit failed and audit rules violated by the text content.
[0062] Exemplarily, when the audit result of the text content includes audit passed, and its reference audit result includes audit failed, it is considered that the audit result of the text content is inconsistent with the reference audit result.
[0063] Exemplarily, when the audit result of the text content is that the audit failed, and its reference audit result is that the audit passed, it is considered that the audit result of the text content is inconsistent with the reference audit result.
[0064] Exemplarily, when the review result of the text content is consistent with the reference review result, the above steps S120 to S140 are not executed.
[0065] It can be understood that the first audit rule is the audit rule involved in the audit result or the reference audit result, which can be understood as the audit rule included in the audit result or the audit rule included in the reference audit result.
[0066] For example, for the audit error text content A1, the audit result is "audit failed, violation of rule a1", and the reference audit result is: "audit passed", then the first audit rule is rule a1.
[0067] For another example, for the review of the erroneous text content B1, the review result is "review passed", and the reference review result is: "review failed, violation of rule b1", then the first review rule is rule b1.
[0068] It can be understood that the reference audit result is a pre-marked audit result for the text content. For example, multiple text contents are processed in advance and divided into two categories, one is the text content with a reference audit result of passed audit, and the other is the text content with a reference audit result of failed audit. The reference audit result of the text content should also include the audit rules violated by the failed audit.
[0069] For example, for the text content C1: "After purchasing "xxx", the user no longer feels tired when working overtime.", its reference review result is approved.
[0070] For example, for the text content D1: "The product can regulate enterohepatic circulation and help delay liver aging.", the reference review result is failure to pass, which violates rule d1.
[0071] It can be understood that the reference review result is a result of manual marking or a result of machine marking.
[0072] Exemplarily, in the above step S130, based on the type of the audit error text content under the first audit rule, the audit error text content is determined as a compliant text example or an illegal text example under the first audit rule.
[0073] Exemplarily, in the above-mentioned step S130, the audit error text content is added to the audit error text content set under the first audit rule. When the number of audit error text content sets exceeds the set number, the target audit error text content is determined in the audit error text content set, and thus, based on the target audit error text content, the audit example under the first audit rule is determined.
[0074] Exemplarily, the scheme may also include counting the number of text contents whose audit results are consistent with the reference audit results, and when the ratio between the above number and the total number of text contents participating in the above step S110 reaches a preset ratio condition, determining the target audit error text content in the audit error text content set, thereby determining the audit example under the first audit rule based on the target audit error text content.
[0075] Exemplarily, based on the audit examples under the first audit rule, updating the audit prompt words of the large language model under the first audit rule may include replacing the audit examples included in the audit prompt words of the large language model under the first audit rule with the audit examples under the first audit rule. Alternatively, it may include adding the audit examples under the first audit rule to the audit prompt words of the large language model under the first audit rule.
[0076] According to the above implementation, based on the large language model and the audit prompt words under each audit rule, the text content is audited to obtain the audit result of the text content; when the audit result of the text content is inconsistent with the reference audit result, based on the text content, the audit error text content under the first audit rule is determined, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result; based on the audit error text content under the first audit rule, the audit example under the first audit rule is determined; based on the audit example under the first audit rule, the audit prompt words of the large language model under the first audit rule are updated. In this way, when the large model audits the text content, the audit can be improved by using these updated audit prompt words to improve the audit accuracy and reduce the number of audit errors.
[0077] In one embodiment, the above method may further include: when the audit result of the text content is inconsistent with the reference audit result, determining the type of audit error text content based on the audit result and the reference audit result.
[0078] It can be understood that when the audit result of the text content is inconsistent with the reference audit result, the text content is audit error text content, which includes two types: it is audit error data but is actually compliant data of the corresponding rules, or it is audit error data and is actually violation data of the corresponding rules.
[0079] For example, if the audit result is general and the reference audit result is failed, the type of the audit error text content is illegal text content.
[0080] For another example, if the audit result is failure and the reference audit result is passing, then the type of the audit error text content is compliant text content.
[0081] According to the above-mentioned implementation manner, based on the audit result and the reference audit result, it can be determined whether the type of the audit error text content is illegal text content or compliant text content.
[0082] In one embodiment, based on the audit result and the reference audit result, the type of the audit error text is determined, including: when the audit result of the text content is audit failure and the reference audit result is audit passing, determining the type of the audit error text content is compliant text content.
[0083] In one embodiment, based on the audit result and the reference audit result, the type of the audit error text content is determined, including: when the audit result of the text content is audit passed and the reference audit result is audit failed, determining the type of the audit error text content as illegal text content.
[0084] According to the above-mentioned implementation manner, based on the audit result and the reference audit result, it can be determined whether the type of the audit error text content is illegal text content or compliant text content.
[0085] In one embodiment, based on the audit error text content under the first audit rule, determining the audit example under the first audit rule includes: when the type of the audit error text content under the first audit rule is compliant text content, determining the compliant text example under the first audit rule based on the audit error text content.
[0086] It can be understood that when the type of the audited erroneous text content is compliant text content, a compliant text example under the first audit rule can be generated based on the compliant text content itself.
[0087] For example, if the first audit rule is rule a1, and the compliance text content is "After purchasing "xxx", the feedback is that working overtime does not make you tired anymore", then the corresponding compliance text example is "Rule a1: Compliance text example: 1. "After purchasing "xxx", the feedback is that working overtime does not make you tired anymore"".
[0088] Exemplarily, the audit error text content may also be added to the corresponding set, and then the set may be used to generate multiple compliant text examples under the first audit rule.
[0089] According to the above embodiment, when the type of the audit error text content under the first audit rule is compliant text content, a compliant text example under the first audit rule can be generated based on the audit error text content.
[0090] In one embodiment, based on the audit error text content, determining a compliant text example under the first audit rule includes: adding the audit error text content to the compliant text content set under the first audit rule; based on the number of repeated occurrences of each compliant text content in the compliant text content set, determining the target compliant text content in the compliant text content set; based on the target compliant text content, determining a compliant text example under the first audit rule.
[0091] It can be understood that the compliant text content set includes one or more compliant text content.
[0092] It can be understood that the target compliance text content can be one or more.
[0093] Exemplarily, when the total number of compliant text contents in the compliant text content set exceeds a preset number, target compliant text contents are determined in the compliant text content set based on the number of repetitions of each compliant text content in the compliant text content set.
[0094] For example, the compliant text contents in the compliant text content set are sorted according to the number of repetitions, and the top three compliant text contents are obtained as the target compliant text contents. In this way, three compliant text examples under the corresponding rules can be generated. Of course, it can also be ranked first or ranked top five.
[0095] Exemplarily, if the target compliance text content includes multiple ones, then based on each target compliance text content, each compliance text example under the first audit rule is generated respectively.
[0096] According to the above implementation, a compliant text content set is used to collect and review erroneous text contents, and then the target compliant text content is determined in the compliant text content set using the number of repeated occurrences of each compliant text content in the compliant text content set. Then, the target compliant text content is used to determine a compliant text example under the first review rule, which can improve the accuracy of generating compliant text examples.
[0097] In one embodiment, based on the audit error text content under the first audit rule, determining the audit example under the first audit rule includes: when the type of the audit error text content under the first audit rule is illegal text content, determining the illegal text example under the first audit rule based on the audit error text content.
[0098] It can be understood that when the type of the audited erroneous text content is illegal text content, an illegal text example under the first audit rule can be generated based on the illegal text content itself.
[0099] For example, the first review rule is rule b1, and the illegal text content is "The product can regulate enterohepatic circulation and help delay liver aging." The corresponding illegal text example is "Rule b1: Illegal text example: 1. "The product can regulate enterohepatic circulation and help delay liver aging."".
[0100] Exemplarily, the audit error text content may also be added to the corresponding set, and then the set is used to generate multiple violation text examples under the first audit rule.
[0101] According to the above embodiment, when the type of the audit error text content under the first audit rule is illegal text content, an illegal text example under the first audit rule can be generated based on the audit error text content.
[0102] In one embodiment, based on the audit error text content, determining the violation text example under the first audit rule includes: adding the audit error text content to the violation text content set under the first audit rule; based on the number of repeated occurrences of each violation text content in the violation text content set, determining the target violation text content in the violation text content set; based on the target violation text content, determining the violation text example under the first audit rule.
[0103] It can be understood that the illegal text content set includes one or more illegal text contents.
[0104] It can be understood that the target illegal text content can be one or more.
[0105] Exemplarily, when the total number of illegal text contents in the illegal text content set exceeds a preset number, target illegal text contents are determined in the illegal text content set based on the number of repeated occurrences of each illegal text content in the illegal text content set.
[0106] For example, the illegal text contents in the illegal text content set are sorted according to the number of repetitions, and the top three illegal text contents are obtained as the target illegal text contents. In this way, three illegal text examples under the corresponding rules can be generated. Of course, it can also be ranked first or ranked top five.
[0107] Exemplarily, if the target illegal text content includes multiple ones, then based on each target illegal text content, each illegal text example under the first audit rule is generated respectively.
[0108] According to the above implementation mode, a violation text content set is used to collect and review erroneous text contents, and then, the target violation text content is determined in the violation text content set using the number of repeated occurrences of each violation text content in the violation text content set. Then, the target violation text content is used to determine a valid violation text example under the first review rule, which can improve the accuracy of generating violation text examples.
[0109] In one embodiment, the above-mentioned auditing of text content based on the large language model and the audit prompt words under each audit rule to obtain the audit result of the text content may also include: auditing multiple text contents based on the large language model and the audit prompt words under each audit rule to obtain the audit result of each of the text contents.
[0110] It can be understood that the large language model can review multiple text contents one by one and output the review results one by one.
[0111] In this example, a large language model and review prompt words are used to obtain the review results of each text content for multiple text contents. Subsequently, the review results of multiple text contents and reference review results can be used to determine more accurate review examples to improve the accuracy of the review prompt words.
[0112] In one embodiment, when the audit result of the text content is inconsistent with the reference audit result, determining the audit error text content under the first audit rule based on the text content includes:
[0113] In the case where there is at least one text content whose audit result is inconsistent with the reference audit result, a first number of the text contents whose audit results are consistent with the reference audit result and a second number of the text contents whose audit results are inconsistent with the reference audit result are counted; based on the first number and the second number, target text content is determined in at least one text content whose audit result is inconsistent with the reference audit result; based on the target text content, the audit error text content under the first audit rule is determined.
[0114] It can be understood that the first audit rule is the audit result of the target text content and the audit rules involved in the reference audit result.
[0115] It is understandable that if the audit results of each text content are consistent with the reference audit results, the quantity statistics of the above text content will not be performed.
[0116] Exemplarily, when a large language model reviews multiple text contents one by one and outputs the review results one by one, each time a review result of a text content is obtained, the review result is compared with the corresponding reference review result. If the two are inconsistent, the first number and the second number are accumulated one by one starting from zero. Otherwise, the statistics of the first number and the second number are not started.
[0117] In this example, multiple text contents are reviewed one by one. If the review result of the reviewed text content is inconsistent with the reference review result, the number of consistent and inconsistent review results is triggered. In this way, the statistical workload can be reduced and the efficiency of determining the review error text can be improved.
[0118] Exemplarily, when the second quantity is greater than a preset quantity threshold and the ratio of the second quantity to the first quantity satisfies a preset ratio condition, the target text content is determined in at least one of the text contents whose audit result is inconsistent with the reference audit result.
[0119] Exemplarily, when the ratio of the second number to the sum of the first number and the second number satisfies a preset ratio condition, the target text content is determined in at least one of the text contents whose audit result is inconsistent with the reference audit result;
[0120] Exemplarily, when a ratio of the first quantity to the sum of the first quantity and the second quantity satisfies a preset ratio condition, the target text content is determined in at least one of the text contents whose audit results are inconsistent with the reference audit results.
[0121] According to the above implementation, if the ratio or quantitative relationship between the number of correct and incorrect audits meets the preset conditions, the target text content is selected from the multiple text contents, and the audit error text is determined using the target text content. In this way, on the one hand, the efficiency of determining the audit error text can be improved, and on the other hand, using multiple texts to determine the final audit error text can make the subsequent audit examples and audit prompts more accurate.
[0122] Figure 2 It is a flowchart of a large model prompt word processing method according to another embodiment of the present disclosure.
[0123] like Figure 2 As shown, the method includes the following processes:
[0124] S210, data processing. The details are as follows:
[0125] First, prepare the training data set in text format. Divide the data set into two categories: one is the content that can pass the review, and the other is the content that cannot pass the review. Then label the data set, add the label of "passed" to the data that can pass the review, and add the label of "failed" to the data that cannot pass the review, and mark the serial number of the violation.
[0126] For example: Data 1: "After buying "xxx", I no longer feel tired when working overtime.", approved. Data 2: "The product can regulate enterohepatic circulation and help delay liver aging.", not approved, violates Rule 1.
[0127] S220, review data. After data preparation, review the data. Use prompts to review the data in sequence. Once all data reviews are completed, it is considered a training session. The default training session is 3 times. You can customize the number of training sessions. After setting the number of training sessions, start reviewing.
[0128] Here is a detailed description of each data review process. First, get the current review data, combine the data with the prompt and submit it to the big model for review. After obtaining the review result, compare the review result with the label to determine whether the review is correct. If it is correct, count the number of correct reviews. If it fails, there are two situations:
[0129] The first case: If the data label is passed, but the audit result is failed, the data is recorded in the error data and marked as compliant data for the corresponding rules.
[0130] The second case: If the data label is audit failure, but the audit result is passed, the data will also be recorded in the error data and marked as violation data of the corresponding rule.
[0131] Until the training is completed.
[0132] For example: If the audit result of the above data 1 is "failed to pass the audit, violated rule 1", then data 1 will be recorded in the error data set, and the record will be: Rule 1 compliant data: Data 1: "After buying "xxx", the feedback is that I don't feel tired even when working overtime."; If the audit result of the above data 2 is "passed", then data 2 will be recorded in the error data set, and the record will be: Rule 1 violation data: Data 2: "The product can regulate enterohepatic circulation and help delay liver aging."
[0133] S230, obtain cases. After data training is completed, obtain the error data set, sort the compliance and violation data of each rule in the data set from high to low according to the number of repeated occurrences of the data, and obtain the top three data, which are the audit cases of the rule. Finally, each rule obtains a maximum of 6 audit cases, including three compliance cases and three violation cases. Then add the generated audit cases to Prompt to assist in the audit.
[0134] For example: Audit rule 1: Compliance case: 1. "After purchasing "xxx", I feel no more tired when working overtime." Violation case: 1. "The product can regulate enterohepatic circulation and help delay liver aging."
[0135] According to the above implementation, audit examples for various audit scenarios of the large language model can be generated and used as prompt words to improve the audit accuracy of the large language model.
[0136] Figure 3 It is a flowchart of a large model content review method according to an embodiment of the present disclosure.
[0137] like Figure 3 As shown, the method may include:
[0138] S310, determining corresponding review prompt words based on the text content;
[0139] S320, segmenting the text content to obtain multiple text paragraphs;
[0140] S330, concatenating the review prompt words with each text paragraph to obtain the target prompt words corresponding to each text paragraph;
[0141] S340, inputting each target prompt word into the large language model respectively, and obtaining the review results corresponding to each text paragraph output by the large language model;
[0142] S350, determining the review result of the text content based on the review result corresponding to each text paragraph.
[0143] Exemplarily, the general structure of prompt words provided by the embodiment of the present disclosure is used to design corresponding review prompt words for the text content to be reviewed. For example, if the text content to be reviewed is advertising content, the review prompt words corresponding to the advertising field are used, and the overall structure of the review prompt words is consistent with the general structure of prompt words.
[0144] Exemplarily, according to the above-mentioned prompt word processing method, the review prompt words in the above-mentioned step S310 are updated, which can improve the review ability and comprehension ability of the large language model.
[0145] Exemplarily, the audit prompt words include at least one of the following: the audit role definition for auditing text content, the audit task description, the audit process, the audit rules, and the output standard of the audit results.
[0146] Exemplarily, the audit prompt words of the above step S310 include not only the audit prompt words under the various audit rules in the above embodiments, but also other audit prompt words for audit role definition, audit task description, audit process and output standards of audit results for auditing text content.
[0147] Understandably, the role of the review prompt words is to assist the large language model in reviewing the text content. For different text contents in the same field, the review role definition, review task description, review process and the output standard of the review results are relatively fixed. Of course, it can also be adjusted as needed, such as adjusting the review rules.
[0148] Exemplarily, the audit rules also include at least one of an audit blacklist, an audit whitelist, and an audit example.
[0149] Exemplarily, the review role is XX content review specialist.
[0150] For example, the review role is defined as follows: XXX App has an independent review mechanism. As a core member of the commercial content review team, you are responsible for reviewing articles promoting commercial products published on the XXX App platform. Your main task is to review the surface text of the article, which does not include the review of the article's orientation and implications. You need to focus on identifying and evaluating the effect description of the promoted product in the article, and you can accurately judge whether the content violates the review rules.
[0151] For example, the audit task description includes the audit object, audit focus and audit intensity. For example:
[0152] 1. Review commercial promotion articles and carefully review the articles according to the established review process.
[0153] (Audit Object)
[0154] 2. The focus of the review is to identify and evaluate the effect statements of the products that are intentionally promoted in the article, rather than all the products mentioned in the article. (Review focus)
[0155] 3. The review will be limited to the direct meaning of the text. No interpretation or review will be conducted on obscure or suggestive content to avoid over-interpretation of the article and ensure the objectivity and accuracy of the review.
[0156] (Audit intensity).
[0157] For example, the audit process is obtained by summarizing and optimizing the audit process by manual auditors. At the same time, the audit process can also be optimized based on the audit results returned by the large language model. For example, when it is found during testing that the audit results of the large language model are more stringent than manual audits, restrictions are added to the audit process to restrict the large model to strictly follow the rules and avoid subjective assumptions.
[0158] For example, the review process is as follows:
[0159] 1. You need to carefully study and understand the following audit rules, audit cases, audit intensity, exemption words, and prohibited words to master the audit standards.
[0160] 2. Carefully review the input articles according to the review intensity and review rules. When judging violations, please strictly follow the review rules and avoid subjective assumptions or over-interpretation.
[0161] 3. Compare each content with each rule to avoid omissions.
[0162] 4. Output the audit results according to the output format.
[0163] For example, the process of customizing audit rules is similar to the audit process. The first version of the audit rules is customized according to the audit rules summarized by the current auditors. However, various problems were found when using the first version of the rules. For example, some rules can be understood by humans, but the big model cannot understand them, so it is necessary to use words that the big model can understand; some rules may be ambiguous and audit errors may occur. Therefore, for each audit rule, it is necessary to go through repeated debugging to reach a rule that can be used for audit. The formulation of audit rules is a process of continuous debugging and optimization.
[0164] For example:
[0165] ##Audit rules:
[0166] 1. Exemption terms can be expressed.
[0167] 2. Prohibited words cannot be used.
[0168] 3. Rules for expressing product usage effects and experimental conclusions:
[0169] Permitted expressions:
[0170] a. Health assistance: improving the quality of life and promoting health.
[0171] Not allowed to express:
[0172] a. Inhibit aging and prolong life: Do not state that life can be prolonged in product effects or test results.
[0173] For example, blacklists and whitelists are used to assist large language models in understanding audit rules. For example, when certain words are prohibited or certain words can be expressed, using blacklists and whitelists is more flexible and efficient.
[0174] For example, false advertising is not allowed in advertising and marketing articles. In the anti-aging field, promoting a product as having anti-aging effects is considered false advertising, and the term "anti-aging" can be added to the blacklist.
[0175] For example:
[0176] ##Exempt words:
[0177] Promote health.
[0178] ## Prohibited words:
[0179] Inhibit aging.
[0180] Exemplarily, the audit intensity is mainly determined based on the return results of the big model. For example, when the audit results of the big model are stricter than those of human review, it is necessary to tell the big model what audit intensity to use.
[0181] For example:
[0182] ##Audit Strength
[0183] When conducting content review, we should follow the review rules and only check the literal meaning without involving associations and implied meanings, and avoid subjective assumptions and over-interpretation of the implied meanings in the article. Only when the literal meaning clearly violates the rules should it be judged as a violation.
[0184] Exemplarily, the audit examples may include positive audit examples (compliant text examples) and negative audit examples (illegal text examples).
[0185] Understandably, audit examples are used to assist the large language model in understanding audit rules. Using specific examples can help you understand audit rules more clearly. For rules, positive and negative examples are needed to allow the large model to fully understand them.
[0186] The generation of review examples can be: when a certain text content has been reviewed incorrectly by the large language model and the large model cannot understand it, the corresponding review examples are generated using the text content and added to the review prompt words. Then, when encountering such problems, the large model can easily understand them.
[0187] For example:
[0188] Audit Rule 1:
[0189] Example of compliant text:
[0190] 1. After purchasing "xxx", the feedback was "Now I feel better, I don't feel tired even when working overtime, and my skin feels much better." The article mentioned the role of promoting health, which meets the requirements.
[0191] Example of offending text:
[0192] 1. xxx has therapeutic potential in cardiovascular protection, neuromodulation and protection, and improvement of metabolic diseases. The “cardiovascular protection, neuromodulation, and improvement of diseases” in the article involve disease treatment, which violates the rules.
[0193] Understandably, the output standard is also the result of continuous debugging based on the content returned by the large model, and is mainly used to constrain the content returned by the large model.
[0194] For example:
[0195] ## Output requirements
[0196] 1. For articles that meet the standards, just mark them as “approved”.
[0197] 2. For articles with problems, you need to clearly point out the review guidelines that have been violated and provide specific modification suggestions to help the author improve the content.
[0198] 3. When listing violations, please include only those rules that were actually violated. Rules that were not triggered do not need to be mentioned. This is intended to increase review efficiency while ensuring that the author can clearly understand the specific areas that need improvement.
[0199] Exemplarily, the output standard also includes an output format.
[0200] For example:
[0201] ## Output format example:
[0202] Example of approved review:
[0203] [Audit results]
[0204] Review passed.
[0205] Examples of failed audits:
[0206] [Audit results]
[0207] The review failed.
[0208]
Violation point n
[0209] 1. Violation of rules: Rule n, "" (output rule details)
[0210] 2. Trigger content: """" (output the content that violates the rules. If there are multiple, output them together. If there are none, do not output the violation point)
[0211] 3. Reason for violation: "" (output the reason for violation of the rule)
[0212] 4. Suggestions for modification: "" (provide specific suggestions for modification)
[0213] Figure 4 It is a structural block diagram of a large model content review system according to an embodiment of the present disclosure.
[0214] like Figure 4 As shown, it includes a text content processing module, a prompt word general architecture module and a model training module. The details are as follows:
[0215] 1. Text content processing module
[0216] The system receives the user's text content and determines whether it is an article link or article ID based on the text content. If it is, the system automatically obtains the article content to avoid the tedious manual pasting. For long articles, the system splits the text content into multiple paragraphs and then provides the text content to the Prompt general architecture module.
[0217] 2. Prompt general architecture module
[0218] In order to meet the stringent requirements for advertising and marketing content review, we conducted an in-depth analysis of existing review scenarios and refined Prompt into multiple functional modules:
[0219] a. Role definition and task description: used to clarify the role positioning and assist the large language model in understanding the role through background information. At the same time, clearly set the goals and expected results to help the large language model understand the problems to be solved, so as to generate content related to the goals in a more focused and targeted manner.
[0220] b. Refinement of processes and rules: used to set up the advertising review process, so that the large language model can conduct a comprehensive review according to the established steps to avoid omissions. At the same time, it supports the flexible definition of clear and accurate review rules, so that the large language model can quickly grasp the review standards. In addition, according to the characteristics of different review links, the rules can be flexibly adjusted to meet diverse review needs.
[0221] c. Improved flexibility and efficiency: By establishing a text blacklist and whitelist, while ensuring the principle of auditing, a certain degree of flexibility is given to the auditing process, enabling more efficient handling of special situations.
[0222] d. Case assistance and audit intensity adjustment: In order to enable the large language model to understand and apply the rules more deeply and avoid ambiguity, positive and negative cases are provided for each rule. These cases will provide a strong reference, especially when facing complex or ambiguous situations. At the same time, the audit intensity can be dynamically adjusted according to actual needs to ensure that the audit is neither too strict nor too loose.
[0223] e. Output standardization and templateization: Establish unified output requirements and format examples to ensure the consistency and readability of audit results, thereby improving the professionalism of the overall audit process.
[0224] On this basis, adhering to the clear and logical design principles, a complete framework including role definition, task description, audit process, audit rules, exempt words, prohibited words, audit cases, audit intensity, output requirements, output format examples and input core elements was produced. This framework is committed to building a flexible and rigorous audit system so that it can be quickly adjusted according to different scenarios and needs, thereby ensuring efficient operation while maintaining the accuracy of audit results.
[0225] With the help of the Prompt universal architecture module, the text content from the content processing module can be seamlessly received and intelligently spliced with the preset prompt. The large model is called through the API interface for in-depth review. After the review is completed, the review results of the large language model are sent back to the user in real time, thus realizing full-process automated review.
[0226] 3. Model training module
[0227] By collecting a large number of positive and negative cases from actual audit results and converting them into data sets, we fine-tuned the parameters of the large language model using the SFT (Supervised Fine-Tuning) technology to more accurately audit advertising and marketing content. In order to improve the accuracy of the system, in actual applications, we collect current audit error cases through actual business feedback and include them in the data set for model training. Through this series of processes, we can achieve a high-accuracy audit effect.
[0228] According to the above implementation, the commercial review pass rate and the hot product review pass rate can be improved. At the same time, the review time is greatly reduced from the original two hours to less than ten minutes. These optimization measures will significantly improve the review efficiency and thus help increase business revenue.
[0229] Figure 5 It is a structural block diagram of a large model prompt word processing device according to an embodiment of the present disclosure.
[0230] like Figure 5 As shown, the prompt word processing device may include:
[0231] The text content review module 510 is used to review the text content based on the large language model and the review prompt words under each review rule to obtain the review result of the text content;
[0232] A sample screening module 520, configured to determine, based on the text content, the audit error text content under a first audit rule when the audit result of the text content is inconsistent with the reference audit result, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result;
[0233] An audit example determination module 530, configured to determine an audit example under the first audit rule based on the audit error text content under the first audit rule;
[0234] The prompt word determination module 540 is used to update the audit prompt words of the large language model under the first audit rule based on the audit examples under the first audit rule.
[0235] Figure 6 It is a structural block diagram of a large model prompt word processing device according to another embodiment of the present disclosure.
[0236] like Figure 5 and Figure 6 As shown, Figure 6 The text content review module 610, the sample screening module 620, the review example determination module 630 and the prompt word determination module 640 are respectively connected to Figure 5 The text content review module 510, sample screening module 520, review example determination module 530 and prompt word determination module 540 have the same structure and function and will not be described in detail here.
[0237] In one embodiment, the above device further includes:
[0238] The type determination module 650 is used to determine the type of the audit error text content based on the audit result and the reference audit result when the audit result of the text content is inconsistent with the reference audit result.
[0239] In one implementation, the type determination module 650 includes:
[0240] The compliance type determination unit 651 is used to determine the type of the audit error text content as compliant text content when the audit result of the text content is audit failure and the reference audit result is audit passing.
[0241] In one implementation, the type determination module 650 includes:
[0242] The violation type determination unit 652 is used to determine the type of the audit error text content as violation text content when the audit result of the text content is audit passed and the reference audit result is audit failed.
[0243] In one implementation, the audit example determination module 630 includes:
[0244] The compliance example determination unit 631 is used to determine a compliant text example under the first audit rule based on the audit error text content when the type of the audit error text content under the first audit rule is compliant text content.
[0245] In one implementation, the compliance example determination unit 631 is specifically configured to:
[0246] Add the audit error text content to the compliance text content set under the first audit rule;
[0247] Determining target compliant text content in the compliant text content set based on the number of repetitions of each compliant text content in the compliant text content set;
[0248] Based on the target compliant text content, determine a compliant text example under the first audit rule.
[0249] In one implementation, the audit example determination module 630 includes:
[0250] The violation example determination unit 632 is used to determine a violation text example under the first audit rule based on the audit error text content when the type of the audit error text content under the first audit rule is violation text content.
[0251] In one implementation, the violation example determination unit 632 is specifically configured to:
[0252] Add the audit error text content to the set of illegal text content under the first audit rule;
[0253] Determining target illegal text content in the illegal text content set based on the number of repeated occurrences of each illegal text content in the illegal text content set;
[0254] Based on the target illegal text content, an illegal text example under the first audit rule is determined.
[0255] Figure 7 It is a structural block diagram of a large model content review device according to an embodiment of the present disclosure.
[0256] like Figure 7 As shown, the content review device includes:
[0257] The prompt word determination module 710 is used to determine the corresponding review prompt word based on the text content;
[0258] Segmentation module 720, used to segment the text content to obtain multiple text paragraphs;
[0259] A text concatenation module 730 is used to concatenate the audit prompt word with each of the text paragraphs to obtain a target prompt word corresponding to each of the text paragraphs;
[0260] A text review module 740 is used to input each of the target prompt words into a large language model to obtain a review result corresponding to each of the text paragraphs output by the large language model;
[0261] The review result determination module 750 is used to determine the review result of the text content based on the review results corresponding to each of the text paragraphs.
[0262] Figure 8 It is a structural block diagram of a large model content review device according to another embodiment of the present invention.
[0263] like Figure 7 and Figure 8 As shown, Figure 8 The prompt word determination module 810, the segmentation module 820, the text splicing module 830, the text review module 840 and the review result determination module 850 are respectively connected with Figure 7 The prompt word determination module 710, segmentation module 720, text splicing module 730, text review module 740 and review result determination module 750 in the embodiment have the same structure and function, which will not be described in detail here.
[0264] In one embodiment, the above device may further include:
[0265] The prompt word updating module 860 is used to update the audit prompt word according to any prompt word processing method of the embodiment of the present disclosure.
[0266] In one embodiment, the audit prompt includes at least one of the following: an audit role definition, an audit task description, an audit process, an audit rule, and an output standard of an audit result for auditing the text content.
[0267] In one embodiment, the audit rules further include at least one of an audit blacklist, an audit whitelist, and an audit example.
[0268] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.
[0269] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0270] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product. The electronic device may be a server, and the server may also include any power supply control device in the embodiment of the present disclosure.
[0271] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0272] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 907 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0273] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 906, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0274] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a large model prompt word processing method or a large model content audit method. For example, in some embodiments, a large model prompt word processing method or a large model content audit method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 907. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps in the large model prompt word processing method or the large model content audit method described above may be executed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute a large model prompt word processing method or a large model content review method in any other appropriate manner (for example, by means of firmware).
[0275] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0276] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0277] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0278] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0279] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0280] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0281] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0282] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A large model prompt word processing method, comprising: Based on the large language model and the audit prompt words under each audit rule, the text content is audited to obtain the audit result of the text content; In the case where the audit result of the text content is inconsistent with the reference audit result, based on the text content, determining the audit error text content under the first audit rule, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result; Determine a review example under the first review rule based on the review error text content under the first review rule; Based on the review example under the first review rule, the review prompt words of the large language model under the first review rule are updated.
2. The method according to claim 1, further comprising: In the case that the audit result of the text content is inconsistent with the reference audit result, the type of the audit error text content is determined based on the audit result and the reference audit result.
3. The method according to claim 2, wherein: The determining the type of the audit error text content based on the audit result and the reference audit result includes: When the audit result of the text content is that the audit fails, and the reference audit result is that the audit passes, the type of the audit error text content is determined to be compliant text content.
4. The method according to claim 2, wherein: The determining the type of the audit error text content based on the audit result and the reference audit result includes: When the audit result of the text content is that the audit is passed, and the reference audit result is that the audit is not passed, the type of the audit error text content is determined to be illegal text content.
5. The method according to claim 3, wherein: The determining of the audit example under the first audit rule based on the audit error text content under the first audit rule includes: In a case where the type of the audit error text content under the first audit rule is compliant text content, a compliant text example under the first audit rule is determined based on the audit error text content.
6. The method according to claim 5, wherein: The determining, based on the audit error text content, a compliant text example under the first audit rule includes: Adding the audit error text content to the compliance text content set under the first audit rule; Determining target compliant text content in the compliant text content set based on the number of repetitions of each compliant text content in the compliant text content set; Based on the target compliant text content, determine a compliant text example under the first audit rule.
7. The method according to claim 4, wherein: The determining of the audit example under the first audit rule based on the audit error text content under the first audit rule includes: In the case where the type of the audit error text content under the first audit rule is illegal text content, an illegal text example under the first audit rule is determined based on the audit error text content.
8. The method according to claim 7, wherein: The determining, based on the audit error text content, an example of a violation text under the first audit rule includes: Adding the audit error text content to the set of illegal text content under the first audit rule; Determining target illegal text content in the illegal text content set based on the number of repeated occurrences of each illegal text content in the illegal text content set; Based on the target illegal text content, an illegal text example under the first audit rule is determined.
9. The method according to claim 1, wherein: The text content is audited based on the large language model and the audit prompt words under each audit rule to obtain the audit result of the text content, including: Based on the large language model and the audit prompt words under each audit rule, multiple text contents are audited to obtain the audit results of each of the text contents.
10. The method according to claim 9, wherein: When the audit result of the text content is inconsistent with the reference audit result, determining the audit error text content under the first audit rule based on the text content includes: In the case where there is a case where the audit result of at least one of the text contents is inconsistent with the reference audit result, counting a first number of the text contents whose audit results are consistent with the reference audit result, and counting a second number of the text contents whose audit results are inconsistent with the reference audit result; Based on the first number and the second number, determining a target text content in at least one of the text contents whose audit result is inconsistent with a reference audit result; Based on the target text content, the audit error text content under the first audit rule is determined.
11. The method according to claim 10, wherein: The determining, based on the first number and the second number, target text content in at least one of the text contents whose audit result is inconsistent with the reference audit result, comprises: When a ratio of the first quantity to the sum of the first quantity and the second quantity satisfies a preset ratio condition, a target text content is determined in at least one of the text contents whose audit result is inconsistent with a reference audit result.
12. A large model content audit method, comprising: Based on the text content, determine the corresponding review prompt words; Segment the text content to obtain multiple text paragraphs; The audit prompt words are respectively concatenated with each of the text paragraphs to obtain the target prompt words corresponding to each of the text paragraphs; Inputting each of the target prompt words into a large language model respectively, and obtaining a review result corresponding to each of the text paragraphs output by the large language model; Based on the review results corresponding to each of the text paragraphs, the review results of the text content are determined.
13. The method according to claim 12, further comprising: According to the method described in any one of claims 1-11, the audit prompt word is updated.
14. The method according to claim 12 or 13, wherein: The audit prompt words include at least one of the following: the audit role definition, audit task description, audit process, audit rules and output standards of the audit results for auditing the text content.
15. The method according to claim 14, wherein: The audit rules also include at least one of an audit blacklist, an audit whitelist, and an audit example.
16. A large model prompt word processing device, comprising: A text content review module is used to review the text content based on the large language model and the review prompt words under various review rules to obtain the review result of the text content; A sample screening module, for determining, based on the text content, audit error text content under a first audit rule when the audit result of the text content is inconsistent with the reference audit result, wherein the first audit rule is the audit rule involved in the audit result or the reference audit result; An audit example determination module, configured to determine an audit example under the first audit rule based on the audit error text content under the first audit rule; A prompt word determination module is used to update the audit prompt words of the large language model under the first audit rule based on the audit examples under the first audit rule.
17. The apparatus according to claim 16, further comprising: The type determination module is used to determine the type of the audit error text content based on the audit result and the reference audit result when the audit result of the text content is inconsistent with the reference audit result.
18. The device according to claim 16, wherein: The type determination module comprises: The compliance type determination unit is used to determine the type of the audit error text content as compliant text content when the audit result of the text content is audit failure and the reference audit result is audit passing.
19. The device according to claim 16, wherein: The type determination module comprises: The violation type determination unit is used to determine the type of the audit error text content as violation text content when the audit result of the text content is audit passed and the reference audit result is audit failed.
20. The device according to claim 16, wherein: The audit example determination module includes: A compliance example determination unit is used to determine a compliance text example under the first audit rule based on the audit error text content when the type of the audit error text content under the first audit rule is compliance text content.
21. The device according to claim 20, wherein: The compliance example determination unit is specifically used to: Add the audit error text content to the compliance text content set under the first audit rule; Determining target compliant text content in the compliant text content set based on the number of repetitions of each compliant text content in the compliant text content set; Based on the target compliant text content, determine a compliant text example under the first audit rule.
22. The device according to claim 19, wherein The audit example determination module includes: A violation example determination unit is used to determine a violation text example under the first audit rule based on the audit error text content when the type of the audit error text content under the first audit rule is violation text content.
23. The device according to claim 22, wherein: The violation example determination unit is specifically used to: Add the audit error text content to the set of illegal text content under the first audit rule; Determining target illegal text content in the illegal text content set based on the number of repeated occurrences of each illegal text content in the illegal text content set; Based on the target illegal text content, an illegal text example under the first audit rule is determined.
24. A large model content review device, comprising: A prompt word determination module is used to determine the corresponding review prompt word based on the text content; A segmentation module, used to segment the text content to obtain multiple text paragraphs; A text splicing module, used to splice the audit prompt word with each of the text paragraphs to obtain the target prompt word corresponding to each of the text paragraphs; A text review module, used to input each of the target prompt words into a large language model respectively, and obtain a review result corresponding to each of the text paragraphs output by the large language model; The audit result determination module is used to determine the audit result of the text content based on the audit results corresponding to each of the text paragraphs.
25. The apparatus according to claim 24, further comprising: A prompt word updating module is used to update the audit prompt word according to the method according to any one of claims 1-8.
26. The device according to claim 24 or 25, wherein: The audit prompt words include at least one of the following: the audit role definition, audit task description, audit process, audit rules and output standards of the audit results for auditing the text content.
27. The device according to claim 26, wherein The audit rules also include at least one of an audit blacklist, an audit whitelist, and an audit example.
28. An electronic device comprising: at least one central processing unit; as well as A memory in communication with the at least one central processing unit; wherein, The memory stores instructions that can be executed by the at least one central processor, and the instructions are executed by the at least one central processor to enable the at least one central processor to perform the method according to any one of claims 1 to 15.
29. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-15.
30. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 15.
Citation Information
Cited By
Text auxiliary auditing and proofreading method and system based on large language model
CN119599004A
A Text-Assisted Proofreading Method and System Based on a Large Language Model
CN119599004B
Text processing method and device based on large model, electronic equipment and storage medium
CN119647465A
AI output presentation governance system, method, program
JP7891640B1
AI output presentation governance system, method, program
JP7930549B1
