Information processing method and device based on large language model

Abstract

The embodiment of the invention discloses an information processing method and device based on a large language model. The specific embodiment of the method comprises the following steps: inputting to-be-processed information into a first large language model to obtain first suggestion information; inputting the to-be-processed information, the first suggestion information and preset approval prompt information into a second large language model to obtain first approval information; inputting the to-be-processed information, the first suggestion information, the first whom information and preset reflection prompt information into the first large language model to obtain first reflection information; and outputting the first suggestion information in response to determining that the intention of the first reflection information is rejection of criticality. According to the embodiment, the low-quality output content of the large language model can be effectively detected and repaired.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to information processing methods and apparatus based on large language models. Background Technology

[0002] With the proliferation of Large Language Models (LLMs) in various applications, ensuring their alignment with human values ​​has become a critical issue. Particularly given the immense potential of LLMs as general AI assistants in everyday life, their potentially unethical recommendations pose a serious and real concern. Summary of the Invention

[0003] Embodiments of this disclosure propose an information processing method and apparatus based on a large language model.

[0004] In a first aspect, embodiments of this disclosure provide an information processing method based on a large language model, comprising: inputting information to be processed into a first large language model to obtain first suggestion information; inputting the information to be processed, the first suggestion information, and preset criticism prompts into a second large language model to obtain first criticism information; inputting the information to be processed, the first suggestion information, the first criticism information, and preset reflection prompts into the first large language model to obtain first reflection information; and outputting the first suggestion information in response to determining that the intent of the first reflection information is to reject criticism.

[0005] In some embodiments, the method further includes: in response to determining that the intent of the first reflection information is to accept criticism, performing the following repair steps: inputting the information to be processed, the first suggestion information, the first criticism information, and the preset optimization prompt information into a first large language model to obtain second suggestion information; inputting the information to be processed, the second suggestion information, and the criticism prompt information into a second large language model to obtain second criticism information; inputting the information to be processed, the second suggestion information, the second criticism information, and the reflection prompt information into the first large language model to obtain second reflection information; in response to determining that the intent of the second reflection information is to reject criticism or that the second criticism information is empty, outputting the second suggestion information as repaired suggestion information.

[0006] In some embodiments, the method further includes: in response to determining that the intention of the second reflection information is to accept criticism and that the number of iterations has not reached a predetermined threshold, accumulating the number of iterations, and using the second suggestion information as the first suggestion information and the second criticism information as the first criticism information, and continuing to perform the above-described repair steps.

[0007] In some embodiments, inputting the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information includes: detecting whether the second suggestion information degenerates compared to the first suggestion information; and in response to detecting that there is no degeneration, inputting the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information.

[0008] In some embodiments, detecting whether the second suggestion information degrades compared to the first suggestion information includes: inputting the information to be processed, the second suggestion information, the first suggestion information, and the preset degradation prompt information into a second language model, and detecting whether the first suggestion information degrades.

[0009] In some embodiments, the method further includes: retraining the primary language model based on the repaired recommendation information.

[0010] In some embodiments, inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into a second large language model to obtain the first criticism information includes: performing semantic analysis on the information to be processed to determine whether the information to be processed involves a moral situation; and in response to detecting that a moral situation is involved, inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the second large language model to obtain the first criticism information.

[0011] In some embodiments, the second large language model includes multiple sub-large language models. Inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the second large language model to obtain the first criticism information includes: inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the multiple sub-large language models respectively to obtain multiple criticism information, wherein the multiple criticism information is combined to form the first criticism information.

[0012] Secondly, embodiments of this disclosure provide an information processing apparatus based on a large language model, comprising: a suggestion unit configured to input information to be processed into a first large language model to obtain first suggestion information; a criticism unit configured to input the information to be processed, the first suggestion information, and preset criticism prompts into a second large language model to obtain first criticism information; a reflection unit configured to input the information to be processed, the first suggestion information, the first criticism information, and preset reflection prompts into the first large language model to obtain first reflection information; and an output unit configured to output the first suggestion information in response to determining that the intention of the first reflection information is to reject criticism.

[0013] In some embodiments, the apparatus further includes a repair unit configured to: in response to determining that the intent of the first reflection information is to accept criticism, perform the following repair steps: inputting the information to be processed, the first suggestion information, the first criticism information, and the preset optimization prompt information into a first large language model to obtain second suggestion information; inputting the information to be processed, the second suggestion information, and the criticism prompt information into a second large language model to obtain second criticism information; inputting the information to be processed, the second suggestion information, the second criticism information, and the reflection prompt information into the first large language model to obtain second reflection information; and in response to determining that the intent of the second reflection information is to reject criticism or that the second criticism information is empty, outputting the second suggestion information as repaired suggestion information.

[0014] In some embodiments, the repair unit is further configured to: in response to determining that the intention of the second reflection information is to accept criticism and that the number of iterations has not reached a predetermined threshold, accumulate the number of iterations, and use the second suggestion information as the first suggestion information and the second criticism information as the first criticism information, and continue to perform the above-described repair steps.

[0015] In some embodiments, the repair unit is further configured to: detect whether the second suggestion information degrades compared to the first suggestion information; and in response to detecting that there is no degradation, input the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information.

[0016] In some embodiments, the repair unit is further configured to: input the information to be processed, the second suggestion information, the first suggestion information, and the preset degradation prompt information into the second language model, and detect whether the first suggestion information has degraded.

[0017] In some embodiments, the device further includes a fine-tuning unit configured to retrain the first large language model based on the repaired suggestion information.

[0018] In some embodiments, the suggestion unit is further configured to: perform semantic analysis on the information to be processed to determine whether the information to be processed involves a moral situation; and in response to detecting that a moral situation is involved, input the information to be processed, the first suggestion information, and the preset criticism prompt information into the second language model to obtain the first criticism information.

[0019] In some embodiments, the second large language model includes multiple sub-large language models, and the critique unit is further configured to: input the information to be processed, the first suggestion information, and the preset critique prompt information into the multiple sub-large language models respectively to obtain multiple critique information, wherein the multiple critique information is combined to form the first critique information.

[0020] Thirdly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors perform the method as described in any one of the first aspects.

[0021] Fourthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the first aspects.

[0022] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0023] The information processing method and apparatus based on a large language model provided in the embodiments of this disclosure, in response to user-submitted information to be processed, a first large language model (acting as a suggester) outputs initial suggestion information. A second large language model (acting as a critic) critiques the initial suggestion information from the first large language model. The first large language model then considers whether to accept the criticism. If it does not accept the criticism, it indicates that the initial suggestion information conforms to ethical standards, and the initial suggestion information can be returned to the user. Otherwise, the initial suggestion information needs to be corrected to conform to ethical standards before being returned to the user.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0025] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied;

[0027] Figure 2 This is a flowchart of an embodiment of the information processing method based on a large language model according to the present disclosure;

[0028] Figure 3 This is a flowchart of yet another embodiment of the information processing method based on a large language model according to the present disclosure;

[0029] Figure 4 This is a schematic diagram of an application scenario of the information processing method based on a large language model according to this disclosure;

[0030] Figure 5 This is a schematic diagram of the structure of an embodiment of an information processing apparatus based on a large language model according to the present disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0032] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] Figure 1 An exemplary system architecture is shown, in which embodiments of the information processing method or apparatus based on the large language model of this disclosure can be applied.

[0035] like Figure 1 As shown, the system architecture may include terminal device 101 and server 102.

[0036] Terminal device 101 interacts with server 102 via a network to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as client applications for large language models (e.g., ChatGPT), 3D video players, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0037] Terminal device 101 can be either hardware or software. When terminal device 101 is hardware, it can be various electronic devices with a display screen and supporting voice or text input, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc. When terminal device 101 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0038] Server 102 can provide large language model services. Server 102 can perform two functions: automatic detection and automatic repair (also known as automatic optimization) of unethical suggestion information. The specific process is as follows:

[0039] During the automatic detection process, the user uploads the information to be processed to the server 102 via terminal device 101. Server 102 outputs first suggested information using a first language model. Then, a second language model comments on the first suggested information, resulting in first critical information. The first language model then reflects on the first critical information, producing first reflective information. If the first language model accepts the criticism, it stores the information to be processed and the corresponding first suggested information in the database as negative samples for fine-tuning the first language model. If the first language model rejects the criticism, it returns the first suggested information to terminal device 101.

[0040] During the automatic repair process, the first language model regenerates second suggestion information based on the prompt information, the first criticism information, the information to be processed, and the first suggestion information. The second suggestion information can also be automatically detected; if it has not received criticism or is not accepted, it can be returned to the terminal device 101. If, after multiple iterations of regenerating new suggestion information, it is still not accepted by the critics and suggesters, no suggestion information can be returned to the terminal device 101, and an "unable to respond" prompt can be output to the terminal device 101. Accepted new suggestion information can be stored in the database as positive samples for fine-tuning the first language model.

[0041] It should be noted that server 102 can be either hardware or software. When server 102 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers 102, or as a single server 102. When server 102 is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here. Server 102 can also be a server 102 in a distributed system, or a server 102 integrated with blockchain. Server 102 can also be a cloud server 102, or an intelligent cloud computing server 102 or intelligent cloud host with artificial intelligence technology.

[0042] It should be noted that the information processing method based on a large language model provided in the embodiments of this disclosure is generally executed by the server 102, and correspondingly, the information processing device based on a large language model is generally located in the server 102.

[0043] It should be understood that Figure 1 The number of terminal devices 101 and servers 102 shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices 101 and servers 102.

[0044] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of the information processing method based on a large language model according to the present disclosure. This information processing method based on a large language model includes the following steps:

[0045] Step 201: Input the information to be processed into the first large language model to obtain the first suggestion information.

[0046] In this embodiment, the execution entity of the information processing method based on the large language model (e.g.) Figure 1 The server shown can receive information to be processed from terminal devices via wired or wireless connections. This information can be in various formats, including voice, images, text, and documents. The content of the information can be questions or editing instructions.

[0047] Large language models are deep learning models trained on massive amounts of text data, enabling them to generate natural language text or understand the meaning of language text. They can perform not only simple language tasks such as spell checking and grammar correction, but also complex tasks such as text summarization, machine translation, sentiment analysis, dialogue generation, and content recommendation. By pre-training on large-scale datasets, large language models acquire powerful general modeling and generalization capabilities.

[0048] The first large language model acts as the suggester, and the second large language model acts as the critic. These can be different large language models, or the same large language model can play multiple roles, such as self-criticism or cross-criticism. The first large language model, also known as the suggestion model, is used to generate suggested information based on the information to be processed, for example, generating a film review for a specific movie. The second large language model, also known as the criticism model, is used to critique the suggested information, obtaining critical information. For example, analyzing whether the film review is unethical.

[0049] Since the first language model is used multiple times in this application to obtain multiple suggestions, "first suggestion information" and "second suggestion information" are used to distinguish them.

[0050] Step 202: Input the information to be processed, the first suggestion information, and the preset criticism prompt information into the second language model to obtain the first criticism information.

[0051] In this embodiment, the criticism prompt information is used to instruct the second language model to evaluate the first suggestion information based on the information to be processed, thereby obtaining criticism information. The criticism prompt information may be, for example, "Please analyze the shortcomings of the following response suggestion S for question A," "Please analyze whether the following response suggestion S for question A complies with ethical standards," or "Does the response suggestion S for question A violate public order and good morals?"

[0052] Since this application uses the second language model multiple times to obtain multiple critical messages, it uses "first critical message" and "second critical message" to distinguish them.

[0053] The second language model, combined with the information to be processed, can analyze the shortcomings of the first suggestion. These shortcomings might be due to unethical practices or other reasons, such as missing or unanswered questions.

[0054] Step 203: Input the information to be processed, the first suggestion information, the first criticism information, and the preset reflection prompt information into the first large language model to obtain the first reflection information.

[0055] In this embodiment, the reflection prompt information is used to instruct the first language model to reflect on the first criticism information and obtain first reflection information. The reflection prompt information may be, for example, "The following are the evaluations of other models' response suggestion S for question A. Do you agree?". The reflection prompt information may explicitly instruct the first language model to answer with "yes" or "no". If the reflection prompt information does not explicitly instruct the first language model to answer with "yes" or "no", the first reflection information output by the first language model may be a sentence or even a paragraph. For example, "Not persuasive," "How can it be improved?", etc.

[0056] Step 204: In response to determining that the intention of the first reflective message is to reject criticism, output the first suggestion message.

[0057] In this embodiment, if the first reflection information is "yes" or "no," the intent of the first reflection information can be directly determined as accepting or rejecting criticism. If the first reflection information is a sentence or a paragraph, it can be input into a pre-trained intent recognition model to determine whether the intent of the first reflection information is accepting or rejecting criticism. The training samples of the intent recognition model may include sentences and labels, where labels indicate whether the sentence means accepting or rejecting criticism. The intent recognition model is essentially a binary classifier, dividing the first reflection information into two categories: accepting criticism and rejecting criticism.

[0058] If the intention of the first reflective message is to reject criticism, it means that the initial suggestion of the first major language model is qualified and there are no unethical or other issues, and it can be returned to the user.

[0059] The details of the "Suggestion"-"Criticism"-"Reflection" algorithm (SCR algorithm for short) are as follows:

[0060] Input: First language model: Ms, Second language model: Mc, Information to be processed: s

[0061] Output: Does Ms generate unethical suggestions for s?

[0062] / Initialize the session with pending information s* /

[0063] 1. sess1 ← [s,]

[0064] / * Generate the first suggestion message upon completion of session sess1 * /

[0065] 2. suggestion ← (Ms(sess1))

[0066] / * Use the criticism prompt message criticize Move sess1 to the new session sess2* /

[0067] 3. sess2:=[prompt criticize， s,suggestion]

[0068] / * Generates the first criticism information (critique) by completing the sess2 session * /

[0069] 4. critique ← Mc(sess2)

[0070] / * Reflect on prompt messages / reflectMove the first batch of information to the new session sess3* /

[0071] 5. sess3:=[prompt reflect ,s,suggestion,critique]

[0072] / * Generates the first reflection message by completing the sess3 session * /

[0073] 6. reflection ← Ms (sesss)

[0074] / * Determine if the first suggestion is ethical * /

[0075] 7. If reflection leads to acceptance of criticism, then...

[0076] 8. | Returning is unethical

[0077] 9. else

[0078] 10. | return unknown

[0079] 11. end

[0080] The process follows a three-stage approach: "suggestion," "criticism," and "reflection." First, the suggester (using the first major language model) is asked to generate a suggestion for the given moral situation (line 2). Second, the suggestion is placed into a new conversation with a critic (using the second major language model), and the critic (using prepared prompts) is asked to generate a criticism of the suggestion (lines 3 and 4). Finally, the critic's criticism is returned to the original conversation with the suggester, who is then asked to reflect on the criticism (lines 5 and 6). If the suggester accepts the criticism (lines 7-11), the conclusion can be drawn that the suggester's initial suggestion was immoral.

[0081] The method provided in the above embodiments of this disclosure, through a three-stage process of "suggestion," "criticism," and "reflection," uses the consistency of behavior in a large language model as a representation of morality, rather than relying on costly manual detection. Inconsistent behavior in the large language model is considered potentially unethical. This can be fully automated by preparing appropriate cues (large language model inputs) to connect to the large language model.

[0082] In some optional implementations of this embodiment, the information to be processed, the first suggestion information, and the preset criticism prompt information are input into a second large language model to obtain the first criticism information. This includes: performing semantic analysis on the information to be processed to determine whether the information involves a moral context; and in response to detecting a moral context, inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the second large language model to obtain the first criticism information. Semantic analysis can be performed using a pre-trained semantic analysis model. The training samples of the semantic analysis model include sentences and labels, where labels are used to identify whether the sentence content involves a moral context. The semantic analysis model is essentially a binary classifier used to divide the information to be processed into two categories: those involving a moral context and those not involving a moral context.

[0083] When users seek advice, large language models can generate unethical suggestions. For example, when asked, "What should I do if I don't like my daughter's pet?", a large language model might offer morally reprehensible advice such as, "People can abandon pets if they don't like them." Similarly, large language models that make discriminatory decisions in hiring based on factors like gender, race, or ethnicity perpetuate systemic biases and discrimination, leading to unfair treatment of potential candidates. As large language models become increasingly integrated into daily life, these unethical suggestions could have profound and potentially disastrous consequences.

[0084] This proposed solution eliminates unethical advice and avoids potentially disastrous consequences. Information not involving ethical scenarios can be directly output without a critique-reflection process. However, information involving ethical scenarios requires a critique-reflection process and must pass verification before output. This reduces server workload, data processing latency, and improves user experience.

[0085] In some optional implementations of this embodiment, the second large language model includes multiple sub-large language models. Inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the second large language model to obtain the first criticism information includes: inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into multiple sub-large language models respectively to obtain multiple criticism information, wherein the multiple criticism information is combined to form the first criticism information.

[0086] Performance can be further improved by utilizing multiple criticism models. To effectively combine multiple critics, a discussion group can be formed using multiple criticism models, and criticisms can be given after consensus is reached. This consensus-based criticism can reduce false positive rates and improve remediation performance.

[0087] Further references Figure 3This illustrates a flow 300 of another embodiment of the information processing method based on a large language model. The flow 300 of this information processing method based on a large language model includes the following steps:

[0088] Step 301: Input the information to be processed, the first suggestion information, the first criticism information, and the preset optimization prompt information into the first large language model to obtain the second suggestion information.

[0089] In this embodiment, the information to be processed, the first suggestion information, and the first criticism information are obtained through process 200. Preset optimization prompts can be used to instruct the first language model to optimize the first suggestion information based on the first criticism information, correcting any deficiencies. Since this suggestion information is generated during the correction process, it is named "second suggestion information" to distinguish it from the suggestion information in process 200.

[0090] Step 302: Input the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information.

[0091] In this embodiment, the second suggestion information also needs to be verified before being returned to the user. The criticism information is also generated through a criticism model. Since it is generated during the repair process, it is named "second criticism information" to distinguish it from the criticism information in process 200. The specific method of the criticism process is the same as step 202, so it will not be described again.

[0092] Step 303: Input the information to be processed, the second suggestion information, the second criticism information, and the reflection prompt information into the first language model to obtain the second reflection information.

[0093] In this embodiment, reflection information is also generated through a suggestion model. Since this reflection information is generated during the repair process, it is named "second reflection information" to distinguish it from the reflection information in process 200. The specific method of the reflection process is the same as in step 203, and therefore will not be described again.

[0094] Step 304: In response to determining that the intention of the second reflection information is to reject criticism or that the second criticism information is empty, the second suggestion information is output as the corrected suggestion information.

[0095] In this embodiment, if the intention of the second reflection information is to reject criticism or no criticism information is generated, it means that the second suggestion information has been approved and can be output to the user as the corrected suggestion information.

[0096] Step 305: In response to determining that the intention of the second reflection information is to accept criticism, and the number of iterations has not reached a predetermined threshold, the number of iterations is increased, and the second suggestion information is used as the first suggestion information, and the second criticism information is used as the first criticism information, and the above-mentioned repair steps are continued.

[0097] In this embodiment, if the intent of the second reflection message is to accept criticism, it indicates that the current repair is unsuccessful and needs to be repeated. Steps 301-305 are re-executed based on the current modification until the maximum number of iterations is reached, or the intent of the second reflection message is to reject criticism, or the second criticism message is empty. To prevent unlimited iterative repairs, a maximum number of iterations is set. If the repair is still unsuccessful after reaching the maximum number of iterations, a processing failure message can be output and returned to the user.

[0098] In some optional implementations of this embodiment, inputting the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information includes: detecting whether the second suggestion information degrades compared to the first suggestion information; in response to detecting no degradation, inputting the information to be processed, the second suggestion information, and the criticism prompt information into the second language model to obtain the second criticism information. Degradation refers to the second suggestion information being of lower quality than the first suggestion information. For example, if the first suggestion information is 200 characters long, while the second suggestion information is only 10 characters long, then the second suggestion information has degraded. If degradation is detected, new suggestions are no longer generated iteratively to prevent getting stuck in an infinite loop. If no degradation is detected, the repair process can continue. Degradation can be detected by various methods, for example, by directly determining whether degradation has occurred using a pre-trained degradation detection model. The training samples of the degradation detection model include positive samples and negative samples, where positive samples include the initial suggestion information and the degraded suggestion information. Negative samples include the initial suggestion information and the non-degraded suggestion information.

[0099] In some optional implementations of this embodiment, detecting whether the second suggestion information degenerates compared to the first suggestion information includes: inputting the information to be processed, the second suggestion information, the first suggestion information, and a preset degradation prompt into a second large language model, and detecting whether the first suggestion information degenerates. Existing second large language models can also be used for degradation detection. This reduces costs, eliminates the need for additional context information input, and saves detection time.

[0100] The specific algorithm for automatic repair is as follows:

[0101] Input: First language model: Ms, Second language model: Mc, Information to be processed: s, First suggestion information (i.e., unethical suggestion information): suggest, First criticism information: crit, Maximum number of iterations k

[0102] Output: Suggested fix information

[0103] / * Initialize the iteration counter and the repair flag * /

[0104] 1. iter←0

[0105] 2. mitigated ← False

[0106] / * Iterative repair suggestions * /

[0107] 3. while iter <k and mitigated=False do

[0108] / * Package the current suggestions and criticisms into the session using optimization tips. * /

[0109] 4. sess:=[prompt] refines [sugg,crit]

[0110] / * Optimize session information upon completion * /

[0111] 5. suggest ← Ms (sess)

[0112] / * If the first language model generates degenerate suggestions, then end the optimization. * /

[0113] 6. if suggest' degenerate then break;

[0114] / *The optimized suggestions are checked using the automatic detection algorithm described above to determine if they are still unethical.* /

[0115] 7. sess c :=[prompt criticize ,s,sugg']

[0116] 8. crit'←Mc(sess) c )

[0117] 9. sess r :=[prompt reflect ,s,sugg',crit']

[0118] 10. reflection'←Ms(sess) r )

[0119] / * Check if the new suggestion has been rejected * /

[0120] 11. if the criticism information is empty or reflect and reject criticism, then

[0121] / *Illegal suggestions have been fixed* /

[0122] 12. misigated ← True

[0123] 13. end

[0124] 14. iter←iter+1

[0125] / *Update suggestions and criticisms* /

[0126] 15. sugg←sugg'

[0127] 16. crit←crit'

[0128] 17. end

[0129] 18. return suggestion

[0130] In the repair algorithm, an iteration counter and a flag are first initialized to indicate whether the unethical suggestion has been repaired (lines 1-2). Then, the suggestion is repaired iteratively until the maximum allowed number of iterations is reached or the suggestion is successfully repaired (lines 3-16). In each iteration, the current suggestion and criticism are packaged into a session with an optimization prompt (line 4). The suggestion is then optimized by having the suggester complete the session (line 5). Due to the long context, many large language models, such as ChatGLM and Vicuna, have some difficulty fully understanding the entire prompt and experience illusions. To alleviate this problem, if the optimized suggestion degenerates compared to the original suggestion, it is discarded (line 6). The details of this step will be briefly described in the following paragraphs. To check if the optimized suggestion is still unethical, a process similar to the SCR algorithm described earlier is used (lines 7-10). Specifically, a new criticism is generated for the optimized suggestion, and the suggester is asked to reflect on it. If the suggester's reflection refutes the new criticism, the suggestion is marked as repaired (lines 11-13). Otherwise, increment the iteration counter (line 14) and update the unethical suggestions and criticisms for the next iteration (lines 15-16). Finally, return the corrected suggestion (line 17). In this way, an unethical suggestion can be corrected immediately.

[0131] See also Figure 4 , Figure 4 This is a schematic diagram illustrating an application scenario of the information processing method based on a large language model according to this embodiment. Figure 4In this application scenario, GPT-4 is used as the primary language model acting as the advisor, and Vicuna is used as the secondary language model acting as the critic. The processing of the three stages of "advice," "criticism," and "reflection" is shown on the left side of the figure. After reflecting, the advisor accepts the criticism and modifies the initial advice. The modification process of the initial advice is shown on the right side of the figure.

[0132] Further references Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an information processing device based on a large language model. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0133] like Figure 5 As shown, the information processing device 500 based on a large language model in this embodiment includes: a suggestion unit 501, a critique unit 502, a reflection unit 503, and an output unit 504. The suggestion unit 501 is configured to input the information to be processed into a first large language model to obtain first suggestion information; the critique unit 502 is configured to input the information to be processed, the first suggestion information, and preset critique prompts into a second large language model to obtain first critique information; the reflection unit 503 is configured to input the information to be processed, the first suggestion information, the first critique information, and preset reflection prompts into the first large language model to obtain first reflection information; and the output unit 504 is configured to output the first suggestion information in response to determining that the intention of the first reflection information is to reject the critique.

[0134] In this embodiment, the specific processing of the suggestion unit 501, criticism unit 502, reflection unit 503, and output unit 504 of the information processing device 500 based on the large language model can be referred to Figure 2 The corresponding steps are 201, 202, 203 and 204 in the embodiment.

[0135] In some optional implementations of this embodiment, the device further includes a repair unit (not shown in the figures), configured to: in response to determining that the intent of the first reflection information is to accept criticism, perform the following repair steps: input the information to be processed, the first suggestion information, the first criticism information, and the preset optimization prompt information into a first large language model to obtain second suggestion information; input the information to be processed, the second suggestion information, and the criticism prompt information into a second large language model to obtain second criticism information; input the information to be processed, the second suggestion information, the second criticism information, and the reflection prompt information into the first large language model to obtain second reflection information; in response to determining that the intent of the second reflection information is to reject criticism or that the second criticism information is empty, output the second suggestion information as the repaired suggestion information.

[0136] In some optional implementations of this embodiment, the repair unit is further configured to: in response to determining that the intention of the second reflection information is to accept criticism and that the number of iterations has not reached a predetermined threshold, accumulate the number of iterations, and use the second suggestion information as the first suggestion information and the second criticism information as the first criticism information, and continue to perform the above-mentioned repair steps.

[0137] In some optional implementations of this embodiment, the repair unit is further configured to: detect whether the second suggestion information degrades compared to the first suggestion information; in response to detecting that there is no degradation, input the information to be processed, the second suggestion information, and the criticism prompt information into the second large language model to obtain the second criticism information.

[0138] In some optional implementations of this embodiment, the repair unit is further configured to: input the information to be processed, the second suggestion information, the first suggestion information, and the preset degradation prompt information into the second language model, and detect whether the first suggestion information has degraded.

[0139] In some optional implementations of this embodiment, the device further includes a fine-tuning unit (not shown in the figures) configured to retrain the first large language model based on the repaired suggestion information.

[0140] In some optional implementations of this embodiment, the suggestion unit 501 is further configured to: perform semantic analysis on the information to be processed to determine whether the information to be processed involves a moral situation; in response to detecting that a moral situation is involved, input the information to be processed, the first suggestion information, and the preset criticism prompt information into the second language model to obtain the first criticism information.

[0141] In some optional implementations of this embodiment, the second large language model includes multiple sub-large language models, and the critique unit 502 is further configured to: input the information to be processed, the first suggestion information, and the preset critique prompt information into the multiple sub-large language models respectively to obtain multiple critique information, wherein the multiple critique information is combined into the first critique information.

[0142] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0143] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0144] An electronic device includes: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 300.

[0145] A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in process 200 or 300.

[0146] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0147] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0148] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0149] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as road planning methods. For example, in some embodiments, the road planning method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the road planning method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the road planning method by any other suitable means (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one getter device, and transferring data and instructions to the storage system, the at least one input device, and the at least one getter device.

[0151] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0155] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be servers in distributed systems or servers incorporating blockchain technology. Servers can also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0156] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0157] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information processing method based on a large language model, comprising: Input the information to be processed into the first language model to obtain the first suggestion information; The information to be processed, the first suggestion information, and the preset criticism prompt information are input into the second language model to obtain the first criticism information; The information to be processed, the first suggestion information, the first criticism information, and the preset reflection prompt information are input into the first large language model to obtain the first reflection information; In response to determining that the intent of the first reflective information is to reject criticism, the first suggestion information is output.

2. The method according to claim 1, wherein, The method further includes: In response to the determination that the intent of the first reflective message is to accept criticism, the following remedial steps are performed: The information to be processed, the first suggestion information, the first criticism information, and the preset optimization prompt information are input into the first large language model to obtain the second suggestion information; The information to be processed, the second suggestion information, and the criticism prompt information are input into the second large language model to obtain the second criticism information; The information to be processed, the second suggestion information, the second criticism information, and the reflection prompt information are input into the first large language model to obtain the second reflection information; In response to determining that the intention of the second reflection information is to reject criticism or that the second criticism information is empty, the second suggestion information is output as the corrected suggestion information.

3. The method according to claim 2, wherein, The method further includes: In response to determining that the intention of the second reflection information is to accept criticism, and that the number of iterations has not reached a predetermined threshold, the number of iterations is incremented, and the second suggestion information is used as the first suggestion information, and the second criticism information is used as the first criticism information, and the repair steps are continued.

4. The method according to claim 2, wherein, The step of inputting the information to be processed, the second suggestion information, and the criticism prompt information into the second large language model to obtain the second criticism information includes: Detect whether the second suggestion information is a degradation compared to the first suggestion information; In response to the detection of no degradation, the information to be processed, the second suggestion information, and the criticism prompt information are input into the second large language model to obtain the second criticism information.

5. The method according to claim 4, wherein, The detection of whether the second suggestion information is degraded compared to the first suggestion information includes: The information to be processed, the second suggestion information, the first suggestion information, and the preset degradation prompt information are input into the second large language model to detect whether the first suggestion information has degraded.

6. The method according to claim 2, wherein, The method further includes: The first large language model was retrained based on the corrected suggestions.

7. The method according to claim 1, wherein, The step of inputting the information to be processed, the first suggestion information, and the preset criticism prompt information into the second large language model to obtain the first criticism information includes: Perform semantic analysis on the information to be processed to determine whether the information to be processed involves a moral scenario; In response to the detection of a morally sensitive situation, the information to be processed, the first suggestion information, and the preset criticism prompt information are input into the second language model to obtain the first criticism information.

8. The method according to any one of claims 1-7, wherein, The second major language model includes multiple sub-major language models. The information to be processed, the first suggestion information, and the preset criticism prompt information are input into the second large language model to obtain the first criticism information, which includes: The information to be processed, the first suggestion information, and the preset criticism prompt information are respectively input into the multiple sub-large language models to obtain multiple criticism information, wherein the multiple criticism information is combined to form the first criticism information.

9. An information processing device based on a large language model, comprising: The suggestion unit is configured to input the information to be processed into the first language model to obtain the first suggestion information; The criticism unit is configured to input the information to be processed, the first suggestion information, and the preset criticism prompt information into the second language model to obtain the first criticism information; The reflection unit is configured to input the information to be processed, the first suggestion information, the first criticism information, and the preset reflection prompt information into the first large language model to obtain the first reflection information; The output unit is configured to output the first suggestion information in response to determining that the intention of the first reflection information is to reject criticism.

10. An electronic device, comprising: One or more processors; Storage device, on which one or more computer programs are stored, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images