Method and apparatus for information processing, device, and storage medium

Abstract

Embodiments of the present disclosure relate to a method and apparatus for information processing, a device, and a storage medium. The method proposed herein comprises: acquiring an input message; in an interactive interface, presenting a plurality of pieces of response content generated on the basis of the input message by a plurality of virtual objects; and providing evaluation information for the plurality of pieces of response content.

Description

Information processing methods, apparatus, equipment and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411464112.2, filed on October 18, 2024, entitled "Method, Apparatus, Device and Storage Medium for Information Processing", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The exemplary embodiments disclosed herein generally relate to the field of computers, and more particularly to methods, apparatus, devices and computer-readable storage media for information processing. Background Technology

[0003] With the development of artificial intelligence technology, generative AI technology has been widely applied to various applications. For example, developers can create applications such as agents or bots based on generative models. Different agents or bots may produce different outputs for the same input. Summary of the Invention

[0004] In a first aspect of this disclosure, an information processing method is provided. The method includes: acquiring an input message; presenting, in an interactive interface, multiple response contents generated by multiple virtual objects based on the input message; and providing evaluation information for the multiple response contents.

[0005] In a second aspect of this disclosure, an apparatus for information processing is provided. The apparatus includes: an acquisition module configured to acquire an input message; a presentation module configured to present, in an interactive interface, multiple response contents generated by multiple virtual objects based on the input message; and a providing module configured to provide evaluation information for the multiple response contents.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;

[0012] Figures 2A to 2F illustrate example interfaces according to some embodiments of the present disclosure;

[0013] Figure 3 illustrates a flowchart of an example process for information processing according to some embodiments of the present disclosure;

[0014] Figure 4 shows a schematic structural block diagram of an example apparatus for information processing according to some embodiments of the present disclosure; and

[0015] Figure 5 shows a block diagram of an electronic device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0020] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0021] As mentioned above, different agents or machine programs may produce different outputs for the same input. In this situation, on the one hand, the quality of the results obtained by the user may be poor when using a single agent or machine program. On the other hand, the user may need to repeatedly switch between different agents or machine programs, which increases the user's interaction costs.

[0022] Embodiments of this disclosure propose a scheme for information processing. The scheme includes: acquiring an input message; presenting multiple response contents generated by multiple virtual objects based on the input message in an interactive interface; and providing evaluation information for the multiple response contents.

[0023] In this way, embodiments of this disclosure can provide users with the output results of multiple virtual objects (e.g., intelligent agents or machine programs) for the same request, thereby improving the user's information acquisition efficiency. Furthermore, by providing evaluation information on the response content, embodiments of this disclosure can also help users better perceive the differences between the output results of different virtual objects.

[0024] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0025] Example Environment

[0026] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in Figure 1, the example environment 100 may include an electronic device 110.

[0027] In this example environment 100, electronic device 110 may run an application 120 that supports information processing. Application 120 may be any suitable type of application for information processing, examples of which may include, but are not limited to, instant messaging applications, agent dialogue applications, large model question-and-answer applications, or other suitable applications. User 140 may interact with application 120 via electronic device 110 and / or its attached devices.

[0028] In environment 100 of Figure 1, if application 120 is active, electronic device 110 can present interface 150 for supporting information processing through application 120.

[0029] In some embodiments, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0030] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 that support information processing in electronic device 110.

[0031] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, server 130 and electronic device 110 can achieve signaling interaction through the communication connection between them.

[0032] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0033] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0034] Example Interaction

[0035] Figures 2A to 2F illustrate example interfaces 200A to 200F according to some embodiments of the present disclosure. Interfaces 200A to 200F may be provided, for example, by the electronic device 110 shown in Figure 1.

[0036] In some embodiments, the electronic device 110 may first present a configuration interface for virtual objects. For example, the electronic device 110 may first present a configuration interface for selecting virtual objects, through which the electronic device 110 can receive configuration operations from the user to add any available virtual objects. After configuring the virtual object to be used for dialogue or testing via the configuration interface, the electronic device 110 may present an interactive interface such as interface 200A, allowing the user to begin a dialogue with the selected virtual object.

[0037] In some scenarios, virtual objects can also be referred to as virtual entities. Examples of such entities include, but are not limited to, agents, bots, or other processing entities that support content interaction triggered by dialogue. Such virtual objects can, for example, be implemented using appropriate generative models to generate interactive content based on received prompts.

[0038] In some embodiments, the virtual object selected by the user may include an intelligent agent associated with the user. For example, the user may add a virtual object created by the user, a virtual object edited by the user, a virtual object from the user's favorites, etc. For example, the electronic device 110 may present the user with multiple candidate virtual objects to choose from.

[0039] In some embodiments, referring to FIG2A, the electronic device 110 may present an interface 200A, which may include an object configuration area 250 for presenting virtual objects and a message area 215.

[0040] In some embodiments, the virtual objects configured in the configuration interface described above can be displayed in the object configuration area 250. In other embodiments, the object configuration area 250 may also display some preset virtual objects. The electronic device 110 can receive further configurations from the user for the virtual objects via the object configuration area 250, and change the virtual objects displayed in the object configuration area 250 based on the further configurations.

[0041] In some embodiments, the electronic device 110 may present multiple virtual objects in the object configuration area 250. For example, as shown in interface 200A, the object configuration area 250 may present object 1 210-1, object 2 210-2, and object 3 210-3. Furthermore, the object configuration area 250 may also present object description information for the multiple virtual objects. For example, the object configuration area 250 may also present the "number of times" of each virtual object and the corresponding curve showing the change in the number of times it is called.

[0042] In some embodiments, the object configuration area 250 may also display an add control 212, which users can click to add new virtual objects.

[0043] For example, electronic device 110 can receive messages from users in the input box 202 of message area 215 and receive a trigger operation from send control 204 to input the message to a virtual object, which then provides a corresponding response. Furthermore, electronic device 110 can also receive a trigger operation from share control 220 to generate shareable content, such as a report, for this set of inputs and responses. This shareable content may include the message input by the user and the responses or results generated by different virtual objects in response to the input message. Additionally, the shareable content may include evaluation information for different responses.

[0044] In some scenarios, the message interface 215 can also display sample input messages, such as Example 206-1 and Example 206-2, for user reference. It's important to note that the user's input in the input box 202 can be text or other types of input, such as image input, URL input, or voice input. Virtual objects with corresponding processing capabilities can provide appropriate responses based on the type of input. For example, if a user inputs a question or message via voice, the virtual object can generate the speech and provide it to the user as the result.

[0045] In some embodiments, referring to FIG2B, electronic device 110 may present interface 200B, which illustrates a scenario of a user engaging in a single-turn dialogue.

[0046] In some embodiments, the electronic device 110 can acquire input messages. The electronic device 110 can present multiple response contents generated by multiple virtual objects based on the input messages in an interactive interface. Additionally, the electronic device 110 can also provide evaluation information for the multiple response contents.

[0047] As shown in Figure 2B, the electronic device 110 can acquire input message 212-1, and input message 212-1 is presented in message area 215. Object configuration area 250 presents pre-configured virtual objects: object 1 210-1, object 2 210-2, and object 3 210-3. These three virtual objects generate response content 214-1, response content 214-2, and response content 214-3 respectively in response to input message 212-1.

[0048] The electronic device 110 can provide multiple messages generated by multiple virtual objects based on input message 212-1 in parallel within the session area (i.e., message area 215) of the interactive interface. These multiple messages correspond to multiple response contents. For example, response contents 214-1, 214-2, and 214-3 can be displayed together in area 215 as multiple messages.

[0049] It should be noted that although the example response content presented in message area 215 is in text form, these response contents can also be in other forms, such as voice messages, which the user can trigger the electronic device 110 to play through its playback device by clicking on response content 214-1.

[0050] In some embodiments, the electronic device 110 also provides evaluation information for multiple response contents. For example, the interface 200B also presents evaluation information 216-1 corresponding to response content 214-1, evaluation information 216-2 corresponding to response content 214-2, and evaluation information 216-3 corresponding to response content 214-3.

[0051] In some embodiments, evaluation information for multiple response contents is generated by providing input messages and multiple response contents to a target model. For example, by inputting input message 212-1 and response contents 214-1, 214-2, and 214-3 to a target model for generating evaluation information, the electronic device 110 can present corresponding evaluation information. Here, the target model can be a model with analytical capabilities or an intelligent agent with analytical capabilities.

[0052] In some embodiments, the aforementioned evaluation information may include the score of the corresponding response content and / or the evaluation text of the corresponding response content and / or the evaluation result of the corresponding response content regarding a preset evaluation dimension. For example, evaluation information 216-1 may present a score of "90" for response content 214-1, and may also present the evaluation text of response content 214-1 as well as the evaluation results of response content 214-1 in the preset evaluation dimension, namely "Solution-5" and "Context-5".

[0053] In some scenarios, examples of evaluation dimensions may include, but are not limited to: the memory size occupied by the virtual object when generating response content, the capabilities invoked by the virtual object when generating response content, the workflow of the virtual object, the feasibility of the solution provided in the response content, and the relevance of the response content to the context of the dialogue. As shown in Figure 2B, response content 214-1 is deficient in the above two dimensions, "solution" and "context," thus incurring corresponding deductions.

[0054] In some embodiments, the evaluation information for multiple response contents can be aggregated evaluation content for multiple response contents and / or multiple evaluation contents corresponding to multiple response contents. In the scenario presented in Figure 2B, the interface 200B displays the multiple evaluation contents corresponding to multiple response contents. However, in other embodiments, multiple response contents can be aggregated or combined into aggregated evaluation content, which includes evaluation content for multiple response contents and can comprehensively analyze the specific advantages and disadvantages of each response, and provide corresponding scores and evaluation results for evaluation dimensions. This aggregated evaluation can be presented in the interface space occupied by evaluation information 216-1, evaluation information 216-2, and evaluation information 216-3.

[0055] In some embodiments, the target evaluation content among these multiple evaluation contents is based on the response content corresponding to the target evaluation content and at least one other response content among the multiple response contents. For example, the target evaluation content in evaluation information 216-1 may be an evaluation based solely on the corresponding response content 214-1. In this example, the evaluation is generated without referencing response content generated by other virtual objects, but only for the current response content.

[0056] However, in some embodiments, the target evaluation content in evaluation information 216-1 can be generated based on at least one of the other response content items. For example, the target evaluation content in evaluation information 216-1 can be generated based on response content 214-1, response content 214-2, and response content 214-3. In this generation process, all three response content items are used as references, and the generated target evaluation content can be generated by comparing different response content items, and the corresponding score can also be adjusted based on other response content items. For example, if the overall evaluation of response content 216-1 is lower than that of response content 216-2, then the score of response content 216-1 can be lower than that of response content 216-2.

[0057] In some embodiments, based on the evaluation information of multiple response contents, the electronic device 110 can display a target response content among the multiple response contents in a target style on the interactive interface, wherein the target evaluation of the target response content meets preset conditions. As a non-limiting example, the preset conditions may be that the score of the response content is higher than a preset threshold, or that the score of the response content is the highest among all response contents. For example, if the score of response content 216-2 is 95 points, which is the highest score, then response content 216-2 can be displayed in a different style, such as displaying response content 216-2 in a highlighted color, and its score "95" can be in bold font. Alternatively, response content 216-2 may present an indicator to indicate that its overall evaluation is the best.

[0058] In some embodiments, referring to FIG2C, electronic device 110 may present interface 200C, which shows a scenario in which a user engages in a multi-turn dialogue and in which the user does not add or delete virtual objects used to generate response content.

[0059] In the scenario shown in Figure 2C, after receiving input message 212-1, electronic device 110 can further receive input message 212-2, and electronic device 110 can provide multiple responses generated by one or more virtual objects in response to input message 212-2. As an example, this scenario can be referred to as a "multi-turn dialogue scenario".

[0060] In some embodiments, input message 212-1 may be a first input message, multiple response contents (response contents 214-1, 214-2 and 214-3) may be multiple first response contents, and the evaluation information shown in FIG2B (evaluation information 216-1, 216-2 and 216-3 in FIG2B) may be first evaluation information.

[0061] In some scenarios, electronic device 110 can acquire a second input message. For example, electronic device 110 can acquire input message 212-2.

[0062] In the interactive interface, the electronic device 110 can present a set of response content generated based on the second input message (i.e., input message 212-2). The set of response content includes at least one second response content generated by at least one virtual object among a plurality of virtual objects based on the second input message. For example, the set of response content could be response content 214-4, response content 214-5, and response content 214-6, and these response contents could be second response contents.

[0063] In some embodiments, response content generated by the first virtual object based on a first input message and / or response content generated based on a second input message is displayed in the target area corresponding to the first virtual object in the interactive interface. Here, taking object 1 210-1 as an example, response content 214-1 generated by object 1 210-1 based on input message 212-1 (i.e., the first input message) and response content 214-4 generated by object 1 210-1 based on input message 212-2 (i.e., the second input message) are displayed in the area 260 (i.e., the target area) corresponding to object 1 210-1.

[0064] It should be understood that, in order to clearly display the correspondence between multiple rounds of response content, interface 200C displays the response content generated by the same virtual object in different rounds in a single column (i.e., area 260). This aligned display method reflects the correspondence between the response content and the virtual object. In other examples, the response content from different rounds of the same virtual object can also be displayed, for example, in other directions, such as horizontal rows. Furthermore, evaluation information can also be aligned with the response content to present a correspondence.

[0065] Referring again to Figure 2C, the electronic device 110 can provide second evaluation information generated based on a set of response content. For example, the second evaluation information may be evaluation information 216-4, evaluation information 216-5, and evaluation information 216-6.

[0066] In some embodiments, the second evaluation information for at least one second response content is determined based on at least one second response content and at least one first response content generated by at least one virtual object based on the first input message. Here, the second response content can be any one of response content 214-4, 214-5, and 214-6, and the first response content can be any one of response content 214-1, 214-2, and 214-3. Specifically, taking evaluation information 216-4 as the second evaluation information as an example, evaluation information 216-4 can be generated solely based on response content 214-4 (i.e., the second response content).

[0067] In other embodiments, evaluation information 216-4 may be generated based on response content 214-4 (i.e., the second response content) and response content 214-1 (i.e., the first response content). In some embodiments, evaluation information 216-4 may also present the aforementioned "aggregated evaluation content" or the aforementioned evaluation information based on multiple response contents. In this case, the generation of evaluation information 216-4 may be based on, for example, response content 214-4 and response content 214-5, or the generation of evaluation information 216-4 may be based on, for example, response content 214-4, response content 214-5, and response content 214-1 and response content 214-2. As yet another example, evaluation information 216-4 may be based on all previously generated response contents.

[0068] In some embodiments, the electronic device 110 may present first evaluation information and second evaluation information in an interactive interface. Alternatively, the electronic device 110 may update the first evaluation information to the second evaluation information.

[0069] For example, in a multi-turn dialogue scenario, the evaluation information in the interactive interface can be updated. For instance, when switching from a single-turn dialogue scenario shown in interface 200B to a multi-turn dialogue scenario shown in interface 200C, after providing the response content generated based on the user-sent input message 212-2, the electronic device 110 can, for example, update the evaluation information. As shown in Figure 2C, the electronic device 110 can present updated second evaluation information, such as evaluation information 216-4, 216-5, and 216-6.

[0070] In other embodiments, the evaluation information for each round can be retained. For example, the evaluation information 216-1, 216-2 and 216-3 (i.e., the first evaluation information) in FIG2B can be presented together with the evaluation information 216-4, 216-5 and 216-6 (i.e., the second evaluation information) in FIG2C in the interface 200C.

[0071] In some embodiments, referring to FIG2D, electronic device 110 may present interface 200D, which shows a scenario in which a user engages in a multi-turn dialogue, and the user adds a new virtual object in the second round of dialogue.

[0072] In some embodiments, the electronic device 110 can obtain at least one configuration operation via the object configuration area 250 in the interactive interface. The electronic device 110 can also determine multiple virtual objects based on at least one configuration operation. For example, the electronic device 110 can obtain a user's configuration operation to add a virtual object via the add control 212. Specifically, the user can click the add control 212 to add a virtual object to the object configuration area 250. In other embodiments, the user can also delete a virtual object through interactive operations (clicking or long-pressing, etc.) with the virtual object presented in the object configuration area 250.

[0073] In some embodiments, the electronic device 110 may determine a set of target virtual objects based on adjustment operations for a plurality of virtual objects. Here, the adjustment operations include deletion operations for deleting existing virtual objects from the plurality of virtual objects and / or addition operations for adding additional virtual objects. For example, in response to a user adding or deleting virtual objects via object configuration area 250, the electronic device 110 may determine the virtual objects ultimately used to generate response content based on these user adjustment operations.

[0074] In some non-limiting examples, as shown in Figure 2D, after receiving input message 212-1, electronic device 110 may further receive an addition operation for adding a virtual object. Electronic device 110 additionally presents the added object 4 210-4 and the details of the virtual object (i.e., "call count" data, etc.) in object configuration area 250.

[0075] In some embodiments, the electronic device 110 may present a set of response content generated by a set of target virtual objects based on a second input message in an interactive interface.

[0076] For example, when the user sends input message 212-2, since a new virtual object has been added, the interface of the electronic device 110 can be adjusted accordingly to accommodate all the response content and evaluation information. Additionally, object 4210-4 can also generate corresponding response content 214-7 after receiving input message 212-2 (i.e., the second input message). Alternatively or additionally, the target model will also generate corresponding evaluation information 216-7 based on the response content 214-7. The newly generated response content 214-7 and evaluation information 216-7 can also be presented in the interactive interface.

[0077] In some scenarios, if the number of virtual objects is large and the existing interface cannot accommodate all the response content and / or evaluation information, the electronic device 110 can, for example, enable scroll bars and present only a portion of the response content and / or evaluation information in the interactive interface. Users can browse other response content and / or evaluation information that are not presented by dragging the scroll bars or switching display interfaces.

[0078] In some embodiments, referring to FIG2E, electronic device 110 may present interface 200E, which shows a scenario in which a user engages in a multi-turn dialogue and in which the user deletes a virtual object configured in the first round of the dialogue in the second round of the dialogue.

[0079] In the scenario shown in Figure 2E, the electronic device 110 can receive a deletion operation for object 3 210-3 through the object configuration area 250. Therefore, after receiving the input message 212-2, the electronic device 110 may no longer display the response content and corresponding evaluation information corresponding to the deleted object 3 210-3. Furthermore, in other scenarios, if object 3 210-3 is deleted, the electronic device 110 can also delete or hide the response content 214-3 corresponding to object 3 210-3. That is, the electronic device 110 can delete or hide elements associated with object 3 210-3 in the interface.

[0080] In some examples, the area 270 corresponding to the deleted virtual object may be blank. However, in other examples, the electronic device 110 may adjust the interface style or layout to hide or delete the area 270.

[0081] In some embodiments, referring to FIG2F, electronic device 110 may present interface 200F, which shows an interface for users to browse review information details.

[0082] If the electronic device 110 receives a preset operation for the evaluation information, the electronic device 110 can display the evaluation viewing window 230. For example, the user can bring up the evaluation viewing window 230 by clicking on the area where the evaluation information 216-3 is located. The evaluation viewing window 230 can display at least part of the evaluation information and at least one additional evaluation.

[0083] The evaluation viewing window 230 can also display additional evaluation content, which may be, for example, the evaluation dimensions or evaluation criteria of the evaluation information. In some examples, this additional evaluation content may be graphical evaluation content, such as a radar chart.

[0084] In some embodiments, the additional evaluation content presented in the evaluation viewing window 230 can be evaluation information displayed in different formats, such as charts or graphs. If the evaluation content is too long, only a summary of the evaluation content can be displayed in the evaluation information 216-3, while the complete evaluation content can be presented in the evaluation viewing window 230.

[0085] In some embodiments, additional comments can display evaluations based on different evaluation dimensions. For example, evaluation information 216-3 only evaluates response content 214-3 based on the dimension of "solution," but the evaluation viewing window can evaluate the response content based on five dimensions: "speed," "accuracy," "coherence," "reasonableness," and "feasibility," to present a more detailed evaluation.

[0086] In other embodiments, the evaluation content in evaluation information 216-3 may have a different granularity than the evaluation content in evaluation viewing window 230. For example, the evaluation content presented in evaluation information 216-3 may be more granular, that is, only providing a general or comprehensive evaluation. The evaluation content presented in evaluation viewing window 230 may provide a more detailed analysis with finer granularity. For example, the additional evaluation content in evaluation viewing window 230 may provide a detailed analysis of response content 214-3 from multiple perspectives.

[0087] In some embodiments, the technical solutions disclosed herein can also be used in virtual object debugging scenarios, such as electronic device 110 supporting users to test, debug and optimize virtual objects.

[0088] In some scenarios, multiple virtual objects include second virtual objects to be tested (or debugged). For example, these second virtual objects to be tested may be virtual objects presented in object configuration area 250. In some embodiments, the virtual objects to be tested may include user-created virtual objects, such as user-created agents. Electronic device 110 can help the user better optimize the created virtual objects by comparing the different processing results of user-created virtual objects and other reference virtual objects for the same input message.

[0089] As shown in Figure 2F, input message 212-1 can be a test message received by electronic device 110 for a virtual object to be tested. For example, this test message can test the virtual object's ability to solve a specific problem. This ability of the virtual object can be analyzed based on its generated response content. For example, in response to input message 212-1 (i.e., the test message), object 3 210-3 can generate response content 214-3. Based on this response content 214-3, electronic device 110 can also provide evaluation information 216-3, which can present the test results for the object. The test results can be presented in the form of a score and evaluation content.

[0090] For example, evaluation information 216-3 could present a score of "85" for object 3 210-3, indicating the object's overall score in this round of dialogue. Evaluation information 216-3 could also present evaluation content for this round of testing. For example, evaluation information 216-3 could indicate that in this round of dialogue, regarding the "solution" dimension, object 3 210-3's generated response did not meet expectations and was deducted 15 points.

[0091] Electronic device 110 can provide optimization suggestions for the second virtual object, which are generated based on multiple response contents and / or evaluation information.

[0092] In some embodiments, the virtual object can provide optimization suggestions based on user input messages, response messages from other virtual objects to the input messages, and / or corresponding evaluation information. These optimization suggestions can also be displayed in the evaluation viewing window 230.

[0093] For example, optimization suggestions for the tested object 3 210-3 can be presented in the evaluation viewing window 230. The above describes a sample test of object 3 210-3; after this test, corresponding optimization suggestions can be presented in the evaluation viewing window 230. These optimization suggestions may include suggestions for optimizing the configuration parameters of the virtual object under test. For example, if the response generated by object 3 210-3 has significant shortcomings in terms of "feasibility," then suggestions for adjusting the configuration parameters of the object's logical modules can be presented in the evaluation viewing window 230 to further optimize object 3 210-3.

[0094] Alternatively or additionally, the electronic device 110 may also present a modification entry for the virtual object being tested / debugged. This modification entry may be presented, for example, in the area of ​​evaluation information 216-3 or in the evaluation viewing window 230. If an interactive operation is received regarding this modification entry, the electronic device 110 may further present a modification interface for the virtual object to be modified, allowing the user to modify the virtual object. Alternatively, in response to receiving an interactive operation regarding this modification entry, the electronic device 110 may trigger automatic modification of the virtual object, which may correspond, for example, to the presented adjustment suggestions. As examples, the above modifications include, but are not limited to: modifications to the parameters of the virtual object, modifications to the plugins or tools invoked by the virtual object, and modifications to the knowledge or application domain of the virtual object.

[0095] In some embodiments, the modification interface may also display evaluation information and corresponding optimization suggestions for the modified virtual object, so that users can refer to such evaluation information and / or optimization suggestions to further modify the virtual object. Additionally, the modified virtual object can continue the aforementioned testing / debugging process to further improve its capabilities and related configurations.

[0096] In this way, embodiments of this disclosure can provide users with the output results of multiple virtual objects (e.g., intelligent agents or machine programs) for the same request, thereby improving the user's information acquisition efficiency. Furthermore, by providing evaluation information on the response content, embodiments of this disclosure can also help users better perceive the differences between the output results of different virtual objects.

[0097] Example process

[0098] Figure 3 shows a flowchart of an example process 300 for information processing according to some embodiments of the present disclosure. Process 300 can be implemented at electronic device 110. Process 300 will now be described with reference to Figure 1.

[0099] As shown in Figure 3, in box 310, electronic device 110 acquires input messages.

[0100] In box 320, electronic device 110 presents multiple virtual objects generating multiple response contents based on input messages in the interactive interface.

[0101] In box 330, electronic device 110 provides evaluation information for multiple response contents.

[0102] In some embodiments, the input message is a first input message, the multiple response contents are multiple first response contents, and the evaluation information is first evaluation information. The method further includes: obtaining a second input message; presenting a set of response contents generated based on the second input message in an interactive interface, the set of response contents including at least one second response content generated by at least one virtual object among a plurality of virtual objects based on the second input message; and providing second evaluation information generated based on the set of response contents.

[0103] In some embodiments, presenting a set of response content generated based on the second input message in the interactive interface includes: determining a set of target virtual objects based on adjustment operations for multiple virtual objects; and presenting a set of response content generated by the set of target virtual objects based on the second input message in the interactive interface.

[0104] In some embodiments, the adjustment operation includes: a deletion operation for deleting an existing virtual object from a plurality of virtual objects; and / or an addition operation for adding additional virtual objects.

[0105] In some embodiments, the second evaluation information for at least one second response content is determined based on the following information: at least one second response content; and at least one first response content generated by at least one virtual object based on the first input message.

[0106] In some embodiments, response content generated by the first virtual object based on the first input message and / or response content generated based on the second input message is displayed in the target area corresponding to the first virtual object in the interactive interface.

[0107] In some embodiments, providing second evaluation information generated based on a set of response content includes: presenting first evaluation information and second evaluation information in an interactive interface; or updating the first evaluation information to the second evaluation information.

[0108] In some embodiments, process 300 further includes: obtaining at least one configuration operation via an object configuration area in an interactive interface; and determining a plurality of virtual objects based on the at least one configuration operation.

[0109] In some embodiments, the object configuration area presents object description information for multiple virtual objects.

[0110] In some embodiments, process 300 further includes: displaying the target response content among the multiple response contents in a target style on an interactive interface based on the evaluation information of the multiple response contents, wherein the target evaluation of the target response content meets preset conditions.

[0111] In some embodiments, the evaluation information for multiple response contents includes: aggregated evaluation content for multiple response contents; and / or multiple evaluation contents corresponding to multiple response contents.

[0112] In some embodiments, the target evaluation content among the multiple evaluation contents is generated based on the following information: the response content corresponding to the target evaluation content; and at least one other response content among the multiple response contents.

[0113] In some embodiments, the evaluation information indicates at least one of the following: the score of the corresponding response content; the evaluation text of the corresponding response content; and the evaluation result of the corresponding response content on a preset evaluation dimension.

[0114] In some embodiments, process 300 further includes: in response to receiving a preset operation for the evaluation information, presenting an evaluation viewing window; and presenting at least part of the evaluation information and at least one additional evaluation content in the evaluation viewing window.

[0115] In some embodiments, the plurality of virtual objects includes a second virtual object to be tested, and the method further includes: providing optimization suggestions for the second virtual object, the optimization suggestions being generated based on a plurality of response contents and / or evaluation information.

[0116] In some embodiments, presenting multiple response contents generated by multiple virtual objects based on input messages in the interactive interface includes: providing multiple messages generated by multiple virtual objects based on input messages in parallel in the session area of ​​the interactive interface, wherein the multiple messages correspond to multiple response contents.

[0117] In some embodiments, the evaluation information is generated based on the process of providing input messages and multiple response contents to a target model to generate evaluation information for the multiple response contents.

[0118] In some embodiments, the target model is configured to generate evaluation information based on received prompts, wherein the prompts indicate at least one evaluation dimension for generating the evaluation information.

[0119] Example devices and equipment

[0120] Embodiments of this disclosure also provide corresponding apparatus for implementing the methods or processes described above. FIG4 shows a schematic structural block diagram of an example apparatus 400 for information processing according to certain embodiments of this disclosure. Apparatus 400 may be implemented as or included in electronic device 110. The various modules / components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0121] As shown in Figure 4, the device 400 includes: an acquisition module 410 configured to acquire an input message; a presentation module 420 configured to present multiple response contents generated by multiple virtual objects based on the input message in an interactive interface; and a providing module 430 configured to provide evaluation information for the multiple response contents.

[0122] In some embodiments, the input message is a first input message, the multiple response contents are multiple first response contents, the evaluation information is first evaluation information, and the acquisition module 410 is further configured to: acquire a second input message; present a set of response contents generated based on the second input message in the interactive interface, the set of response contents including at least one second response content generated by at least one virtual object among multiple virtual objects based on the second input message; and provide second evaluation information generated based on the set of response contents.

[0123] In some embodiments, the presentation module 420 is further configured to: determine a set of target virtual objects based on adjustment operations for multiple virtual objects; and present a set of response content generated by the set of target virtual objects based on a second input message in an interactive interface.

[0124] In some embodiments, the adjustment operation includes: a deletion operation for deleting an existing virtual object from a plurality of virtual objects; and / or an addition operation for adding additional virtual objects.

[0125] In some embodiments, the second evaluation information for at least one second response content is determined based on the following information: at least one second response content; and at least one first response content generated by at least one virtual object based on the first input message.

[0126] In some embodiments, response content generated by the first virtual object based on the first input message and / or response content generated based on the second input message is displayed in the target area corresponding to the first virtual object in the interactive interface.

[0127] In some embodiments, the presentation module 420 is further configured to: present first evaluation information and second evaluation information in the interactive interface; or update the first evaluation information to the second evaluation information.

[0128] In some embodiments, the apparatus 400 further includes a determining module configured to: acquire at least one configuration operation via an object configuration area in an interactive interface; and determine a plurality of virtual objects based on the at least one configuration operation.

[0129] In some embodiments, the object configuration area presents object description information for multiple virtual objects.

[0130] In some embodiments, the device 400 further includes a display module configured to: display the target response content among the multiple response contents in a target style on an interactive interface based on the evaluation information of the multiple response contents, wherein the target evaluation of the target response content meets preset conditions.

[0131] In some embodiments, the evaluation information for multiple response contents includes: aggregated evaluation content for multiple response contents; and / or multiple evaluation contents corresponding to multiple response contents.

[0132] In some embodiments, the target evaluation content among the multiple evaluation contents is generated based on the following information: the response content corresponding to the target evaluation content; and at least one other response content among the multiple response contents.

[0133] In some embodiments, the evaluation information indicates at least one of the following: the score of the corresponding response content; the evaluation text of the corresponding response content; and the evaluation result of the corresponding response content on a preset evaluation dimension.

[0134] In some embodiments, the device 400 further includes a window presentation module, which is configured to: present an evaluation viewing window in response to receiving a preset operation for the evaluation information; and present at least part of the evaluation information and at least one additional evaluation content in the evaluation viewing window.

[0135] In some embodiments, the providing module 430 is further configured to provide optimization suggestions for the second virtual object, the optimization suggestions being generated based on multiple response contents and / or evaluation information.

[0136] In some embodiments, the providing module 430 is further configured to provide, in the session area of ​​the interactive interface, multiple messages generated by multiple virtual objects based on the input message, wherein the multiple messages correspond to multiple response contents.

[0137] In some embodiments, the evaluation information is generated based on the process of providing input messages and multiple response contents to a target model to generate evaluation information for the multiple response contents.

[0138] In some embodiments, the target model is configured to generate evaluation information based on received prompts, wherein the prompts indicate at least one evaluation dimension for generating the evaluation information.

[0139] The units included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0140] As shown in Figure 5, the electronic device 500 is in the form of a general-purpose electronic device. Components of the electronic device 500 may include, but are not limited to, one or more processing units or processors 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processor 510 may be a physical or virtual processor and is capable of performing various processes according to the programs stored in the memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0141] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0142] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0143] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0144] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0145] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0146] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0147] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0148] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0150] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An information processing method, comprising: Get the input message; The interactive interface displays multiple response contents generated by multiple virtual objects based on the input message; as well as Provide evaluation information for the aforementioned multiple response contents.

2. The method according to claim 1, wherein the input message is a first input message, the plurality of response contents are a plurality of first response contents, and the evaluation information is first evaluation information, the method further comprising: Get the second input message; The interactive interface presents a set of response content generated based on the second input message, the set of response content including at least one second response content generated by at least one of the plurality of virtual objects based on the second input message; and Provide second evaluation information generated based on the set of response contents.

3. The method according to claim 2, wherein presenting a set of response content generated based on the second input message in the interactive interface includes: Based on the adjustment operations performed on the multiple virtual objects, a set of target virtual objects is determined; as well as The interactive interface presents the set of response contents generated by the set of target virtual objects based on the second input message.

4. The method according to claim 3, wherein the adjustment operation includes: Deletion operation used to delete existing virtual objects among the plurality of virtual objects; or The addition operation is used to add additional virtual objects.

5. The method of claim 2, wherein the second evaluation information for the at least one second response content is determined based on the following information: The at least one of the second response contents; and At least one first response content generated by the at least one virtual object based on the first input message.

6. The method according to claim 2, wherein the response content generated by the first virtual object based on the first input message or the response content generated based on the second input message is displayed in the target area of ​​the interactive interface corresponding to the first virtual object.

7. The method of claim 2, wherein providing the second evaluation information generated based on the set of response contents includes: The first evaluation information and the second evaluation information are presented in the interactive interface; or Update the first evaluation information to the second evaluation information.

8. The method according to claim 1, further comprising: At least one configuration operation is obtained through the object configuration area in the interactive interface; as well as Based on the at least one configuration operation, the plurality of virtual objects are determined.

9. The method according to claim 8, wherein the object configuration area presents object description information of the plurality of virtual objects.

10. The method according to claim 1, further comprising: Based on the evaluation information of the multiple response contents, the target response content among the multiple response contents is displayed in a target style on the interactive interface, wherein the target evaluation of the target response content meets preset conditions.

11. The method of claim 1, wherein the evaluation information for the plurality of response contents includes: Aggregated evaluation content for the multiple response contents; or Multiple evaluation contents corresponding to the multiple response contents.

12. The method of claim 11, wherein the target evaluation content among the plurality of evaluation contents is generated based on the following information: The response content corresponding to the target evaluation content; and At least one of the other response contents among the multiple response contents.

13. The method of claim 1, wherein the evaluation information indicates at least one of the following: The score for the corresponding response content; Evaluation text of the corresponding response content; The corresponding response content includes the evaluation results for the preset evaluation dimensions.

14. The method according to claim 1, further comprising: In response to receiving a preset operation for the evaluation information, an evaluation viewing window is displayed; as well as The evaluation viewing window displays at least part of the evaluation information and at least one additional evaluation item.

15. The method of claim 1, wherein the plurality of virtual objects includes a second virtual object to be tested, and the method further comprises: Optimization suggestions are provided for the second virtual object, which are generated based on the multiple response contents or the evaluation information.

16. The method according to claim 1, wherein presenting multiple response contents generated by multiple virtual objects based on the input message in the interactive interface includes: In the session area of ​​the interactive interface, multiple virtual objects generate multiple messages based on the input message, and the multiple messages correspond to the multiple response contents.

17. The method of claim 1, wherein the evaluation information is generated based on the following process: The input message and the multiple response contents are provided to the target model to generate the evaluation information for the multiple response contents.

18. The method of claim 17, wherein the target model is configured to generate the evaluation information based on received prompts, wherein the prompts indicate at least one evaluation dimension for generating the evaluation information.

19. An apparatus for information processing, comprising: The acquisition module is configured to acquire input messages; The presentation module is configured to present multiple response contents generated by multiple virtual objects based on the input message in the interactive interface; as well as A module is configured to provide evaluation information for the multiple response contents.

20. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 18 when executed by the at least one processor.

21. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 18.

22. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 18.

Images