Information processing method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to an information processing method and device, equipment and a storage medium. The method provided by the invention comprises the following steps: acquiring an input message; in an interactive interface, presenting multiple response contents generated by multiple virtual objects based on the input message; and providing evaluation information for the plurality of response contents. In this way, according to the embodiment of the invention, output results of a plurality of virtual objects (such as an intelligent agent or a machine program) for the same request can be provided for the user, so that the information acquisition efficiency of the user is improved. In addition, by providing the evaluation information of the response content, the embodiment of the invention can also help the user to better perceive the difference between the output results of different virtual objects.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to information processing methods, apparatuses, devices, and computer-readable storage media. Background Art

[0002] With the development of artificial intelligence technology, generative artificial intelligence technology has been widely used in various applications. For example, developers can create applications such as agents or bots based on generative models. Different agents or bots may produce different outputs for the same input. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for information processing is provided, which includes: obtaining an input message; presenting, in an interactive interface, multiple response contents generated by multiple virtual objects based on the input message; and providing evaluation information for the multiple response contents.

[0004] In a second aspect of the present disclosure, a device for information processing is provided. The device includes: an acquisition module configured to acquire an input message; a presentation module configured to present, in an interactive interface, multiple response contents generated by multiple virtual objects based on the input message; and a provision module configured to provide evaluation information for the multiple response contents.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1A schematic diagram showing an example environment in which embodiments according to the present disclosure may be implemented;

[0010] FIG. 2A to FIG. 2F shows an example interface according to some embodiments of the present disclosure;

[0011] Figure 3 A flowchart illustrating an example process of information processing according to some embodiments of the present disclosure;

[0012] Figure 4 A schematic structural block diagram showing an example apparatus for information processing according to some embodiments of the present disclosure; and

[0013] Figure 5 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0016] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0017] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0018] In this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or it is necessary to perform a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0019] As mentioned above, different agents or machine programs may produce different outputs for the same input. In this case, on the one hand, the quality of the results obtained by the user may be poor when using a single agent or machine program. On the other hand, the user may need to repeatedly switch between different agents or machine programs, which increases the user's interaction cost.

[0020] The embodiment of the present disclosure proposes a scheme for information processing, which includes: obtaining an input message; presenting multiple response contents generated by multiple virtual objects based on the input message in an interactive interface; and providing evaluation information for the multiple response contents.

[0021] In this way, the embodiments of the present disclosure can provide users with the output results of multiple virtual objects (e.g., agents or machine programs) for the same request, thereby improving the user's information acquisition efficiency. In addition, by providing evaluation information of the response content, the embodiments of the present disclosure can also help users better perceive the differences between the output results of different virtual objects.

[0022] Various example implementations of the solution are described in detail below in conjunction with the accompanying drawings.

[0023] Example Environment

[0024] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, example environment 100 may include electronic device 110 .

[0025] In this example environment 100, the electronic device 110 may run an application 120 that supports information processing. The application 120 may be any appropriate type of application for information processing, examples of which may include, but are not limited to, instant messaging applications, agent dialogue applications, large model question and answer applications, or other appropriate applications. The user 140 may interact with the application 120 via the electronic device 110 and / or its attached devices.

[0026] exist Figure 1 In the environment 100 , if the application 120 is in an active state, the electronic device 110 may present an interface 150 for supporting information processing through the application 120 .

[0027] In some embodiments, the electronic device 110 communicates with the server 130 to provide services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a handheld computer, a portable game terminal, a VR / AR device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an e-book device, a game device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.).

[0028] The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 130 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc. The server 130 may provide background services for the application 120 that supports information processing in the electronic device 110.

[0029] A communication connection may be established between the server 130 and the electronic device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 130 and the electronic device 110 may implement signaling interaction through the communication connection between the two.

[0030] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0031] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0032] Example Interaction

[0033] FIG. 2A to FIG. 2F 200A to 200F according to some embodiments of the present disclosure. The interfaces 200A to 200F may be composed of, for example, Figure 1 The electronic device 110 is provided as shown.

[0034] In some embodiments, the electronic device 110 may first present a configuration interface for a virtual object. For example, the electronic device 110 may first present a configuration interface for selecting a virtual object, and the electronic device 110 may receive a configuration operation from a user to add any available virtual object through the configuration interface. After configuring a virtual object to be used for a conversation or for a test via the configuration interface, the electronic device 110 may present an interactive interface such as interface 200A, so that the user can start a conversation with the selected virtual object.

[0035] In some scenarios, virtual objects may also be referred to as virtual entities, examples of which may include, but are not limited to, agents, bots, or other processing entities that support triggering content interaction through dialogue, etc. Such virtual objects may be implemented, for example, using appropriate generative models to generate interactive content accordingly based on received prompts.

[0036] In some embodiments, the virtual object selected by the user may include an agent associated with the user. As an example, the user may add a virtual object created by the user, a virtual object edited by the user, a virtual object collected by the user, etc. As an example, the electronic device 110 may present a plurality of candidate virtual objects that are allowed to be added to the user for selection.

[0037] In some embodiments, reference Figure 2A , the electronic device 110 may present an interface 200A, and the interface 200A may include an object configuration area 250 for presenting a virtual object and a message area 215 .

[0038] In some embodiments, the virtual objects configured in the above configuration interface may be presented in the object configuration area 250. In other embodiments, some preset virtual objects may also be presented in the object configuration area 250. The electronic device 110 may receive further configuration of the virtual object by the user via the object configuration area 250, and change the virtual object presented in the object configuration area 250 based on the further configuration.

[0039] In some embodiments, the electronic device 110 may present multiple virtual objects in the object configuration area 250. For example, as shown in the interface 200A, the object configuration area 250 presents object 1 210-1, object 2 210-2, and object 3 210-3. In addition, the object configuration area 250 may also present object description information of the multiple virtual objects. For example, the object configuration area 250 also presents the "call count" of the virtual object and the corresponding call count change curve.

[0040] In some embodiments, an add control 212 may also be presented in the object configuration area 250 , and the user may click the add control 212 to add a new virtual object.

[0041] For example, the electronic device 110 may receive a message from a user in the input box 202 of the message area 215 and receive a trigger operation of the send control 204 to input the message to the virtual object, and the virtual object may give a corresponding response. In addition, the electronic device 110 may also receive a trigger operation of the share control 220 to generate shared content for this set of inputs and corresponding responses, such as a report. The shared content may include the message input by the user and the responses or results generated by different virtual objects for the input message. In addition, the shared content may also include evaluation information for different responses.

[0042] In some scenarios, some examples of input messages, such as example 206-1 and example 206-2, may also be presented in the message interface 215 for user reference. It should be noted that the user's input message in the input box 202 may be text input or other types of input, such as picture input, URL input or voice input. Virtual objects with corresponding processing capabilities can give corresponding types of responses according to the type of input. For example, if the user inputs his own questions or messages by voice, the virtual object can generate voice and provide the voice as a result to the user.

[0043] In some embodiments, reference Figure 2B, the electronic device 110 may present an interface 200B, where the interface 200B shows a scenario in which the user is conducting a single-round conversation.

[0044] In some embodiments, the electronic device 110 may obtain an input message. The electronic device 110 may present multiple response contents generated by multiple virtual objects based on the input message in an interactive interface. Additionally, the electronic device 110 may also provide evaluation information for the multiple response contents.

[0045] like Figure 2B As shown, the electronic device 110 can obtain the input message 212-1, and the input message 212-1 is presented in the message area 215. The object configuration area 250 is presented with pre-configured virtual objects: object 1 210-1, object 2 210-2 and object 3 210-3. These three virtual objects generate response content 214-1, response content 214-2 and response content 214-3 for the input message 212-1 respectively.

[0046] The electronic device 110 can provide multiple messages generated by multiple virtual objects based on the input message 212-1 in parallel in the conversation area (i.e., message area 215) of the interactive interface. The multiple messages correspond to multiple response contents. For example, response content 214-1, response content 214-2, and response content 214-3 can be displayed together in area 215 in the form of multiple messages.

[0047] It should be noted that although the example response content presented in the message area 215 is in the form of text, the response content may also be in other forms, for example, the response content may be a voice message, and the user clicks on the response content 214-1 to trigger the electronic device 110 to play the voice message through its playback device.

[0048] In some embodiments, the electronic device 110 also provides evaluation information for multiple response contents. For example, the interface 200B also presents evaluation information 216-1 corresponding to the response content 214-1, evaluation information 216-2 corresponding to the response content 214-2, and evaluation information 216-3 corresponding to the response content 214-3.

[0049] In some embodiments, the evaluation information for multiple response contents is generated by providing the input message and multiple response contents to the target model. For example, by inputting the input message 212-1 and the response contents 214-1, 214-2, and 214-3 into the target model for generating the evaluation information, the electronic device 110 can present the corresponding evaluation information. Here, the target model can be some models with analysis capabilities or intelligent agents with analysis capabilities.

[0050] In some embodiments, the above evaluation information may include the score of the corresponding response content and / or the evaluation text of the corresponding response content and / or the evaluation result of the corresponding response content on the preset evaluation dimension. For example, the evaluation information 216-1 may present that the score of the response content 214-1 is "90", and the evaluation information 216-1 may also present the evaluation text of the response content 214-1 and the evaluation results of the preset evaluation dimension of the response content 214-1, "Solution-5" and "Context-5".

[0051] In some scenarios, examples of evaluation dimensions may include, but are not limited to: the memory size occupied by the virtual object when generating the response content, the capabilities invoked by the virtual object when generating the response content, the workflow of the virtual object, the feasibility of the solution provided in the response content, the relevance of the response content to the context of the conversation, etc. Figure 2B As shown in , the response content 214 - 1 is deficient in the above two dimensions of “solution” and “context”, so corresponding points are deducted.

[0052] In some embodiments, the evaluation information for the multiple response contents may be aggregated evaluation contents for the multiple response contents and / or multiple evaluation contents corresponding to the multiple response contents. Figure 2B In the scenario presented in , the multiple evaluation contents corresponding to the multiple response contents are presented in interface 200B. However, in other embodiments, the multiple response contents can be aggregated or combined into aggregate evaluation contents, which include evaluation contents of the multiple response contents, and can comprehensively analyze the specific advantages and disadvantages of each response, and give corresponding scores and evaluation results of evaluation dimensions. The aggregate evaluation can be presented in the interface space occupied by evaluation information 216-1, evaluation information 216-2, and evaluation information 216-3.

[0053] In some embodiments, the target evaluation content in the above-mentioned multiple evaluation contents is based on the response content corresponding to the target evaluation content and at least one other response content in the multiple response contents. For example, the target evaluation content in the evaluation information 216-1 can be an evaluation based only on the corresponding response content 214-1. In this example, the evaluation will not refer to the response content generated by other virtual objects when it is generated, but only evaluates the current response content.

[0054] However, in other embodiments, the target evaluation content in the evaluation information 216-1 can be generated based on at least one other response content in the multiple response contents. For example, the target evaluation content in the evaluation information 216-1 can be generated based on the response content 214-1, the response content 214-2 and the response content 214-3. In the generation process, the three response contents are all used as references, and the generated target evaluation content can be generated by comparing different response contents, and the corresponding scores can also be adjusted accordingly based on other response contents. For example, if the overall evaluation of the response content 216-1 is lower than that of the response content 216-2, the score of the response content 216-1 can be lower than that of the response content 216-2.

[0055] In some embodiments, based on the evaluation information of multiple response contents, the electronic device 110 can display the target response content among the multiple response contents in a target style on the interactive interface, wherein the target evaluation for the target response content satisfies a preset condition. As a non-limiting example, the preset condition may be that the score of the response content is higher than a preset threshold, or that the score of the response content is the highest among all response contents. For example, the score of the response content 216-2 is 95 points, which is the highest score, then the response content 216-2 can be displayed in different styles, for example, the response content 216-2 can be displayed in a highlighted color, and its score "95" can be in a bold font. Alternatively, an identification may be presented in the response content 216-2 to indicate that its overall evaluation is the best.

[0056] In some embodiments, reference Figure 2C , the electronic device 110 may present an interface 200C, which shows a scenario in which the user conducts multiple rounds of conversations, and in this scenario the user does not add or delete virtual objects used to generate response content.

[0057] exist Figure 2C In the illustrated scenario, after receiving input message 212-1, electronic device 110 may further receive input message 212-2, and electronic device 110 may provide multiple responses generated by one or more virtual objects to input message 212-2. As an example, this scenario may be referred to as a "multi-round conversation scenario."

[0058] In some embodiments, the input message 212-1 may be a first input message, the plurality of response contents (response contents 214-1, 214-2, and 214-3) may be a plurality of first response contents, and Figure 2B The evaluation information shown in ( Figure 2B The evaluation information 216-1, 216-2 and 216-3) in the example may be the first evaluation information.

[0059] In some scenarios, the electronic device 110 may obtain a second input message. For example, the electronic device 110 may obtain the input message 212-2.

[0060] In the interactive interface, the electronic device 110 may present a set of response contents generated based on the second input message (i.e., input message 212-2), wherein the set of response contents includes at least one second response content generated by at least one virtual object among the plurality of virtual objects based on the second input message. For example, the set of response contents may be response content 214-4, response content 214-5, and response content 214-6, and these response contents may be second response contents.

[0061] In some embodiments, the response content generated by the first virtual object based on the first input message and / or the response content generated based on the second input message is displayed in the target area corresponding to the first virtual object in the interactive interface. Here, taking the first virtual object as object 1 210-1 as an example, the response content 214-1 generated by object 1 210-1 based on input message 212-1 (i.e., the first input message) and the response content 214-4 generated by object 1 210-1 based on input message 212-2 (i.e., the second input message) are displayed in the area 260 (i.e., the target area) corresponding to object 1 210-1.

[0062] It should be understood that in order to clearly display the correspondence between multiple rounds of response content, the response content generated by the same virtual object in different rounds is displayed in one column (i.e., area 260) in interface 200C. This aligned display method can reflect the correspondence between the response content and the virtual object. In some other examples, the response content from different rounds of the same virtual object can also be displayed in other directions, such as horizontal rows. In addition, the evaluation information can also be aligned with the response content to show the correspondence.

[0063] Continue to refer Figure 2C , the electronic device 110 may provide second evaluation information generated based on a set of response contents. For example, the second evaluation information may be evaluation information 216-4, evaluation information 216-5, and evaluation information 216-6.

[0064] In some embodiments, the second evaluation information for at least one second response content is determined based on at least one second response content and at least one first response content generated by at least one virtual object based on the first input message. Here, the second response content can be any one of the response contents 214-4, 214-5, and 214-6, and the first response content can be any one of the response contents 214-1, 214-2, and 214-3. Specifically, taking the evaluation information 216-4 as the second evaluation information as an example, the evaluation information 216-4 can be generated only based on the response content 214-4 (i.e., the second response content).

[0065] In other embodiments, evaluation information 216-4 may be generated based on response content 214-4 (i.e., the second response content) and response content 214-1 (i.e., the first response content). In some embodiments, evaluation information 216-4 may also present the above-mentioned "aggregate evaluation content" or the above-mentioned evaluation information based on multiple response contents. In this case, generation of evaluation information 216-4 may be based on, for example, response content 214-4 and response content 214-5, or generation of evaluation information 216-4 may be based on, for example, response content 214-4, response content 214-5, response content 214-1, and response content 214-2. As another example, evaluation information 216-4 may be based on all previously generated response contents.

[0066] In some embodiments, the electronic device 110 may present the first evaluation information and the second evaluation information in the interactive interface, or the electronic device 110 may update the first evaluation information to the second evaluation information.

[0067] For example, in a multi-round dialogue scenario, the evaluation information in the interactive interface can be updated. For example, when switching from the single-round dialogue scenario shown in interface 200B to the multi-round dialogue scenario shown in interface 200C, after providing the response content generated based on the input message 212-2 sent by the user, the electronic device 110 can update the evaluation information. Figure 2C As shown, the electronic device 110 may present updated second evaluation information, such as evaluation information 216 - 4 , 216 - 5 , and 216 - 6 .

[0068] In other embodiments, the evaluation information of each round may be retained, for example Figure 2B The evaluation information 216-1, 216-2 and 216-3 (i.e., the first evaluation information) in Figure 2C The evaluation information 216 - 4 , 216 - 5 and 216 - 6 (ie, the second evaluation information) are presented together in the interface 200C.

[0069] In some embodiments, reference Figure 2D, the electronic device 110 may present an interface 200D, where the interface 200D shows a scenario in which the user conducts multiple rounds of conversations, and the user adds a new virtual object in the second round of conversation.

[0070] In some embodiments, the electronic device 110 may obtain at least one configuration operation via the object configuration area 250 in the interactive interface. The electronic device 110 may also determine multiple virtual objects based on at least one configuration operation. For example, the electronic device 110 may obtain a user's configuration operation of adding a virtual object via the add control 212. Specifically, the user may click the add control 212 to add a virtual object to the object configuration area 250. In other embodiments, the user may also delete a virtual object by interacting with the virtual object presented in the object configuration area 250 (clicking or long pressing operations, etc.).

[0071] In some embodiments, the electronic device 110 may determine a set of target virtual objects based on adjustment operations for multiple virtual objects. Here, the adjustment operation includes a deletion operation for deleting an existing virtual object from the multiple virtual objects and / or an addition operation for adding an additional virtual object. For example, in response to a user adding or deleting a virtual object via the object configuration area 250, the electronic device 110 may determine the virtual object that is ultimately used to generate the response content based on these adjustment operations of the user.

[0072] In some non-limiting examples, Figure 2D As shown in FIG. 2 , after receiving the input message 212-1, the electronic device 110 may further receive an adding operation for adding a virtual object. The electronic device 110 additionally presents the added object 4 210-4 and detailed information of the virtual object (ie, “call count” data, etc.) in the object configuration area 250.

[0073] In some embodiments, the electronic device 110 may present in the interactive interface a group of response contents generated by a group of target virtual objects based on the second input message.

[0074] For example, when the user sends an input message 212-2, since a new virtual object has been added, the interface of the electronic device 110 can be adjusted accordingly to accommodate all response content and evaluation information. Additionally, object 4210-4 can also generate corresponding response content 214-7 after receiving the input message 212-2 (i.e., the second input message). Alternatively or additionally, the target model will also generate corresponding evaluation information 216-7 based on the response content 214-7. The newly generated response content 214-7 and evaluation information 216-7 can also be presented in the interactive interface.

[0075] In some scenarios, if the number of virtual objects is large and the original interface cannot accommodate all the response content and / or evaluation information, the electronic device 110 may, for example, enable a scroll bar and only present part of the response content and / or evaluation information in the interactive interface. The user may browse other response content and / or evaluation information that cannot be presented by dragging the scroll bar or switching the display interface.

[0076] In some embodiments, reference Figure 2E , the electronic device 110 may present an interface 200E, where the interface 200E shows a scenario in which a user conducts multiple rounds of conversations, and in this scenario, the user deletes a virtual object configured in a first round of conversation in a second round of conversation.

[0077] exist Figure 2E In the illustrated scenario, the electronic device 110 may receive a deletion operation for the object 3 210-3 through the object configuration area 250. Therefore, after receiving the input message 212-2, the electronic device 110 may no longer present the response content and corresponding evaluation information corresponding to the deleted object 3 210-3. In addition, in other scenarios, if the object 3 210-3 is deleted, the electronic device 110 may also delete or hide the response content 214-3 corresponding to the object 3 210-3. That is, the electronic device 110 may delete or hide the elements associated with the object 3 210-3 in the interface.

[0078] In some examples, the area 270 corresponding to the deleted virtual object may be blank. However, in other examples, the electronic device 110 may adjust the interface style or layout to hide or delete the area 270.

[0079] In some embodiments, reference Figure 2F , the electronic device 110 may present an interface 200F, where the interface 200F shows an interface for a user to browse details of evaluation information.

[0080] If the electronic device 110 receives a preset operation for the evaluation information, the electronic device 110 may present the evaluation viewing window 230. For example, the user may call up the evaluation viewing window 230 by clicking the area where the evaluation information 216-3 is located. The evaluation viewing window 230 may present at least part of the evaluation information and at least one additional evaluation content.

[0081] Additional evaluation content may also be presented in the evaluation viewing window 230, and the additional evaluation content may be, for example, an evaluation dimension or evaluation standard of the evaluation information. In some examples, the additional evaluation content may be graphical evaluation content, such as a radar chart.

[0082] In some embodiments, the additional evaluation content presented in the evaluation viewing window 230 may be evaluation information in different display forms, for example, the additional evaluation content may display the evaluation information in the form of a chart. If the evaluation content is too long, only a summary of the evaluation content may be displayed in the evaluation information 216-3, and the complete evaluation content may be presented in the evaluation viewing window 230.

[0083] In some embodiments, the additional comment content may display evaluation content in different evaluation dimensions. For example, the evaluation information 216-3 only evaluates the response content 214-3 in the dimension of "solution", but in the evaluation viewing window, the response content may be evaluated in five dimensions of "speed", "accuracy", "coherence", "rationality" and "feasibility" to present a more detailed evaluation.

[0084] In other embodiments, the evaluation content in the evaluation information 216-3 may have different granularity than the evaluation content in the evaluation viewing window 230. For example, the evaluation content presented in the evaluation information 216-3 may have a larger granularity, that is, only a relatively general or comprehensive evaluation is given. The evaluation content presented in the evaluation viewing window 230 may provide a more detailed analysis with a smaller granularity. For example, the additional evaluation content in the evaluation viewing window 230 may provide a detailed analysis of the response content 214-3 in multiple aspects.

[0085] In some embodiments, the technical solution of the present disclosure can also be used in debugging scenarios of virtual objects. For example, the electronic device 110 can support users in testing, debugging and optimizing virtual objects.

[0086] In some scenarios, the plurality of virtual objects include a second virtual object to be tested (or to be debugged). For example, these second virtual objects to be tested may be virtual objects presented in the object configuration area 250. In some embodiments, the virtual objects to be tested may include user-created virtual objects, such as agents created by the user. The electronic device 110 may help the user to better optimize the created virtual object by comparing the different processing results of the user-created virtual object and other reference virtual objects for the same input message.

[0087] like Figure 2FAs shown in , the input message 212-1 can be a test message for the virtual object to be tested received by the electronic device 110. For example, the test message can test the ability of the virtual object to solve a specific problem. This ability of the virtual object can be analyzed according to the response content generated by it. For example, for the input message 212-1 (i.e., the test message), the object 3 210-3 can generate a response content 214-3. According to the response content 214-3, the electronic device 110 can also provide evaluation information 216-3, and the evaluation information 216-3 can present the test result for the object. The test result can be presented in the form of a score and an evaluation content.

[0088] For example, the evaluation information 216-3 may present the score "85" of the object 3 210-3, which indicates the overall score of the object in this round of dialogue. The evaluation information 216-3 may also present the evaluation content for this round of testing. For example, the evaluation information 216-3 may indicate that in this round of dialogue, for the dimension of "solution", the generated response content of the object 3 210-3 did not meet expectations and was deducted 15 points.

[0089] The electronic device 110 may provide optimization suggestions for the second virtual object, where the optimization suggestions are generated based on multiple response contents and / or evaluation information.

[0090] In some embodiments, the virtual object may provide optimization suggestions for the virtual object based on the user's input message, the response message of other virtual objects to the input message, and / or the corresponding evaluation information. The optimization suggestions may also be presented in the evaluation viewing window 230.

[0091] For example, the optimization suggestion of the object 3 210-3 to be tested can be presented in the evaluation viewing window 230. A round of example testing of the object 3 210-3 is described above, and after the round of testing, the corresponding optimization suggestion can be presented in the evaluation viewing window 230. The optimization suggestion can include the optimization suggestion of the configuration parameters of the virtual object to be tested. For example, if the response content generated by the object 3 210-3 is greatly lacking in the aspect of "feasibility", the adjustment suggestion of the configuration parameters of the logic module of the object can be presented in the evaluation viewing window 230 to further optimize the object 3 210-3.

[0092] Alternatively or additionally, the electronic device 110 can also present a modification entry for the virtual object being tested / debugged. The modification entry can be, for example, presented in the area of ​​the evaluation information 216-3 or presented in the evaluation viewing window 230. If an interactive operation for the modification entry is received, the electronic device 110 can further present a modification interface of the modified virtual object so that the user can modify the virtual object. Alternatively, in response to receiving an interactive operation for the modification entry, the electronic device 110 can trigger an automatic modification of the virtual object, which can correspond to the presented adjustment suggestion, for example. As an example, the above-mentioned modification includes, but is not limited to: modification of parameters for the virtual object, modification of plug-ins or tools called for the virtual object, modification of knowledge or application fields for the virtual object, etc.

[0093] In some embodiments, the modification interface may also present evaluation information and corresponding optimization suggestions for the modified virtual object, so that the user can refer to such evaluation information and / or optimization suggestions to further modify the virtual object. Additionally, the modified virtual object may continue the above-mentioned testing / debugging process to make the capabilities and related configurations of the virtual object more complete.

[0094] In this way, the embodiments of the present disclosure can provide users with the output results of multiple virtual objects (e.g., agents or machine programs) for the same request, thereby improving the user's information acquisition efficiency. In addition, by providing evaluation information of the response content, the embodiments of the present disclosure can also help users better perceive the differences between the output results of different virtual objects.

[0095] Example Process

[0096] Figure 3 1 is a flowchart showing an example process 300 of information processing according to some embodiments of the present disclosure. The process 300 may be implemented at the electronic device 110. Figure 1 The process 300 is described below.

[0097] like Figure 3 As shown, in block 310 , the electronic device 110 obtains an input message.

[0098] In box 320, the electronic device 110 presents multiple response contents generated by multiple virtual objects based on the input message in the interactive interface.

[0099] In block 330 , the electronic device 110 provides evaluation information for the plurality of response contents.

[0100] In some embodiments, the input message is a first input message, the multiple response contents are multiple first response contents, and the evaluation information is first evaluation information. The method also includes: obtaining a second input message; presenting a group of response contents generated based on the second input message in the interactive interface, the group of response contents including at least one second response content generated by at least one virtual object among the multiple virtual objects based on the second input message; and providing second evaluation information generated based on the group of response contents.

[0101] In some embodiments, presenting a set of response content generated based on the second input message in the interactive interface includes: determining a set of target virtual objects based on adjustment operations on multiple virtual objects; and presenting a set of response content generated by the set of target virtual objects based on the second input message in the interactive interface.

[0102] In some embodiments, the adjustment operation includes: a deletion operation for deleting an existing virtual object from among the plurality of virtual objects; and / or an addition operation for adding an additional virtual object.

[0103] In some embodiments, the second evaluation information for the at least one second response content is determined based on the following information: the at least one second response content; and the at least one first response content generated by the at least one virtual object based on the first input message.

[0104] In some embodiments, response content generated by the first virtual object based on the first input message and / or response content generated based on the second input message is displayed in a target area corresponding to the first virtual object in the interaction interface.

[0105] In some embodiments, providing the second evaluation information generated based on a set of response content includes: presenting the first evaluation information and the second evaluation information in an interactive interface; or updating the first evaluation information to the second evaluation information.

[0106] In some embodiments, process 300 further includes: acquiring at least one configuration operation via an object configuration area in the interactive interface; and determining a plurality of virtual objects based on the at least one configuration operation.

[0107] In some embodiments, the object configuration area presents object description information for multiple virtual objects.

[0108] In some embodiments, process 300 further includes: based on the evaluation information of the multiple response contents, displaying target response contents among the multiple response contents in a target style on the interactive interface, wherein the target evaluation for the target response contents meets a preset condition.

[0109] In some embodiments, the evaluation information for the multiple response contents includes: aggregated evaluation content for the multiple response contents; and / or multiple evaluation contents corresponding to the multiple response contents.

[0110] In some embodiments, the target evaluation content among the multiple evaluation contents is generated based on the following information: the response content corresponding to the target evaluation content; and at least one other response content among the multiple response contents.

[0111] In some embodiments, the evaluation information indicates at least one of the following: a score of the corresponding response content; an evaluation text of the corresponding response content; an evaluation result of the corresponding response content with respect to a preset evaluation dimension.

[0112] In some embodiments, the process 300 further includes: in response to receiving a preset operation for the evaluation information, presenting a evaluation viewing window; and presenting at least a portion of the evaluation information and at least one additional evaluation content in the evaluation viewing window.

[0113] In some embodiments, the plurality of virtual objects include a second virtual object to be tested, and the method further includes: providing optimization suggestions for the second virtual object, wherein the optimization suggestions are generated based on a plurality of response contents and / or evaluation information.

[0114] In some embodiments, presenting multiple response contents generated by multiple virtual objects based on input messages in an interactive interface includes: in a conversation area of ​​the interactive interface, providing multiple messages generated by multiple virtual objects based on input messages in parallel, the multiple messages corresponding to the multiple response contents.

[0115] In some embodiments, the evaluation information is generated based on the following process: providing an input message and a plurality of response contents to a target model to generate evaluation information for the plurality of response contents.

[0116] In some embodiments, the target model is configured to generate rating information based on a received prompt item, wherein the prompt item at least indicates at least one rating dimension used to generate the rating information.

[0117] Example devices and equipment

[0118] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an example apparatus 400 for information processing according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0119] like Figure 4As shown, the device 400 includes: an acquisition module 410, configured to acquire an input message; a presentation module 420, configured to present, in an interactive interface, multiple response contents generated by multiple virtual objects based on the input message; and a provision module 430, configured to provide evaluation information for the multiple response contents.

[0120] In some embodiments, the input message is a first input message, the multiple response contents are multiple first response contents, and the evaluation information is first evaluation information, and the acquisition module 410 is also configured to: obtain a second input message; present a set of response contents generated based on the second input message in the interactive interface, the set of response contents including at least one second response content generated by at least one virtual object among the multiple virtual objects based on the second input message; and provide second evaluation information generated based on a set of response contents.

[0121] In some embodiments, the presentation module 420 is further configured to: determine a group of target virtual objects based on adjustment operations on multiple virtual objects; and present a group of response content generated by the group of target virtual objects based on the second input message in the interactive interface.

[0122] In some embodiments, the adjustment operation includes: a deletion operation for deleting an existing virtual object from among the plurality of virtual objects; and / or an addition operation for adding an additional virtual object.

[0123] In some embodiments, the second evaluation information for the at least one second response content is determined based on the following information: the at least one second response content; and the at least one first response content generated by the at least one virtual object based on the first input message.

[0124] In some embodiments, response content generated by the first virtual object based on the first input message and / or response content generated based on the second input message is displayed in a target area corresponding to the first virtual object in the interaction interface.

[0125] In some embodiments, the presentation module 420 is further configured to: present the first evaluation information and the second evaluation information in the interactive interface; or update the first evaluation information to the second evaluation information.

[0126] In some embodiments, the apparatus 400 further includes a determination module, which is configured to: obtain at least one configuration operation via an object configuration area in the interactive interface; and determine a plurality of virtual objects based on the at least one configuration operation.

[0127] In some embodiments, the object configuration area presents object description information for multiple virtual objects.

[0128] In some embodiments, the device 400 also includes a display module, which is configured to: based on the evaluation information of the multiple response contents, display the target response content among the multiple response contents in a target style on the interactive interface, wherein the target evaluation for the target response content meets a preset condition.

[0129] In some embodiments, the evaluation information for the multiple response contents includes: aggregated evaluation content for the multiple response contents; and / or multiple evaluation contents corresponding to the multiple response contents.

[0130] In some embodiments, the target evaluation content among the multiple evaluation contents is generated based on the following information: the response content corresponding to the target evaluation content; and at least one other response content among the multiple response contents.

[0131] In some embodiments, the evaluation information indicates at least one of the following: a score of the corresponding response content; an evaluation text of the corresponding response content; an evaluation result of the corresponding response content with respect to a preset evaluation dimension.

[0132] In some embodiments, the device 400 also includes a window presentation module, which is configured to: present an evaluation viewing window in response to receiving a preset operation for the evaluation information; and present at least part of the evaluation information and at least one additional evaluation content in the evaluation viewing window.

[0133] In some embodiments, the providing module 430 is further configured to provide optimization suggestions for the second virtual object, where the optimization suggestions are generated based on multiple response contents and / or evaluation information.

[0134] In some embodiments, the providing module 430 is further configured to: in the conversation area of ​​the interactive interface, provide in parallel a plurality of messages generated by a plurality of virtual objects based on an input message, the plurality of messages corresponding to a plurality of response contents.

[0135] In some embodiments, the evaluation information is generated based on the following process: providing an input message and a plurality of response contents to a target model to generate evaluation information for the plurality of response contents.

[0136] In some embodiments, the target model is configured to generate rating information based on a received prompt item, wherein the prompt item at least indicates at least one rating dimension used to generate the rating information.

[0137] The units included in the device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 400 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0138] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0139] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 500.

[0140] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0141] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0142] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0143] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0144] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0145] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0146] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0147] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0148] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method of information processing, comprising: Get input message; In the interactive interface, multiple response contents generated by multiple virtual objects based on the input message are presented; as well as Provide evaluation information for the plurality of response contents.

2. The method according to claim 1, wherein the input message is a first input message, the plurality of response contents are a plurality of first response contents, the evaluation information is first evaluation information, and the method further comprises: Get the second input message; In the interactive interface, presenting a group of response contents generated based on the second input message, the group of response contents including at least one second response content generated by at least one virtual object among the plurality of virtual objects based on the second input message; and Second evaluation information generated based on the set of response contents is provided.

3. The method according to claim 2, wherein presenting a set of response contents generated based on the second input message in the interactive interface comprises: Determining a set of target virtual objects based on the adjustment operations on the plurality of virtual objects; as well as The group of response contents generated by the group of target virtual objects based on the second input message are presented in the interactive interface.

4. The method according to claim 3, wherein the adjusting operation comprises: A deletion operation for deleting an existing virtual object among the plurality of virtual objects; and / or Add operation for adding additional virtual objects.

5. The method according to claim 2, wherein the second evaluation information for the at least one second response content is determined based on the following information: the at least one second response content; and At least one first response content is generated by the at least one virtual object based on the first input message.

6. The method according to claim 2, wherein response content generated by the first virtual object based on the first input message and / or response content generated based on the second input message is displayed in a target area corresponding to the first virtual object in the interaction interface.

7. The method according to claim 2, wherein providing second evaluation information generated based on the set of response contents comprises: presenting the first evaluation information and the second evaluation information in the interactive interface; or The first evaluation information is updated to the second evaluation information.

8. The method according to claim 1, further comprising: Obtaining at least one configuration operation via an object configuration area in the interactive interface; as well as Based on the at least one configuration operation, the plurality of virtual objects are determined. 9 . The method according to claim 8 , wherein the object configuration area presents object description information of the plurality of virtual objects.

10. The method according to claim 1, further comprising: Based on the evaluation information of the multiple response contents, target response contents among the multiple response contents are displayed in a target style on the interactive interface, wherein the target evaluation for the target response content meets a preset condition.

11. The method according to claim 1, wherein the evaluation information for the plurality of response contents comprises: Aggregate evaluation content for the multiple response contents; and / or The plurality of evaluation contents corresponding to the plurality of response contents.

12. The method according to claim 11, wherein the target evaluation content among the plurality of evaluation contents is generated based on the following information: Response content corresponding to the target evaluation content; and At least one other response content among the multiple response contents.

13. The method according to claim 1, wherein the evaluation information indicates at least one of the following: The score of the corresponding response content; The evaluation text of the corresponding response content; The corresponding response content is the evaluation result of the preset evaluation dimension.

14. The method according to claim 1, further comprising: In response to receiving a preset operation for the evaluation information, presenting an evaluation viewing window; as well as At least part of the review information and at least one additional review content are presented in the review viewing window.

15. The method of claim 1, wherein the plurality of virtual objects includes a second virtual object to be tested, the method further comprising: Providing optimization suggestions for the second virtual object, wherein the optimization suggestions are generated based on the multiple response contents and / or the evaluation information.

16. The method according to claim 1, wherein presenting in the interactive interface a plurality of response contents generated by a plurality of virtual objects based on the input message comprises: In the conversation area of ​​the interactive interface, multiple messages generated by multiple virtual objects based on the input message are provided in parallel, and the multiple messages correspond to the multiple response contents.

17. The method according to claim 1, wherein the evaluation information is generated based on the following process: The input message and the plurality of response contents are provided to a target model to generate the evaluation information for the plurality of response contents.

18. The method according to claim 17, wherein the target model is configured to generate the evaluation information based on a received prompt item, wherein the prompt item at least indicates at least one evaluation dimension used to generate the evaluation information.

19. An apparatus for information processing, comprising: An acquisition module, configured to acquire an input message; A presentation module, configured to present, in an interactive interface, a plurality of response contents generated by a plurality of virtual objects based on the input message; as well as The providing module is configured to provide evaluation information for the plurality of response contents.

20. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 18 when executed by the at least one processing unit.

21. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 18.

Images