Method and apparatus for processing interactive information, device, medium, and program product

Abstract

Embodiments of the present disclosure provide a method and apparatus for processing interactive information, a device, a medium, and a program product. The method comprises: detecting interactive information to be evaluated of a digital assistant, the interactive information being used for commenting on the digital assistant; and determining an evaluation result of the interactive information on the basis of the interactive information and configuration information of the digital assistant, the evaluation result indicating the tendency of the interactive information in commenting on the digital assistant.

Description

Method, device, equipment, medium and program product for interactive information processing

[0001] The present application claims priority to the Chinese patent application No. 202411304030.1, filed on September 18, 2024, entitled “Method, device, equipment, medium and program product for interactive information processing”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Example embodiments of the present disclosure generally relate to the field of computer, and in particular, to a method, device, equipment, computer readable storage medium and computer program product for interactive information processing. BACKGROUND

[0003] Digital assistants are provided to assist users in various task processing needs in different applications and scenarios. Digital assistants usually have intelligent conversation and task processing capabilities. In the interaction with the digital assistant, the user inputs an interactive message, and the digital assistant provides a reply message in response to the user input. With more and more digital assistants being developed and used, the evaluation of the conversation quality and the task processing quality of different digital assistants is a problem that needs attention. SUMMARY

[0004] In a first aspect of the present disclosure, a method for interactive information processing is provided. The method comprises: detecting interactive information to be evaluated of a digital assistant, the interactive information being used to comment on the digital assistant; and determining an evaluation result of the interactive information based on the interactive information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interactive information to comment on the digital assistant.

[0005] In a second aspect of the present disclosure, a device for interactive information processing is provided. The device comprises: a detection module configured to detect interactive information to be evaluated of a digital assistant, the interactive information being used to comment on the digital assistant; and a determination module configured to determine an evaluation result of the interactive information based on the interactive information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interactive information to comment on the digital assistant.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The device comprises at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the electronic device to perform the method of the first aspect.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0008] In a fifth aspect of the present disclosure, a computer program product is provided. The computer program product includes computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to the first aspect of the present disclosure.

[0009] It should be understood that all statements herein made regarding the exemplary embodiments of the present disclosure are intended to encompass both structural and functional equivalents thereof. Additionally, it should be understood that although the exemplary embodiments of the present disclosure have been specifically illustrated, various alternatives, modifications and equivalents can be used. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, aspects and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numbers represent like elements throughout. In the drawings:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG. 2 shows a flowchart of a process for interactive information processing according to some embodiments of the present disclosure;

[0013] FIG. 3 shows a flowchart of a signaling flow for interactive information processing according to some embodiments of the present disclosure;

[0014] FIG. 4 shows a schematic diagram of an example of an interactive information management interface according to some embodiments of the present disclosure;

[0015] FIG. 5 shows a schematic structural block diagram of an apparatus for interactive information processing according to some embodiments of the present disclosure; and

[0016] FIG. 6 shows a block diagram of an electronic device that can implement one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described herein below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for illustrative purposes and should not be construed as limiting the scope of the present disclosure.

[0018] In the description of embodiments of the disclosure, the term "comprising" and similar terms are to be interpreted as open-ended, i.e., "including but not limited to". The term "based on" is to be interpreted as "based, at least in part, on". The term "one embodiment" or "the embodiment" is to be interpreted as "at least one embodiment". The term "some embodiments" is to be interpreted as "at least some embodiments". Other explicit and implicit definitions can also be included below.

[0019] In this document, unless explicitly stated, performing a step "in response to A" does not mean performing the step immediately after A, but can include one or more intermediate steps.

[0020] It can be understood that the data involved in the technical solutions of the present application (including but not limited to the data itself, obtaining, using, storing or deleting) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0021] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of information involved in the present disclosure, the scope of use, the use scenario, etc. should be informed to the relevant user and the authorization of the relevant user should be obtained by appropriate means according to relevant laws and regulations, wherein the relevant user can include any type of right subject, such as individual, enterprise, group.

[0022] For example, in response to receiving the active request of the user, the prompt information is sent to the relevant user to explicitly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can voluntarily choose whether to provide information to the software or hardware such as electronic device, application program, server or storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0023] As an optional but non-limiting implementation manner, in response to receiving the active request of the relevant user, the prompt information is sent to the relevant user, for example, in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide information to the electronic device.

[0024] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0025] As used herein, the term “model” can learn the relationship between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes input and provides a corresponding output by using multiple layers of processing units. The neural network model is one example of a model based on deep learning. In this document, “model” can also be referred to as “machine learning model”, “learning model”, “machine learning network” or “learning network”, which are used interchangeably herein.

[0026] A “neural network” is a machine learning network based on deep learning. The neural network is capable of processing input and providing a corresponding output, which generally includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. The neural network used in deep learning applications generally includes many hidden layers, thereby increasing the depth of the network. The layers of the neural network are connected in sequence, so that the output of the previous layer is provided as the input of the next layer, where the input layer receives the input of the neural network, and the output of the output layer is the final output of the neural network. Each layer of the neural network includes one or more nodes (also known as processing nodes or neurons), each of which processes the input from the previous layer.

[0027] Generally, machine learning can include three stages, namely a training stage, a testing stage and an application stage (also known as an inference stage). In the training stage, a given model can be trained using a large amount of training data, iteratively updating the parameter values until the model can obtain consistent inference from the training data that meets the expected target. Through training, the model can be considered to be able to learn the relationship between input and output (also known as the mapping from input to output) from the training data. The parameter values of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values obtained by training to determine the corresponding output.

[0028] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In this example environment 100, a terminal device 110 has an application installed therein. A user 140 can interact with the application via the terminal device 110 and / or an attached device of the terminal device 110.

[0029] In embodiments of the present disclosure, an application can provide a digital assistant 120 to assist a user 140 in processing tasks. The digital assistant 120 can have intelligent conversation and task processing capabilities. Generally, the digital assistant 120 is capable of supporting the user 140 to input a question in a natural language manner, and perform a task and provide a reply based on an understanding of the natural language input and logical reasoning capabilities. For example, the digital assistant 120 can support a text conversation service, a voice conversation service, and a content conversation in other modalities with the user 140.

[0030] In some embodiments, the digital assistant 120 can utilize a machine learning model 160 (which can include one or more machine learning models, e.g., can include a machine learning model 160-1, a machine learning model 160-2, …, a machine learning model 160-N, etc., where N is a positive integer) to support interactions with the user 140. For example, the digital assistant can utilize one or more machine learning models 160 to provide a question and answer service to the user 140.

[0031] In the environment 100, the terminal device 110 can present a user interface 150 of the digital assistant 120 if the digital assistant 120 is in an active state. The user interface 150 can include, for example, a conversation interface of the digital assistant 120 (in which a current conversation and a historical conversation, including text conversation content, can be presented), etc. In some embodiments, the terminal device 110 can present text 152 and play speech in the user interface 150. The speech can include, for example, speech from the user 140 or speech for a reply to the speech.

[0032] The machine learning model 160 can be different types of models. In some embodiments, one or more machine learning models 160 can be built based on a language model (LM). The machine learning model used is a content generative model, which is capable of generating a corresponding output based on a model input. In some embodiments, a language model based machine learning model is capable of receiving a model input in a text modality (e.g., natural language and / or machine language) and / or a model input in a non-text modality (e.g., image, speech, video, etc.), and is capable of generating a desired output according to the model input and a prompt word. The prompt word here is used to guide the machine learning model to generate an output that can solve a user demand indicated by the model input. In an application scenario for supporting user conversation, an input of the user 140 can be provided to the machine learning model 160 as at least a part of the model input (other parts can include a prompt word). The user input is considered as a question. Based on the model output, a corresponding reply can be generated to be provided to the user 140.

[0033] In some embodiments, the terminal device 110 communicates with the service-side device 130 to implement provisioning of services of the application 120. As shown in FIG. 1, the service-side device 130 can invoke the machine learning model 160 to support the human-machine conversation function between the application 120 and the user 140 based on the output of the machine learning model 160. The terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media player, a multimedia player, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including an accessory or peripheral device of any of the foregoing, or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface to the user (such as a "wearable" circuit, etc.). The service-side device 130 can be various types of computing systems / servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, etc. The service-side device 130 can be implemented based on a cloud environment, for example.

[0034] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only, without implying any limitation on the scope of the present disclosure.

[0035] As mentioned previously, digital assistants are provided to assist users in various task processing needs in different applications and scenarios. A digital assistant generally has intelligent conversation and task processing capabilities. In the process of interaction with the digital assistant, the user inputs an interaction message, and the digital assistant provides a reply message in response to the user input.

[0036] The creator of the digital assistant can create digital assistants with different functions according to task needs or for different user groups, and the conversation quality and task processing quality of different digital assistants can also differ. Users can post comments (e.g., posts) on the digital assistant according to the use experience. These comments can reflect the quality of the digital assistant to some extent and can provide a reference for the quality evaluation of the digital assistant. However, there are many comments published by users and they can contain various contents and discussions. It is desirable to analyze more useful information from the comments on the digital assistant to help improve the digital assistant.

[0037] Therefore, the embodiments of the present disclosure provide an improved solution for interactive information processing. In the solution, the interactive information to be evaluated for the digital assistant is detected, the interactive information being used for commenting on the digital assistant. Based on the interactive information and configuration information of the digital assistant, an evaluation result of the interactive information is determined, the evaluation result indicating a tendency of the interactive information to comment on the digital assistant.

[0038] In the embodiments of the present disclosure, the tendency of the interactive information to comment on the digital assistant can be determined, which can provide a reference for the promotion, application and improvement of the digital assistant, and thus the quality of the digital assistant can be improved.

[0039] Some example embodiments of the present disclosure will be described in detail below with reference to examples of the accompanying drawings.

[0040] FIG. 2 shows a flowchart of a process 200 for interactive information processing according to some embodiments of the present disclosure. The process 200 can be implemented on the server device 130, and the process 200 shown in FIG. 2 will be described below in conjunction with FIG. 1.

[0041] In block 210 of the process 200, the server device 130 detects the interactive information to be evaluated for the digital assistant 120, the interactive information being used for commenting on the digital assistant 120. The interactive information can have various data types. For example, the interactive information can include text, voice, video, pictures, etc.

[0042] The interactive information can have various publishing manners. In one example, the digital assistant 120 can be configured to have a comment entry. The terminal device 110 can publish the interactive information to the digital assistant 120 through the comment entry. For example, the digital assistant 120 can be provided with a comment area, and the user 140 can input the interactive information to the comment area through the terminal device 110 or an attached device of the terminal device 110. The terminal device 110 sends the interactive information (e.g., a comment post) to the server device 110 in response to a publishing determination operation of the interactive information. The comment post can include a comment post of the user for the digital assistant 120, or a reply content for the comment post.

[0043] In another example, the server device 130 can receive the interactive information sent by the user 140 via a dialogue interface of the digital assistant 120. For example, the user 140 can input the interactive information for the interaction quality or the task execution quality of the digital assistant 120 during the interaction with the digital assistant 120. Of course, in actual application, the interactive information can also be published through other manners or channels.

[0044] Alternatively or additionally, the server device 130 can detect interaction information of all users with the digital assistant 120, or can detect interaction information of a specific user with the digital assistant 120. For example, the users can be divided into a creating party, an operating party and a using party, and the server device 130 can detect interaction information published by the creating party, the operating party and the using party, or can detect interaction information published by, for example, the using party.

[0045] In some embodiments, the server device 130 can be deployed with a comment system, and the server device 130 can detect interaction information for commenting on the digital assistant 120 by using the comment system. To illustrate the process of the server device 130 detecting the interaction information, the following will be described in combination with FIG. 3. FIG. 3 shows a flowchart of a signaling flow 300 of user interaction information processing, according to some embodiments of the present disclosure, the signaling flow 300 involving the terminal device 110, the server device 130 and the machine learning model 160, the server device 130 including a comment system 131, a task queue 132, an evaluation system 133 and a management system 134. As shown in the signaling flow 300, the terminal device 110 can send 301, in response to a comment operation of a user, interaction information for commenting on the digital assistant 120 to the comment system 131 of the server device 130, and the comment system 131 can save 303 the interaction information after receiving 302 the interaction information.

[0046] At block 220 of the process 200, the server device 130 determines an evaluation result of the interaction information based on the interaction information and configuration information of the digital assistant 120, in response to detecting the interaction information to be evaluated for commenting on the digital assistant 120. Alternatively or additionally, the server device 130 can determine that the interaction information to be evaluated is detected, in response to detecting new interaction information. The server device 130 can also periodically detect interaction information of the digital assistant 120, and determine whether there is un-evaluated interaction information in the saved interaction information. The configuration information of the digital assistant 120 can include, but is not limited to, a name, a number, description information and the like of the digital assistant 120, and the description information can be used to describe functions or capabilities of the digital assistant 120.

[0047] The evaluation result can indicate a tendency of the interaction information on the comment of the digital assistant 120. Alternatively or additionally, the evaluation result can at least indicate whether the interaction information is a positive comment or a negative comment on the digital assistant 120. It can be understood that the positive comment generally refers to the interaction information for praising or acknowledging the performance of the digital assistant 120 in terms of interaction quality, task processing quality, etc., and the negative comment generally refers to the interaction information for criticizing the performance of the digital assistant 120 in terms of interaction quality, task processing quality, etc. Of course, the evaluation result can also indicate a neutral comment of the interaction information on the digital assistant 120, or the evaluation result can also indicate the tendency of the comment from other dimensions. Embodiments of the present disclosure are not limited in this regard. In actual application, the comment result can indicate the tendency of the comment of the interaction information on the digital assistant 120 in various ways.

[0048] In some embodiments, the evaluation result can include a score and an evaluation reason corresponding to the score. Based on the score, the interaction information can be divided into a positive comment or a negative comment on the digital assistant. The evaluation reason can include an explanation for the current score of the interaction information, an explanation for the comment opinion of the interaction information, an explanation for the advantages of the digital assistant 120, an explanation for the defects of the digital assistant 120, and the like. For example, the score interval of the score can be determined as 1 to 100, 100 representing that the interaction information is completely positive, 1 representing that the interaction information is completely negative, and 50 representing that the interaction information is a neutral comment. Of course, the evaluation result is not limited to indicating the comment tendency by the score, but can also be indicated by various ways such as grade, advantage and disadvantage, word explanation, etc.

[0049] In some embodiments, the server device 130 can utilize the machine learning model 160 to determine the evaluation result of the interaction information. Specifically, the server device 130 can construct a model input of the machine learning model 160 based on the interaction information and the configuration information of the digital assistant 120 in response to detecting the interaction information to be evaluated. The model input is provided to the machine learning model 160 to obtain a model output of the machine learning model 160. Then, the server device 130 determines the evaluation result of the interaction information based on the model output. For example, the server device 130 can extract the evaluation result of the interaction information from a specific field of the model output.

[0050] In some embodiments, the server device 130 can add a comment event to the first task queue in response to detecting the interaction information to be evaluated. Then, in response to the comment event being removed from the first task queue, the interaction information and the configuration information of the digital assistant 120 are provided to the machine learning model 160 to obtain the evaluation result of the interaction information.

[0051] Exemplarily, as shown in the signaling flow 300, the comment system 131 can save 303 the interaction information in response to receiving 302 the interaction information. Then, the comment system 131 can add 304 the comment event to the task queue 132. The evaluation system 133 can move 305 the comment event from the task queue 132 in response to the comment event, provide 309 the model input to the machine learning model 160 based on the interaction information and the configuration information of the digital assistant 120. The machine learning model 160 can feedback 311 the model output to the evaluation system 133 in response to receiving 310 the model input. The evaluation system 133 can determine 313 the evaluation result based on the model output. Then, the evaluation system 133 can send 314 the evaluation result to the comment system 131. The comment system 131 can save 315 the evaluation result in response to receiving 314 the evaluation result. In this way, the comment system 131 and the evaluation system 133 can perform the tasks asynchronously, and the process of determining the evaluation result by the evaluation system 133 does not affect the comment system 131 receiving the interaction information.

[0052] In some embodiments, the server device 130 can determine the user type of the user who publishes the interaction information in response to detecting the interaction information to be evaluated. If it is determined that the user type does not belong to the predetermined type, the predetermined evaluation result is determined as the evaluation result of the interaction information. If it is determined that the user type is the predetermined type, the interaction information and the configuration information of the digital assistant 120 are provided to the machine learning model 160 to determine the evaluation result of the interaction information by using the machine learning model 160.

[0053] The user type of the user can be divided from multiple dimensions, such as the relationship between the user and the digital assistant 120, the attributes of the user, and the like. For example, the user can be divided into a creator, an operator, and a user according to the relationship between the user and the digital assistant 120, and the predetermined type can include the user of the digital assistant.

[0054] The predetermined evaluation result can be used to indicate that the interaction information is a neutral comment on the digital assistant 120. That is, the predetermined evaluation result indicates that the interaction information is neither a positive comment on the digital assistant 120 nor a negative comment on the digital assistant 120, but a neutral comment between the positive comment and the negative comment. Of course, the predetermined evaluation result can also contain other contents, for example, the predetermined evaluation result can also be used to indicate the user type of the user, or indicate that the user type of the user does not belong to the predetermined type.

[0055] Exemplarily, as shown in the signaling flow 300, the evaluation system 133 can remove (305) the comment event from the task queue 132 in response to the comment event, determine (306) the user type of the user. If it is determined that the user publishing the interactive information belongs to the creator or the operator, the evaluation system 133 can send (307) the predetermined evaluation result to the comment system 131, and the comment system 131 can save the score of the interactive information. For example, the middle value or the score close to the middle value in the score interval can be determined as the score of the interactive information. In the case where the score interval is 1-100, the score of 49, 50, 51, etc. can be taken as the score of the interactive information. In this way, the interactive information published by the user not belonging to the predetermined type can be avoided to be provided to the machine learning model 160, the resource of the machine learning model 160 can be saved, and the evaluation cost of the digital assistant 120 can be reduced.

[0056] In some embodiments, the service-side device 110 can provide the interactive information and the configuration information of the digital assistant 120 to the machine learning model 160 as a first model input to obtain a first model output of the machine learning model 160. The service-side device 110 can provide a second model input based on the interactive information and the configuration information of the digital assistant 120 to the machine learning model 160 to obtain a second model output generated by the machine learning model 160 in response to that the evaluation result of the interactive information is not detected from the first model output of the machine learning model 160. Then, the evaluation result of the interactive information is determined based on the second model output. That is, in the case where the machine learning model 160 fails to evaluate, the machine learning model 160 is invoked again based on the interactive information and the configuration information of the digital assistant 120 to request the machine learning model 160 to re-evaluate. In this way, the machine learning model 160 can output the evaluation result correctly in some cases.

[0057] In some embodiments, the service-side device 110 can generate a second model input meeting an input condition based on the interactive information, the configuration information of the digital assistant 120 and the reference information in response to that the evaluation result of the interactive information is not detected from the first model output of the machine learning model 160. Then, the second model input meeting the input condition is provided to the machine learning model 160 to obtain a second model output generated by the machine learning model 160. The input condition can include a condition that can enable the machine learning model 160 to output the evaluation result correctly. The input condition can include but is not limited to a content condition, a format condition, etc. of the model input. The content condition can indicate the content contained in the model input, for example, the content condition can indicate that the prompt word provided to the machine learning model 160 includes one or more prompt contents. The format condition can indicate the data structure, content layout, etc. of the model input.

[0058] The reference information here can include various information that can provide reference for the server device 160 and enable the server device 160 to correctly generate the model input. For example, the model input can include prompt information conforming to a JavaScript Object Notation (JSON) data structure, which can include, for example, a task description, input content, output requirements, and the like. The task description indicates that the machine learning model 160 determines the evaluation result of the interactive information and describes the requirements for determining the evaluation result. The input content can include, for example, the interactive information, configuration information of the digital assistant, and the like. The output requirements can indicate the content and format contained in the model output, for example, indicating that the model output of the machine learning model 160 should include a score and an evaluation reason, and the format requirements of the score and the evaluation reason, and the like. The server device 160 can generate the model input based on the interactive information and the configuration information of the digital assistant 120, and if it is determined that the generated model input does not conform to the input condition (for example, missing content), it can find a reference case of the model input from a case set of the model input. The server device 160 can supplement the generated model input with the reference case to form a model input conforming to the JSON structure. Then, the model input is provided to the machine learning model 160. In this way, the machine learning model 160 can output the evaluation result correctly in some cases.

[0059] In some embodiments, the server device 130 can also perform a predetermined operation on the digital assistant 120 based on the evaluation result. In some embodiments, the server device 130 adds an evaluation event to a second task queue based on the evaluation result. The server device 130 can be removed from the second task queue in response to the evaluation event, and perform a predetermined operation on the digital assistant based on the evaluation result. The second task queue here can be the same task queue as the first task queue, or can be a different task queue. For example, as shown in the signaling flow 300, the comment system 131 adds (316) an evaluation event to the task queue 132 in response to receiving (315) the evaluation result. The management system 134 performs a predetermined operation on the digital assistant in response to the evaluation event being removed (317) from the task queue 132. In this way, the comment system 131 and the management system 134 can perform task processing operations asynchronously.

[0060] The predetermined operation herein can include various operations related to the evaluation result. In one example, the predetermined operation can include a recommendation operation for the digital assistant 120. The server-side device 130 can determine, based on the evaluation result of the interaction information in the target time range or the target number, whether the interaction information indicating a positive comment for the digital assistant meets a recommendation condition. In response to determining that the interaction information meets the recommendation condition, the recommendation operation for the digital assistant is performed. The recommendation condition can include, but is not limited to, a positive number threshold for the interaction information of the positive comment, a positive score threshold, and the like.

[0061] The target time range can be a basic time range for the server-side device 130 to determine whether the interaction information of the positive comment meets the recommendation condition. For example, the management system 134 can be configured to periodically determine whether the number of interaction information of the positive comment in each time period exceeds a positive number threshold. If the number of interaction information of the positive comment exceeds the positive number threshold, it indicates to some extent that the interaction quality, task processing quality, and the like of the digital assistant 120 are good, and the management system 134 can perform a recommendation operation for the digital assistant.

[0062] The target number can be a basic number for the server-side device 130 to determine whether the interaction information of the positive comment meets the recommendation condition. For example, the management system 134 can be configured to determine whether the number of interaction information of the positive comment in the target number exceeds a positive number threshold each time the number of score events reaches the target number. If the number of interaction information of the positive comment exceeds the positive number threshold, the management system 134 can perform a recommendation operation for the digital assistant. The recommendation operation herein can include, but is not limited to, sending a recommendation notification to the terminal device 110, performing a recommendation display on a specific interface or platform, and the like. Of course, the recommendation operation can also include other recommendation operations, and in actual application, any appropriate recommendation operation can be selected to recommend the digital assistant 120 according to actual needs. In this way, it is conducive to the popularization and application of the digital assistant 120.

[0063] In another example, the predetermined operation can include an alarm operation for the digital assistant 120. The server device 130 can determine whether the interaction information indicating the negative comment for the digital assistant meets an alarm condition based on the evaluation result of the interaction information in the target time range or the target number of interactions. Then, in response to determining that the interaction information meets the alarm condition, the alarm operation for the digital assistant is performed. The alarm condition can include, but is not limited to, a negative number threshold of the negative comment interaction information, a negative score threshold, and the like. For example, it can be determined whether the number of negative comment interaction information exceeds the negative number threshold, and if so, it is determined that the alarm condition is met. Also for example, it can be determined whether the average score or the lowest score of the negative comment interaction information is lower than the negative score threshold, and if so, it is determined that the alarm condition is met. The alarm operation here can be an alarm operation for the operator or an alarm operation for the creator.

[0064] Exemplarily, as shown in FIG. 4, FIG. 4 shows a schematic diagram of an example 400 of an interaction information management interface according to some embodiments of the present disclosure. The example 400 includes interaction information number 401 of the interaction information, digital assistant number (ID) 402, number of replies, score 403, and publishing time 404, and the like. Of course, the comment management interface can also include, for example, the number, name, and the like of the digital assistant to which the interaction information is directed. The management system 134 can be removed from the task queue 132 in response to the evaluation event, and aggregate (318) the evaluation result of the target time range (e.g., a specific time range before the current time) based on the publishing time 404. According to the score 403, the number of positive comment interaction information and the number of negative comment interaction information in the target time range can be determined.

[0065] As shown in the signaling flow 300, if the number of negative comment interaction information exceeds the negative number threshold, the management system 134 performs (319) the alarm operation. If the number of positive comment interaction information exceeds the positive number threshold, the management system 134 can send (320) a recommendation message to the terminal device 110. The terminal device 110 receives (321) the recommendation message and can present the recommendation message. In this way, it is beneficial to the promotion application and improvement and upgrading of the digital assistant 120. It should be noted that the terminal device 110 receiving the recommendation message and the terminal device 110 publishing the interaction information can be the same terminal device or different terminal devices.

[0066] In yet another example, the predetermined operation can include an operation of feeding back the evaluation result to a creator of the digital assistant. In this way, the reference for promotion and improvement of the digital assistant 120 can be provided for the creator, which is beneficial to improve the quality of the digital assistant 120. Illustratively, the server device 110 can feed back the evaluation result to the creator in response to a request of the creator, an interaction information of a positive comment triggering a recommendation condition, an interaction information of a negative comment triggering a recommendation condition, and the like.

[0067] It should be understood that the predetermined operation is only illustrative, and should not be understood as being limited to the above operation. In actual application, any appropriate operation can be performed on the digital assistant 120 according to the evaluation result, and the type of the predetermined operation is not limited herein.

[0068] In summary, according to the embodiments of the present disclosure, the tendency of the interaction information to comment on the digital assistant can be determined, which can provide a reference for the promotion and improvement of the digital assistant, and is beneficial to improve the quality of the digital assistant.

[0069] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 5 shows a schematic structural block diagram of an apparatus 500 for interaction information processing according to some embodiments of the present disclosure. The apparatus 500 can be implemented in or included in the server device 130, for example. Various modules / components in the apparatus 500 can be implemented by hardware, software, firmware, or any combination thereof.

[0070] As shown, the apparatus 500 can include a detection module 510 and a determination module 520. The detection module 510 is configured to detect interaction information to be evaluated for a digital assistant, the comment information being used to comment on the digital assistant; and the determination module 520 is configured to determine an evaluation result of the interaction information based on the interaction information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interaction information to comment on the digital assistant.

[0071] In some embodiments, the determination module 520 is further configured to: in response to detecting the interaction information to be evaluated, add a comment event to a first task queue; and in response to the comment event being removed from the first task queue, provide the interaction information and the configuration information of the digital assistant to a machine learning model to obtain the evaluation result of the interaction information.

[0072] In some embodiments, the determination module 520 is further configured to: in response to detecting the interaction information to be evaluated, determine a user type of a user publishing the interaction information; and in response to the user type being a predetermined type, provide the interaction information and the configuration information of the digital assistant to the machine learning model to determine the evaluation result of the interaction information by using the machine learning model.

[0073] In some embodiments, the determining module 520 is further configured to determine the predetermined evaluation result as the evaluation result of the interaction information, in response to the user type not belonging to the predetermined type.

[0074] In some embodiments, the determining module 520 is further configured to provide a first model input to the machine learning model based on the interaction information and the configuration information of the digital assistant to obtain a first model output of the machine learning model; in response to the evaluation result of the interaction information not being detected from the first model output of the machine learning model, provide a second model input to the machine learning model based on the interaction information and the configuration information of the digital assistant to obtain a second model output generated by the machine learning model; and determine the evaluation result of the interaction information based on the second model output.

[0075] In some embodiments, the determining module 520 is further configured to, in response to the evaluation result of the interaction information not being detected from the first model output of the machine learning model, generate the second model input meeting the input condition based on the interaction information, the configuration information of the digital assistant and the reference information; and provide the second model input to the machine learning model to obtain the second model output generated by the machine learning model.

[0076] In some embodiments, the evaluation result includes a score and an evaluation reason corresponding to the score, the score indicating that the interaction information is a positive comment or a negative comment on the digital assistant.

[0077] In some embodiments, the apparatus 500 further includes an executing module configured to perform a predetermined operation on the digital assistant based on the evaluation result.

[0078] In some embodiments, the executing module is further configured to add an evaluation event to the second task queue based on the evaluation result; and in response to the evaluation event being removed from the second task queue, perform the predetermined operation on the digital assistant based on the evaluation result.

[0079] In some embodiments, the executing module is further configured to, in response to the interaction information of the positive comment on the digital assistant meeting a recommendation condition based on the evaluation result, perform a recommendation operation on the digital assistant; or in response to the interaction information of the negative comment on the digital assistant meeting an alarm condition based on the evaluation result, perform an alarm operation on the digital assistant; or feed back the evaluation result to a creator of the digital assistant.

[0080] In some embodiments, the executing module is further configured to feed back the evaluation result to a creator of the digital assistant.

[0081] The units and / or modules included in the apparatus 500 can be implemented utilizing various means, including software, hardware, and / or firmware. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, e.g., machine-executable instructions stored on a storage medium. In addition to or alternatively, some or all of the units and / or modules in the apparatus 500 can be implemented at least partially by one or more hardware logic components. As an example and not by way of limitation, example types of hardware logic components that can be used include Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs), System-on-a-Chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0082] FIG. 6 illustrates a block diagram of an electronic device 600 in which one or more embodiments of the disclosure can be implemented. It should be understood that the electronic device 600 illustrated in FIG. 6 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 600 illustrated in FIG. 6 can include or be implemented as the server device 130 of FIG. 1, or the apparatus 500 of FIG. 5.

[0083] As illustrated in FIG. 6, the electronic device 600 is in the form of a general electronic device. Components of the electronic device 600 can include, but are not limited to, one or more processing units or processors 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processor 610 can be a real or virtual processor and capable of performing various processes according to programs stored in the memory 620. In a multi-processor system, multiple processors perform computer-executable instructions in parallel to improve parallel processing capabilities of the electronic device 600.

[0084] The electronic device 600 typically includes a number of computer storage media. Such media can be any available media that is accessible by the electronic device 600 and includes both volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 can be a removable or non-removable media and can include machine-readable media, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data and that can be accessed by the electronic device 600.

[0085] The electronic device 600 can further include additional detachable / non-detachable, volatile / non-volatile storage media. Although not shown in FIG. 6, a disk drive for reading from or writing to a detachable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a detachable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In these cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 620 can include a computer program product 625 having one or more program modules configured to carry out the various methods or acts of the various embodiments of the present disclosure.

[0086] The communication unit 640 enables communication with other electronic devices over communication media. Additionally, the functionality of the components of the electronic device 600 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0087] The input device 650 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 660 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc., one or more devices that enable a user to interact with the electronic device 600, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 600 to communicate with one or more other electronic devices, as needed, through the communication unit 640. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0088] According to an example implementation of the present disclosure, a computer readable storage medium having computer executable instructions stored thereon is provided, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0089] Various aspects of the disclosure are now described with reference to the drawings. In general, the drawings described below are diagrammatic and schematic representations of actual or conceptual structures and processes, and are not limiting of the scope of the present disclosure. In the drawings, the size and relative positioning of components can be exaggerated for clarity and / or descriptive purposes. Also, the drawings represent examples of apparatuses and / or methods in accordance with the present disclosure. In some instances, various aspects of the disclosure can be shown in a diagram, or by a series of diagrams, and can include a set of one or more apparatuses or a combination of apparatuses and methods.

[0090] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0091] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0092] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0093] implementations of the present disclosure have been described above, the description is illustrative only and not restrictive ones, and is not limited to the disclosed implementations. Numerous modifications and variations will become apparent to those skilled in the art in light of the above teachings. The terminology used is for the purpose of describing the various implementations, and is not intended to limit the scope of the present disclosure.

Claims

1. A method for interactive information processing, comprising: detecting an interactive information to be evaluated for a digital assistant, the interactive information being for commenting on the digital assistant; and determining an evaluation result of the interactive information based on the interactive information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interactive information on the comment on the digital assistant.

2. The method of claim 1, wherein determining the evaluation result of the interactive information comprises: adding a comment event to a first task queue in response to detecting the interactive information to be evaluated; and providing the interactive information and the configuration information of the digital assistant to a machine learning model to obtain the evaluation result of the interactive information in response to the comment event being removed from the first task queue.

3. The method of claim 1, wherein determining the evaluation result of the interactive information comprises: determining a user type of a user publishing the interactive information in response to detecting the interactive information to be evaluated; and providing the interactive information and the configuration information of the digital assistant to a machine learning model to determine the evaluation result of the interactive information with the machine learning model in response to the user type being a predetermined type.

4. The method of claim 3, wherein determining the evaluation result of the interactive information further comprises: determining a predetermined evaluation result as the evaluation result of the interactive information in response to the user type not belonging to the predetermined type.

5. The method of claim 1, wherein determining the evaluation result of the interactive information comprises: providing a first model input to a machine learning model based on the interactive information and the configuration information of the digital assistant to obtain a first model output of the machine learning model; providing a second model input to the machine learning model based on the interactive information and the configuration information of the digital assistant to obtain a second model output generated by the machine learning model in response to the evaluation result of the interactive information not being detected from the first model output of the machine learning model; and determining the evaluation result of the interactive information based on the second model output.

6. The method of claim 5, wherein providing the second model input to the machine learning model based on the interactive information and the configuration information of the digital assistant comprises: generating a second model input meeting an input condition based on the interactive information, the configuration information of the digital assistant and reference information in response to the evaluation result of the interactive information not being detected from the first model output of the machine learning model; and providing the second model input to the machine learning model to obtain the second model output generated by the machine learning model.

7. The method of claim 1, wherein the evaluation result comprises a score and an evaluation reason corresponding to the score, the score indicating that the interactive information is a positive comment or a negative comment on the digital assistant.

8. The method of claim 1, further comprising: performing a predetermined operation on the digital assistant based on the evaluation result. ​ ​ ​ ​ ​ 9.The method of claim 8, wherein performing the predetermined operation on the digital assistant comprises: adding, to a second task queue, an evaluation event based on the evaluation result; and performing, based on the evaluation result, the predetermined operation on the digital assistant in response to the evaluation event being removed from the second task queue. 10.The method of claim 8, wherein performing the predetermined operation on the digital assistant comprises at least one of: in response to determining, based on the evaluation result, that the interaction information of the positive comment on the digital assistant meets a recommendation condition, performing a recommendation operation on the digital assistant; or performing, in response to determining that the interaction information of the negative comment on the digital assistant meets an alarm condition based on the evaluation result, an alarm operation on the digital assistant; or or feeding back the evaluation result to a creator of the digital assistant. 11.An apparatus for interaction information processing, comprising: a detecting module configured to detect interaction information of a digital assistant to be evaluated, the interaction information being for commenting on the digital assistant; a determining module configured to determine an evaluation result of the interaction information based on the interaction information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interaction information on the comment of the digital assistant. 12.An electronic device, comprising: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1 to 10. 13.A computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions being executable by a processor to implement the method according to any one of claims 1 to 10. 14.A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images