Evaluation method for interactive system, evaluation model construction method and device

Abstract

One or more embodiments of the invention provide an evaluation method for an interactive system, an evaluation model construction method and device, electronic equipment, a computer readable storage medium and a computer program product. The evaluation method for the interactive system comprises the steps of obtaining at least one piece of configuration information of the interactive system to be evaluated in at least one configuration type; determining an evaluation index of the to-be-evaluated interaction system in at least one evaluation dimension according to at least one piece of configuration information of the to-be-evaluated interaction system in the at least one configuration type; and inputting the evaluation index of the to-be-evaluated interaction system in the at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the to-be-evaluated interaction system according to the input evaluation index and outputs an evaluation result. The performance of the interaction system is automatically and quickly evaluated.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of cloud computing technology, and in particular, to a method for evaluating an interaction system, a method for constructing an evaluation model, a device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In recent years, the cloud service market has shown a trend of rapid growth, and its industrial scale and the number of users have continued to increase at a high speed. With the popularization of cloud services and the expansion of the application scope, the interaction systems involving cloud services are constantly iterating and updating.

[0003] An interaction system involving cloud services usually consists of a cloud server and a client deployed in a terminal device. The client is responsible for interacting with users and providing services, while the cloud server undertakes important tasks such as data processing, storage, and application logic.

[0004] In related technologies, manual testing means are usually adopted to monitor the performance of the continuously iterating interaction system. However, manual testing requires a large amount of time and effort. Especially in the case of frequent updates of the interaction system, the testing work often cannot keep up with the iteration speed. Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide a method for evaluating an interaction system, a method for constructing an evaluation model, a device, an electronic device, and a computer-readable storage medium.

[0006] To achieve the above object, one or more embodiments of this specification provide the following technical solutions:

[0007] According to a first aspect of one or more embodiments of this specification, a method for evaluating an interaction system is proposed. The interaction system includes a cloud server and a client deployed in a terminal device. The method includes:

[0008] Obtain at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type;

[0009] Determine the evaluation indexes of the interaction system to be evaluated in at least one evaluation dimension according to at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type;

[0010] Input the evaluation indexes of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the interaction system to be evaluated according to the input evaluation indexes and outputs an evaluation result.

[0011] According to a second aspect of one or more embodiments of this specification, a method for constructing an evaluation model is proposed. The evaluation model is used to evaluate an interaction system. The interaction system includes a cloud server and a client deployed in a terminal device. The method includes:

[0012] Obtain at least one configuration information of a number of interaction systems in at least one configuration type and the evaluation result labels corresponding to each interaction system;

[0013] Determine the evaluation indicators of each interaction system in at least one evaluation dimension according to at least one configuration information of each interaction system in at least one configuration type;

[0014] Use the evaluation indicators of each interaction system in at least one evaluation dimension and the evaluation result labels among the number of interaction systems as training samples for supervised training to obtain an evaluation model;

[0015] Among them, the evaluation model is used to evaluate the interaction system according to the evaluation indicators of the input interaction system in at least one evaluation dimension to output an evaluation result.

[0016] According to a third aspect of the embodiments of the present disclosure, an evaluation device for an interaction system is provided. The interaction system includes a cloud server and a client deployed in a terminal device. The device includes:

[0017] A configuration information acquisition module, configured to obtain at least one configuration information of the interaction system to be evaluated in at least one configuration type;

[0018] An evaluation index determination module, configured to determine the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension according to at least one configuration information of the interaction system to be evaluated in at least one configuration type;

[0019] An evaluation module, configured to input the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model evaluates the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result.

[0020] According to a fourth aspect of the embodiments of the present disclosure, a device for constructing an evaluation model is provided. The evaluation model is used to evaluate an interaction system. The interaction system includes a cloud server and a client deployed in a terminal device. The device includes:

[0021] A configuration information acquisition module, configured to obtain at least one configuration information of a number of interaction systems in at least one configuration type and the evaluation result labels corresponding to each interaction system;

[0022] An evaluation index determination module, configured to determine evaluation indexes of each of the interaction systems in at least one evaluation dimension according to at least one piece of configuration information of each of the interaction systems in at least one configuration type;

[0023] A model training module, configured to perform supervised training on the evaluation indexes of each of the interaction systems in at least one evaluation dimension and the evaluation result labels of the several interaction systems as training samples to obtain an evaluation model; wherein, the evaluation model is configured to perform an evaluation process on an interaction system according to the evaluation indexes of the interaction system in at least one evaluation dimension to output an evaluation result.

[0024] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0025] A processor;

[0026] A memory for storing executable instructions of the processor;

[0027] Wherein, when the processor executes the executable instructions, it is configured to implement the method described in the first aspect or the second aspect.

[0028] According to a sixth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the method described in the first aspect or the second aspect.

[0029] According to a seventh aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect or the second aspect.

[0030] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0031] In the embodiments of the present disclosure, a method for evaluating an interaction system is provided. The method uses machine learning technology to establish an evaluation model to replace traditional manual testing. In the process of executing the method for evaluating an interaction system, first, at least one piece of configuration information of the interaction system to be evaluated is obtained; then, according to the obtained configuration information, evaluation indicators of the interaction system in at least one evaluation dimension are determined, so that the configuration of the interaction system is evaluated in one or more aspects; then, the evaluation indicators of the interaction system in at least one evaluation dimension are input into a pre-trained evaluation model, and the evaluation model performs evaluation processing on the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result. The evaluation result can reflect the performance of the interaction system in each evaluation dimension. Through the above process, the performance of the interaction system can be accurately and comprehensively evaluated, so as to objectively and accurately provide guidance for the optimization direction of the interaction system; and through the evaluation model, the performance of the interaction system under different configuration types can be automatically and quickly evaluated, avoiding the cumbersome and time-consuming manual testing, saving time and resources, and improving the evaluation efficiency.

[0032] A method for constructing an evaluation model is also provided. The evaluation model is obtained by supervised training based on a training data set. The training data set includes a number of training samples. Each training sample includes evaluation indicators of the interaction system in at least one evaluation dimension and an evaluation result label. The configuration information of the interaction systems pointed to by different training samples is different. Through a large number of sample data, the relationship between the performance and configuration of the interaction system can be captured more accurately. Furthermore, the trained evaluation model can be used to automatically and accurately evaluate the interaction system, avoiding the cumbersome and time-consuming manual testing.

[0033] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 FIG. is a schematic structural diagram of a terminal cloudification system provided by an exemplary embodiment. [[ID=!4]]

[0035] Figure 2A FIG. is a schematic flowchart of a method for constructing an evaluation model provided by an exemplary embodiment.

[0036] Figure 2B FIG. is a schematic diagram of an evaluation framework provided by an exemplary embodiment

[0037] Figure 3 FIG. is a schematic diagram of evaluation dimensions affected by various configuration types provided by an exemplary embodiment.

[0038] Figure 4 FIG. is a schematic diagram of evaluation model training provided by an exemplary embodiment. ‍

[0039] Figure 5 is a schematic flowchart of an evaluation method for a terminal cloudification system provided by an exemplary embodiment.

[0040] Figure 6 is a schematic structural diagram of an electronic device provided by an exemplary embodiment.

[0041] Figure 7 is a block diagram of an evaluation device for a terminal cloudification system provided by an exemplary embodiment.

[0042] Figure 8 is a block diagram of a device for constructing an evaluation model provided by an exemplary embodiment. Detailed implementation manners

[0043] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0044] It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0045] In recent years, the cloud service market has shown a trend of rapid growth, and its industrial scale and the number of users have continued to increase at a high speed. With the popularization of cloud services and the expansion of the application scope, the interaction systems involving cloud services are also constantly iterating and updating.

[0046] The interaction system involving cloud services usually consists of a cloud server and a client deployed in a terminal device. The client is responsible for interacting with users and providing services, while the cloud server undertakes important tasks such as data processing, storage, and application logic.

[0047] The interaction system involving cloud services involves multiple application scenarios, such as terminal cloudification scenarios, big data analysis scenarios, virtualization and containerization scenarios, artificial intelligence and machine learning scenarios, etc., but is not limited thereto.

[0048] The following takes the scenario of terminal cloudification as an example for illustrative purposes: Terminal cloudification is a technology that migrates tasks such as application programs, data processing, and storage on terminal devices from local to the cloud or edge cloud to improve the performance of terminal devices and reduce power consumption. This technology is applicable to various scenarios. The following are some examples of terminal cloudification:

[0049] (1) Cloud mobile phone: Users use ordinary terminal devices (such as smartphones), but all computing tasks and data storage are performed on the cloud server. It can save the resources of terminal devices, enable users to seamlessly switch between different devices, and provide larger storage space and computing power. It can be applied to scenarios such as mobile office, gaming, and entertainment.

[0050] (2) Cloud gaming: The graphics processing and computing tasks of the game are completed on the cloud server, and users only need to smoothly play the game through the terminal device. It does not require powerful local gaming devices and is suitable for users of lightweight devices and low - configuration devices. It can be applied to scenarios such as game entertainment, game live broadcast, and multi - platform game experience.

[0051] (3) Cloud desktop: The user's desktop environment, application programs, and files are stored on the cloud server and can be accessed through the client in the terminal device. It improves the flexibility of office work, supports cross - device desktop synchronization and remote work. It can be applied to scenarios such as enterprise office, remote work, and education.

[0052] The terminal cloudification market has shown a rapid growth trend in recent years. The industrial scale and the number of users represented by cloud gaming and cloud mobile phones have continued to grow at a high speed, which has also led to the continuous iteration of the terminal cloudification system.

[0053] Please refer to Figure 1 , the terminal cloudification system includes a cloud server and a client deployed in the terminal device. The cloud server is used to execute computing tasks and return computing results in response to control instructions sent by the client.

[0054] Terminal devices include but are not limited to smartphones, tablets, laptops, Internet of Things devices, etc. Various terminal devices used by users can be connected to the cloud server through the network.

[0055] The client is a lightweight client program running on the terminal device, responsible for the user interface and basic functions of communicating with the cloud server. Exemplarily, the client has a simple interface and communicates with the cloud server through network protocols.

[0056] The cloud server can be an edge cloud server located near the terminal device, which is used to process the computing tasks offloaded from the terminal device, providing a faster response speed and reducing data transmission latency. The edge cloud server is usually deployed closer to the user and can be a cloud data center physically closer to the terminal device. Exemplarily, the operating system on the terminal device can be deployed in the cloud server through cloudification technology, enabling it to run on the cloud server, providing powerful computing power and being able to run application programs (such as Android applications) without discrimination.

[0057] Exemplarily, the cloud server can be an edge heterogeneous server. An edge heterogeneous server refers to a server in which the hardware daughter boards for terminal devices are integrated in a high-density manner, such as manufacturing an SoC chip into a high-density array server through a series of designs. Among them, SoC (System on Chip) is a system-level chip that integrates a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a VPU (Video Processing Unit), an NPU (Neural Network Processing Unit), etc. on an integrated circuit.

[0058] The calculation result can be presented in at least one of text, picture, audio, and video. Taking audio and video as an example, a push stream service for audio and video acquisition and encoding can be deployed in the cloud server. After the cloud server finishes executing the computing task and generates audio and video data, it can transmit the audio and video data to the client based on streaming technology; a pull stream service for audio and video can be deployed in the client, and the client receives the audio and video data through the network and renders the audio and video data.

[0059] In the related art, manual testing means are usually adopted to monitor the performance of the continuously iterated terminal cloudification system, which is cumbersome and time-consuming. In manual testing, the tester needs to actually operate the terminal cloudification system, execute specific tasks or scenarios, and record the performance of the system. For example, in the field of cloud games, the tester may need to perform actual game operations and observe the response speed, frame rate stability, etc. of the system, which requires a lot of time and effort and is prone to human errors. Especially when the terminal cloudification system is updated frequently, the testing work often cannot keep up with the iteration speed.

[0060] Based on this, the embodiments of this specification provide an evaluation method for an interaction system, which uses machine learning technology to establish an evaluation model to replace the traditional manual testing, thereby shortening the evaluation time, improving the evaluation efficiency and accuracy, better meeting the market demand and user expectations, and can greatly improve the development efficiency of the interaction system.

[0061] In the process of executing the evaluation method for an interactive system, first, obtain at least one piece of configuration information of the interactive system to be evaluated in at least one configuration type; then, based on the obtained configuration information, determine the evaluation indicators of the interactive system in at least one evaluation dimension, so that the configuration of the interactive system is evaluated in one or more aspects; next, input the evaluation indicators of the interactive system in at least one evaluation dimension into a pre-trained evaluation model, and the evaluation model performs an evaluation process on the interactive system to be evaluated according to the input evaluation indicators and outputs an evaluation result. The evaluation result can reflect the performance of the interactive system in each evaluation dimension. Through the above process, the performance of the interactive system can be accurately and comprehensively evaluated, so as to objectively and accurately provide guidance for the optimization direction of the interactive system. And through the evaluation model, the performance of the interactive system under different configuration types can be automatically and quickly evaluated, avoiding the cumbersome and time-consuming manual testing, saving time and resources, and improving the evaluation efficiency.

[0062] The embodiment of this specification also provides a method for constructing an evaluation model. Using the method of supervised learning, a supervised training is performed based on a training data set to obtain an evaluation model. The training data set includes a number of training samples, and each training sample includes the evaluation indicators of the interactive system in at least one evaluation dimension and an evaluation result label. The configuration information of the interactive systems pointed to by different training samples is different. Through a large number of sample data, the relationship between the performance of the interactive system and the configuration can be captured more accurately. Furthermore, the trained evaluation model can be used to automatically and accurately evaluate the interactive system, avoiding the cumbersome and time-consuming manual testing.

[0063] In some embodiments, the evaluation method for the interactive system and the method for constructing the evaluation model provided in the embodiment of this specification can be executed by the same electronic device or by different electronic devices, and this embodiment does not impose any restrictions on this. Exemplarily, the method for constructing the evaluation model provided in the embodiment of this specification can be executed by a first electronic device to obtain a trained evaluation model, and then it is deployed in a second electronic device for executing the evaluation method for the interactive system.

[0064] Please refer to Figure 2A , and next, an exemplary description of a method for constructing an evaluation model provided in the embodiment of this specification will be given first: The method for constructing the evaluation model is applied to an electronic device; the evaluation model is used to evaluate an interactive system; the interactive system includes a cloud server and a client deployed in a terminal device; this method includes S101 to S103.

[0065] In S101, obtain at least one piece of configuration information of a number of interactive systems in at least one configuration type and the evaluation result label corresponding to each interactive system.

[0066] Exemplarily, please refer toFigure 2B The interactive system includes a cloud server and a client, so it involves aspects such as the hardware configuration of the cloud server, software configuration, the configuration of the communication link between the client and the cloud server, and the configuration of the client. Therefore, the configuration types of the configuration information include at least one of the following: client configuration type, communication link configuration type between the client and the cloud server, software configuration type of the cloud server, and hardware configuration type of the cloud server.

[0067] By obtaining the configuration information of the interactive system in at least one of the aspects of the client, communication link, cloud server software, and hardware, the configuration status of the interactive system in each aspect can be concerned. The configuration information under different configuration types can be selected to evaluate the performance of the interactive system under different configuration types, so as to understand the performance of the system in different aspects and help users make more informed choices; or the configuration information of the interactive system under all configuration types can be comprehensively considered to comprehensively evaluate the performance of the interactive system; specific selection can be made according to the actual application scenario.

[0068] In S102, according to at least one piece of configuration information of each interactive system in at least one configuration type, the evaluation indexes of each interactive system in at least one evaluation dimension are determined.

[0069] Exemplarily, the evaluation dimension includes at least one of the following: cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension. But it is not limited to this.

[0070] The cost dimension mainly focuses on the cost issues of the interactive system, including hardware, software, and operation costs. For example, the hardware expenditure cost of the cloud server, the software deployment cost of the cloud server, etc.

[0071] The performance dimension mainly focuses on the performance performance of the interactive system, including response speed, stability, load-bearing capacity, etc. For example, how the response time and frame rate stability of the interactive system will affect the user experience, and how much load the system can withstand.

[0072] The function dimension mainly focuses on the functional characteristics of the interactive system, including complete functions, easy to use, easy to maintain, etc. For example, whether the interactive system supports multiple operating systems and applications, and whether it is easy to configure and manage.

[0073] The compatibility dimension mainly focuses on the compatibility issues of the interactive system, including compatibility with different operating systems, applications, and hardware devices. For example, whether the interactive system is compatible with various browsers, mobile devices, and desktop platforms.

[0074] The security dimension mainly focuses on the security issues of the interactive system, including data security, network security, authentication, etc. For example, whether the interactive system adopts encryption measures when transmitting data, and how to prevent hacker attacks and malware.

[0075] The latency dimension mainly focuses on the response time issues of the interactive system, including the delay during data transmission and the response speed. For example, how the latency time of the interactive system will affect the user experience, and how to improve the response speed.

[0076] The ease of operation dimension mainly focuses on the ease of operation of the interactive system, including aspects such as user interface design, menu settings, and instruction operations. For example, whether the operation interface of the interactive system is friendly and easy to use.

[0077] In S103, the evaluation indicators and evaluation result labels of each interactive system in several interactive systems in at least one evaluation dimension are used as training samples for supervised training to obtain an evaluation model; wherein, the evaluation model is used to evaluate the interactive system according to the evaluation indicators of the input interactive system in at least one evaluation dimension to output an evaluation result.

[0078] This embodiment uses the method of supervised learning, considers the configuration of the interactive system from multiple perspectives, constructs an evaluation model that can be used for evaluation and optimization, can be more accurate and objective when evaluating the performance of the interactive system, the evaluation model is obtained based on a large number of training samples, has a high prediction accuracy, and is more reliable than the traditional subjective evaluation method.

[0079] For S101, please refer to Figure 2B , taking the terminal cloudification system as an interactive system as an example, the configuration information of the terminal cloudification system in each configuration type is exemplarily described:

[0080] In some possible implementation manners, the configuration information of the terminal cloudification system in the hardware configuration type of the cloud server includes but is not limited to:

[0081] (1) Configuration information regarding firmware selection, which involves the selection of the processor and peripheral external devices; the firmware includes but is not limited to the central processing unit (CPU), graphics processing unit (GPU), storage device, network card, and external devices (such as cameras, sensors), etc.

[0082] (2) Configuration information regarding server hardware design. The server hardware design determines how to enable each firmware to fully exert its performance through architecture design and comprehensively generate higher overall benefits. Server hardware design involves density, power consumption control, design of the server's motherboard / backplane / switch board, BMC (Baseboard Management Controller) control board, manufacturing quality (yield rate) of hardware design, etc.

[0083] (3) Configuration information regarding network access standards, which refers to the network access specifications that the server deployment must comply with. It involves wiring and networking solutions, control component solutions, out-of-band management solutions, etc.

[0084] In some possible implementation manners, the configuration information of the software configuration type of the terminal cloudification system includes but is not limited to:

[0085] (1) Configuration information regarding virtualization technology types. Virtualization technology types include full virtualization type, para-virtualization type, and application-level virtualization type. The full virtualization type means that the underlying hardware is fully emulated, and the GuestOS (the operating system running in the virtual machine in the virtualization environment) does not need to be modified. The para-virtualization type means that part of the underlying hardware is emulated, and the GuestOS needs to be modified to handle special instructions. The application-level virtualization type means that virtualization is performed at the application level rather than the entire operating system. This virtualization method is lighter and more flexible, but requires the support of the application.

[0086] (2) Configuration information regarding the cloudification of the operating system in the terminal. The cloudification of the operating system in the terminal means that the operating system running on the terminal device can stably run on the cloud server through a series of cloudification technologies. The configuration information regarding the cloudification of the operating system in the terminal involves peripheral device virtualization, real machine emulation, performance acceleration, etc. Among them, peripheral device virtualization means that the peripheral devices (such as cameras, sensors, etc.) on the terminal device are virtualized to the cloud server through virtualization technology. Real machine emulation means that the real environment of the terminal device is simulated to the cloud server by using emulation technology, so that the operating system of the terminal device can stably run in an emulated manner on the cloud server, reducing the dependence on hardware. Performance acceleration means that through various optimization and acceleration technologies, the performance of the operating system of the terminal device on the cloud server is improved, including optimization in terms of response speed, stability, etc.

[0087] In some possible implementation manners, the communication link between the client and the cloud server is used to transmit the calculation results generated by the cloud server after completing the calculation tasks, such as audio and video data, image data, etc. The configuration information of the configuration type of the communication link between the client and the cloud server in the terminal cloudification system includes but is not limited to: data sending algorithms, data splitting and packaging technologies, network transmission protocols, and anti-weak network technologies. Among them, the data sending algorithms include information in aspects such as data compression, bandwidth optimization, and transmission strategies; the data splitting and packaging technologies include how to perform data slicing, frame rate control, data buffering, etc.; the network transmission protocols such as TCP, UDP, etc.; the anti-weak network technologies include packet retransmission, network adaptive adjustment, network quality detection, etc.

[0088] In some possible implementation manners, the configuration information of the configuration type of the terminal cloudification system in the client includes but is not limited to: audio and video acquisition and codec technologies, terminal device model and operating system adaptation (the adaptation degree of the model of the terminal device and the operating system), external device adaptation (the adaptation situation between external devices (such as mice, keyboards, cameras, etc.) and the client), and control input instruction acquisition (the manner in which the client program acquires control instructions, including speech recognition, gesture recognition, mouse and keyboard input, etc.).

[0089] It can be understood that the number of items of the configuration information in each configuration type of the terminal cloudification system can be dynamically increased or decreased according to the iterative process of the terminal cloudification system. For example, in the iterative process of the terminal cloudification system, if an optimization method is added to the software configuration type of the cloud server, then the number of items of the configuration information corresponding to the software configuration type of the cloud server to be measured increases by 1 item. Another example is that in the iterative process of the terminal cloudification system, in order to adapt to a new scenario, one item of the configuration information in the communication link configuration type is deleted, then the number of items of the configuration information corresponding to the communication link configuration type to be measured decreases by 1 item.

[0090] In some embodiments, the evaluation result labels corresponding to each interaction system are determined based on at least one of the market selection information of the interaction system and the user's usage evaluation; wherein, the market selection information is used to represent the actual usage or sales situation of the interaction system. Exemplarily, the market selection information includes the usage times and / or purchase times of the interaction system; among them, the values of the evaluation result labels corresponding to each interaction system are respectively positively correlated with the usage times, purchase times, and the number of positive evaluation times of the user. For example, the higher the usage times of the interaction system, the higher the value of the evaluation result label corresponding to the interaction system, and vice versa.

[0091] Exemplarily, the numerical value of the evaluation result label of the interaction system can be jointly determined by integrating the market selection information of the interaction system and the user's usage evaluation. For example, different weights can be set for the market selection information of the interaction system and the user's usage evaluation (such as the weight of the market selection information is greater than the weight of the user's usage evaluation), and the numerical value of the evaluation result label of the interaction system can be obtained by using the weighted summation method.

[0092] For S102, the evaluation dimensions include at least one of the following: cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension; but not limited to this. Among them, the evaluation dimensions affected by different configuration types are not exactly the same.

[0093] Exemplarily, taking the interaction system as the terminal cloudification system as an example, please refer to Figure 3 , the hardware configuration type of the cloud server will affect 5 evaluation dimensions, namely the cost dimension, performance dimension, function dimension, compatibility dimension, and security dimension. For example: high-density design will affect the cost; the selection of firmware will affect the function, performance, and compatibility (such as the ARM or x86 architecture will affect the performance and system application compatibility); the access node network architecture standard will affect the performance (network topology design, number of data forwarding times), security (access rights, isolation scheme), etc.

[0094] The software configuration type of the cloud server will affect 7 dimensions, namely the cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension. For example, the selection of the virtualization technology type will affect the evaluation dimensions such as cost, performance, function, and security, and the cloudification of the terminal operating system will affect the dimensions such as function, performance, compatibility, and security.

[0095] The communication link configuration type between the client and the cloud server will affect 3 dimensions, namely the security dimension, latency dimension, and ease of operation dimension. For example, whether the network transmission is encrypted affects the security dimension, and the anti-weak network scheme affects the latency dimension and ease of operation dimension, etc.

[0096] The client configuration type will affect 4 dimensions, namely the function dimension, compatibility dimension, latency dimension, and ease of operation dimension. For example, the external device adaptation function, audio and video acquisition, and encoding and decoding technology affect the latency dimension, and the acquisition of control input instructions affects the ease of operation dimension, etc.

[0097] In some embodiments, considering that the evaluation dimensions affected by different configuration types are not exactly the same, in order to improve the operation efficiency of evaluation indicators, at least one piece of configuration information in at least one configuration type of each interaction system is divided into different configuration information sets according to different configuration types. For each configuration type, according to each piece of configuration information in the configuration information set corresponding to this configuration type, the evaluation indicators corresponding to each piece of configuration information in at least one evaluation dimension affected by this configuration type are determined. In this embodiment, dividing the obtained configuration information into corresponding configuration information sets according to different configuration types can improve the operation efficiency of evaluation indicators. Because the influence range of each configuration type is different, only by determining the evaluation indicators of the corresponding evaluation dimensions for the configuration information set of each configuration type can the performance of each configuration type in the affected evaluation dimensions be evaluated more accurately.

[0098] Taking the hardware configuration type of a cloud server as an example, assume that the configuration information set corresponding to the hardware configuration type of the cloud server includes 3 pieces of configuration information, which are respectively about firmware selection, server hardware design, and network access standard; the hardware configuration type of the cloud server affects 5 evaluation dimensions, including cost dimension, performance dimension, function dimension, compatibility dimension, and security dimension. The 3 pieces of configuration information respectively correspond to evaluation indicators in the 5 evaluation dimensions. For example, the configuration information about firmware selection has a cost indicator in the cost dimension, a performance indicator in the performance dimension, a function indicator in the function dimension, a compatibility indicator in the compatibility dimension, and a security indicator in the security dimension. Similarly, the other pieces of configuration information respectively correspond to evaluation indicators in the 5 evaluation dimensions.

[0099] Exemplarily, in order to further improve the determination efficiency of evaluation indicators, the mapping relationships between different pieces of configuration information corresponding to various configuration types in the above 4 configuration types and the evaluation indicators of each evaluation dimension affected by this configuration type can be pre-stored. Then, in the process of determining evaluation indicators, for each configuration type, according to each piece of configuration information in the configuration information set corresponding to this configuration type, and the pre-stored mapping relationships between different pieces of configuration information corresponding to this configuration type and the evaluation indicators of each evaluation dimension affected by this configuration type, the evaluation indicators corresponding to each piece of configuration information in at least one evaluation dimension affected by this configuration type are determined. In this embodiment, by pre-storing the mapping relationships, the electronic device can quickly access the mapping relationships between the required configuration information and evaluation indicators during the process of determining evaluation indicators, and perform matching and calculation, thus greatly improving the calculation speed.

[0100] In some embodiments, for S103, after determining the evaluation metrics corresponding to each item of configuration information in the set of configuration information corresponding to each configuration type in at least one evaluation dimension affected by the configuration type, the electronic device may use the evaluation metrics and evaluation result labels of each interaction system in several interaction systems in at least one evaluation dimension as training samples for supervised training to obtain an evaluation model.

[0101] Exemplarily, please refer to Figure 4 , the evaluation metrics and evaluation result labels of each interaction system in several interaction systems in at least one evaluation dimension among the cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension can be used as training samples and input into the evaluation model to be trained. The evaluation model performs evaluation processing based on the evaluation metrics of the interaction system in at least one evaluation dimension to obtain a predicted evaluation result, and then trains the evaluation model with the goal of minimizing the error between the predicted evaluation result and the evaluation result label corresponding to the interaction system.

[0102] Among them, the embodiments of this specification do not impose any restrictions on the representation forms of the predicted evaluation result and the evaluation result label, which can be represented in the form of numerical values, vectors, or text. For example, it can be represented by numbers from 0 to 100, and the higher the numerical value, the higher the evaluation effect; or it can be represented by text descriptions of different levels such as {poor, average, good, and excellent}.

[0103] Exemplarily, to improve the training efficiency, for each interaction system, after determining the evaluation metrics corresponding to each item of configuration information in the set of configuration information corresponding to each configuration type in at least one evaluation dimension affected by the configuration type, the evaluation metrics corresponding to each item of configuration information in all configuration information in at least one evaluation dimension can be classified according to the evaluation dimension to which the evaluation metrics belong, and an evaluation metric set of the interaction system to be evaluated in each evaluation dimension is obtained; and based on the number of evaluation metrics in the evaluation metric set of each evaluation dimension, the number of configuration information items affecting the evaluation dimension is determined; finally, the electronic device may use the evaluation metric sets of each interaction system in several interaction systems in at least one evaluation dimension, the number of configuration information items affecting the evaluation dimension, and the evaluation result labels as training samples for supervised training to obtain an evaluation model. In this embodiment, the evaluation metrics corresponding to each item of configuration information are classified according to their respective evaluation dimensions, and a fine-grained evaluation metric set can be obtained, so as to more comprehensively understand the performance of the interaction system in each evaluation dimension; and by determining the number of configuration information items affecting each evaluation dimension, it can help understand the influence degree of different configuration information on the performance of the interaction system, which is helpful for optimizing and adjusting the configuration to achieve better performance.

[0104] Exemplarily, the parameters in the evaluation model include influence coefficients and weight coefficients.

[0105] Among them, each evaluation dimension corresponds to a weight coefficient; the weight coefficients of each evaluation dimension are used to represent the importance of this evaluation dimension in the overall evaluation. Different evaluation dimensions may have different important impacts on the performance of the interactive system, so different weights need to be assigned to them to reflect their contributions in the overall evaluation.

[0106] For at least one evaluation dimension affected by any configuration type, each configuration information in the set of configuration information corresponding to this configuration type respectively corresponds to an influence coefficient in at least one evaluation dimension. Each configuration information will have a corresponding influence coefficient in at least one evaluation dimension, and this coefficient can represent the influence intensity of this configuration information in this evaluation dimension.

[0107] The initial values of the influence coefficients and weight coefficients can be preset based on the requirements of the actual application scenario, and then the influence coefficients and weight coefficients are adjusted during the training process of the evaluation model to achieve the optimization goal of the evaluation model.

[0108] Exemplarily, after inputting the set of evaluation indicators of each interactive system in several interactive systems in at least one evaluation dimension and the number of configuration information items affecting this evaluation dimension into the evaluation model to be trained, this evaluation model is used to, for each evaluation dimension, according to the set of evaluation indicators of this evaluation dimension, the number of configuration information items corresponding to this evaluation dimension, and the influence coefficients corresponding to each configuration information affecting this evaluation dimension, determine the target evaluation indicator corresponding to this evaluation dimension, and then according to the target evaluation indicators corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension, determine the predicted evaluation result of the interactive system. Through comprehensively considering the weights and indicators of multiple evaluation dimensions, and the influence degree of each configuration information on different evaluation dimensions in this embodiment, a more comprehensive evaluation result of the interactive system performance can be obtained; and by using the evaluation model to predict the evaluation result, a large amount of actual testing and evaluation work can be saved, thereby improving the efficiency of performance evaluation.

[0109] For example, after inputting the evaluation index sets of each interaction system in several interaction systems in at least one evaluation dimension and the number of configuration information items affecting this evaluation dimension into the evaluation model to be trained, the evaluation model is used to calculate, for each evaluation dimension, the ratio of the first product corresponding to each evaluation index in the evaluation index set of the evaluation dimension to the number of configuration information items corresponding to this evaluation dimension. The first product corresponding to each evaluation index in the evaluation index set of the evaluation dimension is the product of each evaluation index in the evaluation index set of the evaluation dimension and its corresponding influence coefficient, and the sum of the ratios corresponding to all evaluation indexes in the evaluation index set of the evaluation dimension is determined as the target evaluation index corresponding to this evaluation dimension; then, the weighted sum of the target evaluation indexes corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension is processed to obtain the evaluation result of the interaction system.

[0110] In other words, the evaluation model can be expressed by the following formula:

[0111]

[0112]

[0113] Among them, score represents the evaluation result output by the evaluation model, n represents the number of evaluation dimensions, EP i represents the target evaluation index corresponding to the i-th evaluation dimension, IC i represents the weight coefficient corresponding to the i-th evaluation dimension, 1 ≤ i ≤ n;

[0114] eP j represents the j-th evaluation index in the evaluation index set of the i-th evaluation dimension, TN i represents the number of configuration information items affecting the i-th evaluation dimension, CW j represents the influence coefficient corresponding to the configuration information for determining the j-th evaluation index, 1 ≤ j ≤ TN i .

[0115] Taking the cost dimension as an example, assume that the software configuration type and hardware configuration type of the cloud server affect the cost dimension. The configuration information set corresponding to the software configuration type of the cloud server contains 1 item of configuration information, its cost index in the cost dimension is eP1, and the corresponding influence coefficient is CW1; the configuration information set corresponding to the hardware configuration type of the cloud server contains 3 items of configuration information, its cost indexes in the cost dimension are eP2, eP3, and eP4, and the corresponding influence coefficients are CW2, CW3, and CW4 respectively. The number of configuration information items TN affecting the cost dimension is 4, then the target evaluation index corresponding to the cost evaluation dimension The weight coefficient corresponding to the cost dimension is IC, then the evaluation result score output by the evaluation model is score = EP * IC.

[0116] After the evaluation model is trained, the trained evaluation model can be deployed in an electronic device for executing an evaluation method for an interaction system.

[0117] Please refer to Figure 5 , this embodiment of the specification provides an evaluation method for an interaction system. The interaction system includes a cloud server and a client deployed in a terminal device. The cloud server is used to execute a computing task in response to a control instruction sent by the client and return a computing result. The method includes:

[0118] In S201, obtain at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type.

[0119] In S202, determine the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension according to at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type.

[0120] In S203, input the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result.

[0121] In this embodiment, the evaluation model performs evaluation processing on the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result. The evaluation result can reflect the performance of the interaction system in each evaluation dimension. Through the above process, the performance of the interaction system can be accurately and comprehensively evaluated, so as to objectively and accurately provide guidance for the optimization direction of the interaction system. And through the evaluation model, the performance of the interaction system under different configuration types can be quickly evaluated, avoiding the tediousness and time consumption of manual testing, saving time and resources, and improving the evaluation efficiency.

[0122] Among them, the evaluation result can be represented by a score or by a text description. This embodiment does not limit this.

[0123] In some embodiments, the configuration types of at least one piece of configuration information include at least one of the following: client configuration type, communication link configuration type between the client and the cloud server, software configuration type of the cloud server, and hardware configuration type of the cloud server.

[0124] In some embodiments, the evaluation dimensions include at least one of the following: cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension.

[0125] In some embodiments, at least one piece of configuration information is divided into different configuration information sets according to different configuration types; the evaluation dimensions affected by different configuration types are different. According to at least one piece of configuration information of the interactive system to be evaluated in at least one configuration type, the evaluation indicators of the interactive system to be evaluated in at least one evaluation dimension are determined, including: for each configuration type, according to each piece of configuration information in the configuration information set corresponding to the configuration type, determine the evaluation indicators corresponding to each piece of configuration information in at least one evaluation dimension affected by the configuration type. In this embodiment, dividing the obtained configuration information into corresponding configuration information sets according to different configuration types can improve the operation efficiency of the evaluation indicators. Since the evaluation dimensions affected by each configuration type are different, only the evaluation indicators of the corresponding evaluation dimensions need to be determined for the configuration information set of each configuration type, and the performance of each configuration type in the affected evaluation dimensions can be evaluated more accurately.

[0126] In some embodiments, for each configuration type, according to each piece of configuration information in the configuration information set corresponding to the configuration type, determine the evaluation indicators corresponding to each piece of configuration information in at least one evaluation dimension affected by the configuration type, including: for each configuration type, according to each piece of configuration information in the configuration information set corresponding to the configuration type and the pre-stored mapping relationship between different pieces of configuration information corresponding to the configuration type and the evaluation indicators of each evaluation dimension affected by the configuration type, determine the evaluation indicators corresponding to each piece of configuration information in at least one evaluation dimension affected by the configuration type. In this embodiment, by pre-storing the mapping relationship, the electronic device can quickly access the mapping relationship between the required configuration information and the evaluation indicators during the process of determining the evaluation indicators, and perform matching and calculation, thereby greatly improving the calculation speed.

[0127] In some embodiments, the evaluation metrics of the interactive system to be evaluated in at least one evaluation dimension are input into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the interactive system to be evaluated according to the input evaluation metrics and outputs an evaluation result, including: classifying the configuration information items in all the configuration information according to the evaluation dimensions to which the evaluation metrics belong, to obtain the evaluation metric sets of the interactive system to be evaluated in each evaluation dimension; determining the number of configuration information items affecting the evaluation dimension based on the number of evaluation metrics in the evaluation metric sets of each evaluation dimension; inputting the evaluation metric sets of the interactive system to be evaluated in each evaluation dimension and the number of configuration information items corresponding to each evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the interactive system to be evaluated according to the input data and outputs an evaluation result. In this embodiment, the evaluation metrics corresponding to the configuration information items are classified according to their respective evaluation dimensions, and a fine-grained evaluation metric set can be obtained, so as to more comprehensively understand the performance of the interactive system in each evaluation dimension; and by determining the number of configuration information items affecting each evaluation dimension, it helps to understand the influence degree of different configuration information on the performance of the interactive system, which is helpful for optimizing and adjusting the configuration to achieve better performance.

[0128] In some embodiments, the parameters in the evaluation model include influence coefficients and weight coefficients; among them, for at least one evaluation dimension affected by any one configuration type, each configuration information item in the configuration information set corresponding to the configuration type has an influence coefficient corresponding to it in at least one evaluation dimension; each evaluation dimension corresponds to a weight coefficient.

[0129] The evaluation model performs evaluation processing on the interactive system to be evaluated according to the input data and outputs an evaluation result, including: for each evaluation dimension, determining the target evaluation metric corresponding to the evaluation dimension according to the evaluation metric set of the evaluation dimension, the number of configuration information items corresponding to the evaluation dimension, and the influence coefficients corresponding to the configuration information items affecting the evaluation dimension; determining the evaluation result of the interactive system to be evaluated according to the target evaluation metrics corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension. In this embodiment, by comprehensively considering the weights and metrics of multiple evaluation dimensions, as well as the influence degree of each configuration information on different evaluation dimensions, a more comprehensive evaluation result of the interactive system performance can be obtained; and by using the evaluation model to predict the evaluation result, a large amount of actual testing and evaluation work can be saved, thereby improving the efficiency of performance evaluation.

[0130] In some embodiments, for each evaluation dimension, according to the set of evaluation indicators of the evaluation dimension, the number of configuration information items corresponding to the evaluation dimension, and the influence coefficients corresponding to the configuration information items affecting the evaluation dimension, the target evaluation indicator corresponding to the evaluation dimension is determined, including: for each evaluation dimension, calculating the ratio of the first product corresponding to each evaluation indicator in the set of evaluation indicators of the evaluation dimension to the number of configuration information items corresponding to the evaluation dimension, where the first product corresponding to each evaluation indicator in the set of evaluation indicators of the evaluation dimension is the product of each evaluation indicator in the set of evaluation indicators of the evaluation dimension and its corresponding influence coefficient, and determining the sum of the ratios corresponding to all the evaluation indicators in the set of evaluation indicators of the evaluation dimension as the target evaluation indicator corresponding to the evaluation dimension.

[0131] According to the target evaluation indicators corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension, the evaluation result of the interactive system to be evaluated is determined, including: performing a weighted sum processing on the target evaluation indicators corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension to obtain the evaluation result of the interactive system to be evaluated.

[0132] In some embodiments, the evaluation model is expressed by the following formula:

[0133]

[0134]

[0135] where score represents the evaluation result output by the evaluation model, n represents the number of evaluation dimensions, EP i represents the target evaluation indicator corresponding to the i-th evaluation dimension, IC i represents the weight coefficient corresponding to the i-th evaluation dimension, 1 ≤ i ≤ n;

[0136] eP j represents the j-th evaluation indicator in the set of evaluation indicators of the i-th evaluation dimension, TN i represents the number of configuration information items affecting the i-th evaluation dimension, CW j represents the influence coefficient corresponding to the configuration information used to determine the j-th evaluation indicator, 1 ≤ j ≤ TN i .

[0137] In some embodiments, the interactive system to be evaluated is obtained by changing at least one piece of configuration information in at least one configuration type of the original interactive system. The electronic device can also determine the evaluation result of the original interactive system based on the evaluation model. Exemplarily, at least one piece of configuration information of the original interactive system in at least one configuration type can be obtained, and then, based on at least one piece of configuration information of the original interactive system in at least one configuration type, the evaluation indexes of the original interactive system in at least one evaluation dimension can be determined; the evaluation indexes of the original interactive system in at least one evaluation dimension are input into a pre-constructed evaluation model, so that the evaluation model can perform an evaluation process on the original interactive system according to the input evaluation indexes and output an evaluation result.

[0138] Furthermore, the electronic device can output optimization guidance information for the original interactive system according to the difference between the evaluation result of the original interactive system and the evaluation result of the interactive system to be evaluated. By comparing the evaluation results of the original interactive system and the interactive system to be evaluated in this embodiment, it helps developers understand which changes in configuration information have a positive or negative impact on system performance, and how these configuration information should be adjusted to achieve performance optimization. Through the output of the evaluation model, developers can make optimization decisions based on objective data rather than just subjective judgment, which helps to avoid blind adjustments and optimizations, and improves the scientificity and reliability of the decision-making. By outputting optimization guidance information, developers can targetedly optimize and adjust the original interactive system, thereby improving the performance and stability of the system. This data-driven optimization process can ensure the correctness of the optimization direction, avoid unnecessary trial and error, and improve the efficiency and effect of the optimization.

[0139] Exemplarily, the optimization guidance information includes but is not limited to configuration adjustment suggestions, resource allocation optimization suggestions, fault tolerance and reliability improvement suggestions, performance tuning strategy suggestions, etc.

[0140] If the evaluation score of the interactive system to be evaluated in a certain configuration type is lower than the evaluation score of the original interactive system, then the optimization guidance information can include configuration adjustment suggestions to suggest adjusting the relevant parameters of this configuration type.

[0141] If it is determined based on the above differences that the interactive system to be evaluated performs poorly in terms of resource utilization, resource allocation optimization suggestions can be provided. For example, if the memory utilization score is low, it is recommended to increase the system memory capacity or optimize the memory management algorithm.

[0142] If it is determined based on the above differences that the interactive system to be evaluated has problems in terms of fault tolerance and reliability, fault tolerance and reliability improvement suggestions can be provided. For example, if the disaster tolerance ability score of the system is low, it is recommended to increase redundant backups, implement a failover mechanism, or improve the data backup strategy.

[0143] If it is determined based on the above differences that the interaction system to be evaluated performs poorly in a certain performance metric, performance tuning strategy suggestions can be provided. For example, if the response time score of the system is poor, it is recommended to optimize the code logic, increase the server processing capacity, or improve the network transmission efficiency, etc. The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also belongs to the scope disclosed in this specification.

[0144] Figure 6 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Please refer to Figure 6 , at the hardware level, the electronic device includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile memory 610. Of course, it may also include other hardware required for other services. One or more embodiments of this specification can be implemented in software. For example, the processor 602 reads the corresponding computer program from the non-volatile memory 610 into the memory 608 and then runs it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0145] Please refer to Figure 7 , the evaluation device for the interaction system can be applied to an electronic device as shown in Figure 6 to implement the technical solutions of this specification. Among them, the evaluation device for the interaction system can include:

[0146] A configuration information acquisition module 301, configured to acquire at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type.

[0147] An evaluation metric determination module 302, configured to determine the evaluation metrics of the interaction system to be evaluated in at least one evaluation dimension according to at least one piece of configuration information of the interaction system to be evaluated in at least one configuration type;

[0148] An evaluation module 303, configured to input the evaluation metrics of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs evaluation processing on the interaction system to be evaluated according to the input evaluation metrics and outputs an evaluation result.

[0149] The implementation processes of the functions and roles of each module in the above device are specifically described in the implementation processes of the corresponding steps in the above method, and will not be repeated here.

[0150] Please refer toFigure 8 , the construction device of the evaluation model can be applied to an electronic device as shown in Figure 6 to implement the technical solutions of this specification. Among them, the construction device of the evaluation model may include:

[0151] A configuration information acquisition module 401, configured to acquire at least one piece of configuration information of several interactive systems in at least one configuration type and evaluation result tags corresponding to each interactive system.

[0152] An evaluation index determination module 402, configured to determine evaluation indexes of each of the interactive systems in at least one evaluation dimension according to at least one piece of configuration information of each of the interactive systems in at least one configuration type.

[0153] A model training module 403, configured to perform supervised training on the evaluation indexes of each of the several interactive systems in at least one evaluation dimension and the evaluation result tags as training samples to obtain an evaluation model; wherein, the evaluation model is used to perform an evaluation process on an interactive system according to the evaluation indexes of the interactive system input in at least one evaluation dimension to output an evaluation result.

[0154] The implementation processes of the functions and roles of each module in the above device are specifically described in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0155] In some embodiments, this specification embodiment also provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor runs the executable instructions to implement the method described in any one of the above.

[0156] In some embodiments, this specification embodiment also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0157] In some embodiments, this specification embodiment also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0159] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0160] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0161] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0162] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0163] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, the element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, commodity or device comprising the element.

[0164] The above description has been made of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0165] The terms used in one or more embodiments of this specification are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0166] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "upon" or "in response to determining".

[0167] The above is only the preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included within the scope of protection of one or more embodiments of this specification.

Claims

1. An evaluation method for an interaction system, the interaction system including a cloud server and a client deployed in a terminal device; the method includes: Obtaining at least one configuration information of the interaction system to be evaluated in at least one configuration type; Determining evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension according to the at least one configuration information of the interaction system to be evaluated in at least one configuration type; Inputting the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs an evaluation process on the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result.

2. The method according to claim 1, wherein the at least one configuration information is divided into different configuration information sets according to different configuration types; the evaluation dimensions affected by different configuration types are not completely the same; The determining the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension according to the at least one configuration information of the interaction system to be evaluated in at least one configuration type includes: For each configuration type, determining the evaluation indicators respectively corresponding to the at least one evaluation dimension affected by this configuration type according to each configuration information in the configuration information set corresponding to this configuration type.

3. The method according to claim 2, wherein the determining the evaluation indicators respectively corresponding to the at least one evaluation dimension affected by this configuration type according to each configuration information in the configuration information set corresponding to this configuration type for each configuration type includes: For each configuration type, determining the evaluation indicators respectively corresponding to the at least one evaluation dimension affected by this configuration type according to each configuration information in the configuration information set corresponding to this configuration type and the mapping relationship between different configuration information corresponding to this configuration type stored in advance and the evaluation indicators of each evaluation dimension affected by this configuration type.

4. The method according to any one of claims 1 to 3, wherein the inputting the evaluation indicators of the interaction system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model performs an evaluation process on the interaction system to be evaluated according to the input evaluation indicators and outputs an evaluation result includes: Classifying the evaluation indicators respectively corresponding to the at least one evaluation dimension of each configuration information in all configuration information according to the evaluation dimension to which each evaluation indicator belongs, to obtain an evaluation indicator set of the interaction system to be evaluated in each evaluation dimension; Determining the number of configuration information items affecting this evaluation dimension based on the number of evaluation indicators in the evaluation indicator set of each evaluation dimension; Inputting the evaluation indicator set of the interaction system to be evaluated in each evaluation dimension and the number of configuration information items corresponding to each evaluation dimension into the pre-constructed evaluation model, so that the evaluation model performs an evaluation process on the interaction system to be evaluated according to the input data and outputs the evaluation result.

5. The method according to claim 4, wherein the parameters in the evaluation model include an influence coefficient and a weight coefficient; Among them, For at least one evaluation dimension affected by any one configuration type, each configuration information in the configuration information set corresponding to the configuration type has an influence coefficient corresponding thereto in the at least one evaluation dimension; Each evaluation dimension corresponds to a weight coefficient; The evaluation by the evaluation model of the interactive system to be evaluated based on the input data and outputting an evaluation result includes: For each evaluation dimension, determining a target evaluation index corresponding to the evaluation dimension according to the evaluation index set of the evaluation dimension, the number of configuration information items corresponding to the evaluation dimension, and the influence coefficients corresponding to the configuration information items affecting the evaluation dimension; Determining the evaluation result of the interactive system to be evaluated according to the target evaluation indexes corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension.

6. The method according to claim 5, wherein the determining, for each evaluation dimension, a target evaluation index corresponding to the evaluation dimension according to the evaluation index set of the evaluation dimension, the number of configuration information items corresponding to the evaluation dimension, and the influence coefficients corresponding to the configuration information items affecting the evaluation dimension includes: For each evaluation dimension, calculating the ratio of the first product corresponding to each evaluation index in the evaluation index set of the evaluation dimension to the number of configuration information items corresponding to the evaluation dimension, wherein the first product corresponding to each evaluation index in the evaluation index set of the evaluation dimension is the product of each evaluation index in the evaluation index set of the evaluation dimension and its corresponding influence coefficient, and determining the sum of the ratios respectively corresponding to all the evaluation indexes in the evaluation index set of the evaluation dimension as the target evaluation index corresponding to the evaluation dimension; The determining the evaluation result of the interactive system to be evaluated according to the target evaluation indexes corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension includes: Performing a weighted summation process on the target evaluation indexes corresponding to each evaluation dimension and the weight coefficients corresponding to each evaluation dimension to obtain the evaluation result of the interactive system to be evaluated.

7. The method according to claim 1, wherein the interactive system to be evaluated is obtained by changing at least one piece of configuration information in at least one configuration type in the original interactive system; The method further includes: Determining the evaluation result of the original interactive system based on the evaluation model; Outputting optimization guidance information for the original interactive system according to the difference between the evaluation result of the original interactive system and the evaluation result of the interactive system to be evaluated.

8. The method according to claim 1, wherein the configuration type of the at least one piece of configuration information includes at least one of the following: client configuration type, communication link configuration type between the client and the cloud server, software configuration type of the cloud server, and hardware configuration type of the cloud server; and / or, The evaluation dimension includes at least one of the following: cost dimension, performance dimension, function dimension, compatibility dimension, security dimension, latency dimension, and ease of operation dimension.

9. A method for constructing an evaluation model, where the evaluation model is used to evaluate an interactive system; The interactive system includes a cloud server and a client deployed in a terminal device; the method includes: Obtaining at least one configuration information of a plurality of interactive systems in at least one configuration type and evaluation result labels corresponding to each interactive system; Determining evaluation metrics of each of the interactive systems in at least one evaluation dimension according to at least one configuration information of each of the interactive systems in at least one configuration type; Using the evaluation metrics of each of the interactive systems in at least one evaluation dimension and the evaluation result labels among the plurality of interactive systems as training samples for supervised training to obtain an evaluation model; Wherein, the evaluation model is used to evaluate an interactive system according to the input evaluation metrics of the interactive system in at least one evaluation dimension to output an evaluation result.

10. The evaluation result label corresponding to each interaction system is determined based on at least one of the market selection information of the interaction system and the user's usage evaluation in the method according to claim 9; wherein, The market selection information includes the usage times and / or purchase times of the interactive system; Wherein, the values of the evaluation result labels corresponding to each interactive system are respectively positively correlated with the usage times, purchase times, and the number of positive evaluations of the users of the interactive system.

11. An evaluation device for an interactive system, the interactive system includes a cloud server and a client deployed in a terminal device; the device includes: A configuration information acquisition module, configured to acquire at least one configuration information of an interactive system to be evaluated in at least one configuration type; An evaluation metric determination module, configured to determine evaluation metrics of the interactive system to be evaluated in at least one evaluation dimension according to at least one configuration information of the interactive system to be evaluated in at least one configuration type; An evaluation module, configured to input the evaluation metrics of the interactive system to be evaluated in at least one evaluation dimension into a pre-constructed evaluation model, so that the evaluation model evaluates the interactive system to be evaluated according to the input evaluation metrics and outputs an evaluation result; wherein, the evaluation model is obtained through supervised training based on a training data set, the training data set includes a plurality of training samples, the training samples include evaluation metrics of an interactive system in at least one evaluation dimension and evaluation result labels, and the configuration information of the interactive systems pointed to by different training samples is different.

12. An electronic device, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor realizes the method according to any one of claims 1 to 10 by running the executable instructions.

13. A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.

14. A computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.

Images