User experience test method and device, equipment and storage medium

Abstract

The invention relates to a user experience test method and device, equipment and a storage medium, and the method comprises the steps: obtaining behavior data and physiological data recorded by a user in a cloud product experience process, the physiological data comprising eye movement data and / or emotion data; performing feature extraction on the behavior data and the physiological data based on preset evaluation dimensions, and determining a dimension score corresponding to each evaluation dimension based on a feature extraction result; and determining a comprehensive test score based on the dimension score corresponding to each evaluation dimension and the adaptive optimization weight. According to the method, the behavior data and the physiological data of the user in the cloud product experience process are obtained, feature extraction is performed, the dimension score of each evaluation dimension is obtained, the comprehensive test score is finally obtained by using the dimension scores and the adaptive optimization weight, multi-evaluation-dimension, multi-data-source and quantifiable analysis can be performed on the user experience, and the user experience is improved. And the accuracy of the evaluation result of the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of product testing, and particularly relates to a user experience testing method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of cloud computing technology, cloud products have been widely applied in storage, computing, network, security and other fields. The functions covered by cloud products are continuously expanding, and the competition between products is becoming increasingly fierce. In order to improve the service quality of cloud products, user experience testing has become an indispensable link. The traditional user experience testing method mainly relies on manual observation or questionnaire survey, but the above methods are easily affected by subjective bias, and it is difficult to quantitatively analyze. Meanwhile, the automatic testing tool focuses on the collection of click stream data, and can only record user operations, so the data source is single, which leads to the evaluation result of user experience not being accurate enough. Therefore, how to optimize the user experience testing method and improve the accuracy of the evaluation result has become a technical problem to be solved at present. SUMMARY

[0003] In order to solve the above technical problems, the present disclosure provides a user experience testing method, device, equipment and storage medium.

[0004] The first aspect of the embodiment of the present disclosure provides a user experience testing method, which comprises:

[0005] obtaining behavior data and physiological data recorded by a user in a cloud product experience process, the physiological data comprising eye movement data and / or emotional data;

[0006] performing feature extraction on the behavior data and the physiological data based on a preset evaluation dimension, and determining a dimension score corresponding to each evaluation dimension based on a feature extraction result;

[0007] determining a comprehensive test score based on the dimension score corresponding to each evaluation dimension and an adaptive optimization weight.

[0008] The second aspect of the embodiment of the present disclosure provides a user experience testing device, which comprises:

[0009] an obtaining module, configured to obtain behavior data and physiological data recorded by a user in a cloud product experience process, the physiological data comprising eye movement data and / or emotional data;

[0010] an extracting module, configured to perform feature extraction on the behavior data and the physiological data based on a preset evaluation dimension, and determine a dimension score corresponding to each evaluation dimension based on a feature extraction result;

[0011] a scoring module, configured to determine a comprehensive test score based on the dimension score corresponding to each evaluation dimension and an adaptive optimization weight.

[0012] A third aspect of the embodiments of the present disclosure provides a computer device, comprising a memory and a processor, and a computer program, wherein the memory stores the computer program, and when the computer program is executed by the processor, the user experience testing method of the first aspect is implemented.

[0013] A fourth aspect of the embodiments of the present disclosure provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the user experience testing method of the first aspect is implemented.

[0014] Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have the following advantages:

[0015] In the user experience testing method, device, equipment and storage medium provided by the embodiments of the present disclosure, the behavior data and physiological data recorded by the user in the cloud product experience process are obtained, the physiological data includes eye movement data and / or emotional data, the feature extraction is performed on the behavior data and physiological data based on the preset evaluation dimensions, the dimension score corresponding to each evaluation dimension is determined based on the feature extraction result, and the comprehensive test score is determined based on the dimension score corresponding to each evaluation dimension and the adaptive optimization weight. The user experience can be analyzed and evaluated quantitatively in multiple evaluation dimensions and multiple data sources, so that the evaluation result of the user experience is more comprehensive, and the accuracy of the evaluation result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings incorporated in the specification and forming a part thereof illustrate embodiments consistent with the present disclosure and together with the description serve to explain the principles of the present disclosure.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0018] Figure 1 is a flowchart of a user experience testing method provided by the embodiments of the present disclosure;

[0019] Figure 2 is a flowchart of a method for determining adaptive optimization weight provided by the embodiments of the present disclosure;

[0020] Figure 3 is a flowchart of another method for determining adaptive optimization weight provided by the embodiments of the present disclosure;

[0021] Figure 4 is a flowchart of a method for determining optimization measures provided by the embodiments of the present disclosure;

[0022] Figure 5 is a flowchart of a method for generating cloud product optimization suggestions provided by an embodiment of the present disclosure.

[0023] Figure 6 is a structural schematic diagram of a user experience testing device provided by an embodiment of the present disclosure.

[0024] Figure 7 is a structural schematic diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0026] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, not all the embodiments.

[0027] It should be understood that each step recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] Figure 1 is a flowchart of a user experience testing method provided by an embodiment of the present disclosure, which can be executed by a user experience testing device, which can be implemented in software and / or hardware, and can be configured in an electronic device, such as a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc. As shown in Figure 1 the user experience testing method provided by the embodiment includes the following steps:

[0029] S101, obtaining behavior data and physiological data recorded by a user in a cloud product experience process, wherein the physiological data includes eye movement data and / or emotional data.

[0030] The behavior data in the embodiments of the present disclosure can be understood as behavior data related to operations performed by the user on the cloud product. For example, the behavior data can include click coordinate page dwell time, application programming interface (API) call time consumption, resource utilization, throughput, etc., which are not limited herein.

[0031] The physiological data in the embodiments of the present disclosure can be understood as physiological information capable of reflecting user experience. The physiological data can include at least one of eye movement data and emotional data, wherein the eye movement data can include gaze point coordinates, pupil diameter change amount, etc., the emotional data can include facial emotional data, voice emotional data, etc., and the physiological data can further include other types of data such as heartbeat data, breathing data, facial color data, etc., which are not limited herein.

[0032] In the embodiments of the present disclosure, the user experience testing device can acquire the behavior data and physiological data recorded in the process of the user experiencing the cloud product. Specifically, the behavior data can be acquired through a software development kit (SDK) embedded in the front end in advance. For the eye movement data, the user experience testing device can calculate the gaze point coordinates through the corneal reflection light spot combined with a pre-established user personal eye movement model through an infrared camera, or can estimate the gaze direction through a normal camera + OpenCV algorithm to generate the gaze point coordinates, and then determine the saccade speed according to the gaze point coordinates at different times, and can further determine the pupil diameter change amount according to the pupil diameter recognized from the collected eye image. For the emotional data, the user experience testing device can analyze the facial action through a facial recognition model (such as a residual network model) to determine the facial emotion score, can extract acoustic features (such as tone, speed) through a voice emotion analysis model and map them to the emotion dimension to obtain the voice emotion score, and can further obtain the eye emotion score according to the pupil diameter change amount.

[0033] In an exemplary embodiment of the present disclosure, after the user experience testing device acquires the gaze point coordinates, the user experience testing device can match the gaze point coordinates with the page element coordinate system to determine the page element corresponding to at least part of the gaze point coordinates.

[0034] In S102, the behavior data and the physiological data are subjected to feature extraction based on the preset evaluation dimensions, and dimension scores corresponding to each evaluation dimension are determined based on the feature extraction results.

[0035] The evaluation dimensions in the embodiments of the present disclosure can include efficiency, ease of use, emotion, and cognitive load evaluation dimensions.

[0036] In the embodiments of the present disclosure, after the user experience testing device acquires the behavior data and the physiological data, the user experience testing device can perform feature extraction on the behavior data and the physiological data according to the feature information required by the preset evaluation dimensions, and then determine the dimension scores corresponding to each evaluation dimension based on the feature extraction results corresponding to each evaluation dimension.

[0037] In an exemplary embodiment of the present disclosure, for the efficiency evaluation dimension, the user experience test device can perform feature extraction on the task completion time, resource utilization, and throughput in the behavior data to obtain the feature extraction result corresponding to the efficiency evaluation dimension, and determine the dimension score corresponding to the efficiency evaluation dimension based on the score calculation manner of the efficiency evaluation dimension; for the ease of use evaluation dimension, the user experience test device can perform feature extraction on the number of user operation steps and error rate in the behavior data, and perform natural language processing on the text corresponding to the user voice in the emotional data to obtain the feature extraction result corresponding to the ease of use evaluation dimension, and determine the dimension score corresponding to the ease of use evaluation dimension based on the score calculation manner of the ease of use evaluation dimension; for the emotional evaluation dimension, the user experience test device can perform feature extraction on the facial emotional data, voice emotional data, and pupil diameter change amount in the eye movement data in the emotional data to obtain the feature extraction result corresponding to the emotional evaluation dimension, and determine the dimension score corresponding to the emotional evaluation dimension based on the score calculation manner of the emotional evaluation dimension; for the cognitive load evaluation dimension, the user experience test device can perform feature extraction on the pupil diameter change amount in the eye movement data to obtain the feature extraction result corresponding to the cognitive load evaluation dimension, and determine the dimension score corresponding to the cognitive load evaluation dimension based on the score calculation manner of the cognitive load evaluation dimension.

[0038] S103, determining a comprehensive test score based on the dimension score corresponding to each evaluation dimension and the adaptive optimization weight.

[0039] In the present disclosure, after determining the dimension score corresponding to each evaluation dimension, the user experience test device can perform weighted summation processing based on the dimension score corresponding to each evaluation dimension and the adaptive optimization weight corresponding to each evaluation dimension, and determine the summation result as the comprehensive test score.

[0040] In an exemplary embodiment of the present disclosure, the user experience test device can input the behavior data and physiological data into the pre-trained weight optimization model, and obtain the adaptive optimization weight output by the weight optimization model after optimizing the basic weight configuration scheme based on the recorded behavior data and physiological data this time, and then determine the comprehensive test score based on the dimension score corresponding to each evaluation dimension and the adaptive optimization weight.

[0041] The embodiment of the present disclosure can acquire the behavior data and physiological data recorded by the user in the cloud product experience process, the physiological data including eye movement data and / or emotional data, perform feature extraction on the behavior data and physiological data based on the preset evaluation dimensions, determine the dimension scores corresponding to each evaluation dimension based on the feature extraction result, and determine the comprehensive test score based on the dimension scores corresponding to each evaluation dimension and the adaptive optimization weight. The user experience can be analyzed and evaluated in multiple evaluation dimensions, multiple data sources and quantifiable manner, so that the evaluation result of the user experience is more comprehensive, and the accuracy of the evaluation result is improved.

[0042] Figure 2 is a flowchart of a method for determining an adaptive optimization weight provided by the embodiment of the present disclosure, as shown in the above embodiment, the adaptive optimization weight can be determined by the following method. Figure 2

[0043] S201, determining a task type of a task completed by the user in the cloud product experience process based on the behavior data and the physiological data, the task type including a simple task and a complex task.

[0044] In the embodiment of the present disclosure, the user experience test device can determine whether the task type of the task completed by the user in the cloud product experience process is a simple task or a complex task according to the behavior data and the physiological data after acquiring the behavior data and the physiological data.

[0045] In an exemplary embodiment of the embodiment of the present disclosure, the behavior data can further include functions finally realized by the user, such as creating a virtual machine, deleting a virtual machine, service deployment, etc. The user experience test device can determine the task type of the task completed by the user in the cloud product experience process according to the functions finally realized by the user. For example, creating a virtual machine and deleting a virtual machine belong to a simple task, and service deployment belongs to a complex task.

[0046] In another exemplary embodiment of the embodiment of the present disclosure, the behavior data can further include a page stay duration, and the physiological data includes a saccade speed determined according to the fixation point coordinates in the eye movement data. The user experience test device can compare the page stay duration and the saccade speed with the corresponding stay duration threshold and saccade speed threshold respectively. If the page stay duration is greater than the stay duration threshold and the saccade speed is less than the saccade speed threshold, it is determined that the task type is a complex task, otherwise it is determined that the task type is a simple task.

[0047] S202, determining the adaptive optimization weight corresponding to each evaluation dimension based on the task type and the preset weight configuration scheme.

[0048] ​In the embodiments of the present disclosure, the user experience test device can determine the adaptive optimization weight of each evaluation dimension according to the task type and the preset weight configuration scheme after determining the task type.

[0049] The embodiments of the present disclosure can determine the task type of the task completed by the user in the cloud product experience process based on the behavior data and the physiological data, the task type including a simple task and a complex task, determine the adaptive optimization weight corresponding to each evaluation dimension based on the task type and the preset weight configuration scheme, set different scoring weights for each evaluation dimension according to different task types, thereby more accurately identifying key factors affecting user experience in different task scenarios, avoiding evaluation deviation caused by uniform weights, and further improving the accuracy of the evaluation result.

[0050] Figure 3 is a flowchart of another method for determining adaptive optimization weight provided by the embodiments of the present disclosure, as shown in Figure 3 On the basis of the above-mentioned embodiments, the adaptive optimization weight can be determined by the following method, wherein the evaluation dimension includes an emotional evaluation dimension and an efficiency evaluation dimension.

[0051] S301, determining the initial weight corresponding to each evaluation dimension under the task type based on the task type and the weight configuration scheme.

[0052] In the embodiments of the present disclosure, the user experience test device can determine the adaptive optimization weight of each evaluation dimension according to the task type and the preset weight configuration scheme after determining the task type.

[0053] S302, performing correlation analysis on the behavior data and the emotional data, and if the correlation coefficient of the behavior data and the emotional data is greater than a preset threshold, adjusting the initial weight of the emotional evaluation dimension corresponding to each evaluation dimension.

[0054] In the embodiments of the present disclosure, the user experience test device can further perform correlation analysis on the behavior data and the emotional data after determining the initial weight corresponding to each evaluation dimension under the current task type, determine the correlation coefficient of the behavior data and the emotional data, and compare the correlation coefficient with the preset threshold, if the correlation coefficient is greater than the preset threshold, it is determined that the behavior data and the emotional data have strong correlation, and the emotional evaluation dimension is more critical, so the initial weight of the emotional evaluation dimension corresponding to each evaluation dimension is adjusted, and specifically, the initial weight of the emotional evaluation dimension in each evaluation dimension can be increased based on the correlation coefficient.

[0055] In an exemplary embodiment of the present disclosure, the user experience testing apparatus can use cross-correlation analysis to calculate the lag relationship between the emotional data and the behavior data, and find the maximum correlation lag point of the two, and determine the correlation coefficient of the behavior data and the emotional data based on the maximum correlation lag point when performing correlation analysis on the behavior data and the emotional data.

[0056] S303, determine the emotion type of the emotional data, if the emotion type is negative emotion, extract the operation efficiency data of the user from the behavior data, and determine the influence degree of the emotional data on the operation efficiency data.

[0057] In the present disclosure, the user experience testing apparatus can further identify the emotion type of the emotional data table, determine whether the emotion type of the emotional data is positive emotion or negative emotion, if it is negative emotion, extract the operation efficiency data of the user from the behavior data, and analyze and determine the influence degree of the emotional data on the operation efficiency data.

[0058] In an exemplary embodiment of the present disclosure, the user experience testing apparatus can first determine the transition moment when the emotion type of the user changes to negative emotion, determine the operation efficiency data before and after the emotion type of the user changes to negative emotion according to the page stay duration and the operation number before and after the transition moment, and then compare and analyze the operation efficiency data under different emotion types to determine the influence degree of the emotional data on the operation efficiency data.

[0059] In an exemplary embodiment of the present disclosure, the user experience testing apparatus can establish a linear model of the influence degree of the emotional data on the operation efficiency based on the pre-acquired emotional data and operation efficiency data of the historical users with negative emotion type, and determine the influence degree of the emotional data of the user based on the linear model when determining that the emotion type of the emotional data of the user is negative emotion.

[0060] S304, based on the influence degree of the emotional data on the operation efficiency data, adjust the initial weight of the efficiency evaluation dimension corresponding to each evaluation dimension.

[0061] In the present disclosure, the user experience testing apparatus can determine the influence degree of the emotional data on the operation efficiency data, adjust the initial weight of the efficiency evaluation dimension corresponding to each evaluation dimension based on the influence degree, specifically, determine the weight adjustment amount based on the influence degree, and then reduce the initial weight of the efficiency evaluation dimension in each evaluation dimension based on the weight adjustment amount.

[0062] S305, determine the initial weight of each evaluation dimension after adjustment as the adaptive optimization weight corresponding to each evaluation dimension.

[0063] In the embodiments of the present disclosure, the user experience test device can determine the adjusted initial weight of each evaluation dimension as the adaptive optimization weight of each evaluation dimension.

[0064] The embodiments of the present disclosure can determine the initial weight of each evaluation dimension under the task type based on the task type and the weight configuration scheme, analyze the correlation between the behavior data and the emotional data, adjust the initial weight of the emotional evaluation dimension in each evaluation dimension if the correlation coefficient of the behavior data and the emotional data is greater than a preset threshold, determine the emotion type of the emotional data, extract the operation efficiency data of the user from the behavior data if the emotion type is a negative emotion, determine the influence degree of the emotional data on the operation efficiency data, adjust the initial weight of the efficiency evaluation dimension in each evaluation dimension based on the influence degree of the emotional data on the operation efficiency data, determine the adjusted initial weight of each evaluation dimension as the adaptive optimization weight of each evaluation dimension, capture the potential correlation between the behavior data and the emotional data, and the influence of the negative emotion type on the operation efficiency, timely adjust the weight proportion of the emotional evaluation dimension and the efficiency evaluation dimension in the comprehensive test score, realize the dynamic optimization of the comprehensive test score, make it more suitable for the real situation, and further improve the accuracy of the evaluation result.

[0065] Figure 4 is a flowchart of a method for determining an optimization measure provided by the embodiments of the present disclosure. As shown in Figure 4 on the basis of the above-mentioned embodiments, the optimization measure can be determined by the following method.

[0066] S401, compare the comprehensive test score with a preset score.

[0067] In the embodiments of the present disclosure, the user experience test device can compare the comprehensive test score with a preset score after determining the comprehensive test score of the user in the cloud product experience process, and judge whether the user experiences well in the cloud product experience process.

[0068] S402, if the comprehensive test score is less than the preset score, determine the low score reason of the comprehensive test score based on the pre-trained decision tree model and the behavior data and the physiological data, and the decision tree model is trained based on the pre-collected historical behavior data, historical physiological data and corresponding low score reason label.

[0069] In the embodiments of the present disclosure, when it is determined that the comprehensive test score is less than the preset score, the user experience test device can determine that the user has a poor experience in the current cloud product experience process and needs to use the pre-trained decision tree model to perform cause analysis. Specifically, key features required for cause analysis can be extracted from the behavior data and the physiological data, and the extracted key features can be matched with the decision tree model to determine the low-score reason of the comprehensive test score. For example, the low-score reason can include high operation complexity, unreasonable interface design, excessive cognitive load, and the like.

[0070] The decision tree model can be trained based on the pre-collected historical behavior data, historical physiological data, and corresponding low-score reason labels. Specifically, after the historical behavior data and the historical physiological data are used to determine the historical comprehensive test score in a manner similar to S101-S103, if the historical comprehensive test score is less than the preset score, the historical comprehensive test score is used as training data of the decision tree model, and the decision tree model is trained in combination with the corresponding low-score reason label until the model performance meets the preset requirement.

[0071] In an exemplary embodiment of the present disclosure, the user experience test device can analyze the dependency relationship between different evaluation dimensions. Specifically, after the dimension features in the historical behavior data and the historical physiological data are extracted for each evaluation dimension, the Granger causality verification method can be used to verify whether there is a causal relationship between each evaluation dimension. When dividing the nodes in the decision tree model, the dependency relationship between different evaluation dimensions is considered as a factor to construct the decision tree model.

[0072] S403, based on the corresponding relationship between each type of low-score reason and the optimization measure obtained in advance, determining the target optimization measure corresponding to the low-score reason of the comprehensive test score.

[0073] In the embodiments of the present disclosure, after the low-score reason of the comprehensive test score is determined, the user experience test device can find and determine the target optimization measure corresponding to the low-score reason of the comprehensive test score based on the corresponding relationship between each type of low-score reason and the optimization measure obtained in advance. For example, when the low-score reason is that the instance type description term is obscure, the corresponding target optimization measure can be to suggest adding a graphic and text description.

[0074] The embodiment of the present disclosure compares the comprehensive test score with the preset score, and if the comprehensive test score is less than the preset score, determines the low-score reason of the comprehensive test score based on a pre-trained decision tree model and behavior data and physiological data, the decision tree model is trained based on pre-collected historical behavior data, historical physiological data and corresponding low-score reason labels, based on the corresponding relationship between the various low-score reasons and optimization measures obtained in advance, the target optimization measure corresponding to the low-score reason of the comprehensive test score is determined, the specific reason leading to the low score can be mined, and the corresponding optimization measure is automatically found, which facilitates the related personnel to collate, summarize and deeply analyze the low-score reason, and realizes the optimization of the cloud product.

[0075] In some embodiments, the user experience test device can identify and remove noise data in the behavior data after acquiring the behavior data and physiological data recorded by the user during the experience of the cloud product; the behavior data and physiological data are time-synchronized; based on the pre-acquired page element information of the cloud product, the behavior data and physiological data corresponding to the same page element are associated.

[0076] Specifically, the user experience test device can preprocess the behavior data and physiological data, which includes identifying and removing noise data in the behavior data, such as a mis-touch click operation, if the user clicks to jump to a new page and then backtracks, and the stay time on the new page is short, it can be determined that the click operation is a mis-touch click; the preprocessing operation also includes time-synchronized processing of the behavior data and physiological data, which can specifically use an interpolation algorithm to align data with different sampling rates; the preprocessing operation also includes integration and association of the behavior data and physiological data, which can specifically map gaze point coordinates, click coordinates, etc. to page elements based on the pre-acquired page element information of the cloud product, and associate the behavior data and physiological data corresponding to the same page element acquired at the same time.

[0077] Figure 5 is a flowchart of a method for generating cloud product optimization suggestions provided by the embodiment of the present disclosure, as Figure 5 shown in the above embodiment, the cloud product optimization suggestions can be generated by the following method.

[0078] S501, create a virtual role, and abstract the cloud product into a state space and an action space.

[0079] The virtual role in the embodiment of the present disclosure can be understood as an intelligent agent or rule script set for imitating the operation behavior of a real user on a cloud product, thereby expanding the user behavior path and corresponding operation result data.

[0080] The state space in the embodiments of the present disclosure can be understood as a set of all possible states in the cloud product, and can specifically include pages and page elements and the calling relationship between the pages and the page elements, such as a uniform resource locator (URL) tree and a document object model (DOM) tree. The action space can be understood as a set of operation behaviors supported by the cloud product, such as clicking and inputting.

[0081] In the embodiments of the present disclosure, the user experience test apparatus can create a virtual role for performing a training task for the cloud product, and abstract the cloud product into a state space and an action space. Specifically, the URL pages in the cloud product and the DOM elements in each page can be identified, and attribute information of the URL pages and the DOM elements can be obtained, and then the state space and the action space corresponding to the cloud product can be constructed based on the attribute information of the URL pages and the DOM elements.

[0082] S502, control the virtual role to perform a predefined training task based on the state space, the action space and a predefined role behavior template, construct a reward function with the maximum task completion rate as an optimization target, and train the virtual role in a reinforcement learning manner.

[0083] The task in the embodiments of the present disclosure can be understood as an operation task determined according to the functions supported by the cloud product, and can specifically be divided into a training task performed in a training phase of the virtual role and a test task performed after the training of the virtual role is completed.

[0084] In the embodiments of the present disclosure, the user experience test apparatus can obtain a predefined role behavior template after the virtual role, the state space and the action space are constructed, construct a reward function with the maximum task completion rate as an optimization target, give a large positive reward when the task is completed, give a negative punishment for each operation when the task is not completed, control the virtual role to perform operations according to the predefined role behavior template in a task environment composed of the state space and the action space, complete the predefined training task, and train the virtual role in a reinforcement learning manner until the task completion rate or the task completion speed of the virtual role meets a preset requirement.

[0085] S503, collect role behavior data of the virtual role when performing a predefined test task after the training is completed.

[0086] The role behavior data in the embodiments of the present disclosure can be understood as operations performed by the virtual role and corresponding operation results.

[0087] In the embodiments of the present disclosure, the user experience test device can control the virtual character to perform a predefined test task based on the predefined character behavior template after the virtual character is trained, and record character behavior data generated in the execution process of the test task.

[0088] S504, generating a cloud product optimization suggestion based on the character behavior data.

[0089] In the embodiments of the present disclosure, the user experience test device can analyze the character behavior data after obtaining the character behavior data, and generate a cloud product optimization suggestion according to the analysis result. Specifically, the user experience test device can filter out high-frequency operation behaviors and error information, generate a cloud product optimization suggestion for function improvement for the high-frequency operation behaviors, such as setting the page elements corresponding to the high-frequency operations in a more obvious position, adding an automatic paste function to the input box, and the like, analyze the reasons for the high-frequency error information, and generate a cloud product optimization suggestion for fault repair, and the like.

[0090] The embodiments of the present disclosure can simulate user behavior, efficiently and quickly execute test tasks and obtain task execution data, reduce the time cost and labor cost required for testing, and automatically generate a cloud product optimization suggestion, which facilitates the operation and maintenance personnel to further optimize the cloud product based on the cloud product optimization suggestion, thereby improving the user experience of real users in the process of using the cloud product.

[0091] In some embodiments, the user experience test device can record the failure execution path of the virtual character in the execution process of the training task and / or the test task, and send the sorted failure execution path to the operation and maintenance personnel.

[0092] Specifically, for at least one of the training task and the test task, the user experience test device can monitor the task execution process of the virtual character, determine whether a failure execution path, such as an execution path trapped in a dead loop, exists in the execution process, and if so, record information of the failure execution path. Specifically, the user experience test device can record the training task or the test task to which the failure execution path belongs, the page URL, the DOM element, the specific operation for each DOM element, and the like involved in the failure execution path, and periodically aggregate and sort the recorded failure execution paths, and send them to the operation and maintenance personnel, so that the operation and maintenance personnel can timely discover and repair design defects.

[0093] Figure 6 is a structural schematic diagram of a user experience test device provided by an embodiment of the present disclosure. As shown in the figure, the user experience test device 600 includes an acquisition module 610, an extraction module 620, and a scoring module 630. Figure 6 The acquisition module 610 is configured to acquire behavior data and physiological data recorded by a user in a cloud product experience process, wherein the physiological data includes eye movement data and / or emotional data. The extraction module 620 is configured to perform feature extraction on the behavior data and the physiological data based on a preset evaluation dimension, and determine a dimension score corresponding to each evaluation dimension based on a feature extraction result. The scoring module 630 is configured to determine a comprehensive test score based on the dimension score corresponding to each evaluation dimension and an adaptive optimization weight.

[0094] Optionally, the user experience test device 600 further includes a first determination module configured to determine a task type of a task completed by the user in the cloud product experience process based on the behavior data and the physiological data, wherein the task type includes a simple task and a complex task. The user experience test device 600 further includes a second determination module configured to determine the adaptive optimization weight corresponding to each evaluation dimension based on the task type and a preset weight configuration scheme.

[0095] Optionally, the evaluation dimension includes an emotional evaluation dimension and an efficiency evaluation dimension. The second determination module includes a first determination unit configured to determine an initial weight corresponding to each evaluation dimension under the task type based on the task type and the weight configuration scheme. A first adjustment unit is configured to perform correlation analysis on the behavior data and the emotional data. If a correlation coefficient of the behavior data and the emotional data is greater than a preset threshold, the initial weight corresponding to the emotional evaluation dimension in the evaluation dimension is adjusted. A second determination unit is configured to determine an emotion type of the emotional data. If the emotion type is a negative emotion, operation efficiency data of the user is extracted from the behavior data, and an influence degree of the emotional data on the operation efficiency data is determined. A second adjustment unit is configured to adjust the initial weight corresponding to the efficiency evaluation dimension in the evaluation dimension based on the influence degree of the emotional data on the operation efficiency data. A third determination unit is configured to determine the initial weight corresponding to each evaluation dimension after adjustment as the adaptive optimization weight corresponding to each evaluation dimension.

[0096] Optionally, the user experience testing device 600 further includes: a third determining module, used to compare the comprehensive test score with a preset score; a fourth determining module, used to determine the reason for the low score of the comprehensive test score based on a pre-trained decision tree model and the behavioral data and the physiological data if the comprehensive test score is less than the preset score, wherein the decision tree model is trained based on pre-collected historical behavioral data, historical physiological data and corresponding low score reason labels; and a fifth determining module, used to determine the target optimization measures corresponding to the low score reasons of the comprehensive test score based on the pre-acquired correspondence between various low score reasons and optimization measures.

[0097] Optionally, the user experience testing device 600 further includes: a rejection module for identifying and rejecting noise data in the behavioral data; a synchronization module for performing time synchronization processing on the behavioral data and the physiological data; and an association module for associating behavioral data and physiological data corresponding to the same page element based on pre-acquired page element information of the cloud product.

[0098] Optionally, the user experience testing device 600 further includes: a creation module for creating a virtual character and abstracting the cloud product into a state space and an action space; a training module for controlling the virtual character to execute a predefined training task based on the state space, the action space, and a predefined character behavior template, constructing a reward function with maximizing the task completion rate as the optimization objective, and training the virtual character using reinforcement learning; a collection module for collecting character behavior data of the virtual character when executing a predefined test task after training; and a generation module for generating cloud product optimization suggestions based on the character behavior data.

[0099] Optionally, the user experience testing device 600 further includes a recording module, used to record the failed execution paths of the virtual character during the execution of the training task and / or the test task, and send the organized failed execution paths to the operation and maintenance personnel.

[0100] The user experience testing device provided in this embodiment can execute the methods described in any of the above embodiments. Its execution method and beneficial effects are similar, and will not be repeated here.

[0101] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0102] like Figure 7 As shown, the computer device may include a processor 710 and a memory 720 storing computer program instructions.

[0103] In particular, the processor 710 can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or can be configured as one or more integrated circuits that implement embodiments of the present application.

[0104] The memory 720 can include mass storage for information or instructions. For example, and without limitation, the memory 720 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage memory 720 can include removable or non-removable (or fixed) media, as appropriate. Storage memory 720 can be internal or external, as appropriate. In particular embodiments, the storage memory 720 is non-volatile solid-state memory. In particular embodiments, the storage memory 720 includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0105] The processor 710 performs the steps of the user experience test method provided by the embodiments of the present disclosure by reading and executing computer program instructions stored in the memory 720.

[0106] In one example, the computer device can also include a transceiver 730 and a bus 740. As shown, the processor 710, the memory 720, and the transceiver 730 are connected and complete communication with each other through the bus 740. Figure 7

[0107] ​Bus 740 includes a hardware, software, or both that couples components of computer system 700 to each other. As an example without limitation, bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side BUS (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or some other suitable bus or interconnect, or a combination of two or more of these. Where appropriate, bus 740 can include one or more buses. Although this application describes and shows a particular bus, this application contemplates any suitable bus or interconnect.

[0108] The embodiment of the present disclosure further provides a computer readable storage medium, which can store a computer program. When the computer program is executed by a processor, the processor implements the user experience test method provided by the embodiment of the present disclosure.

[0109] The storage medium described above may, for example, include a memory 720 of computer program instructions that are executable by the processor 710 of the user experience testing device to implement the user experience testing method provided by the embodiments of the present disclosure. Optionally, the storage medium may be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium may be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc. The computer program described above can be written in any combination of one or more programming languages implementing the operations of the embodiments of the present disclosure, including an object-oriented programming language, such as Java, C++, etc., and a conventional procedural programming language, such as a "C" language or a similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0110] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0111] The above description is merely one implementation of the present disclosure, and those skilled in the art will be able to make various modifications to the embodiments without departing from the spirit or scope of the present disclosure. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A user experience testing method, characterized in that, The method includes: Acquire behavioral and physiological data recorded by users during their cloud product experience, including eye-tracking data and / or emotional data; Based on preset evaluation dimensions, feature extraction is performed on the behavioral data and the physiological data, and the dimension score corresponding to each evaluation dimension is determined based on the feature extraction results. The comprehensive test score is determined based on the dimension scores and adaptive optimization weights corresponding to each evaluation dimension.

2. The method according to claim 1, characterized in that, Before determining the comprehensive test score based on the dimension scores and adaptive optimization weights corresponding to each evaluation dimension, the method further includes: Based on the behavioral data and the physiological data, the task type of the task completed by the user during the cloud product experience is determined, and the task type includes simple tasks and complex tasks. The adaptive optimization weights for each evaluation dimension are determined based on the task type and the preset weight configuration scheme.

3. The method according to claim 2, characterized in that, The evaluation dimensions include an emotional evaluation dimension and an efficiency evaluation dimension. The process of determining the adaptive optimization weights corresponding to each evaluation dimension based on the task type and a preset weight configuration scheme includes: Based on the task type and the weight configuration scheme, determine the initial weights corresponding to each evaluation dimension under the task type; A correlation analysis is performed on the behavioral data and the emotional data. If the correlation coefficient between the behavioral data and the emotional data is greater than a preset threshold, the initial weight of the emotional evaluation dimension in each evaluation dimension is adjusted. The emotion type of the emotional data is determined. If the emotion type is negative, the user's operational efficiency data is extracted from the behavioral data, and the degree of influence of the emotional data on the operational efficiency data is determined. Based on the degree of influence of the emotional data on the operational efficiency data, the initial weights of the efficiency evaluation dimensions in each evaluation dimension are adjusted. The initial weights corresponding to each of the adjusted evaluation dimensions are determined as the adaptive optimization weights corresponding to each of the evaluation dimensions.

4. The method according to claim 1, characterized in that, After determining the comprehensive test score based on the dimension scores and adaptive optimization weights corresponding to each evaluation dimension, the method further includes: Compare the comprehensive test score with the preset score; If the comprehensive test score is less than the preset score, the reason for the low comprehensive test score is determined based on the pre-trained decision tree model, the behavioral data, and the physiological data. The decision tree model is trained based on pre-collected historical behavioral data, historical physiological data, and corresponding low score reason labels. Based on the pre-obtained correspondence between various low-score reasons and optimization measures, the target optimization measures corresponding to the low-score reasons in the comprehensive test score are determined.

5. The method according to claim 1, characterized in that, After acquiring the behavioral and physiological data recorded by the user during their cloud product experience, the method further includes: Noise data is identified and removed from the behavioral data; The behavioral data and the physiological data are synchronized over time. Based on the pre-acquired page element information of the cloud product, behavioral data and physiological data corresponding to the same page element are associated.

6. The method according to claim 1, characterized in that, The method further includes: Create virtual characters and abstract the cloud products into a state space and an action space; Based on the state space, the action space, and the predefined role behavior template, the virtual character is controlled to perform a predefined training task. The reward function is constructed with maximizing the task completion rate as the optimization objective, and the virtual character is trained using reinforcement learning. Collect the character behavior data of the virtual character when it performs a predefined test task after training is completed; Based on the aforementioned role behavior data, cloud product optimization suggestions are generated.

7. The method according to claim 6, characterized in that, The method further includes: Record the failed execution paths of the virtual character during the execution of the training task and / or the test task, and send the compiled failed execution paths to the operation and maintenance personnel.

8. A user experience testing device, characterized in that, The device includes: The acquisition module is used to acquire behavioral and physiological data recorded by users during their cloud product experience, including eye-tracking data and / or emotional data. The extraction module is used to extract features from the behavioral data and the physiological data based on preset evaluation dimensions, and to determine the dimension score corresponding to each evaluation dimension based on the feature extraction results. The scoring module is used to determine the comprehensive test score based on the dimension scores corresponding to each evaluation dimension and the adaptive optimization weights.

9. A computer device, characterized in that, include: Memory; processor; And a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the user experience testing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the user experience testing method as described in any one of claims 1-7.

Images