Method and device for monitoring Internet user experience

Abstract

The present invention discloses a method and device for monitoring Internet user experience, and relates to the field of computer technology. A specific implementation of the method includes: constructing an eye movement heat map based on a facial image data set and page behavior data of a user during the browsing process of a target page, and fitting the eye movement heat map of each user to obtain a first layer; constructing a page screenshot based on the document tree structure and page data of the target page; fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimized analysis result, and using the user experience cloud map and the optimized analysis result as a user experience monitoring result. This implementation improves the reliability and significance of the data, and the data analysis results are more accurate.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for monitoring Internet user experience. Background Art

[0002] With the development of Internet technology, people are increasingly obtaining the information and data they need through the Internet. Therefore, in the process of Internet users visiting Internet pages, the user experience becomes more and more important. At present, most of them are collected by analyzing the user's behavior data on Internet pages or the experience problems reported by users, so as to monitor and improve the user experience. When collecting user behavior data, it is generally collected by embedding code points in Internet pages to collect the user's interaction behavior data with the page or using professional experimental equipment (such as: eye tracker) to collect the user's eye movement data.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The accuracy of user experience monitoring by analyzing the interaction behavior data between users and pages is low. The user experience monitoring method by analyzing eye movement data collected by professional experimental equipment such as eye trackers has low data credibility because the experimental equipment will interfere with the user, and in the process of collecting eye movement data, changes in the user's head posture will reduce the significance of the collected data, making the data analysis results inaccurate. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for monitoring Internet user experience, which can improve the reliability and significance of data and make the data analysis results more accurate.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for monitoring Internet user experience is provided, comprising:

[0007] An eye movement heat map is constructed based on the facial image dataset and page behavior data of the user during the browsing process of the target page, and the eye movement heat map of each user is fitted to obtain the first layer;

[0008] Constructing a page screenshot according to the document tree structure and page data of the target page;

[0009] The first layer and the page screenshot are fitted to obtain a user experience cloud map, image analysis is performed on the first layer to obtain an optimization analysis result, and the user experience cloud map and the optimization analysis result are used as user experience monitoring results.

[0010] Optionally, an eye movement heat map is constructed based on a facial image dataset and page behavior data of a user during a browsing process of a target page, including: collecting a facial image dataset and page behavior data of a user during a browsing process of a target page; constructing a canvas based on the page behavior data; generating an eye movement trajectory of the user based on the facial image dataset and a preset standard image dataset; and constructing an eye movement heat map on the canvas based on the eye movement trajectory.

[0011] Optionally, the page behavior data includes a page scrolling distance of a user during browsing of a target page; constructing a canvas based on the page behavior data includes: determining a canvas size based on a size of a page display window and a page scrolling distance during browsing of the target page, and constructing a canvas based on the canvas size.

[0012] Optionally, generating the user's eye movement trajectory according to the facial image dataset and a preset standard image dataset includes: determining the user's browsing position on the target page and the browsing order of each browsing position according to the facial image dataset and the preset standard image dataset; generating the user's eye movement trajectory according to the browsing position and the browsing order of each browsing position.

[0013] Optionally, before constructing an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, it also includes: determining whether the similarity between the user's sampled image data and preset standard image data meets a set threshold by performing calibration sampling and tracking process sampling on the user.

[0014] Optionally, calibration sampling is performed on the user, including: dividing a visible area of ​​a first sampling page into a specified number of grid areas; a total number of calibration points appearing in the grid areas in a random manner, with only one calibration point appearing at a time; for each calibration point, collecting first facial image data of the user each time the user gazes at and clicks the calibration point through a network camera; and calculating the similarity between the first facial image data and preset first standard image data.

[0015] Optionally, during the calibration sampling process, the network camera image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

[0016] Optionally, calculating the similarity between the first facial image data and the preset first standard image data includes: constructing a first head posture feature based on the first facial image data; determining a first calibration point that the user gazes at and clicks based on the first head posture feature and the first standard posture feature, wherein the first standard posture feature is constructed based on the first standard image data; using the first standard image data corresponding to the first calibration point as the first standard image data corresponding to the first facial image data; and calculating the similarity between the first facial image data and the first standard image data through a similarity algorithm.

[0017] Optionally, sampling of the tracking process of the user includes: dividing the visible area of ​​the second sampling page into a specified number of grid areas; randomly placing calibration points in a uniform displacement motion state in the grid areas to guide the user's line of sight to move along with the position of the current calibration point on the screen; collecting second facial image data of the user's line of sight moving state through a network camera; and calculating the similarity between the second facial image data and preset second standard image data.

[0018] Optionally, during the sampling process of the tracking process, the network camera image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

[0019] Optionally, calculating the similarity between the second facial image data and preset second standard image data includes: constructing a second head posture feature based on the second facial image data; determining a moving trajectory of a second calibration point that the user is gazing at based on the second head posture feature and the second standard posture feature, wherein the second standard posture feature is constructed based on the second standard image data; acquiring second standard image data corresponding to the second facial image data based on the moving trajectory of the second calibration point; and calculating the similarity between the second facial image data and the second standard image data through a similarity algorithm.

[0020] Optionally, performing image analysis on the first layer to obtain an optimized analysis result includes: assigning different weights to different areas of the target page according to the document tree structure of the target page and the semantics carried by the document nodes; performing color value conversion on the first layer to obtain hue values ​​of the first layer in different areas of the target page; performing weight matching on the hue values ​​of the first layer in different areas of the target page according to preset hue value ranges corresponding to different weights; and generating an optimized analysis result according to the weight matching result.

[0021] According to another aspect of an embodiment of the present invention, a device for monitoring Internet user experience is provided, including:

[0022] A first layer construction module is used to construct an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, and to fit the eye movement heat map of each user to obtain the first layer;

[0023] A page screenshot construction module, used to construct a page screenshot according to the document tree structure and page data of the target page;

[0024] The monitoring, analysis and optimization module is used to fit the first layer and the page screenshot to obtain a user experience cloud map, perform image analysis on the first layer to obtain an optimization analysis result, and use the user experience cloud map and the optimization analysis result as the user experience monitoring result.

[0025] According to another aspect of an embodiment of the present invention, an electronic device for monitoring Internet user experience is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for monitoring Internet user experience provided by an embodiment of the present invention.

[0026] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for monitoring Internet user experience provided by an embodiment of the present invention is implemented.

[0027] An embodiment of the above invention has the following advantages or beneficial effects: by constructing an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, and fitting the eye movement heat map of each user to obtain a first layer; constructing a page screenshot based on the document tree structure and page data of the target page; fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimized analysis result, and using the user experience cloud map and the optimized analysis result as a technical solution for user experience monitoring results, an eye movement heat map can be constructed based on the collected facial image data of the user, and an immersive and imperceptible user experience can be provided. User experience optimization provides continuous image data. There is no need to use professional experimental equipment such as eye trackers to collect data, which avoids interference of experimental equipment on users. The data is highly reliable and easy to collect massive user experience data. At the same time, the collected facial image data can cover the user's head posture characteristics, making the user's head posture more flexible, and the significance of the collected data will not be reduced due to changes in the user's head posture, making the data analysis results more accurate. The first layer is obtained by fitting massive data, and the regression calculation of the data gives the advantage that a single eye tracking device cannot achieve, which improves the reliability and significance of the data and makes the data analysis results more accurate.

[0028] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0030] Figure 1 is a schematic diagram of the main steps of the method for monitoring Internet user experience according to an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a page grid division result according to an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of the display position of the calibration points according to an embodiment of the present invention;

[0033] Figure 4 is a fixed wireframe schematic diagram of an embodiment of the present invention;

[0034] Figure 5 is a schematic diagram of the motion state of a calibration point during the sampling process of a tracking process according to an embodiment of the present invention;

[0035] Figure 6 is a schematic diagram of the HSB color wheel range of an embodiment of the present invention;

[0036] Figure 7 is a schematic diagram of main modules of an apparatus for monitoring Internet user experience according to an embodiment of the present invention;

[0037] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0038] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0040] The present invention provides a method and device for monitoring Internet user experience, which can solve the following three types of technical problems:

[0041] First, the existing experimental equipment (such as eye trackers) that aims to optimize product interaction design and collects and analyzes user eye movement behavior and the data generated by it has its inherent inconveniences. First, it is not convenient to collect massive user experience data. Second, the professional design of the equipment will cause certain interference to users and affect the reliability of the data. The present invention uses a network camera as a hardware input device for collecting massive user data. After completing the data training (sampling) step, it can provide continuous image data for user experience optimization without perception.

[0042] Second, the existing eye movement data collection and analysis solutions based on the Web platform, such as WebGazer.js, use a training method for sampling eye movement behavior that reduces the significance of the data when the user's head posture moves. When collecting massive user eye movement data, it is inevitable that the user's head posture moves. The present invention intends to use several sampling points on the human face that can be used to identify changes in head posture, and optimize the training to improve the data analysis results;

[0043] Third, the existing eye movement data collection and analysis solutions of the Web platform are limited by the reliability and significance of the data, and cannot output reliable user experience reports (such as eye movement heat maps) like professional eye movement experimental equipment. Since the present invention uses massive data as a training pool, the regression calculation of the data gives an advantage that a single eye movement device cannot achieve. Since the present invention takes into account the impact of head posture changes on image data, the reliability and significance of the data are improved. Then the present invention proposes to perform data fitting and data analysis on the server side, and directly output a user experience report (such as an eye movement heat map).

[0044] Figure 1 Schematic diagram of the main steps of the method for monitoring Internet user experience according to an embodiment of the present invention. Figure 1 As shown, the method for monitoring Internet user experience according to the embodiment of the present invention mainly includes the following steps S101 to S103.

[0045] Step S101: construct an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, and fit the eye movement heat map of each user to obtain the first layer. In an embodiment of the present invention, before collecting data from the user, the user will be reminded in an obvious way (for example: pop-up window, dialog box, tips, alert and other conventional interactive ways to remind the user, which can be a modal dialog box) that the network camera will continue to be turned on, and the user will be informed that the system will continue to collect the user's facial image information to construct an eye movement trajectory graph, and provide the user with an interactive way to close the image collection process at any time. In addition, when collecting data, the user is required to display the target web page in full screen to facilitate comparison with the standard image data set, reduce the difficulty of data analysis, and improve the efficiency of data analysis.

[0046] According to the technical solution of the present invention, before data collection is performed on the user, data training is required to verify whether the user meets the data collection requirements. Only when the user meets the data collection requirements during the data training phase, data collection and data analysis operations will be performed on the user. Before data training, it is necessary to determine whether the user has the hardware conditions for data training. Specifically, before the user browses the page, it can be determined whether the user's device is connected to the network camera. If the browser and hardware conditions support it, the user is prompted to turn on the user experience detection service, and turn on the network camera permissions with the user's authorization to enter the data training phase. In the data training phase, it is mainly through calibration sampling and tracking process sampling of the user to determine whether the user meets the data collection requirements. It should be noted that in the introduction of all embodiments of the present invention, the collected user data and information such as character pictures and user facial images are collected with the user's authorization and are obtained through legal means.

[0047] According to an embodiment of the present invention, before constructing an eye movement heat map based on the facial image data set and page behavior data of the user in the process of browsing the target page, it also includes: determining that the similarity between the sampled image data of the user and the preset standard image data meets the set threshold by performing calibration sampling and tracking process sampling on the user. Only users whose sampled image data of the user meets the set threshold in similarity with the preset standard image data are users that meet the data collection requirements.

[0048] According to an embodiment of the present invention, the step of calibrating and sampling the user may specifically include: dividing the visible area of ​​the first sampling page into a specified number of grid areas; a fixed number of calibration points appear in the grid areas in a random manner, and only one calibration point appears at a time; for each calibration point, the first facial image data of the user each time the user gazes at and clicks the calibration point is collected through a network camera; and the similarity between the first facial image data and the preset first standard image data is calculated. In an embodiment of the present invention, the first sampling page is, for example, the page that the user is currently browsing, or it may be a preset sampling page, and so on. When performing calibration sampling and tracking process sampling on the user, the user will also be prompted to display the current web page in full screen to facilitate comparative analysis with the standard image data.

[0049] Specifically, when calculating the similarity between the first facial image data and the preset first standard image data, it can specifically include: constructing a first head posture feature based on the first facial image data; determining a first calibration point that the user gazes at and clicks based on the first head posture feature and the first standard posture feature, wherein the first standard posture feature is constructed based on the first standard image data; using the first standard image data corresponding to the first calibration point as the first standard image data corresponding to the first facial image data; and calculating the similarity between the first facial image data and the first standard image data through a similarity algorithm.

[0050] In one embodiment of the present invention, the process of calibrating and sampling users is as follows:

[0051] 1) Prompt the user to display the web page in full screen, and then divide the visible area of ​​the first sample page into a specified number of grid areas, such as Figure 2 As shown, Figure 2 is a schematic diagram of the page grid division result of an embodiment of the present invention. In this embodiment, for example, the first sample page is divided into 24×16 grid areas. Then, assuming that the width of the visible area of ​​the first sample page is window_width and the height is window_height, the width of each grid area is window_width / 24 and the height is window_height / 16.

[0052] 2) A fixed number of calibration points appear in random positions in the grid area, and only one calibration point appears at a time. The number of calibration points depends on the training effect. If the number of calibration points is too small, the effect will not be good, and if the number of calibration points is too large, the user experience will not be good. According to the literature and experiments, it is expected that the number of calibration points should be set to 9-16. Once the number of calibration points is fixed, it will no longer be easily adjusted when the user uses this function, that is, the total number is fixed. The position of the calibration points depends on the training effect. The position of the expected calibration points will include the following 9 key positions in the visible area of ​​the current first sampling page, namely, upper left, upper middle, upper right, middle left, center, middle right, lower left, lower middle, and lower right. If the total number of calibration points is set to be greater than 9, the positions of the excess calibration points should be randomly distributed on the screen. Figure 3 FIG. 1 is a schematic diagram showing the display position of the calibration points according to an embodiment of the present invention. Figure 3 As shown, there are 16 calibration points in this embodiment, and the figure shows the display positions of the 16 calibration points.

[0053] 3) For each displayed calibration point, the system will guide the user to look at the center of the calibration point that currently appears, and require the user to use a pointer interaction device such as a mouse cursor to click the current calibration point on the screen for a fixed number of times (the initial setting can be 5-10 times). The system uses a network camera to collect the first facial image data of the user each time he looks at and clicks the calibration point. In the implementation process of the present invention, in order to improve the accuracy of the collected data, the system continuously outputs network camera image information to the user during the calibration sampling period, and uses a fixed wireframe to limit and guide the range of movement of the user's head. Figure 4 is a fixed wireframe schematic diagram of an embodiment of the present invention, such as Figure 4 As shown, a fixed wire control may appear in the upper right corner of the page to prompt and guide the user's head movement range. The character image in the figure comes from https: / / this-person-does-not-exist.com, which is a virtual character image generated by AI. In the specific implementation process, the user can guide the movement range of his head according to the user's own head image displayed in the fixed wire frame.

[0054] 4) Calculate the similarity between the first facial image data of the user collected previously and the preset first standard image data, so as to take the user whose similarity meets the set threshold as the user who meets the data collection requirements. According to the first facial image data collected previously, the change of the user's head posture characteristics can be obtained. Specifically, according to each first facial image data, based on the image information of the triangle area between the "nose bridge" and the "nose wing", it is judged whether the user's head has "approached or moved away from the screen", "translated", "raised the head", "lowered the head", "leaned to the side", etc., which can be used to construct the user's head posture characteristics during the calibration sampling process. According to the head posture characteristics, it is possible to determine which calibration point each collected first facial image data corresponds to, and obtain the first standard image data corresponding to the calibration point. By comparing the first facial image data of the same calibration point with the first standard image data, and calculating the similarity of the first facial image data corresponding to all calibration points and the first standard image data, the similarity data during the data calibration sampling period can be obtained and output to the user interface. Among them, the similarity algorithm can be selected according to needs and is not limited to a certain algorithm. For example, you can first use the js-objectdetect algorithm and tracking.js algorithm to detect facial and eye images and analyze the similarity; then, use the clmtrackr algorithm to fit the facial and eye image information, and finally perform linear regression on the collected image information to obtain the similarity.

[0055] In the data training stage, in addition to calibration sampling, the user can also be sampled during the tracking process. According to one embodiment of the present invention, the user can be sampled during the tracking process, which can specifically include: dividing the visible area of ​​the second sampling page into a specified number of grid areas; in the grid areas, calibration points in a uniform displacement motion state appear in a random manner to guide the user's line of sight to follow the position of the current calibration point on the screen; collecting the second facial image data of the user's line of sight in the moving state through a network camera; calculating the similarity between the second facial image data and the preset second standard image data. When sampling the tracking process, the selected second sampling page can be the same web page as the first sampling page or the target page that the user wants to visit, or it can be a pre-set page.

[0056] In an embodiment of the present invention, when calculating the similarity between the second facial image data and the preset second standard image data, it may specifically include: constructing a second head posture feature according to the second facial image data; determining the moving trajectory of the second calibration point that the user is looking at according to the second head posture feature and the second standard posture feature, wherein the second standard posture feature is constructed based on the second standard image data; acquiring the second standard image data corresponding to the second facial image data according to the moving trajectory of the second calibration point; and calculating the similarity between the second facial image data and the second standard image data by a similarity algorithm. During the sampling process of the tracking process, the calibration point is moving, so the second facial image data corresponding to each calibration point collected is a plurality of continuous images corresponding to the moving trajectory of the calibration point. When performing a similarity comparison, only the first and last images corresponding to the moving trajectory of the calibration point can be taken as the second facial image data of the calibration point, so that the amount of data processing calculation can be reduced and the data processing efficiency can be improved while retaining the moving trajectory information of the calibration point as much as possible. Correspondingly, when performing a similarity comparison between the second facial image data and the second standard image data, only the first and last images corresponding to the moving trajectory of a calibration point are selected for comparison.

[0057] In one embodiment of the present invention, the process of sampling the tracking process of the user is as follows:

[0058] 1) Prompt the user to display the web page in full screen, and then divide the visible area of ​​the second sampling page into a specified number of grid areas. The specific division process can be found in step 1) in the calibration sampling process.

[0059] 2) Calibration points in a uniform displacement state appear randomly in the grid area, with a uniform linear speed. The trajectory can be a straight line or a curve. The purpose is to let the user's line of sight track the calibration points and collect more abundant image information to improve the training effect. Figure 5 FIG. 1 is a schematic diagram of the motion state of the calibration point during the sampling process of the tracking process of an embodiment of the present invention. Figure 5 As shown, the calibration point can move at a uniform speed according to a preset moving trajectory.

[0060] 3) The system guides the user's line of sight to follow the position of the current calibration point on the screen. The system uses a network camera to collect dynamic data of the user's facial information to obtain second facial image data. In the implementation of the present invention, in order to improve the accuracy of the collected data, the system continuously outputs network camera image information to the user during the calibration sampling period, and uses a fixed wireframe to limit and guide the range of movement of the user's head.

[0061] 4) Calculate the similarity between the second facial image data of the user collected previously and the preset second standard image data, so as to select the user whose similarity meets the set threshold as the user who meets the data collection requirements. According to the second facial image data collected previously, the change of the user's head posture characteristics can be obtained. The specific implementation method can be found in step 4 of the calibration sampling process). By comparing the second facial image data and the second standard image data of the same calibration point movement trajectory for similarity, and calculating the similarity of the second facial image data and the second standard image data corresponding to all calibration points, the similarity data during the data tracking process sampling period can be obtained and output to the user interface.

[0062] By sampling the user during calibration and tracking, it is possible to detect whether the user meets the data collection requirements. For users who meet the data collection requirements, the user can be prompted to browse the target web page and collect facial image data. When collecting facial image data, in order to maintain data stability, the minimum length of time that users participating in data collection stay on the target web page can be pre-specified (the initial experiment can be set to about 10 seconds). When the user stays on the target web page for more than the minimum length of time, the collected user data is valid. The collected data includes the facial image data set and page behavior data of the user during the browsing process of the target page.

[0063] According to an embodiment of the present invention, an eye movement heat map is constructed based on a facial image data set and page behavior data of a user in the process of browsing a target page, including: collecting a facial image data set and page behavior data of a user in the process of browsing a target page; constructing a canvas based on the page behavior data; generating an eye movement trajectory of the user based on the facial image data set and a preset standard image data set; and constructing an eye movement heat map on the canvas based on the eye movement trajectory. Wherein, the page behavior data includes a page scrolling distance of the user in the process of browsing the target page; and, constructing a canvas based on the page behavior data includes: determining a canvas size based on the size of a page display window and a page scrolling distance in the process of browsing the target page, and constructing a canvas based on the canvas size. In a specific implementation process, a suitable canvas size can be constructed using WebGL technology based on the size data of an actual web page (window_height / window_width) and the distance (scrolling) of a user scrolling the page when accessing the target web page.

[0064] In an embodiment of the present invention, generating the eye movement trajectory of the user according to the facial image data set and the preset standard image data set may specifically include: determining the browsing position of the user on the target page and the browsing order of each browsing position according to the facial image data set and the preset standard image data set; generating the eye movement trajectory of the user according to the browsing position and the browsing order of each browsing position. After generating the eye movement trajectory of the user, an eye movement heat map can be constructed using WebGL technology according to the eye movement trajectory of the user in visiting the target web page. Specifically, the eye movement frequent area is represented by red, and vice versa, it is represented by blue, and a 256-color RGB color spectrum is constructed between red and blue in a color light mixing manner to construct an eye movement heat map. Afterwards, the eye movement heat map of each user who visited the target page is fitted to obtain the first layer. The eye movement heat map of each user who visited the target page is taken as a sample, and the samples collected within a specified time are merged and output to a canvas and fitted and output. The merged output can be realized by WebGL, and the data is transmitted to the front end to construct the HTML5 canvas. When performing image fitting, the image is fitted on the z-axis through the WebGL vector. The width and height of the image are controllable. The vector primitives can be constructed first, and then the color information is rendered. The amount of calculation is controllable. When performing image fitting, if the number of samples is too large, the samples to be fitted can be obtained by sampling. In addition, the RGB color values ​​of different areas of the first layer after fitting can be calculated according to the sample mean.

[0065] Step S102: Construct a page screenshot according to the document tree structure and page data of the target page. According to an embodiment of the present invention, a page screenshot can be constructed based on the DOM document tree structure of a Web page using WebGL technology or SVG method. On the basis of ensuring that the user stays on the target web page for a minimum time, "valid data for the page screenshot" can be provided after the Web page is loaded. That is: after the user stays on the target web page for the minimum time and the page is loaded, the DOM document tree structure and page data of the target page are collected, and a page screenshot containing the page data is constructed.

[0066] Step S103: Fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimization analysis result, and using the user experience cloud map and the optimization analysis result as the user experience monitoring result. After obtaining the first layer and the page screenshot, image fitting can be performed to obtain a user experience cloud map as part of the user experience detection result to provide support for data visualization design.

[0067] In an embodiment of the present invention, image analysis is performed on the first layer to obtain an optimized analysis result, which may specifically include: assigning different weights to different areas of the target page according to the document tree structure of the target page and the semantics carried by the document nodes; performing color value conversion on the first layer to obtain hue values ​​of the first layer in different areas of the target page; performing weight matching on the hue values ​​of the first layer in different areas of the target page according to preset hue value ranges corresponding to different weights; and generating an optimized analysis result based on the weight matching result.

[0068] In the specific implementation process, first, different weights are assigned to different visual areas of the Web page according to the DOM tree structure of the Web page obtained in the data collection and construction steps, and the HTML semantics carried by the specific DOM nodes, as shown in Table 1.

[0069] Table 1

[0070] Web Page Area Weight Logo 5 text 3 Decorative background image 1

[0071] The system provides an API interface for Web development, giving developers the ability to manually increase the weight of one or more Web areas. For example, if the namespace of the data attribute defined in HTML5 is data-pupilcr, the weight of the Logo area can be manually set using "data-pupilcr=5".

[0072] Afterwards, by analyzing the color values ​​of the first layer in different areas of the web page, the RGB color values ​​are converted into HSB color values ​​(with the HSB system in which red is at 0 radians and yellow is at π / 3 radians as the standard), and the following data can be obtained as shown in Table 2.

[0073] Table 2

[0074] Web Page Area Weight The HSB color value of the first layer in this area Logo 5 H:0,S:75%,B:80% text 3 H:2π / 3,S:75%,B:80% Decorative background image 1 H:4π / 3,S:75%,B:80%

[0075] Then, remove the saturation (S) and brightness (B) information, compare the radian of the hue (H) of the first layer with the color scale range (radian) corresponding to the weight, and analyze whether the color value matches. For example, if the HSB color wheel is used as the heat map color scale, Figure 6 3 is a schematic diagram of the HSB color wheel range of an embodiment of the present invention. The total color scale range is 0 to 5π / 3. Assuming that the color scale range of the first layer in a certain area of ​​the Web page is Rc, there is a corresponding relationship between different weights and color scale ranges as shown in Table 3.

[0076] Table 3

[0077] Weight The HSB color value of the first layer in this area 5 0≤Rc<π / 3 4 π / 3≤Rc<2π / 3 3 2π / 3≤Rc<π 2 π≤Rc<4π / 3 1 4π / 3≤Rc≤5π / 3

[0078] If the Rc value does not match the weight, it means that there is room for optimization. The optimization analysis results obtained may be: it may be necessary to optimize the design scheme to make the Rc value of the heat map of the area match the range corresponding to the weight; or it may be necessary to adjust the interactive layout and rearrange the page elements with corresponding weights to return it to the corresponding area where the Rc value matches the weight element.

[0079] Finally, the optimization analysis results obtained from the above analysis are combined with the user experience cloud map and output to the user experience monitoring background in a visual way to form a product optimization needs report as the user experience monitoring result.

[0080] Figure 7 Schematic diagram of the main modules of the device for monitoring Internet user experience according to an embodiment of the present invention. Figure 7 As shown, the Internet user experience monitoring device 700 of the embodiment of the present invention mainly includes a first layer construction module 701, a page screenshot construction module 702 and a monitoring analysis optimization module 703.

[0081] The first layer construction module 701 is used to construct an eye movement heat map based on the facial image data set and page behavior data of the user in the process of browsing the target page, and to fit the eye movement heat map of each user to obtain the first layer;

[0082] A page screenshot construction module 702 is used to construct a page screenshot according to the document tree structure and page data of the target page;

[0083] The monitoring and analysis optimization module 703 is used to fit the first layer and the page screenshot to obtain a user experience cloud map, perform image analysis on the first layer to obtain an optimization analysis result, and use the user experience cloud map and the optimization analysis result as the user experience monitoring result.

[0084] According to one embodiment of the present invention, the first layer construction module 701 can also be used to: collect facial image data sets and page behavior data of the user during the browsing process of the target page; construct a canvas according to the page behavior data; generate the user's eye movement trajectory according to the facial image data set and a preset standard image data set; and construct an eye movement heat map on the canvas according to the eye movement trajectory.

[0085] According to another embodiment of the present invention, the page behavior data includes a page scrolling distance of a user during a browsing process of a target page; when constructing a canvas according to the page behavior data, the first layer construction module 701 can also be used to: determine a canvas size according to the size of the page display window and a page scrolling distance during a browsing process of the target page, and construct a canvas according to the canvas size.

[0086] According to another embodiment of the present invention, when the first layer construction module 701 generates the eye movement trajectory of the user according to the facial image dataset and the preset standard image dataset, it can also be used to: determine the user's browsing position on the target page and the browsing order of each browsing position according to the facial image dataset and the preset standard image dataset; generate the eye movement trajectory of the user according to the browsing position and the browsing order of each browsing position.

[0087] According to another embodiment of the present invention, the Internet user experience monitoring device 700 also includes a sampling training module (not shown in the figure), which is used to: before constructing an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, determine whether the similarity between the user's sampled image data and preset standard image data meets a set threshold by performing calibration sampling and tracking process sampling on the user.

[0088] According to another embodiment of the present invention, when performing calibration sampling on the user, the sampling training module (not shown in the figure) can also be used to: divide the visible area of ​​the first sampling page into a specified number of grid areas; a total number of calibration points appear in the grid areas in a random manner, and only one calibration point appears at a time; for each calibration point, collect the first facial image data of the user each time the user looks at and clicks the calibration point through a network camera; and calculate the similarity between the first facial image data and the preset first standard image data.

[0089] According to yet another embodiment of the present invention, during the calibration sampling process, the network camera image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

[0090] According to another embodiment of the present invention, when calculating the similarity between the first facial image data and the preset first standard image data, the sampling training module (not shown in the figure) can also be used to: construct a first head posture feature based on the first facial image data; determine a first calibration point that the user gazes at and clicks based on the first head posture feature and the first standard posture feature, wherein the first standard posture feature is constructed based on the first standard image data; use the first standard image data corresponding to the first calibration point as the first standard image data corresponding to the first facial image data; and calculate the similarity between the first facial image data and the first standard image data through a similarity algorithm.

[0091] According to another embodiment of the present invention, when sampling the tracking process of the user, the sampling training module (not shown in the figure) can also be used to: divide the visible area of ​​the second sampling page into a specified number of grid areas; randomly position calibration points in a uniform displacement motion state in the grid areas to guide the user's line of sight to move along the position of the current calibration point on the screen; collect the second facial image data of the user's line of sight in a moving state through a network camera; and calculate the similarity between the second facial image data and the preset second standard image data.

[0092] According to yet another embodiment of the present invention, during the sampling process of the tracking process, the network camera image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

[0093] According to another embodiment of the present invention, when calculating the similarity between the second facial image data and the preset second standard image data, the sampling training module (not shown in the figure) can also be used to: construct a second head posture feature based on the second facial image data; determine the movement trajectory of a second calibration point that the user is gazing at based on the second head posture feature and the second standard posture feature, wherein the second standard posture feature is constructed based on the second standard image data; obtain the second standard image data corresponding to the second facial image data based on the movement trajectory of the second calibration point; and calculate the similarity between the second facial image data and the second standard image data through a similarity algorithm.

[0094] According to another embodiment of the present invention, the monitoring, analysis and optimization module 703 can also be used to: assign different weights to different areas of the target page according to the document tree structure of the target page and the semantics carried by the document nodes; perform color value conversion on the first layer to obtain the hue values ​​of the first layer in different areas of the target page; perform weight matching on the hue values ​​of the first layer in different areas of the target page according to the preset hue value ranges corresponding to different weights; and generate optimization analysis results based on the weight matching results.

[0095] According to the technical solution of the embodiment of the present invention, an eye movement heat map is constructed according to the facial image data set and page behavior data of the user in the browsing process of the target page, and the eye movement heat map of each user is fitted to obtain a first layer; a page screenshot is constructed according to the document tree structure and page data of the target page; the first layer and the page screenshot are fitted to obtain a user experience cloud map, image analysis is performed on the first layer to obtain an optimization analysis result, and the user experience cloud map and the optimization analysis result are used as the technical solution of the user experience monitoring result. An eye movement heat map can be constructed according to the collected facial image data of the user, and continuous image data can be provided for user experience optimization in an immersive and imperceptible manner. There is no need to use professional experimental equipment such as an eye tracker to collect data, which avoids interference of the experimental equipment to the user, and the data has high credibility and is convenient for collecting massive user experience data; at the same time, the head posture characteristics of the user can be covered according to the collected facial image data, so that the head posture of the user is more flexible, and the significance of the collected data will not be reduced due to changes in the head posture of the user, so that the data analysis results are more accurate; the first layer is obtained by fitting the massive data, and the regression calculation of the data gives an advantage that a single eye movement device cannot achieve, so that the credibility and significance of the data are improved, and the data analysis results are more accurate.

[0096] Figure 8 An exemplary system architecture 800 is shown to which the method or apparatus for monitoring Internet user experience according to an embodiment of the present invention may be applied.

[0097] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0098] Users can use terminal devices 801, 802, 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, 803, such as image processing applications, image acquisition applications, search applications, etc. (only examples).

[0099] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0100] Server 805 can be a server that provides various services, such as a background management server that provides support for shopping websites browsed by users using terminal devices 801, 802, and 803 (for example only). The background management server can construct an eye movement heat map based on the facial image data set and page behavior data of the user in the browsing process of the target page for the received user experience monitoring request, and fit the eye movement heat map of each user to obtain a first layer; construct a page screenshot based on the document tree structure and page data of the target page; fit the first layer and the page screenshot to obtain a user experience cloud map, perform image analysis on the first layer to obtain an optimization analysis result, process the user experience cloud map and the optimization analysis result as user experience monitoring results, and feed back the processing results (such as user experience cloud map, optimization analysis result - for example only) to the terminal device.

[0101] It should be noted that the method for monitoring Internet user experience provided in the embodiment of the present invention is generally executed by the server 805 , and accordingly, the device for monitoring Internet user experience is generally disposed in the server 805 .

[0102] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0103] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device or server suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0104] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0105] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0106] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0107] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The units or modules involved in the embodiments of the present invention may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a first layer construction module, a page screenshot construction module, and a monitoring and analysis optimization module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves. For example, the page screenshot construction module may also be described as "a module for constructing a page screenshot based on the document tree structure and page data of the target page".

[0110] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: constructing an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, and fitting the eye movement heat map of each user to obtain a first layer; constructing a page screenshot based on the document tree structure and page data of the target page; fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimization analysis result, and using the user experience cloud map and the optimization analysis result as the user experience monitoring result.

[0111] According to the technical solution of the embodiment of the present invention, an eye movement heat map is constructed according to the facial image data set and page behavior data of the user in the browsing process of the target page, and the eye movement heat map of each user is fitted to obtain a first layer; a page screenshot is constructed according to the document tree structure and page data of the target page; the first layer and the page screenshot are fitted to obtain a user experience cloud map, image analysis is performed on the first layer to obtain an optimization analysis result, and the user experience cloud map and the optimization analysis result are used as the technical solution of the user experience monitoring result. An eye movement heat map can be constructed according to the collected facial image data of the user, and continuous image data can be provided for user experience optimization in an immersive and imperceptible manner. There is no need to use professional experimental equipment such as an eye tracker to collect data, which avoids interference of the experimental equipment to the user, and the data has high credibility and is convenient for collecting massive user experience data; at the same time, the head posture characteristics of the user can be covered according to the collected facial image data, so that the head posture of the user is more flexible, and the significance of the collected data will not be reduced due to changes in the head posture of the user, so that the data analysis results are more accurate; the first layer is obtained by fitting the massive data, and the regression calculation of the data gives an advantage that a single eye movement device cannot achieve, so that the credibility and significance of the data are improved, and the data analysis results are more accurate.

[0112] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for monitoring Internet user experience, characterized in that: include: An eye movement heat map is constructed based on the facial image dataset and page behavior data of the user during the browsing process of the target page, and the eye movement heat map of each user is fitted to obtain the first layer; Constructing a page screenshot according to the document tree structure and page data of the target page; Fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimization analysis result, and using the user experience cloud map and the optimization analysis result as user experience monitoring results; Performing image analysis on the first layer to obtain an optimized analysis result includes: assigning different weights to different areas of the target page according to the document tree structure of the target page and the semantics carried by the document nodes; performing color value conversion on the first layer to obtain hue values ​​of the first layer in different areas of the target page; performing weight matching on the hue values ​​of the first layer in different areas of the target page according to preset hue value ranges corresponding to different weights; and generating an optimized analysis result according to the weight matching result.

2. The method according to claim 1, characterized in that The eye movement heat map is constructed based on the facial image dataset and page behavior data of users during their browsing of the target page, including: Collect facial image data sets and page behavior data of users while browsing the target page; Building a canvas according to the page behavior data; Generating an eye movement trajectory of the user according to the facial image dataset and a preset standard image dataset; An eye movement heat map is constructed on the canvas according to the eye movement trajectory.

3. The method according to claim 2, characterized in that The page behavior data includes the page scrolling distance of the user during the browsing process of the target page; A canvas is constructed according to the page behavior data, including: The canvas size is determined according to the size of the page display window and the page scrolling distance during the browsing process of the target page, and the canvas is constructed according to the canvas size.

4. The method according to claim 2, characterized in that: Generating the eye movement trajectory of the user according to the facial image dataset and a preset standard image dataset includes: Determining the browsing position of the user on the target page and the browsing order of each browsing position according to the facial image data set and the preset standard image data set; The eye movement trajectory of the user is generated according to the browsing position and the browsing order of each browsing position.

5. The method according to claim 1, characterized in that Before building an eye movement heat map based on the facial image dataset and page behavior data of users browsing the target page, it also includes: By performing calibration sampling and tracking process sampling on the user, it is determined that the similarity between the sampled image data of the user and the preset standard image data meets a set threshold.

6. The method according to claim 5, characterized in that Calibration sampling is performed on the user, including: Divide the visible area of ​​the first sampling page into a specified number of grid areas; A fixed number of calibration points appear in the grid area in a random manner, and only one calibration point appears at a time; For each calibration point, the first facial image data of the user each time the user gazes at and clicks the calibration point is collected through the network camera; The similarity between the first facial image data and the preset first standard image data is calculated.

7. The method according to claim 6, characterized in that During the calibration sampling process, the network camera image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

8. The method according to claim 6, characterized in that Calculating the similarity between the first facial image data and the preset first standard image data includes: constructing a first head posture feature according to the first facial image data; Determine a first calibration point that the user gazes at and clicks according to the first head posture feature and a first standard posture feature, wherein the first standard posture feature is constructed based on the first standard image data; using the first standard image data corresponding to the first calibration point as the first standard image data corresponding to the first facial image data; The similarity between the first facial image data and the first standard image data is calculated by a similarity algorithm.

9. The method according to claim 5, characterized in that Sampling the tracking process of the user includes: Divide the visible area of ​​the second sampling page into a specified number of grid areas; Calibration points in a uniform displacement motion state appear in random positions in the grid area, guiding the user's line of sight to follow the position of the current calibration point on the screen; Collecting second facial image data of the user in a moving sight state through a network camera; The similarity between the second facial image data and the preset second standard image data is calculated.

10. The method according to claim 9, characterized in that During the sampling process of tracking, the webcam image information is continuously output, and a fixed wireframe is used to limit and guide the range of movement of the user's head.

11. The method according to claim 9, characterized in that Calculating the similarity between the second facial image data and the preset second standard image data includes: constructing a second head posture feature according to the second facial image data; determining a moving trajectory of a second calibration point gazed at by the user according to the second head posture feature and a second standard posture feature, wherein the second standard posture feature is constructed based on the second standard image data; Acquire second standard image data corresponding to the second facial image data according to the moving trajectory of the second calibration point; The similarity between the second facial image data and the second standard image data is calculated by a similarity algorithm.

12. A device for monitoring Internet user experience, characterized in that: include: A first layer construction module is used to construct an eye movement heat map based on the facial image data set and page behavior data of the user during the browsing process of the target page, and to fit the eye movement heat map of each user to obtain the first layer; A page screenshot construction module, used to construct a page screenshot according to the document tree structure and page data of the target page; A monitoring analysis optimization module, used for fitting the first layer and the page screenshot to obtain a user experience cloud map, performing image analysis on the first layer to obtain an optimization analysis result, and using the user experience cloud map and the optimization analysis result as a user experience monitoring result; The monitoring, analyzing and optimizing module is further used to: assign different weights to different areas of the target page according to the document tree structure of the target page and the semantics carried by the document nodes; Performing color value conversion on the first layer to obtain hue values ​​of the first layer in different areas of the target page; performing weight matching on the hue values ​​of the first layer in different areas of the target page according to preset hue value ranges corresponding to different weights; And generate optimization analysis results based on the weight matching results.

13. An electronic device for monitoring Internet user experience, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images