Prediction method and system for possibility distribution of target point selection and landing area with arbitrary shape

Abstract

The present invention provides a method and system for predicting the possibility distribution of the landing point area of an arbitrarily shaped target point selection, belonging to the field of computer graphical user interfaces. A target with an arbitrary shape is selected on the displayed interface content, and the centroid coordinates of the target are obtained; the largest inscribed circle algorithm for polygons is used to find the largest inscribed circle in the target, and then the largest inscribed circle is cut out from the original target, and the target is iteratively divided into multiple inscribed circle regions; the center coordinates and radii of each inscribed circle region are obtained, and the probability of the user clicking on the target and falling into each inscribed circle region is calculated. The present invention aims at the complex polygon target selection task on a plane, divides the area by cutting the largest inscribed circle, calculates the central area of the region and the distance to the centroid of the target, and predicts the possibility distribution of the landing point falling into any region, and the prediction is simpler and more efficient.

Description

Technical Field

[0001] The present invention belongs to the field of computer graphical user interfaces, and particularly relates to a method and system for predicting the possibility distribution of the landing area of a target point selection for an arbitrary shape in a graphical user interface. Background Art

[0002] With the popularization of applications such as large-scale online games, augmented and virtual reality systems, etc., the scenarios where users need to select targets are becoming more and more numerous. For example, first-person shooter games, interactions in the metaverse, and so on. At the same time, with the development and progress in the fields of computer graphics, animation design, etc., the user's screen is no longer simply a list of basic graphics such as rectangles and circles, but is composed of various irregular and complex shapes. For example, in a shooter game, different user characters often correspond to different irregular external contours. In the above-mentioned scenarios, the size and shape factors of the target will have a great impact on the landing distribution when the user performs the target selection task. Therefore, in order to assist the work of interface designers, a simple and clear method for predicting the user's point selection landing distribution of an arbitrary shape target is needed.

[0003] In real interaction scenarios, the selected targets often have bounded sizes in two dimensions. Therefore, many studies have explored the influence of two-dimensional target shapes on users' performance when selecting targets. MacKenzie and Buxton proposed five representations of target width considering the different angles at which users click on the target, thus extending Fitts' law, which was originally only suitable for one-dimensional target selection tasks, to two-dimensional space (Reference: MacKenzie, S. and Buxton, W. (1992). Extending Fitts' law to two-dimensional tasks. ACM CHI Conference on Human Factors in Computing Systems. p. 219-226.). Further, Accot et al. found that for a two-dimensional target of a fixed size, the target has different effects on the amplitude constraint W and the direction constraint H when users perform selection operations, and the amplitude constraint W is more dominant than the direction constraint H. Therefore, they proposed a Euclidean model with a free weight coefficient η to balance these two different effects (Reference: Johnny Accot and Shumin Zhai. 2003. Refining Fitts' law models for bivariate pointing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'03). 193-200.).

[0004] One point that needs to be emphasized is that the research focus of these above-mentioned works is basically concentrated on the task of clicking on rectangular targets. However, with the development of computer graphical interfaces in recent years, it has become very common to click on targets with arbitrary irregular shapes in current user interfaces. Sheikh et al. found in their research on Fitts' law that the shape of the target has a significant impact on the movement time for users to select the target. They used a two-dimensional normal distribution and a "cookie-cutter" method to estimate the shape of the target and significantly improved the accuracy of Fitts' law in predicting the user's click time for targets of different shapes (Reference: Sheikh, I.H., & Hoffmann, E.R. (1994). Effect of target shape on movement time in a Fitts task. Ergonomics, 37(9), 1533-1547). In another notable study, Tovi Grossman et al. proposed a probability-based Fitts' law model, which can relatively accurately predict the movement time required for users to click on a large number of non-rectangular targets in a static target selection task. The core idea of this probability-based Fitts' law model is to map the probability of hitting the target in open-loop movement to the difficulty value index of the original Fitts' law (Reference: Tovi Grossman, Nicholas Kong, and Ravin Balakrishnan. 2007. Modeling pointing at targets of arbitrary shapes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). 463-472.).

[0005] However, none of the above studies on target selection tasks can be directly applied to practical applications. This is mainly due to two reasons. First, these models are all too complex. For example, Grossman's model requires integrating the enclosed area of an object with an arbitrary shape, which is a very difficult operation. Second, although it has been verified that the distribution of the user's click landing points on objects with regular shapes (such as circles and rectangles) conforms to the normal distribution, for objects with irregular shapes, the landing point distribution of the user is very likely to be the superposition of multiple distributions. However, most of the above models' premise assumptions are that the landing points conform to the two-dimensional normal distribution. Therefore, it is necessary to propose a model that can directly give the set of divided regions and the possibility of the landing point falling into any region for any target shape. This model has important guiding significance and practical significance for the development and design of application software with the task of selecting complex polygon targets. Summary of the Invention

[0006] The object of the present invention is to propose a method and system for predicting the possibility distribution of the click landing point area of an object with an arbitrary shape. For the complex polygon target selection task on a plane, the area is divided by cutting the largest inscribed circle, the central area of the region and the distance to the centroid of the target are calculated, and the possibility distribution of the landing point falling into any region is predicted.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for predicting the possibility distribution of the click landing point area of an object with an arbitrary shape, the steps of which include:

[0009] Select an object with an arbitrary shape on the displayed interface content to obtain the centroid coordinates of the object;

[0010] Use the polygon largest inscribed circle algorithm to find the largest inscribed circle in the object, then cut out the largest inscribed circle from the original object, and then find the largest inscribed circle in the remaining figure and cut it out from the figure. Iterate in this way to divide the object into multiple inscribed circle regions;

[0011] Obtain the central coordinates and radii of each inscribed circle region, calculate the distance from the center of each inscribed circle region to the centroid according to the central coordinates and the centroid coordinates, and calculate the probability that the user clicks on the object and falls into each inscribed circle region according to the radius of the inscribed circle region and the distance from the center to the centroid.

[0012] Further, the centroid coordinates of the object are the coordinates in the world coordinate system.

[0013] Further, the polygon largest inscribed circle algorithm is the Max Inscribed Circles algorithm used in ImageJ.

[0014] Further, find the largest inscribed circle in the remaining figure and cut it out from the figure, and iterate in this way until the area of the remaining figure is less than 20% of the area of the original target image.

[0015] Further, the distance from the center of the inscribed circle area to the centroid includes the distances in the tangent and normal directions between the center of the inscribed circle area and the centroid.

[0016] Further, calculate the probability p that the user clicks on the target and falls into each inscribed circle area i The formula for is:

[0017]

[0018] In the formula, r i represents the radius of the inscribed circle area, d t,i and d n,i respectively represent the distances in the tangent and normal directions between the center of the inscribed circle area and the centroid; C is a normalization constant, where K represents the number of inscribed circle areas, and c1, c2, c3 are constant parameters;

[0019] Further, the coefficients a and b are obtained by fitting according to empirical data, and the steps include:

[0020] Specify the target size and shape as experimental conditions;

[0021] Present the initial display interface to the user, record the data of the user's multiple selected landing positions as the empirical data under this experimental condition, and repeat to obtain the empirical data under multiple experimental conditions;

[0022] Record the ratio of the number of actual user landings on each inscribed circle area of the target to the total number of user landings on the target;

[0023] Construct the minimum mean square error between the ratio of the number of actual user landings on each inscribed circle area and the predicted probability calculated by the formula, and then use the gradient descent algorithm to optimize and fit the data to obtain the values of the coefficients a and b.

[0024] A prediction system for the possibility distribution of the landing area of an arbitrarily shaped target point selection includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the program, the steps of the above method are implemented.

[0025] The positive effects of the present invention are as follows: The method of the present invention only needs to know the shape and size of the target on the system interface to predict the probability that the landing point when the user clicks on the target within the system interface falls into any area of the target, without the need to modify the appearance of the original interface or for the user to perform additional operations; the present invention is applicable not only to targets with regular shapes such as circles and rectangles, but also to any irregular shapes. The present invention relies on the equipment, environment, and scenario when performing tasks on the interface. Using the method of the present invention to coarsely predict the click landing point distribution of targets with arbitrary shapes is simpler and more efficient, and can well assist interface designers in completing development tasks. The present invention predicts the landing area of the user, and the final feedback given is the probability that the user falls into each area when clicking on the target. Compared with the prior art that uses a ternary Gaussian model to obtain probability distributions on each area and uses a Gaussian mixture model to superimpose multiple probabilities for complex calculations, the present invention can simply calculate the result through a formula based on the area and distance relationships of each part. The calculation process is simple, easy to operate, and can be directly applied to actual scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 FIG. is a process diagram of cutting a target with an arbitrary shape into multiple regions in an embodiment of the present invention;

[0027] Figure 2 FIG. is a schematic diagram of the distance from the center of the region to the centroid in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] In order to make the above features and advantages of the present invention more obvious and understandable, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0029] This embodiment provides a method for predicting the possibility distribution of the click landing area of a target with an arbitrary shape, and the steps are as follows:

[0030] 1) Display the interface content, select a target (graphic) with an arbitrary shape, and obtain the centroid coordinates of the target;

[0031] 2) Iteratively use the maximum inscribed circle algorithm to divide the complex target into a set of multiple inscribed circle regions;

[0032] 3) Obtain the center coordinates and radii of each inscribed circle region of the target, calculate the distance from the center coordinates to the centroid coordinates, and substitute the distance and radius information into the probability function formula to obtain the probability that the landing point falls into each region.

[0033] In step 1), the interface content is the content displayed by a general interactive system. The specific content form is not limited in this embodiment. Among them, the centroid coordinates are the coordinates C(c x , c y ) in the world coordinate system.

[0034] In step 2), in order to achieve the purpose of dividing the target into a set of multiple regions, the Max Inscribed Circles algorithm is used to obtain the maximum inscribed circle of a shape. This algorithm is an algorithm implementation of the maximum inscribed circle based on the Euclidean distance mapping. In order to divide the target into multiple basic figures, first a) execute the Max Inscribed Circles algorithm to obtain the maximum inscribed circle of a target and use this circle as a basic figure of the target, b) then cut out this inscribed circle from the target, and then use the remaining part of the target in step a) as a new target. Repeat the above steps until the area of the new target is less than about 20% of the original target. The process of dividing the target is as Figure 1 shown, where the numerical label on each circle represents the generation order in the division process.

[0035] In step 3), the radius of each inscribed circle region is represented by ri, and the center coordinates of the region are expressed as A(a x , a y ), d t,i and d n,i respectively represent the distances between the center of the region and the centroid of the target in the tangent and normal directions (as Figure 2 shown). The specific formulas are as follows:

[0036] d t,i = |c x - a x |

[0037] d n,i = |c y - a y |

[0038] The probability function designed in step 3) is related to two shape factors. One element related to the shape is the area of the region divided in step 2). Generally speaking, if a region has a larger area, it often means that there is a greater possibility that more landing points will fall here. Therefore, their weights should logically be greater than those of other regions. Another factor related to the shape is the distance between the center of the region and the centroid of the target. According to the findings of the previous work by Grossman et al., the centroid of the polygon of a shape can be regarded as the aiming center when the user clicks on the target. At the same time, previous research has also found that compared with clicking on the edge region of a target, the subjects participating in the experiment tend to click on the center region of the target more. Based on these works, the distance between the center of any region and the centroid of the target is projected onto the X-axis and Y-axis of the coordinate system (i.e., the tangent and normal directions).

[0039] Therefore, a probability function as shown in the following formula is proposed. Using this formula, it is only necessary to substitute the radius of each region of the target and the distance to the centroid to obtain the probability distribution of the possible landing points of the target with any shape. The numerator in the formula consists of three parts: the area of the inscribed circle and the distances in the x-axis and y-axis directions between the center of the region and the polygon centroid of the target. The denominator is a normalization process, and the purpose of doing this is to ensure that the sum of the probabilities of the landing points falling into each region is 1.

[0040]

[0041] In the formula, r i represents the radius of the inscribed circle region, d t,i and d n,i respectively represent the distances in the tangent and normal directions between the center of the inscribed circle region and the centroid of the target. C is a normalization constant, and it can be expressed as K represents the number of inscribed circles. The purpose of doing this is to ensure that the sum of the probabilities of a landing point falling into all regions of the target is 100%. c1, c2, c3 are constant parameters. During the process of dealing with normalization, the coefficients of d t,i and d n,i are changed to and Because c2, c3, c1π are all constants, new coefficients a, b are respectively used to replace and That is The values of a and b can be obtained by fitting with empirical data.

[0042] Empirical data refers to, for any specific user display interface, before using the method of the present invention, the selection landing point data obtained by the user repeatedly selecting the target in the interface. The acquisition of empirical data is completed through a user experiment, which follows the general steps and criteria of a human-computer interaction user experiment. For the convenience of understanding, it is briefly described as follows:

[0043] i. Match the interface content with the target definition and the space where it is located.

[0044] ii. According to the interface content, specify the shape and size of a target as the experimental condition.

[0045] iii. Present it to N users in the original form of the interface and require them to repeatedly select the target under this condition M times using the original interface system.

[0046] iv. According to the definition of the target and space that match the interface, record the positions of all M selection landing points to obtain the empirical data under the experimental condition.

[0047] v. Repeat the above steps ii to iv to obtain empirical data under other experimental conditions.

[0048] The design and implementation process of the experiment need to consider the influence of factors such as order effect, learning effect, fatigue level, user differences, and sample size. The general principles of human-computer interaction experiment design can be followed to eliminate the influence of these factors.

[0049] For example, according to the size and shape of the specific interface target, 10 target radii and 3 target sizes, a total of 10×3 = 30 experimental conditions can be selected for the above experiment. The number of users N can be 12, and the number of times M that a user repeatedly selects under the same condition can be 30. Thus, 12×30 = 360 selection landing points can be obtained under one condition, forming empirical data composed of 30×360 = 10800 selection landing points.

[0050] The following steps for estimating the model constants by fitting the above empirical data using the model are given:

[0051] i. For a specified experimental condition (fixed target size and shape), divide it into a set of multiple regions according to the above region division method, and then calculate the proportion of the number of landing points in each region to the total number of landing points when the user clicks on the target, and calculate the radius of each region and the distance to the centroid of the target.

[0052] ii. For all other experimental conditions, repeat step i above to obtain the true proportion of user landing points in any region of the target and the probability predicted by the probability model under any condition.

[0053] iii. Construct the minimum mean square error between the true proportion of landing points in each region of the target in all cases and the model predicted probability, and then use the gradient descent algorithm for optimization fitting to obtain the values (estimated values) of the coefficients a and b in the formula.

[0054] Substitute the values of these coefficients into the formula for calculating the probability of landing points in each region, and the probability that the user clicks on the target and lands in any region after dividing the target into regions of any shape can be obtained using this formula.

[0055] Verify the effectiveness of the method of the present invention through the following experiment:

[0056] A total of 12 subjects participated in the entire experiment. The code ran on a Surface Pro 4, and an external 23.8-inch AOC monitor with a resolution of 1920×1080 was used. The mouse used in the experiment was a Dell ms116t. To test the robustness of the method of the present invention under complex conditions, this experiment included three target sizes (96, 192, 384 pixels), three target speeds (96, 192, 384 pixels / second), and 10 target shapes. In order to complete this experiment, each subject needed to perform 10 shapes × 3 sizes × 3 speeds × 10 repetitions = 900 click tasks. In each click task, the subject needed to click the start button in the center of the interface. After that, a target with a specified shape and size would appear on the screen and move in a random direction at a specified speed. In each click task, the subject had only one chance to complete the click. Whether or not the target was hit, the click coordinates would be recorded. Throughout the experiment, 12×900 = 10800 landing points would be generated. To offset the influence of the target speed, when calculating the number of landing points in each area, all landing points would first be moved a certain distance along the direction of the target speed (the distance between the center of the landing point distribution and the centroid of the target).

[0057] The results of the correlation analysis showed that the area of each region and the distance between the center of the region and the centroid of the target had a significant correlation with the number of landing points in the region in the tangential and normal directions of the speed, respectively. The probability of the landing points calculated by the probability function of the present invention falling into each region also had a significant correlation with the number of landing points in the region. The Pearson correlation coefficient between the area of the region and the number of landing points was 0.675 (p<0.001). The distance d t between the center of the region and the centroid of the target in the tangential direction of the speed and the Pearson correlation coefficient value of the number of landing points in the region was -0.211 (p<0.001). The distance d n between the center of the region and the centroid of the target in the normal direction of the speed and the Pearson correlation coefficient value of the number of landing points in the region was -0.295 (p<0.0001). In addition, the Pearson correlation coefficient between the probability of the landing points calculated by the probability function falling into each region and the number of landing points in the region was 0.937 (p<0.001), which also verified the reliability and rationality of the probability function in predicting the landing point distribution of user clicks.

[0058] The method of the present invention has been described in detail above through formal expressions and embodiments, but the specific implementation form of the present invention is not limited to this. Those of ordinary skill in the art can make various obvious changes and modifications without departing from the spirit and principles of the method described in the present invention. The protection scope of the present invention shall be subject to what is described in the claims.

Claims

1. A method for predicting the probability distribution of the landing area of a target point selection with an arbitrary shape, characterized in that the steps Comprising: Select a target with an arbitrary shape on the displayed interface content to obtain the centroid coordinates of the target; Use the polygon maximum inscribed circle algorithm to find the maximum inscribed circle in the target, then cut out the maximum inscribed circle from the original target, and then find the maximum inscribed circle in the remaining figure and cut it out from the figure. Iterate in this way to divide the target into multiple inscribed circle regions; Obtain the center coordinates and radius of each inscribed circle region, calculate the distance from the center of each inscribed circle region to the centroid according to the center coordinates and the centroid coordinates, where the distance includes the distances in the tangent and normal directions between the center of the inscribed circle region and the centroid, and calculate the probability that the user clicks on the target and falls into each inscribed circle region according to the radius of the inscribed circle region and the distance from the center to the centroid; calculate the probability p that the user clicks on the target and falls into each inscribed circle region i The formula is as follows: where r i represents the radius of the inscribed circle region, d t,i and d n,i respectively represent the distances in the tangent and normal directions between the center of the inscribed circle region and the centroid; C is a normalization constant, where K represents the number of inscribed circle regions, and c1, c2, c3 are constant parameters; 2. The method according to claim 1, wherein The centroid coordinates of the target are coordinates in the world coordinate system.

3. The method according to claim 1, characterized in that, The polygon maximum inscribed circle algorithm is the Max Inscribed Circles algorithm adopted in ImageJ.

4. The method according to claim 1, wherein Find the maximum inscribed circle in the remaining figure and cut it out from the figure. Iterate in this way until the area of the remaining figure is less than 20% of the area of the original target image.

5. The method according to claim 1, characterized in that, The coefficients a and b are obtained by fitting according to empirical data. The steps include: Specify the target size and shape as experimental conditions; Present the initial display interface to the user, record the data of the user's multiple selected landing point positions as the empirical data under this experimental condition, and repeat to obtain the empirical data under multiple experimental conditions; Record the ratio of the actual number of user landing points on each inscribed circle region of the target to the total number of user landing points when the user clicks on the target; Construct the least mean square error between the ratio of the actual number of user landing points on each inscribed circle region and the predicted probability calculated by the formula, and then use the gradient descent algorithm to optimize and fit the data to obtain the values of the coefficients a and b.

6. A prediction system for the possibility distribution of the landing area of an arbitrarily shaped target point selection, characterized in that, Including a memory and a processor, a computer program is stored on the memory, and when the processor executes the program, the steps of the method described in any one of claims 1-5 are implemented.

Images