User operation behavior prediction method and device, medium and equipment

Abstract

The invention discloses a user operation behavior prediction method and device, a medium and equipment, and belongs to the field of behavior prediction.The user operation behavior prediction method comprises the steps that user interface layout data containing preset semantic attributes, positions and size information is obtained, and semantic strategy frequency features and visual features are extracted from the user interface layout data; the features respectively reflect the distribution and visual presentation rules of the components. And inputting the characteristics into a prediction model which is finely adjusted based on large-scale historical user behavior data and is trained in combination with a man-machine interaction multi-step analysis framework, and analyzing the response of the user to interface manipulation so as to output a user behavior prediction result. The problem that the operation behavior of the user cannot be accurately and efficiently predicted in the prior art is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of behavior prediction, and more particularly to a method, apparatus, medium, and device for predicting user operation behavior. Background Technology

[0002] In the field of Human-Computer Interaction (HCI), the user interface (UI), as the core carrier of user interaction with digital products, directly impacts user decision-making autonomy and user experience. However, current UI designs commonly suffer from "dark patterns," where deliberately designed interface strategies (such as false urgency prompts and forced subscription guidance) manipulate users into making decisions against their will. This not only infringes on users' right to know and autonomy but may also trigger negative user emotions, erode brand trust, and even impact the ethics and transparency of the digital service ecosystem. Therefore, accurately identifying and avoiding potential manipulative effects during the UI design phase has become a critical issue that the industry urgently needs to address.

[0003] In existing technologies, most methods for assessing the manipulative effects of UI design are reactive and lagging approaches, which are insufficient to meet the real-time decision-making needs of the design phase. A / B testing relies on real online traffic, which is not only costly and has a long iteration cycle, but also unsuitable for agile development models. It can only verify the merits of pre-set solutions and cannot explain the deeper reasons behind user decisions. Laboratory user studies are limited by small sample sizes, resulting in insufficient generalizability. Furthermore, recruiting users and conducting experiments are time-consuming and labor-intensive. Additionally, the "laboratory effect" may cause user behavior to deviate from real-world usage scenarios, limiting the reference value of the evaluation results. Expert heuristic evaluation, on the other hand, relies heavily on the evaluator's personal experience, is highly subjective, and lacks quantifiable and reproducible predictive capabilities. It is difficult to effectively scale up for large-scale, complex UI design scenarios and cannot provide designers with stable and scientific decision support.

[0004] Furthermore, existing methods fail to provide a "dual-dimensional" interpretation of user decisions. They cannot predict potential user behavior strategies in real time (such as whether to convert or avoid), nor can they reveal users' cognitive attributions of their own behavior (i.e., what design factors users believe influence their behavior). This deficiency results in designers lacking effective risk warning tools during the creative phase, forcing them to make adjustments only after the UI goes live, based on user feedback or data review. This not only increases development costs and timelines but also carries the risk of regulatory penalties due to manipulative designs that have already been implemented.

[0005] These shortcomings prevent existing technologies from accurately and efficiently predicting user behavior. Summary of the Invention

[0006] This invention provides a method, apparatus, medium, and device for predicting user operation behavior, in order to solve the problem that existing technologies cannot accurately and efficiently predict user operation behavior.

[0007] Firstly, this application provides a method for predicting user operation behavior, including: Obtain the layout data of the user interface during the design phase; the layout data includes user interface components with preset semantic attributes, position information of user interface components, and size information of user interface components; Semantic strategy frequency features and visual features are extracted from the layout data; wherein, the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, and the visual features correspond to the visual presentation pattern of user interface components. The semantic strategy frequency features and visual features are input into a preset prediction model so that the prediction model can analyze the response of the user interface operation and output the user behavior prediction result. The prediction model is based on parameter fine-tuning of large-scale historical user behavior data and trained on a pre-set multi-step analysis framework in the field of human-computer interaction.

[0008] This application captures key features of the user interface design in real time by acquiring layout data of the user interface during the design phase, including components with preset semantic attributes, position information, and size information. Furthermore, it extracts semantic strategy frequency features and visual features from this layout data. The semantic strategy frequency features reflect the distribution patterns of components with preset semantic attributes, while the visual features reveal the visual presentation patterns of the components. These features are input into a predictive model fine-tuned with large-scale historical user behavior data and trained based on a preset multi-step analysis framework for human-computer interaction. This model can perform in-depth analysis of user responses to user interface manipulation, thereby outputting user behavior prediction results. This process not only achieves real-time prediction of the user interface manipulation effect from the design phase but also provides designers with quantitative and scientific design feedback through scientific data analysis and machine learning techniques, helping them identify and avoid potential "dark modes" before development. This application effectively solves the problem that existing technologies cannot accurately and efficiently predict user operation behavior.

[0009] Furthermore, the acquisition of the layout data of the user interface during the design phase specifically involves: Monitor user actions on the interactive design canvas, obtain the current state of the interactive design canvas after each user action, and generate user interface layout data for each state. The user operations include drag-and-drop, move, and delete operations on user interface components. Based on the user interface layout data in each state, the preset semantic attributes, position coordinate information and size parameter information of the user interface components are recorded. The preset semantic attributes include false urgency, false scarcity, false social proof, and forced subscription.

[0010] This application monitors user actions on an interactive design canvas, acquiring the canvas's current state in real time after each action and generating corresponding user interface layout data. These actions include dragging, dropping, moving, and deleting user interface components. Based on this layout data, the system records the components' preset semantic attributes (such as false urgency, false scarcity, false social proof, and forced subscription), position coordinates, and size parameters. This process enables the system to dynamically capture changes in the interface layout during the design phase, providing real-time and accurate data support for subsequent prediction of manipulative effects. In this way, designers can instantly understand the potential impact of interface design, adjust the layout in advance to avoid poor design patterns, thereby improving design quality and user experience, while reducing business risks and regulatory issues caused by inappropriate design.

[0011] Furthermore, the step of extracting semantic strategy frequency features from the layout data; wherein the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, specifically: After filtering out all user interface components with preset semantic attributes from the layout data, the filtered user interface components are classified according to the type of preset semantic attributes, and the number of user interface components corresponding to each type of preset semantic attribute is counted. The number of components corresponding to various preset semantic attributes obtained from statistics is used as the frequency feature of semantic strategies.

[0012] This application provides crucial data support for predicting the manipulative effects of user interfaces by extracting semantic strategy frequency features from layout data. Specifically, the system first filters all user interface components with preset semantic attributes (such as false urgency, false scarcity, etc.) from the layout data and categorizes them according to the type of these attributes. Next, the system counts the number of components corresponding to each preset semantic attribute category and uses these numbers as semantic strategy frequency features. This process allows the system to quantify the frequency of use of potential manipulative strategies in interface design, providing accurate input data for subsequent predictive models. In this way, designers can intuitively understand the distribution of different manipulative strategies in interface design, thereby adjusting and optimizing the interface layout in a timely manner during the design phase, avoiding potential bad design patterns, and improving the ethical aspects of the design and the user experience.

[0013] Furthermore, the extraction of visual features from the layout data specifically includes: Calculate the ratio of the total area of ​​all user interface components in the layout data to the total area of ​​the interactive design canvas to obtain the visual size characteristics; Divide the interactive design canvas into a preset grid and determine the focal area and discrete areas within the grid; The number of user interface components falling into the focus area is counted to obtain visual focus characteristics; The number of user interface components falling into discrete regions is statistically analyzed to obtain visual dispersion characteristics. The visual size feature, visual focus feature, and visual dispersion feature are integrated to obtain the visual feature.

[0014] This application provides a quantitative basis for evaluating the visual effects of user interface design by extracting visual features from layout data. Specifically, the system first calculates the ratio of the total area of ​​all user interface components to the total area of ​​the interactive design canvas, obtaining a visual size feature that reflects the visual proportion of the interface components in the overall canvas. Next, the system divides the interactive design canvas into a preset grid and identifies focal and discrete regions within it. By counting the number of components falling into the focal region, a visual focus feature is obtained, which helps assess which components attract the user's attention most. Simultaneously, the number of components falling into the discrete regions is counted, obtaining a visual dispersion feature to measure the uniformity of component distribution. Finally, the visual size feature, visual focus feature, and visual dispersion feature are integrated to form a complete visual feature set. This process not only provides designers with a quantitative evaluation of interface visual design but also helps them optimize interface layout during the design phase, improve user experience, and avoid user manipulation effects caused by inappropriate visual design.

[0015] Furthermore, the prediction model is fine-tuned based on large-scale historical user behavior data and trained using a pre-defined multi-step analysis framework in the field of human-computer interaction, specifically as follows: Acquire historical user behavior data; the historical user behavior data includes multiple user interface design schemes and user behavior records corresponding to each design scheme. A pre-defined multi-step analysis framework for the human-computer interaction domain is constructed; the multi-step analysis framework includes rules relating semantic attributes to user behavior; The historical user behavior data is input into a preset initial prediction model, so that the initial prediction model is combined with the multi-step analysis framework and fine-tuned according to the fine-tuning technique to obtain the prediction model.

[0016] This application significantly improves the accuracy and reliability of predicting the manipulative effects of user interfaces by constructing a predictive model fine-tuned based on large-scale historical user behavior data. Specifically, the system first collects historical data containing multiple sets of user interface design schemes and their corresponding user behavior records. This data provides rich training material for the model, covering various design patterns and their actual impact on user behavior. Next, the system constructs a pre-defined multi-step analysis framework in the human-computer interaction domain, which defines in detail the association rules between semantic attributes and user behavior. These rules provide the theoretical basis for the model to analyze and understand user behavior. Subsequently, the system inputs historical user behavior data into the pre-defined initial predictive model, enabling the model to combine with the multi-step analysis framework and adjust parameters through fine-tuning techniques. This process not only optimizes the model's predictive performance but also enables it to more accurately capture the complex psychological motivations behind user behavior. The final predictive model can predict the manipulative effects of user interface design in real time and accurately, providing designers with scientific and quantitative decision support.

[0017] Furthermore, the step of inputting the semantic strategy frequency features and visual features into a preset prediction model, so that the prediction model analyzes the response to user interface manipulation and outputs user behavior prediction results, specifically involves: The extracted semantic strategy frequency features and visual features are converted into a structured input format that the prediction model can recognize; The semantic strategy frequency features and visual features of the structured input format are input into a preset prediction model, so that the prediction model can perform inference analysis on the input features according to the preset multi-step analysis framework of the human-computer interaction field, determine the user's response trend to the user interface manipulation, and generate user behavior prediction results based on the response trend. The user behavior prediction results include user behavior strategy levels and cognitive attribution scores. The user behavior strategy levels include avoidance, exploration and verification inquiry, acceptance of proposals, and conversion. The cognitive attribution scores include semantic attribution scores, visual size attribution scores, and visual location attribution scores.

[0018] This application achieves accurate analysis of user responses to user interface manipulation by converting extracted semantic strategy frequency features and visual features into a structured input format recognizable by the prediction model and then inputting it into a pre-defined prediction model. Specifically, the system first converts these features into a structured input format to ensure the model can effectively process the data. Subsequently, these structured features are input into the prediction model, which performs inference analysis based on a pre-defined multi-step analysis framework in the human-computer interaction domain. This framework defines in detail the association rules between semantic attributes and user behavior, enabling the model to determine user response trends to user interface manipulation and generate user behavior prediction results based on these trends. The prediction results include not only user behavior strategy levels (such as avoidance, exploration and verification inquiry, acceptance of proposals, and conversion) but also cognitive attribution scores (such as semantic attribution scores, visual size attribution scores, and visual position attribution scores). This dual prediction dimension not only predicts user behavioral tendencies but also reveals the psychological motivations behind user behavior, providing designers with comprehensive and in-depth feedback. This allows designers to adjust and optimize interface layouts in a timely manner during the design phase, avoiding potential poor design patterns.

[0019] Secondly, this application provides a user operation behavior prediction device, the user operation behavior prediction device comprising: The acquisition module is used to acquire layout data of the user interface during the design phase; the layout data includes user interface components with preset semantic attributes, position information of the user interface components, and size information of the user interface components. An extraction module is used to extract semantic strategy frequency features and visual features from the layout data; wherein, the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, and the visual features correspond to the visual presentation pattern of user interface components. The prediction module is used to input the frequency features and visual features of the semantic strategy into a preset prediction model, so that the prediction model can analyze the response of the user interface operation and output the user behavior prediction result. The prediction model is based on parameter fine-tuning of large-scale historical user behavior data and trained on a pre-set multi-step analysis framework in the field of human-computer interaction.

[0020] This user behavior prediction device, through the collaborative operation of three core modules, achieves real-time prediction of the manipulative effects of user interfaces, providing designers with scientific and quantitative decision support. Specifically, the acquisition module collects layout data of the user interface during the design phase. This data includes components with preset semantic attributes, location information, and size information, providing a foundation for subsequent analysis. The extraction module extracts semantic strategy frequency features and visual features from this layout data. The semantic strategy frequency features reflect the distribution patterns of components with preset semantic attributes, while the visual features reveal the visual presentation patterns of the components. The prediction module inputs these features into a preset prediction model. This model is fine-tuned based on large-scale historical user behavior data and trained using a preset multi-step analysis framework in the field of human-computer interaction. It can perform in-depth analysis of user responses to user interface manipulation and output prediction results including user behavior strategy levels and cognitive attribution scores. This device not only improves decision-making efficiency during the design phase but also helps designers identify and avoid potential poor design patterns, enhance user experience and design ethics, and reduce business risks and regulatory issues caused by inappropriate design.

[0021] Thirdly, this application provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a user operation behavior prediction method as described above. Its beneficial effects are the same as those of the user operation behavior prediction method provided in the first aspect of this application.

[0022] Fourthly, this application provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a method for predicting any of the user operation behaviors described in the first aspect. Attached Figure Description

[0023] Figure 1 : A schematic flowchart of an embodiment of the user operation behavior prediction method provided in this application; Figure 2 : A schematic diagram of an embodiment of the user operation behavior prediction device provided in this application. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1 Please refer to Figure 1 In order to solve the problem that existing technologies cannot accurately and efficiently predict user operation behavior, this invention provides a method for predicting user operation behavior, including steps S01-S03.

[0026] S01: Obtain the layout data of the user interface during the design phase; the layout data includes user interface components with preset semantic attributes, the position information of the user interface components, and the size information of the user interface components.

[0027] In a preferred embodiment of this invention, the step of obtaining the layout data of the user interface during the design phase specifically involves: In this embodiment, the real-time prediction process of the user interface manipulation effect begins with the acquisition of user interface layout data during the design phase. This process relies on an operating environment that integrates an interactive design canvas and a library of preset attribute components. When conducting UI design, designers can select user interface components with clear semantic attributes from the component library. For example, a "countdown timer" component to create time pressure (corresponding to the semantic attribute of false urgency), a "X items remaining in stock" component to indicate limited resources (corresponding to the semantic attribute of false scarcity), a sign component to display "X people have already selected" (corresponding to the semantic attribute of false social proof), and a subscription button component with "auto-renewal" checked by default (corresponding to the semantic attribute of forced subscription). These components can then be arranged and adjusted on the design canvas through drag-and-drop, moving, or deleting operations.

[0028] This application monitors the state changes of the design canvas in real time every time a designer performs a component operation (including adding, moving, or deleting a component). Upon detecting the completion of the operation, it immediately captures the complete state of the current canvas to generate corresponding layout data. This layout data records three core types of information for each user interface component on the canvas: first, the component's preset semantic attributes, clearly indicating the semantic strategy type to which the component belongs (such as false urgency, false scarcity, etc.); second, the component's position information, establishing a Cartesian coordinate system with the top-left corner of the design canvas as the origin, recording the X-axis and Y-axis coordinates of the component's top-left corner vertex to accurately locate the component's specific position on the canvas; and third, the component's size information, recording the component's width and height values ​​(in pixels), or calculating the proportion of the component's total area to the total area of ​​the design canvas to form quantifiable size parameters. For example, when a designer places a "countdown timer" component in the center of the canvas, the system-generated layout data records the component's semantic attribute as "false urgency," its position coordinates as (300, 200) (assuming a canvas resolution of 800×600 pixels), and its size parameters as width 150 pixels, height 80 pixels, or a total area percentage of 0.03125 (150×80÷(800×600)). Through this method, the system can completely acquire user interface layout data for each state in the design phase, providing fundamental data support for subsequent feature extraction and manipulation effect prediction.

[0029] S02: Extract semantic strategy frequency features and visual features from the layout data; wherein, the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, and the visual features correspond to the visual presentation pattern of user interface components.

[0030] In a preferred embodiment of this invention, the extraction of semantic strategy frequency features and visual features from the layout data; wherein the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, and the visual features correspond to the visual presentation pattern of user interface components, specifically: After obtaining the user interface layout data during the design phase, this application will initiate a feature extraction process to extract semantic strategy frequency features and visual features from the layout data, respectively, in order to quantify the manipulation-related patterns in user interface design.

[0031] To extract the frequency features of semantic strategies, this application first filters out all user interface components with preset semantic attributes from the layout data, and then classifies the filtered components according to the type of semantic attributes (such as false urgency, false scarcity, false social proof, and forced subscription). Subsequently, this application separately counts the number of components corresponding to each type of semantic attribute, and the statistical results constitute the frequency features of semantic strategies. For example, if the layout data contains 1 "countdown" component (false urgency attribute), 2 "inventory remaining prompt" components (false scarcity attribute), 0 "user selection identifier" components (false social proof attribute), and 1 "default selected subscription button" component (forced subscription attribute), then the specific frequency features of semantic strategies are: false urgency 1, false scarcity 2, false social proof 0, and forced subscription 1. This feature directly reflects the distribution density and usage tendency of various manipulative semantic strategies in the interface.

[0032] For the extraction of visual features, this application proceeds from the visual presentation dimension of the components, and is completed in three steps: First, calculate the total area of ​​all user interface components in the layout data (obtained by summing the products of the width and height of each component), and then calculate the ratio of this total area to the total area of ​​the interactive design canvas. This ratio is the visual size feature. For example, when the total canvas area is 480,000 pixels (800 pixels × 600 pixels) and the total component area is 24,000 pixels, the visual size feature is 0.05. Second, divide the design canvas evenly into a 3x3 grid. Define the five cells formed by the overlap of the center row (the middle horizontal row) and the center column (the middle vertical column) in the 3x3 grid as the focal area. Count the number of components falling into the focal area to obtain the visual focal feature. For example, when there are 3 components in the focal area, the visual focal feature is 3. Third, define the four corner cells of the 3x3 grid as discrete areas. Count the number of components falling into the discrete areas to obtain the visual dispersion feature. For example, when there is 1 component in the discrete area, the visual dispersion feature is 1. Finally, this embodiment integrates visual size features, visual focus features, and visual dispersion features to form complete visual features, thereby quantifying the visual prominence and distribution of components in the interface.

[0033] S03: Input the frequency features of the semantic strategy and the visual features into a preset prediction model so that the prediction model can analyze the response of the user interface operation and output the user behavior prediction result.

[0034] The prediction model is based on parameter fine-tuning of large-scale historical user behavior data and trained on a pre-set multi-step analysis framework in the field of human-computer interaction.

[0035] In a preferred embodiment of this invention, the step of inputting the semantic policy frequency features and visual features into a preset prediction model, so that the prediction model analyzes the response to user interface manipulation and outputs user behavior prediction results, specifically involves: After extracting the semantic strategy frequency features and visual features, the two types of features need to be integrated into an input format that the prediction model can recognize. In this embodiment, the input format is a JSON object, and it needs to contain quantified values ​​of 7 independent variables that have been verified by large-scale experiments, specifically corresponding to: The four variables related to the semantic policy frequency feature are: frequency of false urgency policy (i.e., the statistical value of "false urgency" in the semantic policy frequency feature), frequency of false scarcity policy (the statistical value of "false scarcity" in the semantic policy frequency feature), frequency of false social proof policy (the statistical value of "false social proof" in the semantic policy frequency feature), and frequency of forced subscription policy (the statistical value of "forced subscription" in the semantic policy frequency feature). The three variables related to visual features are: visual size (i.e., the calculation result of "visual size feature" in visual features), visual focus (the statistical result of "number of visual focus components" in visual features), and visual dispersion (the statistical result of "number of visual dispersion components" in visual features).

[0036] For example, after the features of a "Double Eleven" promotional pop-up window from an e-commerce platform are transformed, the generated JSON input object is: {"fake_urgency":1,"fake_scarcity":0,"fake_social_proof":0,"forced_subscription":1,"visual_size":0.08,"visual_focus":2,"visual_dispersion":0}, ensuring that the model can accurately read the quantitative information of each independent variable.

[0037] Subsequently, this structured feature is input into a pre-defined prediction model. The core advantage of this model lies in its training process, which integrates large-scale historical user behavior data with a pre-defined multi-step analysis framework in the human-computer interaction domain. The former provides the model with the correspondence between "interface features and user responses" in real-world scenarios (e.g., historical data shows that "the higher the value of the false urgency feature, the stronger the user's conversion tendency"). The latter constructs standardized reasoning logic for the model (e.g., first analyzing the guiding role of semantic attributes in user decision-making, then evaluating the attractiveness of visual features to attention, and finally combining the two types of influence to judge user response). After receiving the feature input, the model follows the multi-step analysis framework, first matching the frequency features of semantic strategies with the correlation patterns of user behavior in historical data to determine the user decision-making tendencies that various semantic attributes may trigger; then combining the visual features such as "the component prominence reflected by the proportion of visual size and the attention concentration reflected by the number of components in the focal area" to adjust the influence weight of semantic attributes; finally, combining the analysis results of the two types of features to generate user behavior prediction results.

[0038] The user behavior prediction results are output in the form of a list of quantified values, containing two core pieces of information: First, the user behavior strategy level, with values ​​from 1 to 4 corresponding to four decision-making tendencies: "avoidance," "exploration and verification inquiry," "acceptance of suggestions," and "conversion." An output value of 4 indicates that the model predicts the user is highly likely to convert. Second, the cognitive attribution score, with values ​​from 1 to 7 corresponding to the user's perceived influence on three design factors: "semantic attributes," "visual size," and "visual position." Scores of 6, 5, and 7 respectively indicate that the model predicts the user believes their decision is strongly influenced by semantic attributes (6 points), moderately influenced by visual size (5 points), and extremely strongly influenced by visual position (7 points). Through this process, the model can output prediction results from two dimensions: "behavioral tendency" and "cognitive root," providing designers with direct evidence to judge the manipulative effects of the interface. The prediction results are presented as a list containing four values, and the list structure is completely consistent with the structure of the "output fields" in the training samples. For example, the output is [4, 6, 5, 7], where: The first value "4" corresponds to the behavioral strategy level (i.e., predicting that the user will perform a "conversion" behavior). The second value "6" corresponds to the semantic attribution score (that is, the prediction that the user believes their behavior is strongly influenced by semantic attributes, with a score of 6 / 7). The third value "5" corresponds to the visual size attribution score (that is, the prediction that the user believes their behavior is moderately affected by visual size, with a score of 5 / 7). The fourth value "7" corresponds to the visual location attribution score (i.e., the prediction that the user believes their behavior is strongly influenced by visual location, with a score of 7 / 7).

[0039] Furthermore, in this embodiment, the preset prediction model needs to be trained through a three-step core process of 'data preparation - sample construction (including multi-step analysis framework embedding) - parameter fine-tuning' to ensure that it can accurately analyze user responses corresponding to user interface manipulation.

[0040] The preset prediction model in this embodiment is transformed into a UI manipulation effect-specific prediction model through a three-step process of "data preparation - sample construction - parameter fine-tuning". The specific process is as follows: Training data preparation: A total of 2,420 valid data entries were obtained through large-scale online user experiments. Each entry includes a complete UI design scheme, the user's actual behavior record under the scheme (such as whether to convert or avoid), and the user's cognitive attribution record of their own behavior (such as the degree to which they believe they are influenced by semantic cues or visual location), to ensure the authenticity and coverage of the model training data.

[0041] Structured sample construction: Each valid data entry is transformed into a training sample that strictly follows the Alpaca data format. Each sample contains three core fields: Instruction field: Assign the human-computer interaction (HCI) expert role to the model and write a multi-step analysis framework based on the statistical findings of 2420 sets of experiments, explicitly including causal association rules, such as "Step 1: Analyze false urgency. The higher the value of false urgency, the higher the value of user conversion behavior; Step 2: Analyze visual focus. The more components in the visual focus area, the higher the user's perception of the visual position's influence score," forcing the model to learn scientifically verified causal relationships; Input fields: Record the quantized values ​​of the 7 independent variables corresponding to this data entry in JSON object format (completely consistent with the "JSON format after feature transformation" in the first step of this step), providing complete contextual information for the model; Output fields: Record the true target values ​​of the four dependent variables corresponding to the data entry in the form of a list containing four integers: behavioral strategy level (1 = avoidance, 2 = exploration, verification, inquiry, 3 = acceptance of proposal, 4 = conversion), semantic attribution score (1-7 points), visual size attribution score (1-7 points), and visual location attribution score (1-7 points), providing the model with the "standard answer" for learning.

[0042] Model parameter fine-tuning: Meta-Llama-3.1-8B-Instruct was selected as the initial base model. The Low-Rank Adaptive (LoRA) method in the Parameter Efficient Fine-Tuning (PEFT) technique was adopted and fine-tuning was managed through the LlamaFactory platform. The 2420 structured samples were divided into a training set (1936 samples) and a test set (484 samples) in an 8:2 ratio. The model parameters were iteratively optimized based on the training set to gradually make the model conform to the correspondence between "UI features and user responses". The model performance was verified based on the test set until the accuracy of the model on the behavior policy prediction task was stably achieved and the root mean square error of the cognitive attribution score met the preset requirements, thus forming a dedicated prediction model.

[0043] In summary, this embodiment acquires layout data of the user interface during the design phase, including components with preset semantic attributes, position information, and size information, enabling real-time capture of key features of the interface design. Furthermore, semantic strategy frequency features and visual features are extracted from this layout data. The semantic strategy frequency features reflect the distribution patterns of components with preset semantic attributes, while the visual features reveal the visual presentation patterns of the components. These features are input into a prediction model fine-tuned with large-scale historical user behavior data and trained based on a preset multi-step analysis framework for human-computer interaction. This model can perform in-depth analysis of user responses to user interface manipulation, thereby outputting user behavior prediction results. This process not only achieves real-time prediction of the user interface manipulation effect from the design phase but also provides designers with quantitative and scientific design feedback through scientific data analysis and machine learning techniques, helping them identify and avoid potential "dark modes" before development. This application effectively solves the problem that existing technologies cannot accurately and efficiently predict user operation behavior.

[0044] Example 2 Please refer to Figure 2 This is a user operation behavior prediction device provided in the embodiments of this application.

[0045] In this embodiment, the user operation behavior prediction device includes an acquisition module 10, an extraction module 20, and a prediction module 30.

[0046] The acquisition module 10 is used to acquire layout data of the user interface during the design phase; the layout data includes user interface components with preset semantic attributes, position information of the user interface components, and size information of the user interface components. Extraction module 20 is used to extract semantic strategy frequency features and visual features from the layout data; wherein, the semantic strategy frequency features correspond to the component distribution pattern of preset semantic attributes, and the visual features correspond to the visual presentation pattern of user interface components. The prediction module 30 is used to input the frequency features of the semantic strategy and the visual features into a preset prediction model so that the prediction model can analyze the response of the user interface operation and output the user behavior prediction result.

[0047] The prediction model is based on parameter fine-tuning of large-scale historical user behavior data and trained on a pre-set multi-step analysis framework in the field of human-computer interaction.

[0048] For ease of description and brevity, the embodiments of the device of the present invention include all the implementation methods in the above-described embodiments of the user operation behavior prediction method, and will not be repeated here.

[0049] Example 3: This application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute a method for predicting user operation behavior. The user operation behavior prediction method, if implemented as a software functional unit and used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0050] Example 4 This embodiment provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements any of the user operation behavior prediction methods described in Embodiment 1.

[0051] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method of predicting user operation behavior, characterized by, The method comprises the following steps: obtaining layout data of a design stage user interface; the layout data comprises user interface components with preset semantic attributes, position information of the user interface components, and size information of the user interface components; extracting semantic strategy frequency features and visual features from the layout data; the semantic strategy frequency features correspond to the distribution of components with preset semantic attributes, and the visual features correspond to the visual presentation rules of the user interface components; inputting the semantic strategy frequency features and the visual features into a preset prediction model to analyze the response of the user interface operation and output a user behavior prediction result; wherein the prediction model is trained based on a large-scale historical user behavior data and a preset multi-step analysis framework in the field of human-computer interaction.

2. The method of predicting user operation behavior according to claim 1, wherein The method for obtaining the layout data of the design stage user interface comprises the following steps: monitoring user operations on an interactive design canvas, obtaining the current state of the interactive design canvas after each user operation, and generating user interface layout data in each state; wherein the user operations include drag-and-drop operations, moving operations, and deleting operations on the user interface components; based on the user interface layout data in each state, recording the preset semantic attributes, position coordinate information, and size parameter information of the user interface components; wherein the preset semantic attributes include false urgency, false scarcity, false social proof, and forced subscription.

3. The method of predicting user operation behavior according to claim 1, wherein The method for extracting semantic strategy frequency features from the layout data comprises the following steps: after screening all user interface components with preset semantic attributes from the layout data, classifying the screened user interface components according to the types of the preset semantic attributes, and counting the number of user interface components corresponding to each type of preset semantic attribute; the number of user interface components corresponding to each type of preset semantic attribute obtained by counting is taken as the semantic strategy frequency features.

4. The method of predicting user operation behavior according to claim 1, wherein The method for extracting visual features from the layout data comprises the following steps: calculating the proportion of the total area of all user interface components in the layout data to the total area of the interactive design canvas to obtain a visual size feature; dividing the interactive design canvas into a preset grid to determine the focus area and the discrete area in the grid; counting the number of user interface components falling into the focus area to obtain a visual focus feature; counting the number of user interface components falling into the discrete area to obtain a visual dispersion degree feature; integrating the visual size feature, the visual focus feature, and the visual dispersion degree feature to obtain the visual features.

5. The method of predicting user operation behavior according to claim 1, wherein The prediction model is trained based on a large-scale historical user behavior data and a preset multi-step analysis framework in the field of human-computer interaction, and the method comprises the following steps: obtaining historical user behavior data; the historical user behavior data comprises multiple user interface design schemes and user behavior records corresponding to each design scheme; constructing a preset multi-step analysis framework in the field of human-computer interaction; the multi-step analysis framework comprises rules associated with semantic attributes and user behavior. inputting the historical user behavior data into a preset initial prediction model, so that the initial prediction model is combined with the multi-step analysis framework and is fine-tuned according to a fine-tuning technique to obtain the prediction model.

6. The method of predicting user operation behavior according to claim 1, wherein The semantic strategy frequency features and the visual features are input into a preset prediction model, so that the prediction model analyzes the response of the user interface operation, and outputs a user behavior prediction result, specifically: The extracted semantic strategy frequency features and the visual features are converted into a structured input format recognizable by the prediction model; The semantic strategy frequency features and the visual features in the structured input format are input into a preset prediction model, so that the prediction model analyzes the input features according to a multi-step analysis framework of a preset human-computer interaction field, determines the response trend of the user to the user interface operation, and generates a user behavior prediction result based on the response trend; The user behavior prediction result includes a user behavior strategy level and a cognitive attribution score, the user behavior strategy level includes avoidance, exploration verification inquiry, acceptance of proposal, and conversion, and the cognitive attribution score includes a semantic attribution score, a visual size attribution score, and a visual position attribution score.

7. A device for predicting a user operation behavior, characterized by, The method comprises the following steps: An acquisition module is configured to acquire layout data of a user interface in a design stage; the layout data comprises user interface components with preset semantic attributes, position information of the user interface components, and size information of the user interface components; An extraction module is configured to extract semantic strategy frequency features and visual features from the layout data; the semantic strategy frequency features correspond to a component distribution rule of the preset semantic attributes, and the visual features correspond to a visual presentation rule of the user interface components; A prediction module is configured to input the semantic strategy frequency features and the visual features into a preset prediction model, so that the prediction model analyzes the response of the user interface operation, and outputs a user behavior prediction result. The prediction model is obtained based on parameter fine-tuning of large-scale historical user behavior data and a multi-step analysis framework of a preset human-computer interaction field.

8. The user operation behavior prediction device according to claim 7, characterized by, The semantic strategy frequency features and the visual features are input into a preset prediction model, so that the prediction model analyzes the response of the user interface operation, and outputs a user behavior prediction result, specifically: The extracted semantic strategy frequency features and the visual features are converted into a structured input format recognizable by the prediction model; The semantic strategy frequency features and the visual features in the structured input format are input into a preset prediction model, so that the prediction model analyzes the input features according to a multi-step analysis framework of a preset human-computer interaction field, determines the response trend of the user to the user interface operation, and generates a user behavior prediction result based on the response trend; The user behavior prediction result includes a user behavior strategy level and a cognitive attribution score, the user behavior strategy level includes avoidance, exploration verification inquiry, acceptance of proposal, and conversion, and the cognitive attribution score includes a semantic attribution score, a visual size attribution score, and a visual position attribution score.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored computer program, wherein the computer readable storage medium controls a device in which the computer readable storage medium is located to execute the user operation behavior prediction method according to any one of claims 1 to 6 when the computer program is running.

10. A terminal device, comprising: A computer program product includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the user operation behavior prediction method according to any one of claims 1 to 6 when the computer program is executed.

Images