Display Interface Layout Control Method and System Based on User Operation Analysis

Abstract

The present invention discloses a display interface layout control method and system based on user operation analysis. The method includes: obtaining historical operation data of a user received by a display device; determining a preference parameter of the user for any display element in the display interface of the display device according to the historical operation data; determining an important parameter of each display element according to the function scenario corresponding to the display device and a preset scenario rule; and determining a layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements by using a dynamic programming algorithm. It can be seen that the present invention can accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for controlling the layout of a display interface based on user operation analysis. Background Art

[0002] With the continuous development of the terminal display industry and the continuous improvement of the technological level, many terminal control devices have emerged. Among them, some devices have relatively low requirements for the display interface and only need to display a simple display of the terminal function operation and confirmation of the operation. Therefore, some low-cost screens such as serial ports are required to implement the operation interface display. Since most low-cost screens are only equipped with processors with poor performance, the degree of display intelligence is generally poor. Most of the display control technologies of low-cost screens in the prior art only display and respond to user control based on preset display interface layout rules, and do not further control the layout of the display interface based on user operation analysis. Therefore, the display effect is poor, the degree of intelligence is low, and a better user experience cannot be provided to users. It can be seen that there are defects in the prior art and it is urgent to solve. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for controlling the layout of a display interface based on user operation analysis, which can accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better user experience.

[0004] To solve the above technical problem, a first aspect of the present invention discloses a method for controlling the layout of a display interface based on user operation analysis, the method comprising:

[0005] Obtaining historical operation data of a user received by a display device;

[0006] Determining a preference parameter of any display element in the display interface of the display device according to the historical operation data;

[0007] Determining an important parameter of each display element according to the function scenario corresponding to the display device and a preset scenario rule;

[0008] Determining a layout scheme corresponding to the display interface based on the preference parameter and the important parameter of all the display elements by means of a dynamic programming algorithm; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

[0009] As an alternative implementation, in the first aspect of the present invention, the historical operation data includes the operations of the user on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0010] As an alternative implementation, in the first aspect of the present invention, determining the preference parameter of any display element in the display interface of the display device according to the historical operation data includes:

[0011] Determine the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device;

[0012] For each display element in the display interface, determine the corresponding element position range;

[0013] Count the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the operation preference corresponding to the display element;

[0014] Count the number of intention influence positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the intention preference corresponding to the display element;

[0015] Calculate the weighted sum average of the operation preference and the intention preference to obtain the preference parameter corresponding to the display element.

[0016] As an alternative implementation, in the first aspect of the present invention, determining the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device includes:

[0017] For each operation in the historical operation data, determine the corresponding operation type;

[0018] When the operation type is the touch screen operation, determine the operation position coordinates corresponding to the operation as the operation position corresponding to the operation;

[0019] When the operation type is the button operation, simulate the operation based on a preset button operation simulation algorithm to obtain the operation position corresponding to the operation;

[0020] Obtain a plurality of consecutive operations by acquiring the operation at the preset number of historical time points before the operation and the historical time point corresponding to the operation;

[0021] Sort the operation positions corresponding to the plurality of consecutive operations from the earliest to the latest based on the corresponding historical time points to obtain an operation position sequence;

[0022] Input the operation position sequence into the trained LSTM neural network to obtain the intention influence position corresponding to the operation; the LSTM neural network is trained by a training data set including a plurality of training operation position sequences and corresponding intention influence position annotations.

[0023] As an alternative implementation, in the first aspect of the present invention, when calculating the preference parameter, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the operation intersection degree; the operation intersection degree is the ratio of the operation intersection quantity to the total operation quantity; the operation intersection quantity is the quantity of the intersection of the sets of all the operations corresponding to the operation preference and all the operations corresponding to the intention preference; the total operation quantity is the total quantity of all the operations corresponding to the operation preference and the intention preference.

[0024] As an alternative implementation, in the first aspect of the present invention, the functional scenario is a vehicle-mounted scenario, a medical scenario, a manufacturing assembly line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario, or an atomizer scenario.

[0025] As an alternative implementation, in the first aspect of the present invention, determining the important parameters of each display element according to the functional scenario corresponding to the display device and a preset scenario rule includes:

[0026] Determine the scenario historical layout data corresponding to the functional scenario of the display device; the scenario historical layout data includes a plurality of historical element layout schemes corresponding to the corresponding functional scenario;

[0027] For each display element, determine the layout position corresponding to the display element in all the historical element layout schemes;

[0028] Calculate the average value of the position distances between each layout position and at least one key position to obtain the key parameter of each layout position; the key position is a position preset on the display interface of the display device that is easily noticed by the user;

[0029] Calculate the weighted summation average value of the key parameters of all the layout positions to obtain the important parameter of the display element; wherein, the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter.

[0030] As an alternative implementation, in the first aspect of the present invention, determining the layout scheme corresponding to the display interface based on the dynamic programming algorithm according to the preference parameters and the important parameters of all the display elements includes:

[0031] For each of the display elements, calculate the weighted sum value of the preference parameter and the important parameter corresponding to the display element to obtain the display priority parameter corresponding to the display element; wherein, the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter;

[0032] Set the objective function to include that the similarity between the layout scheme and the current layout scheme of the display interface reaches the maximum value and the number of display elements in the layout scheme reaches the maximum;

[0033] Set the constraint conditions to include:

[0034] The smaller the positional distance between the layout position of the display element with a higher display priority parameter in the layout scheme and the central position; the central position is the center point of the display interface of the display device;

[0035] The larger the element size of the display element with a higher display priority parameter in the layout scheme;

[0036] In the calculation, the display element can be set in an area outside the display interface;

[0037] In the layout scheme, the display element whose area of the part outside the display interface is greater than the area of the part inside the display interface is excluded;

[0038] Based on the dynamic programming algorithm, according to the objective function and the constraint conditions, perform layout iterative calculation on all the display elements to obtain the optimal layout scheme, which is determined as the layout scheme corresponding to the display interface.

[0039] A second aspect of the embodiments of the present invention discloses a display interface layout control system based on user operation analysis, and the system includes:

[0040] An acquisition module, configured to acquire historical operation data of a user received by a display device;

[0041] A first determination module, configured to determine a preference parameter of any display element in the display interface of the display device according to the historical operation data;

[0042] A second determination module, configured to determine an important parameter of each display element according to the function scenario corresponding to the display device and a preset scenario rule;

[0043] A third determination module, configured to determine a layout scheme corresponding to the display interface according to the preference parameter and the important parameter of all the display elements based on the dynamic programming algorithm; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

[0044] As an alternative embodiment, in the second aspect of the present invention, the historical operation data includes operations of the user on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0045] As an alternative embodiment, in the second aspect of the present invention, the specific manner in which the first determination module determines the preference parameter of any display element in the display interface of the display device according to the historical operation data includes:

[0046] Determine the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device;

[0047] For each display element in the display interface, determine the corresponding element position range;

[0048] Count the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the operation preference corresponding to the display element;

[0049] Count the number of intention influence positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the intention preference corresponding to the display element;

[0050] Calculate the weighted sum average of the operation preference and the intention preference to obtain the preference parameter corresponding to the display element.

[0051] As an alternative embodiment, in the second aspect of the present invention, the specific manner in which the first determination module determines the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device includes:

[0052] For each operation in the historical operation data, determine the corresponding operation type;

[0053] When the operation type is the touch screen operation, determine the operation position coordinates corresponding to the operation as the operation position corresponding to the operation;

[0054] When the operation type is the button operation, simulate the operation based on a preset button operation simulation algorithm to obtain the operation position corresponding to the operation;

[0055] Obtain a plurality of consecutive operations by acquiring the operation and the operations at a preset number of historical time points before the historical time point corresponding to the operation;

[0056] Sort the operation positions corresponding to the multiple consecutive operations from the earliest to the latest based on the corresponding historical time points to obtain an operation position sequence;

[0057] Input the operation position sequence into the trained LSTM neural network to obtain the intention influence position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intention influence position annotations.

[0058] As an optional implementation manner, in the second aspect of the present invention, when calculating the preference parameter, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the operation intersection degree; the operation intersection degree is the ratio of the operation intersection quantity to the total operation quantity; the operation intersection quantity is the quantity of the intersection of the sets of all the operations corresponding to the operation preference and all the operations corresponding to the intention preference; the total operation quantity is the total quantity of all the operations corresponding to the operation preference and the intention preference.

[0059] As an optional implementation manner, in the second aspect of the present invention, the functional scenario is a vehicle-mounted scenario, a medical scenario, a manufacturing assembly line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario, or an atomizer scenario.

[0060] As an optional implementation manner, in the second aspect of the present invention, the specific manner in which the second determination module determines the important parameters of each display element according to the functional scenario corresponding to the display device and a preset scenario rule includes:

[0061] Determine the scenario historical layout data corresponding to the functional scenario of the display device; the scenario historical layout data includes multiple historical element layout schemes corresponding to the corresponding functional scenario;

[0062] For each display element, determine the layout position corresponding to the display element in all the historical element layout schemes;

[0063] Calculate the average value of the position distances between each layout position and at least one key position to obtain the key parameter of each layout position; the key position is a position preset on the display interface of the display device that is easily noticed by the user;

[0064] Calculate the weighted summation average value of the key parameters of all the layout positions to obtain the important parameter of the display element; wherein, the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter.

[0065] As an alternative embodiment, in the second aspect of the present invention, the specific manner in which the third determination module determines the layout scheme corresponding to the display interface based on the preference parameters and the importance parameters of all the display elements and based on the dynamic programming algorithm includes:

[0066] For each of the display elements, calculate the weighted sum value of the preference parameter and the importance parameter corresponding to the display element to obtain the display priority parameter corresponding to the display element; wherein, the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the importance parameter;

[0067] Set the objective function to include that the similarity between the layout scheme and the current layout scheme of the display interface reaches the maximum value and the number of display elements in the layout scheme reaches the maximum;

[0068] Set the constraint conditions to include:

[0069] The smaller the positional distance between the layout position of the display element with a higher display priority parameter in the layout scheme and the central position; the central position is the center point of the display interface of the display device;

[0070] The larger the element size of the display element with a higher display priority parameter in the layout scheme;

[0071] In the calculation, the display element can be set in an area outside the display interface;

[0072] In the layout scheme, the display element whose area of the part outside the display interface is greater than the area of the part inside the display interface is excluded;

[0073] Based on the dynamic programming algorithm, perform layout iterative calculation on all the display elements according to the objective function and the constraint conditions to obtain the optimal layout scheme, which is determined as the layout scheme corresponding to the display interface.

[0074] The third aspect of the present invention discloses another display interface layout control system based on user operation analysis, and the system includes:

[0075] A memory storing executable program code;

[0076] A processor coupled to the memory;

[0077] The processor calls the executable program code stored in the memory and executes some or all of the steps in the display interface layout control method disclosed in the first aspect of the present invention.

[0078] A fourth aspect of the present invention discloses a computer storage medium storing computer instructions which, when called, are used to execute some or all of the steps in the display interface layout control method based on user operation analysis disclosed in the first aspect of the present invention.

[0079] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0080] The present invention can determine the preference parameters of any display element in the display interface of the display device according to historical operation data, and then determine the important parameters of each display element based on the function scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display element, thereby being able to accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0082] Figure 1 FIG. is a schematic flowchart of a display interface layout control method based on user operation analysis disclosed in an embodiment of the present invention.

[0083] Figure 2 FIG. is a schematic structural diagram of a display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention.

[0084] Figure 3 FIG. is a schematic structural diagram of another display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0085] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0086] In the description, claims, and above-mentioned drawings of the present invention, terms such as "first" and "second" are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or equipment.

[0087] Reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0088] The present invention discloses a display interface layout control method and system based on user operation analysis, which can determine the preference parameters of any display element in the display interface of a display device according to historical operation data, and then determine the important parameters of each display element based on the function scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, thereby being able to accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience. The following will be described in detail respectively.

[0089] Embodiment 1

[0090] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a display interface layout control method based on user operation analysis disclosed in an embodiment of the present invention. Among them, Figure 1 the described display interface layout control method based on user operation analysis can be applied to a data processing system / data processing device / data processing server (wherein, the server includes a local processing server or a cloud processing server). As Figure 1 shown, the display interface layout control method based on user operation analysis can include the following operations:

[0091] 101. Obtain the historical operation data of the user received by the display device.

[0092] 102. Determine the preference parameters of any display element in the display interface of the display device according to the historical operation data.

[0093] 103. Determine the important parameters of each display element according to the function scenario corresponding to the display device and the preset scenario rules.

[0094] 104. Determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of all display elements and the dynamic programming algorithm.

[0095] Optionally, the layout scheme is used to be sent to the display device to control the display of at least one display element in the display interface.

[0096] Optionally, when applying the solution of the present invention to the serial screen scenario, the above historical operation data can be obtained based on the communication between the server and the serial screen, and after executing the solution in the present invention locally or on the cloud server to determine the layout scheme, the layout scheme is packaged into data in a format such as XML and sent to the serial screen for layout control.

[0097] It can be seen that the above-mentioned invention embodiments can determine the preference parameters of any display element in the display interface of the display device according to the historical operation data, and then determine the important parameters of each display element based on the function scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, thereby being able to accurately determine a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0098] As an optional embodiment, in the above steps, the historical operation data includes the operations of the user on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0099] It can be seen that through the above optional embodiment, the content of the historical operation data is defined, which can fully represent the operation characteristics of the user, facilitate the subsequent prediction of user preferences and the generation of the layout scheme, and assist in accurately determining a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0100] As an optional embodiment, in the above steps, according to the historical operation data, determining the preference parameters of any display element in the display interface of the display device includes:

[0101] Determine the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device;

[0102] For each display element in the display interface, determine the element position range corresponding to the display element;

[0103] Count the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, and obtain the operation preference corresponding to the display element;

[0104] Count the number of intention influence positions corresponding to all operations in the historical operation data that fall within the element position range, and obtain the intention preference corresponding to the display element;

[0105] Calculate the weighted sum average of the operation preference and the intention preference to obtain the preference parameter corresponding to the display element.

[0106] It can be seen that through the above optional embodiments, based on the correspondence between the operation position and the intention influence position corresponding to each operation in the historical operation data and the position ranges of different display elements, the preference of the user for different display elements can be accurately determined. Subsequently, it is convenient to generate a more accurate layout plan, assist in realizing the accurate determination of a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0107] As an optional embodiment, in the above steps, determining the operation position and the intention influence position corresponding to each operation in the historical operation data in the display interface of the display device includes:

[0108] For each operation in the historical operation data, determine the operation type corresponding to the operation;

[0109] When the operation type is a touch screen operation, determine the operation position coordinates corresponding to the operation as the operation position corresponding to the operation;

[0110] When the operation type is a button operation, based on a preset button operation simulation algorithm, simulate the operation to obtain the operation position corresponding to the operation;

[0111] Obtain the operation and the operations at a preset number of historical time points before the historical time point corresponding to the operation to obtain a plurality of consecutive operations;

[0112] Sort the operation positions corresponding to the plurality of consecutive operations from the earliest to the latest based on the corresponding historical time points to obtain an operation position sequence;

[0113] Input the operation position sequence into the trained LSTM neural network to obtain the intention influence position corresponding to the operation; the LSTM neural network is trained through a training data set including a plurality of training operation position sequences and corresponding intention influence position annotations.

[0114] It can be seen that through the above optional embodiments, it is possible to determine the operation position by determining the type of operation, and predict the intended influence position based on the LSTM network for continuous operation positions. Subsequently, it is convenient to accurately predict user preferences and generate a more accurate layout plan, assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0115] As an optional embodiment, in the above steps, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the operation intersection degree; the operation intersection degree is the ratio of the operation intersection quantity to the total operation quantity; the operation intersection quantity is the quantity of the intersection of the set of all operations corresponding to the operation preference and the set of all operations corresponding to the intention preference; the total operation quantity is the total quantity of all operations corresponding to the operation preference and the intention preference.

[0116] It can be seen that through the above optional embodiments, it is possible to determine the calculation weight rule based on the ratio relationship between the operation quantities of the operation preference and the intention preference, so as to effectively and reasonably correct the proportions of the operation preference and the intention preference in the final user preference. Subsequently, it is convenient to generate a more accurate layout plan, assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0117] As an optional embodiment, in the above steps, the functional scenario is a vehicle-mounted scenario, a medical scenario, a manufacturing assembly line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario, or a nebulizer scenario.

[0118] It can be seen that through the above optional embodiments, the types of functional scenarios are limited to comprehensively and accurately characterize the scenario features of the display device. Subsequently, it is convenient to predict the important parameters of the display elements and generate a layout plan, assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0119] As an optional embodiment, in the above steps, according to the functional scenario corresponding to the display device and the preset scenario rules, determine the important parameters of each display element, including:

[0120] Determine the scenario historical layout data corresponding to the functional scenario corresponding to the display device; optionally, the scenario historical layout data includes multiple historical element layout plans of the corresponding functional scenario;

[0121] For each display element, determine the layout position corresponding to the display element in all historical element layout schemes;

[0122] Calculate the average value of the position distances between each layout position and at least one key position to obtain the key parameter of each layout position; Optionally, the key position is a position on the display interface of the display device that is preset to be easily noticed by the user;

[0123] Calculate the weighted sum average value of the key parameters of all layout positions to obtain the important parameter of the display element; Among them, the weighted calculation weight corresponding to each key parameter is proportional to the number of key positions involved in the calculation process of the key parameter.

[0124] It can be seen that through the above optional embodiments, it is possible to accurately determine the important parameter of each display element by statistically calculating the key degree of the layout positions of different display elements in the historical layout schemes of different functional scenarios. Subsequently, it is convenient to generate a more accurate layout scheme to assist in realizing the accurate determination of a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0125] As an optional embodiment, in the above steps, based on the preference parameters and important parameters of all display elements, and based on the dynamic programming algorithm, determine the layout scheme corresponding to the display interface, including:

[0126] For each display element, calculate the weighted sum value of the preference parameter and the important parameter corresponding to the display element to obtain the display priority parameter corresponding to the display element; Optionally, among them, the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter;

[0127] Set the objective function to include that the similarity between the layout scheme and the current layout scheme of the display interface reaches the maximum value and the number of display elements in the layout scheme reaches the maximum;

[0128] Set the constraint conditions to include:

[0129] The smaller the position distance between the layout position of the display element with a higher display priority parameter in the layout scheme and the central position; Optionally, the central position is the center point of the display interface of the display device;

[0130] The larger the element size of the display element with a higher display priority parameter in the layout scheme;

[0131] In the calculation, the display element can be set in an area outside the display interface;

[0132] Display elements whose area of the part outside the display interface in the layout scheme is larger than the area of the part inside the display interface are excluded;

[0133] Based on the dynamic programming algorithm, according to the objective function and constraints, perform layout iterative calculations on all display elements to obtain the optimal layout scheme, which is determined as the layout scheme corresponding to the display interface.

[0134] It can be seen that through the above optional embodiments, it is possible to accurately determine the display priority parameter of a display element by weighted calculation of the preference parameter and the important parameter of the display element, and then calculate the optimal layout scheme through the dynamic programming algorithm based on the preset objective function and constraints, so as to realize the accurate determination of a more reasonable and intelligent display interface layout based on the analysis of user operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0135] Embodiment Two

[0136] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of a display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention. Among them, Figure 2 The described display interface layout control system based on user operation analysis can be applied to a data processing system / data processing device / data processing server (wherein, the server includes a local processing server or a cloud processing server). As Figure 2 shown, the display interface layout control system based on user operation analysis may include:

[0137] An acquisition module 201, configured to acquire historical operation data of a user received by a display device.

[0138] A first determination module 202, configured to determine a preference parameter of a user for any display element in the display interface of the display device according to the historical operation data.

[0139] A second determination module 203, configured to determine an important parameter of each display element according to the function scenario corresponding to the display device and a preset scenario rule.

[0140] A third determination module 204, configured to determine a layout scheme corresponding to the display interface based on the preference parameters and important parameters of all display elements and based on the dynamic programming algorithm.

[0141] Optionally, the layout scheme is used to be sent to the display device to control the display of at least one display element in the display interface.

[0142] It can be seen that the above-mentioned invention embodiments can determine the preference parameters of any display element in the display interface of the display device according to historical operation data, and then determine the important parameters of each display element based on the function scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display element, thereby being able to accurately determine a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0143] As an optional embodiment, the historical operation data includes the operations of the user on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0144] It can be seen that through the above optional embodiment, the content of the historical operation data is defined, which can fully represent the operation characteristics of the user, facilitate the subsequent prediction of user preferences and the generation of layout schemes, and assist in accurately determining a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0145] As an optional embodiment, the specific manner in which the first determination module determines the preference parameters of any display element in the display interface of the display device according to historical operation data includes:

[0146] Determine the corresponding operation position and intention influence position of each operation in the historical operation data in the display interface of the display device;

[0147] For each display element in the display interface, determine the element position range corresponding to the display element;

[0148] Count the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the operation preference corresponding to the display element;

[0149] Count the number of intention influence positions corresponding to all operations in the historical operation data that fall within the element position range to obtain the intention preference corresponding to the display element;

[0150] Calculate the weighted sum average of the operation preference and the intention preference to obtain the preference parameter corresponding to the display element.

[0151] It can be seen that through the above optional embodiments, it is possible to determine the correspondence between the operation position and the intention influence position corresponding to each operation in the historical operation data and the position ranges of different display elements, so as to accurately determine the user's preference for different display elements. Subsequently, it is convenient to generate a more accurate layout plan, assist in realizing the precise determination of a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0152] As an optional embodiment, the specific manner in which the first determination module determines the operation position and the intention influence position corresponding to each operation in the historical operation data on the display interface of the display device includes:

[0153] For each operation in the historical operation data, determine the operation type corresponding to the operation;

[0154] When the operation type is a touch screen operation, determine the operation position coordinates corresponding to the operation as the operation position corresponding to the operation;

[0155] When the operation type is a button operation, based on a preset button operation simulation algorithm, simulate the operation to obtain the operation position corresponding to the operation;

[0156] Obtain the operation and the operations at a preset number of historical time points before the historical time point corresponding to the operation to obtain a plurality of consecutive operations;

[0157] Sort the operation positions corresponding to the plurality of consecutive operations from the earliest to the latest based on the corresponding historical time points to obtain an operation position sequence;

[0158] Input the operation position sequence into the trained LSTM neural network to obtain the intention influence position corresponding to the operation; the LSTM neural network is trained through a training data set including a plurality of training operation position sequences and corresponding intention influence position annotations.

[0159] It can be seen that through the above optional embodiments, it is possible to determine the operation position by determining the type of the operation, and predict the intention influence position of the consecutive operation positions based on the LSTM network. Subsequently, it is convenient to accurately predict the user's preference and generate a more accurate layout plan, assist in realizing the precise determination of a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0160] As an alternative embodiment, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the operation intersection degree; the operation intersection degree is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all operations corresponding to the operation preference and the set of all operations corresponding to the intention preference; the total number of operations is the total number of all operations corresponding to the operation preference and the intention preference.

[0161] It can be seen that through the above alternative embodiments, the calculation weight rule can be determined by the ratio relationship of the number of operations of the operation preference and the intention preference, so as to effectively and reasonably correct the proportion of the operation preference and the intention preference in the final user preference, which is convenient for generating a more accurate layout scheme subsequently, assisting in realizing a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improving the display effect of low-cost screens, reducing the power consumption of unnecessary displays, and giving users a better use experience.

[0162] As an alternative embodiment, the functional scenario is a vehicle-mounted scenario, a medical scenario, a manufacturing assembly line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario, or an atomizer scenario.

[0163] It can be seen that through the above alternative embodiments, the types of functional scenarios are defined to comprehensively and accurately characterize the scenario characteristics of the display device, which is convenient for predicting the important parameters of the display elements and generating layout schemes subsequently, assisting in realizing a more reasonable and intelligent display interface layout based on the analysis of the user's operations, improving the display effect of low-cost screens, reducing the power consumption of unnecessary displays, and giving users a better use experience.

[0164] As an alternative embodiment, the specific manner in which the second determination module determines the important parameters of each display element according to the functional scenario corresponding to the display device and the preset scenario rules includes:

[0165] Determine the scenario historical layout data corresponding to the functional scenario of the display device; optionally, the scenario historical layout data includes multiple historical element layout schemes corresponding to the corresponding functional scenario;

[0166] For each display element, determine the layout position of the display element in all historical element layout schemes;

[0167] Calculate the average value of the position distances between each layout position and at least one key position to obtain the key parameter of each layout position; optionally, the key position is a position preset on the display interface of the display device that is easily noticed by the user;

[0168] The weighted sum average of the key parameters of all layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of key positions involved in the calculation process of the key parameter.

[0169] It can be seen that through the above-mentioned optional embodiments, it is possible to accurately determine the important parameters of each display element by counting and calculating the criticality of the layout positions of different display elements in the historical layout schemes of different functional scenarios, which is convenient for subsequently generating a more accurate layout scheme, and assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0170] As an optional embodiment, the third determination module determines the specific manner of the layout scheme corresponding to the display interface based on the preference parameters and important parameters of all display elements and the dynamic programming algorithm, including:

[0171] For each display element, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; optionally, the weight of the weighted calculation of the preference parameter is greater than the weighted calculation weight of the important parameter;

[0172] Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme;

[0173] Setting restrictions includes:

[0174] The layout position of a display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; optionally, the central position is the center point of a display interface of the display device;

[0175] The element size of a display element with a higher display priority parameter in the layout scheme is larger;

[0176] During calculation, display elements can be set in areas outside the display interface;

[0177] Display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are removed;

[0178] Based on the dynamic programming algorithm, according to the objective function and constraints, all display elements are iteratively calculated to obtain the optimal layout solution, which is determined as the layout solution corresponding to the display interface.

[0179] It can be seen that through the above optional embodiments, the display priority parameter of a display element can be accurately determined by weighted calculation of the preference parameter and the important parameter of the display element, and then the optimal layout scheme can be calculated through a dynamic programming algorithm based on a preset objective function and constraint conditions, so as to realize the accurate determination of a more reasonable and intelligent display interface layout based on the analysis of user operations, improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better use experience.

[0180] Embodiment III

[0181] Please refer to Figure 3 , Figure 3 which is another display interface layout control system based on user operation analysis disclosed in the embodiments of the present invention. Figure 3 The described display interface layout control system based on user operation analysis is applied to a data processing system / data processing device / data processing server (wherein, the server includes a local processing server or a cloud processing server). As Figure 3 shown, the display interface layout control system based on user operation analysis may include:

[0182] A memory 301 storing executable program code;

[0183] A processor 302 coupled to the memory 301;

[0184] Wherein, the processor 302 calls the executable program code stored in the memory 301 to execute the steps of the display interface layout control method described in Embodiment I.

[0185] Embodiment IV

[0186] The embodiments of the present invention disclose a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the steps of the display interface layout control method described in Embodiment I.

[0187] Embodiment V

[0188] The embodiments of the present invention disclose a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the display interface layout control method described in Embodiment I.

[0189] The above description has been made of specific embodiments of this specification, and other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily have to be performed in the particular order shown or in a sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] The systems, devices, modules, or units illustrated in the above embodiments may specifically be implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0191] For convenience of description, the above devices are described by dividing them into various units according to their functions. Of course, when implementing this specification, the functions of each unit may be implemented in one or more software and / or hardware.

[0192] Those skilled in the art should understand that the embodiments of this specification may be provided as a method, a system, or a computer program product. Therefore, the embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0193] This specification is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of this specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a device for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0194] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more processes and / or blocks Figure 1 in one or more processes and / or blocks Figure 1 specified in the block or blocks.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes and / or blocks Figure 1 in one or more processes and / or blocks Figure 1 specified in the block or blocks.

[0196] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0197] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0198] Computer-readable media includes both permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0199] It should also be noted that the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0200] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0201] Each embodiment in this specification is described in a progressive manner. For parts that are the same or similar among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0202] Finally, it should be noted that the display interface layout control method and system disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A display interface layout control method based on user operation analysis, characterized in that: The method comprises: Obtain the historical operation data of the user received by the display device; Determining, according to the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; According to the functional scenario corresponding to the display device and the preset scenario rules, the important parameters of each display element are determined, including: Determine scene history layout data corresponding to the functional scene corresponding to the display device; the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each of the display elements, determining a layout position corresponding to the display element in all the historical element layout schemes; Calculating an average value of the position distance between each of the layout positions and at least one key position to obtain a key parameter of each of the layout positions; the key position is a preset position on the display interface of the display device; Calculating the weighted sum average of the key parameters of all the layout positions to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter; According to the preference parameters and the important parameters of all the display elements, based on a dynamic programming algorithm, a layout scheme corresponding to the display interface is determined; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

2. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

3. The display interface layout control method based on user operation analysis according to claim 2, characterized in that: The determining, according to the historical operation data, the user's preference parameter for any display element in the display interface of the display device includes: Determining the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device includes: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is the touch screen operation, determining the operation position coordinates corresponding to the operation as the operation position corresponding to the operation; When the operation type is the button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; Sorting the operation positions corresponding to the multiple continuous operations from early to late based on the corresponding historical time points to obtain an operation position sequence; Inputting the operation position sequence into a trained LSTM neural network to obtain the intended impact position corresponding to the operation; the LSTM neural network is trained by a training data set including a plurality of training operation position sequences and corresponding intended impact position annotations; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the operation preference corresponding to the display element; Counting the number of intention-affected positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the intention preference corresponding to the display element; A weighted average value of the operation preference and the intention preference is calculated to obtain a preference parameter corresponding to the display element.

4. The display interface layout control method based on user operation analysis according to claim 3, characterized in that: When calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all the operations corresponding to the operation preference and the set of all the operations corresponding to the intention preference; the total number of operations is the total number of all the operations corresponding to the operation preference and the intention preference.

5. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The functional scenario is a car computer scenario, a medical scenario, a manufacturing line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario or a nebulizer scenario.

6. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The determining, based on the preference parameters and the important parameters of all the display elements and a dynamic programming algorithm, a layout scheme corresponding to the display interface includes: For each of the display elements, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; wherein the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of the display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; the central position is the center point of the display interface of the display device; The element size of the display element with a higher display priority parameter in the layout scheme is larger; The display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are eliminated; Based on a dynamic programming algorithm, according to the objective function and the constraint conditions, an iterative layout calculation is performed on all the display elements to obtain an optimal layout solution, which is determined as the layout solution corresponding to the display interface.

7. A display interface layout control system based on user operation analysis, characterized in that: The system comprises: An acquisition module, used to acquire the historical operation data of the user received by the display device; A first determination module, configured to determine, based on the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; The second determination module is used to determine the important parameters of each of the display elements according to the functional scenario corresponding to the display device and the preset scenario rules, including: Determine scene history layout data corresponding to the functional scene corresponding to the display device; the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each of the display elements, determining a layout position corresponding to the display element in all the historical element layout schemes; Calculating an average value of the position distance between each of the layout positions and at least one key position to obtain a key parameter of each of the layout positions; the key position is a preset position on the display interface of the display device; Calculating the weighted sum average of the key parameters of all the layout positions to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter; The third determination module is used to determine the layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements based on a dynamic programming algorithm; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

8. A display interface layout control system based on user operation analysis, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the display interface layout control method based on user operation analysis as described in any one of claims 1-6.

Images