Display interface layout control method and system based on user operation analysis

Abstract

The invention discloses a display interface layout control method and system based on user operation analysis. The method comprises the following steps: acquiring historical operation data of a user received by display equipment; according to the historical operation data, determining a preference parameter of the user for any display element in a display interface of the display equipment; determining important parameters of each display element according to a function scene corresponding to the display equipment and a preset scene rule; and determining a layout scheme corresponding to the display interface based on a dynamic programming algorithm according to the preference parameters and the important parameters of all the display elements. Visibly, according to the method and the device, a more reasonable and intelligent display interface layout can be accurately determined based on the operation analysis of the user, so that the display effect of a low-cost screen is improved, the power consumption of unnecessary display is reduced, and better use experience is provided for the user.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a display interface layout control method and system based on user operation analysis. Background Art

[0002] With the continuous development of the terminal display industry and the continuous improvement of science and technology, many terminal control devices have been born. Some of these devices have low requirements for the display interface and only need to display the simple display of the terminal function operation function and the confirmation of the operation. Therefore, some low-cost screens such as serial port screens are needed to realize the operation interface display. However, since most low-cost screens are only equipped with processors with poor performance, the intelligence of their display is generally poor. Most of the display control technologies of low-cost screens in the prior art only display and respond to user controls based on preset display interface layout rules, and do not further control the layout of the display interface based on user operation analysis. Therefore, their display effect is poor and their intelligence is low, and they cannot provide users with a better user experience. It can be seen that the prior art has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a display interface layout control method and system based on user operation analysis, which can accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a display interface layout control method based on user operation analysis, the method comprising: Obtain the historical operation data of the user received by the display device; Determining, according to the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; Determine important parameters of each of the display elements according to the functional scenario corresponding to the display device and preset scenario rules; According to the preference parameters and the important parameters of all the display elements, based on a dynamic programming algorithm, a layout scheme corresponding to the display interface is determined; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

[0005] As an optional implementation, in the first aspect of the present invention, the historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0006] As an optional implementation, in the first aspect of the present invention, determining, based on the historical operation data, the user's preference parameter for any display element in the display interface of the display device includes: Determine the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the operation preference corresponding to the display element; Counting the number of intention-affected positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the intention preference corresponding to the display element; A weighted average value of the operation preference and the intention preference is calculated to obtain a preference parameter corresponding to the display element.

[0007] As an optional implementation, in the first aspect of the present invention, determining the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device includes: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is the touch screen operation, determining the operation position coordinates corresponding to the operation as the operation position corresponding to the operation; When the operation type is the button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; Sorting the operation positions corresponding to the multiple continuous operations from early to late based on the corresponding historical time points to obtain an operation position sequence; The operation position sequence is input into a trained LSTM neural network to obtain the intended impact position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intended impact position annotations.

[0008] As an optional embodiment, in the first aspect of the present invention, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all the operations corresponding to the operation preference and the set of all the operations corresponding to the intention preference; the total number of operations is the total number of all the operations corresponding to the operation preference and the intention preference.

[0009] As an optional implementation, in the first aspect of the present invention, the functional scenario is a car scene, a medical scene, a manufacturing line scene, a smart home scene, a smart cooking scene, a clothes drying scene or a nebulizer scene.

[0010] As an optional implementation, in the first aspect of the present invention, determining the important parameters of each of the display elements according to the functional scenario corresponding to the display device and the preset scenario rules includes: Determine scene history layout data corresponding to the functional scene corresponding to the display device; the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each of the display elements, determining a layout position corresponding to the display element in all the historical element layout schemes; Calculating an average value of the position distance between each of the layout positions and at least one key position to obtain a key parameter of each of the layout positions; the key position is a position preset on the display interface of the display device that is easily noticed by the user; The weighted sum average of the key parameters of all the layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter.

[0011] As an optional implementation, in the first aspect of the present invention, determining the layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements based on a dynamic programming algorithm includes: For each of the display elements, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; wherein the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of the display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; the central position is the center point of the display interface of the display device; The element size of the display element with a higher display priority parameter in the layout scheme is larger; During the calculation, the display element may be set in an area outside the display interface; The display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are eliminated; Based on a dynamic programming algorithm, according to the objective function and the constraint conditions, an iterative layout calculation is performed on all the display elements to obtain an optimal layout solution, which is determined as the layout solution corresponding to the display interface.

[0012] A second aspect of an embodiment of the present invention discloses a display interface layout control system based on user operation analysis, the system comprising: An acquisition module, used to acquire the historical operation data of the user received by the display device; A first determination module, configured to determine, based on the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; A second determination module, used to determine important parameters of each of the display elements according to a functional scenario corresponding to the display device and a preset scenario rule; The third determination module is used to determine the layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements based on a dynamic programming algorithm; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

[0013] As an optional implementation, in the second aspect of the present invention, the historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0014] As an optional implementation, in the second aspect of the present invention, the specific manner in which the first determination module determines, based on the historical operation data, the user's preference parameter for any display element in the display interface of the display device includes: Determine the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the operation preference corresponding to the display element; Counting the number of intention-affected positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the intention preference corresponding to the display element; A weighted average value of the operation preference and the intention preference is calculated to obtain a preference parameter corresponding to the display element.

[0015] As an optional implementation, in the second aspect of the present invention, the first determination module determines the specific manner in which the corresponding operation position and the intended impact position of each operation in the historical operation data in the display interface of the display device include: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is the touch screen operation, determining the operation position coordinates corresponding to the operation as the operation position corresponding to the operation; When the operation type is the button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; Sorting the operation positions corresponding to the multiple continuous operations from early to late based on the corresponding historical time points to obtain an operation position sequence; The operation position sequence is input into a trained LSTM neural network to obtain the intended impact position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intended impact position annotations.

[0016] As an optional embodiment, in the second aspect of the present invention, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all the operations corresponding to the operation preference and the set of all the operations corresponding to the intention preference; the total number of operations is the total number of all the operations corresponding to the operation preference and the intention preference.

[0017] As an optional implementation, in the second aspect of the present invention, the functional scenario is a car scene, a medical scene, a manufacturing line scene, a smart home scene, a smart cooking scene, a clothes drying scene or a nebulizer scene.

[0018] As an optional implementation, in the second aspect of the present invention, the second determination module determines the specific manner of the important parameters of each of the display elements according to the functional scenario corresponding to the display device and the preset scenario rules, including: Determine scene history layout data corresponding to the functional scene corresponding to the display device; the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each of the display elements, determining a layout position corresponding to the display element in all the historical element layout schemes; Calculating an average value of the position distance between each of the layout positions and at least one key position to obtain a key parameter of each of the layout positions; the key position is a position preset on the display interface of the display device that is easily noticed by the user; The weighted sum average of the key parameters of all the layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter.

[0019] As an optional implementation, in the second aspect of the present invention, the third determination module determines the specific manner of the layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements based on a dynamic programming algorithm, including: For each of the display elements, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; wherein the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of the display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; the central position is the center point of the display interface of the display device; The element size of the display element with a higher display priority parameter in the layout scheme is larger; During the calculation, the display element may be set in an area outside the display interface; The display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are eliminated; Based on a dynamic programming algorithm, according to the objective function and the constraint conditions, an iterative layout calculation is performed on all the display elements to obtain an optimal layout solution, which is determined as the layout solution corresponding to the display interface.

[0020] The third aspect of the present invention discloses another display interface layout control system based on user operation analysis, the system comprising: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute part or all of the steps in the display interface layout control method based on user operation analysis disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the display interface layout control method based on user operation analysis disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention can determine the user's preference parameters for any display element in the display interface of the display device based on historical operation data, and then determine the important parameters of each display element based on the functional scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, thereby being able to accurately determine a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and provide users with a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of a display interface layout control method based on user operation analysis disclosed in an embodiment of the present invention.

[0025] Figure 2 It is a structural schematic diagram of a display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention.

[0026] Figure 3It is a structural schematic diagram of another display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] The present invention discloses a display interface layout control method and system based on user operation analysis, which can determine the user's preference parameters for any display element in the display interface of a display device based on historical operation data, and then determine the important parameters of each display element based on the functional scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, so as to accurately determine a more reasonable and intelligent display interface layout based on the user's operation analysis, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and give the user a better user experience. The following are detailed descriptions.

[0031] Embodiment 1 See also Figure 1 , Figure 1 is a flow chart of a display interface layout control method based on user operation analysis disclosed in an embodiment of the present invention. Figure 1The display interface layout control method based on user operation analysis described above can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the display interface layout control method based on user operation analysis may include the following operations: 101. Obtain historical operation data of the user received by a display device.

[0032] 102. Determine, based on the historical operation data, a user's preference parameter for any display element in a display interface of the display device.

[0033] 103. Determine important parameters of each display element according to the functional scenario corresponding to the display device and preset scenario rules.

[0034] 104. According to the preference parameters and important parameters of all display elements, based on a dynamic programming algorithm, determine a layout solution corresponding to the display interface.

[0035] Optionally, the layout scheme is used to be sent to a display device to control the display of at least one display element in the display interface.

[0036] Optionally, when the solution of the present invention is applied in a serial port screen scenario, the above-mentioned historical operation data can be obtained based on the communication between the server and the serial port screen, and after the solution of the present invention is executed on a local or cloud server to determine the layout plan, the layout plan is packaged into data in XML format and sent to the serial port screen for layout control.

[0037] It can be seen that the above-mentioned embodiment of the invention can determine the user's preference parameters for any display element in the display interface of the display device based on historical operation data, and then determine the important parameters of each display element based on the functional scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, so as to accurately determine a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and give the user a better user experience.

[0038] As an optional embodiment, in the above steps, the historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0039] It can be seen that through the above-mentioned optional embodiments, the content of historical operation data is limited, which can fully characterize the user's operation characteristics, facilitate the subsequent prediction of user preferences and generation of layout plans, and assist in accurately determining a more reasonable and intelligent display interface layout based on user operation analysis, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better user experience.

[0040] As an optional embodiment, in the above step, determining the user's preference parameter for any display element in the display interface of the display device according to the historical operation data includes: Determine the corresponding operation position and intended impact position of each operation in the display interface of the display device in the historical operation data; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, and obtaining the operation preference corresponding to the display element; Counting the number of intention-affecting positions corresponding to all operations in the historical operation data that fall within the element position range, and obtaining the intention preference corresponding to the display element; The weighted average of the operation preference and the intention preference is calculated to obtain the preference parameter corresponding to the display element.

[0041] It can be seen that through the above-mentioned optional embodiments, it is possible to accurately determine the user's preference for different display elements based on the correspondence between the corresponding operation position and the intended impact position of each operation in the display interface in the historical operation data and the position range of different display elements, which is convenient for subsequently generating a more accurate layout plan, and assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and provide users with a better user experience.

[0042] As an optional embodiment, in the above step, determining the corresponding operation position and intended impact position of each operation in the display interface of the display device in the historical operation data includes: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is a touch screen operation, the operation position coordinates corresponding to the operation are determined as the operation position corresponding to the operation; When the operation type is a button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; The operation positions corresponding to the multiple consecutive operations are sorted from early to late based on the corresponding historical time points to obtain an operation position sequence; The operation position sequence is input into the trained LSTM neural network to obtain the intention impact position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intention impact position annotations.

[0043] It can be seen that through the above optional embodiments, it is possible to determine the operation position by determining the type of operation, and predict the intended impact position of continuous operation positions based on the LSTM network, which is convenient for accurate user preference prediction and generation of more accurate layout solutions, and assists in accurately determining a more reasonable and intelligent display interface layout based on user operation analysis, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better user experience.

[0044] As an optional embodiment, in the above steps, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all operations corresponding to the operation preference and the set of all operations corresponding to the intention preference; the total number of operations is the total number of all operations corresponding to the operation preference and the intention preference.

[0045] It can be seen that through the above-mentioned optional embodiments, the calculation weight rules can be determined by the ratio relationship between the number of operations of the operation preference and the intention preference, so as to effectively and reasonably correct the proportion of the operation preference and the intention preference in the final user preference, and subsequently facilitate the generation of a more accurate layout plan, and assist in realizing a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and give users a better user experience.

[0046] As an optional embodiment, in the above steps, the functional scenario is a car scene, a medical scene, a manufacturing line scene, a smart home scene, a smart cooking scene, a clothes drying scene or a nebulizer scene.

[0047] It can be seen that through the above optional embodiments, the types of functional scenarios are defined to comprehensively and accurately characterize the scenario characteristics of the display device, which is convenient for the subsequent prediction of important parameters of the display elements and the generation of layout plans, and assists in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0048] As an optional embodiment, in the above steps, determining the important parameters of each display element according to the functional scenario corresponding to the display device and the preset scenario rules includes: Determine scene history layout data corresponding to a functional scene corresponding to the display device; optionally, the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each display element, determine the layout position corresponding to the display element in all historical element layout schemes; Calculate the average value of the position distance between each layout position and at least one key position to obtain the key parameter of each layout position; optionally, the key position is a preset position on the display interface of the display device that is easy to be noticed by the user; The weighted sum average of the key parameters of all layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of key positions involved in the calculation process of the key parameter.

[0049] It can be seen that through the above-mentioned optional embodiments, it is possible to accurately determine the important parameters of each display element by counting and calculating the criticality of the layout positions of different display elements in the historical layout schemes of different functional scenarios, which is convenient for subsequently generating a more accurate layout scheme, and assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0050] As an optional embodiment, in the above steps, determining the layout scheme corresponding to the display interface based on the dynamic programming algorithm according to the preference parameters and important parameters of all display elements includes: For each display element, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; optionally, the weight of the weighted calculation of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of a display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; optionally, the central position is the center point of a display interface of the display device; The element size of a display element with a higher display priority parameter in the layout scheme is larger; During calculation, display elements can be set in areas outside the display interface; Display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are removed; Based on the dynamic programming algorithm, according to the objective function and constraints, all display elements are iteratively calculated to obtain the optimal layout solution, which is determined as the layout solution corresponding to the display interface.

[0051] It can be seen that through the above-mentioned optional embodiments, the display priority parameters of the display elements can be accurately determined by weighted calculation of the preference parameters and important parameters of the display elements, and then the optimal layout scheme can be calculated through the dynamic programming algorithm based on the preset objective function and constraints, so as to accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0052] Embodiment 2 See also Figure 2 , Figure 2 : is a schematic diagram of a display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention. Figure 2 The display interface layout control system based on user operation analysis described above can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the display interface layout control system based on user operation analysis may include: The acquisition module 201 is used to acquire the historical operation data of the user received by the display device.

[0053] The first determination module 202 is used to determine the user's preference parameters for any display element in the display interface of the display device according to the historical operation data.

[0054] The second determination module 203 is used to determine the important parameters of each display element according to the functional scenario corresponding to the display device and the preset scenario rules.

[0055] The third determination module 204 is used to determine a layout solution corresponding to the display interface based on the preference parameters and important parameters of all display elements and based on a dynamic programming algorithm.

[0056] Optionally, the layout scheme is used to be sent to a display device to control the display of at least one display element in the display interface.

[0057] It can be seen that the above-mentioned embodiment of the invention can determine the user's preference parameters for any display element in the display interface of the display device based on historical operation data, and then determine the important parameters of each display element based on the functional scenario corresponding to the display device, so as to finally determine the layout scheme corresponding to the display interface based on the preference parameters and important parameters of the display elements, so as to accurately determine a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and give the user a better user experience.

[0058] As an optional embodiment, the historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

[0059] It can be seen that through the above-mentioned optional embodiments, the content of historical operation data is limited, which can fully characterize the user's operation characteristics, facilitate the subsequent prediction of user preferences and generation of layout plans, and assist in accurately determining a more reasonable and intelligent display interface layout based on user operation analysis, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better user experience.

[0060] As an optional embodiment, the first determining module determines the specific manner in which the user's preference parameter for any display element in the display interface of the display device is determined according to the historical operation data, including: Determine the corresponding operation position and intended impact position of each operation in the display interface of the display device in the historical operation data; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, and obtaining the operation preference corresponding to the display element; Counting the number of intention-affecting positions corresponding to all operations in the historical operation data that fall within the element position range, and obtaining the intention preference corresponding to the display element; The weighted average of the operation preference and the intention preference is calculated to obtain the preference parameter corresponding to the display element.

[0061] It can be seen that through the above-mentioned optional embodiments, it is possible to accurately determine the user's preference for different display elements based on the correspondence between the corresponding operation position and the intended impact position of each operation in the display interface in the historical operation data and the position range of different display elements, which is convenient for subsequently generating a more accurate layout plan, and assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and provide users with a better user experience.

[0062] As an optional embodiment, the first determination module determines the specific manner in which the operation position corresponding to each operation in the display interface of the display device and the intended impact position in the historical operation data includes: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is a touch screen operation, the operation position coordinates corresponding to the operation are determined as the operation position corresponding to the operation; When the operation type is a button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; The operation positions corresponding to the multiple consecutive operations are sorted from early to late based on the corresponding historical time points to obtain an operation position sequence; The operation position sequence is input into the trained LSTM neural network to obtain the intention impact position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intention impact position annotations.

[0063] It can be seen that through the above optional embodiments, it is possible to determine the operation position by determining the type of operation, and predict the intended impact position of continuous operation positions based on the LSTM network, which is convenient for accurate user preference prediction and generation of more accurate layout solutions, and assists in accurately determining a more reasonable and intelligent display interface layout based on user operation analysis, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and give users a better user experience.

[0064] As an optional embodiment, when calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all operations corresponding to the operation preference and the set of all operations corresponding to the intention preference; the total number of operations is the total number of all operations corresponding to the operation preference and the intention preference.

[0065] It can be seen that through the above-mentioned optional embodiments, the calculation weight rules can be determined by the ratio relationship between the number of operations of the operation preference and the intention preference, so as to effectively and reasonably correct the proportion of the operation preference and the intention preference in the final user preference, and subsequently facilitate the generation of a more accurate layout plan, and assist in realizing a more reasonable and intelligent display interface layout based on the analysis of the user's operations, so as to improve the display effect of the low-cost screen, reduce the power consumption of unnecessary display, and give users a better user experience.

[0066] As an optional embodiment, the functional scenario is a car scene, a medical scene, a manufacturing line scene, a smart home scene, a smart cooking scene, a clothes drying scene or a nebulizer scene.

[0067] It can be seen that through the above optional embodiments, the types of functional scenarios are defined to comprehensively and accurately characterize the scenario characteristics of the display device, which is convenient for the subsequent prediction of important parameters of the display elements and the generation of layout plans, and assists in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0068] As an optional embodiment, the second determination module determines the specific manner of the important parameters of each display element according to the functional scenario corresponding to the display device and the preset scenario rule, including: Determine scene history layout data corresponding to a functional scene corresponding to the display device; optionally, the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each display element, determine the layout position corresponding to the display element in all historical element layout schemes; Calculate the average value of the position distance between each layout position and at least one key position to obtain the key parameter of each layout position; optionally, the key position is a preset position on the display interface of the display device that is easy to be noticed by the user; The weighted sum average of the key parameters of all layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of key positions involved in the calculation process of the key parameter.

[0069] It can be seen that through the above-mentioned optional embodiments, it is possible to accurately determine the important parameters of each display element by counting and calculating the criticality of the layout positions of different display elements in the historical layout schemes of different functional scenarios, which is convenient for subsequently generating a more accurate layout scheme, and assisting in accurately determining a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0070] As an optional embodiment, the third determination module determines the specific manner of the layout scheme corresponding to the display interface based on the preference parameters and important parameters of all display elements and the dynamic programming algorithm, including: For each display element, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; optionally, the weight of the weighted calculation of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of a display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; optionally, the central position is the center point of a display interface of the display device; The element size of a display element with a higher display priority parameter in the layout scheme is larger; During calculation, display elements can be set in areas outside the display interface; Display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are removed; Based on the dynamic programming algorithm, according to the objective function and constraints, all display elements are iteratively calculated to obtain the optimal layout solution, which is determined as the layout solution corresponding to the display interface.

[0071] It can be seen that through the above-mentioned optional embodiments, the display priority parameters of the display elements can be accurately determined by weighted calculation of the preference parameters and important parameters of the display elements, and then the optimal layout scheme can be calculated through the dynamic programming algorithm based on the preset objective function and constraints, so as to accurately determine a more reasonable and intelligent display interface layout based on the analysis of user operations, so as to improve the display effect of low-cost screens, reduce the power consumption of unnecessary displays, and provide users with a better user experience.

[0072] Embodiment 3 See also Figure 3 , Figure 3 This is another display interface layout control system based on user operation analysis disclosed in an embodiment of the present invention. Figure 3 The display interface layout control system based on user operation analysis is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the display interface layout control system based on user operation analysis may include: A memory 301 storing executable program codes; a processor 302 coupled to the memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the display interface layout control method based on user operation analysis described in the first embodiment.

[0073] Embodiment 4 An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the display interface layout control method based on user operation analysis described in the first embodiment.

[0074] Embodiment 5 An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the display interface layout control method based on user operation analysis described in the first embodiment.

[0075] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0076] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0077] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0078] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0079] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0082] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0083] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0084] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0085] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0086] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0087] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0088] Finally, it should be noted that the display interface layout control method and system based on user operation analysis disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A display interface layout control method based on user operation analysis, characterized in that: The method comprises: Obtain the historical operation data of the user received by the display device; Determining, according to the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; Determine important parameters of each of the display elements according to the functional scenario corresponding to the display device and preset scenario rules; According to the preference parameters and the important parameters of all the display elements, based on a dynamic programming algorithm, a layout scheme corresponding to the display interface is determined; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

2. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The historical operation data includes the user's operations on the display device at multiple historical time points; the operations are touch screen operations or button operations.

3. The display interface layout control method based on user operation analysis according to claim 2, characterized in that: The determining, according to the historical operation data, the user's preference parameter for any display element in the display interface of the display device includes: Determine the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device; For each display element in the display interface, determining an element position range corresponding to the display element; Counting the number of operation positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the operation preference corresponding to the display element; Counting the number of intention-affected positions corresponding to all operations in the historical operation data that fall within the element position range, to obtain the intention preference corresponding to the display element; A weighted average value of the operation preference and the intention preference is calculated to obtain a preference parameter corresponding to the display element.

4. The display interface layout control method based on user operation analysis according to claim 3, characterized in that: The determining the corresponding operation position and intended impact position of each operation in the historical operation data in the display interface of the display device includes: For each operation in the historical operation data, determining an operation type corresponding to the operation; When the operation type is the touch screen operation, determining the operation position coordinates corresponding to the operation as the operation position corresponding to the operation; When the operation type is the button operation, based on a preset button operation simulation algorithm, the operation is simulated to obtain an operation position corresponding to the operation; Obtain the operation and operations at a preset number of historical time points before the historical time point corresponding to the operation, to obtain multiple continuous operations; Sorting the operation positions corresponding to the multiple continuous operations from early to late based on the corresponding historical time points to obtain an operation position sequence; The operation position sequence is input into a trained LSTM neural network to obtain the intended impact position corresponding to the operation; the LSTM neural network is trained by a training data set including multiple training operation position sequences and corresponding intended impact position annotations.

5. The display interface layout control method based on user operation analysis according to claim 3, characterized in that: When calculating the preference parameters, the weighted calculation weight corresponding to the intention preference is less than the weighted calculation weight corresponding to the operation preference, and the weighted calculation weight corresponding to the intention preference is inversely proportional to the degree of operation intersection; the degree of operation intersection is the ratio of the number of operation intersections to the total number of operations; the number of operation intersections is the number of intersections of the set of all the operations corresponding to the operation preference and the set of all the operations corresponding to the intention preference; the total number of operations is the total number of all the operations corresponding to the operation preference and the intention preference.

6. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The functional scenario is a car computer scenario, a medical scenario, a manufacturing line scenario, a smart home scenario, a smart cooking scenario, a clothes drying scenario or a nebulizer scenario.

7. The display interface layout control method based on user operation analysis according to claim 6, characterized in that: Determining the important parameters of each of the display elements according to the functional scenario corresponding to the display device and the preset scenario rules includes: Determine scene history layout data corresponding to the functional scene corresponding to the display device; the scene history layout data includes a plurality of historical element layout schemes of the corresponding functional scene; For each of the display elements, determining a layout position corresponding to the display element in all the historical element layout schemes; Calculating an average value of the position distance between each of the layout positions and at least one key position to obtain a key parameter of each of the layout positions; the key position is a position preset on the display interface of the display device that is easily noticed by the user; The weighted sum average of the key parameters of all the layout positions is calculated to obtain the important parameters of the display element; wherein the weighted calculation weight corresponding to each key parameter is proportional to the number of the key positions involved in the calculation process of the key parameter.

8. The display interface layout control method based on user operation analysis according to claim 1, characterized in that: The determining, based on the preference parameters and the important parameters of all the display elements and a dynamic programming algorithm, a layout scheme corresponding to the display interface includes: For each of the display elements, a weighted sum of the preference parameter and the important parameter corresponding to the display element is calculated to obtain a display priority parameter corresponding to the display element; wherein the weighted calculation weight of the preference parameter is greater than the weighted calculation weight of the important parameter; Setting the objective function includes maximizing the similarity between the layout scheme and the current layout scheme of the display interface and maximizing the number of display elements in the layout scheme; Setting restrictions includes: The layout position of the display element with a higher display priority parameter in the layout scheme has a smaller distance from the central position; the central position is the center point of the display interface of the display device; The element size of the display element with a higher display priority parameter in the layout scheme is larger; During the calculation, the display element may be set in an area outside the display interface; The display elements in the layout scheme whose area outside the display interface is larger than the area inside the display interface are eliminated; Based on a dynamic programming algorithm, according to the objective function and the constraint conditions, an iterative layout calculation is performed on all the display elements to obtain an optimal layout solution, which is determined as the layout solution corresponding to the display interface.

9. A display interface layout control system based on user operation analysis, characterized in that: The system comprises: An acquisition module, used to acquire the historical operation data of the user received by the display device; A first determination module, configured to determine, based on the historical operation data, a preference parameter of the user for any display element in the display interface of the display device; A second determination module, used to determine important parameters of each of the display elements according to a functional scenario corresponding to the display device and a preset scenario rule; The third determination module is used to determine the layout scheme corresponding to the display interface based on the preference parameters and the important parameters of all the display elements based on a dynamic programming algorithm; the layout scheme is used to be sent to the display device to control the display of at least one of the display elements in the display interface.

10. A display interface layout control system based on user operation analysis, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the display interface layout control method based on user operation analysis as described in any one of claims 1-8.

Images